Video display adjustment method, device, equipment and storage medium

By caching video data and using image restoration model to process keyframe images, the impact of dynamic blur videos during display is solved, and the user's viewing experience and video playback fluency is improved.

CN118175352BActive Publication Date: 2024-12-06SHENZHEN AOLY TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410586228.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-13
Publication Date
2024-12-06
Estimated Expiration
2044-05-13

AI Technical Summary

Technical Problem

The prior art is difficult to effectively solve the impact of dynamic blurred videos on user viewing when displayed, especially the video blur problem caused by shooting objects or camera movements.

Method used

By cached video data, extract keyframe images and use preset image restoration models to restore images, obtain target keyframe images, update the buffered video frame collection, and send it to the video playback for display, reducing the video blur period to view users The impact of

Benefits of technology

It improves the smoothness of video playback and user viewing experience, reduces the workload of video image frame processing, and avoids the impact of dynamic blur on users' normal viewing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118175352B_ABST
    Figure CN118175352B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of video display technology, and in particular to a video display adjustment method, device, equipment and storage medium. The present invention obtains a buffered video image frame set by caching received video data, so as to adjust the image frames in the video before the video is displayed to enhance the fluency of subsequent playback, and then extracts the key frame image in the buffered video image frame set. When the key frame image is the first screen frame, the key frame image is restored by a preset image restoration model to obtain a target key frame image; finally, the buffered video image frame set is updated according to the target key frame image, and the updated buffered video frame set is sent to a video playback end for display, thereby reducing the impact of blurred periods in the video on user viewing, avoiding the technical problem in the prior art that the dynamic blurred video affects the normal viewing of the user when it is displayed, and improving the user's viewing experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of video display technology, and in particular to a video display adjustment method, device, equipment and storage medium. Background Art

[0002] When recording a video, the camera may be affected by the movement of the object being filmed or the movement of the camera, resulting in blurred videos. To eliminate the problem of blurred videos, the video needs to be processed by professionals, but most people do not have this ability. For example, when shooting a moving vehicle, the front and rear ends of the vehicle will be blurred due to the exposure time, which will affect the user's normal viewing during the subsequent video playback.

[0003] Therefore, how to reduce the viewing impact of blur on users when displaying videos with motion blur on a display screen has become an urgent problem to be solved.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide a video display adjustment method, device, equipment and storage medium, aiming to solve the technical problem in the prior art that when displaying a video with dynamic blur, it affects the normal viewing of the user.

[0006] To achieve the above object, the present invention provides a video display adjustment method, the method comprising the following steps:

[0007] Buffering the received video data to obtain a buffered video image frame set;

[0008] Extracting key frame images from the buffered video image frame set;

[0009] When the key frame image is the first screen frame, the key frame image is restored by a preset image restoration model to obtain a target key frame image;

[0010] The buffered video image frame set is updated according to the target key frame image, and the updated buffered video frame set is sent to the video playback end for display.

[0011] Optionally, restoring the key frame image through a preset image restoration model to obtain a target key frame image includes:

[0012] Splitting the key frame image to obtain a plurality of sub-key frame images;

[0013] Determine a first key frame image representing global blur, a second key frame image representing motion blur, and remaining sub-key frame images among the plurality of sub-key frame images;

[0014] respectively determining a first degraded key frame image corresponding to the first key frame image and a second degraded key frame image corresponding to the second key frame image;

[0015] Image combination is performed according to the first degraded key frame image, the second degraded key frame image and the remaining sub-key frame images to obtain a target key frame image.

[0016] Optionally, the respectively determining a first degraded key frame image corresponding to the first key frame image and a second degraded key frame image corresponding to the second key frame image includes:

[0017] Performing Fourier transformation on the first key frame image and the second key frame image respectively to obtain first image transformation data and second key frame image transformation data;

[0018] Performing frequency domain convolution on the first image conversion data and the second key frame image conversion data with a preset degradation function respectively to obtain first convolution data and second convolution data;

[0019] The first convolution data and the second convolution data are inversely transformed to obtain a first degraded key frame image corresponding to the first image conversion data and a second degraded key frame image corresponding to the second image conversion data.

[0020] Optionally, the performing image combination according to the first degraded key frame image, the second degraded key frame image and the remaining sub-key frame images to obtain a target key frame image includes:

[0021] Restoring the first degraded key frame image and the second degraded key frame image to obtain a first restored key frame image and a second restored key frame image;

[0022] The first restored key frame image, the second restored key frame image and the remaining sub-key frame images are combined according to a preset image position relationship to obtain a target key frame image, wherein the preset image position relationship is the position relationship between the first key frame image, the second key frame image and the remaining sub-key frame images.

[0023] Optionally, restoring the degraded key frame image to obtain a restored key frame image includes:

[0024] Calculating convolution data corresponding to the degraded key frame image;

[0025] Determining fuzzy parameters of a preset degradation function and calculating the power spectral density of the convolution data;

[0026] Constructing a restoration filter according to the fuzzy parameter and the power spectrum density;

[0027] The convolution data is restored by the restoration filter to obtain a restored key frame image.

[0028] Optionally, restoring the degraded key frame image to obtain a restored key frame image further includes:

[0029] Calculating convolution data corresponding to the degraded key frame image;

[0030] Iteratively optimize the model parameters of the preset filtering model and determine the optimization constraints;

[0031] Based on the optimization constraint condition, the convolution data is iteratively calculated through a preset filtering model to obtain a restored key frame image.

[0032] Optionally, updating the buffered video image frame set according to the target key frame image, and sending the updated buffered video frame set to the video playback end for display, includes:

[0033] Replacing the key frame image in the buffered video image frame set with the target key frame image to obtain an updated buffered video frame set;

[0034] Obtaining the timestamp of each frame image in the buffered video image frame set;

[0035] Each image frame in the buffered video image frame set is sent to a video playback end for display according to the timestamp of each frame image.

[0036] In addition, to achieve the above-mentioned purpose, the present invention further provides a video display adjustment device, the video display adjustment device comprising:

[0037] A cache module, used for caching received video data to obtain a buffered video image frame set;

[0038] An extraction module, used for extracting key frame images from the buffered video image frame set;

[0039] A restoration module, used for restoring the key frame image through a preset image restoration model to obtain a target key frame image when the key frame image is the first screen frame;

[0040] The display module is used to update the buffered video image frame set according to the target key frame image, and send the updated buffered video frame set to the video playback end for display.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also proposes a video display adjustment device, which includes: a memory, a processor, and a video display adjustment program stored in the memory and executable on the processor, wherein the video display adjustment program is configured to implement the steps of the video display adjustment method described above.

[0042] In addition, to achieve the above-mentioned purpose, the present invention further proposes a storage medium, on which a video display adjustment program is stored, and when the video display adjustment program is executed by a processor, the steps of the video display adjustment method described above are implemented.

[0043] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the video display adjustment method described above are implemented.

[0044] One or more technical solutions proposed in the present application have at least the following technical effects: the present invention obtains a buffered video image frame set by caching the received video data, so as to adjust the image frames in the video before the video is displayed to enhance the smoothness of subsequent playback, and then extract the key frame images in the buffered video image frame set to reduce the workload of video image frame processing; when the key frame image is the first screen frame, the key frame image is restored by a preset image restoration model to obtain a target key frame image; finally, the buffered video image frame set is updated according to the target key frame image, and the updated buffered video frame set is sent to the video playback end for display, thereby reducing the impact of blurred periods in the video on user viewing, avoiding the technical problem in the prior art that dynamic blurred video affects the user's normal viewing when displayed, and improving the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] Figure 1 It is a flow chart of a first embodiment of a video display adjustment method of the present invention;

[0048] Figure 2It is a flow chart of a second embodiment of the video display adjustment method of the present invention;

[0049] Figure 3 A schematic diagram of a key frame image restoration example of an embodiment of a video display adjustment method of the present invention;

[0050] Figure 4 It is a structural block diagram of a first embodiment of a video display adjustment device of the present invention;

[0051] Figure 5 It is a structural schematic diagram of a video display adjustment device in a hardware operating environment involved in an embodiment of the present invention.

[0052] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0054] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0055] The main solution of the embodiment of the present application is: cache the received video data to obtain a buffered video image frame set; extract the key frame image in the buffered video image frame set; when the key frame image is the first screen frame, restore the key frame image through a preset image restoration model to obtain a target key frame image; update the buffered video image frame set according to the target key frame image, and send the updated buffered video frame set to the video playback end for display.

[0056] In this embodiment, for the convenience of description, the following description is made by taking the identification of a device having a display screen as the execution subject.

[0057] In the prior art, when recording a video, the camera will be affected by the motion of the object being filmed or the camera, resulting in blurry videos. To eliminate the blurry video problem, professionals are required to process the video, but most people do not have this capability. For example, when filming a moving vehicle, the front and rear ends of the vehicle will be blurred due to the exposure time, which will affect the user's normal viewing during the subsequent video playback. Therefore, how to reduce the viewing impact of blurry videos on users when they are displayed on a display screen has become an urgent problem to be solved.

[0058] The present application provides a solution, which obtains a buffered video image frame set by caching received video data, so as to adjust the image frames in the video before the video is displayed to enhance the smoothness of subsequent playback, and then extract the key frame images in the buffered video image frame set to reduce the workload of video image frame processing. When the key frame image is the first screen frame, the key frame image is restored by a preset image restoration model to obtain a target key frame image; finally, the buffered video image frame set is updated according to the target key frame image, and the updated buffered video frame set is sent to the video playback end for display, thereby reducing the impact of the blurred period in the video on the user's viewing.

[0059] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, video display adjustment, etc. The following takes video display adjustment as an example to illustrate this embodiment and the following embodiments.

[0060] Based on this, an embodiment of the present invention provides a video display adjustment method, referring to Figure 1 , Figure 1 FIG. 4 is a flow chart of a first embodiment of a video display adjustment method according to the present invention.

[0061] In this embodiment, the video display adjustment method includes:

[0062] Step S10: Buffer the received video data to obtain a buffered video image frame set.

[0063] It should be noted that the executor of the method of this embodiment can be a device with functions such as data processing, program running and video display, such as: a display screen control chip or a processor with a display screen, etc. This embodiment does not make specific restrictions on this. In this embodiment and subsequent embodiments, the display screen control chip will be used as an example for explanation.

[0064] It should be noted that in most cases, videos taken by individuals are affected by factors such as the equipment, shooting angle and object movement, and the captured videos often have motion blur, such as wedding car videos or car video data, and users generally do not have the ability to correct the motion blur, resulting in the video appearing confusing and unclear during subsequent playback, affecting the user's normal viewing.

[0065] Therefore, the present application sets a video display adjustment program at the video display end, that is, the display screen, to pre-process the video data transmitted to the display screen, and then displays the processed video data, thereby reducing the impact of the video quality itself on the playback quality and improving the user's viewing experience. Since the controller of the traditional display screen itself has a certain amount of memory space reserved, the video display adjustment program of the present application does not require additional storage devices to achieve the effect of video display adjustment, and the cost is relatively low.

[0066] In a specific implementation, considering that video data may occupy a large amount of memory and take a long time to process, the video data received by the cache in this embodiment may be the video frame data of the most recent multiple GOPs of the video data stream received by the cache display screen, wherein a video frame of a GOP includes a key frame (each GOP starts with a key frame, so the key frame is also called the first screen frame or I frame) and several P frames or B frames, and the larger the GOP value, the more P and B frames there will be, the corresponding image quality will be better, and the video playback will be smoother. This embodiment subsequently adjusts the image frame, that is, the I frame, wherein the number of stored GOPs is set according to the video playback clarity, and the higher the clarity, the more GOPs are pre-stored to avoid video display freezes.

[0067] The buffered video image frame set includes multiple I frames and P frames or B frames corresponding to each I frame. The entire video data is not pre-stored in order to reduce memory usage, reduce video preprocessing time, and reduce the waiting time for video playback.

[0068] Step S20: extracting key frame images from the buffered video image frame set.

[0069] To extract key frame images from a buffered video image frame set, image key frame extraction technology may be used, and this embodiment does not impose any specific restrictions on this. At the same time, due to the video quality, there may be a large number of image frames in the buffered video image frame set. When extracting key frame images, it is inevitable that non-first screen frames are also determined as key frame images during video playback. Therefore, this embodiment also needs to filter out key frame images from first screen frames before preprocessing the image frame data to improve the efficiency of subsequent video playback and reduce waiting time.

[0070] Step S30: when the key frame image is the first screen frame, the key frame image is restored by using a preset image restoration model to obtain a target key frame image.

[0071] It is worth noting that, in this embodiment, image restoration of key frame images refers to removing or reducing the blur of key frame images through a pre-set image restoration model. Since the blur of image frames in the video is caused by the movement of the object or the movement of the camera during shooting, which fails to capture all the features of the object, resulting in blurriness in some areas when the video is played. Therefore, in this embodiment, the blurred key frame images are degraded to restore the features of each object in the image, and then the features of each object are reconstructed to achieve the function of clarifying or reducing the blur.

[0072] Step S40: updating the buffered video image frame set according to the target key frame image, and sending the updated buffered video frame set to the video playback end for display.

[0073] As mentioned above, since the first screen frame of each GOP during video playback represents the characteristic information of the GOP to the maximum extent, and for high-bitrate videos, there may be multiple GOPs forming a scene, the video playback quality can be maximized by restoring each first screen frame. At the same time, compared with the workload of processing the buffered video frame set of the entire video, the workload of processing the first screen frame is lower.

[0074] Further, updating the buffered video image frame set according to the target key frame image, and sending the updated buffered video frame set to the video playback end for display, includes:

[0075] Replacing the key frame image in the buffered video image frame set with the target key frame image to obtain an updated buffered video frame set;

[0076] Obtaining the timestamp of each frame image in the buffered video image frame set;

[0077] Each image frame in the buffered video image frame set is sent to a video playback end for display according to the timestamp of each frame image.

[0078] In a specific implementation, when caching video data, since the buffered video image frame set contains a large number of frame images, in order to prevent garbled characters or frame skipping during subsequent video playback, this embodiment replaces the key frame image in the buffered video image frame set with the target key frame image. It can then transmit them in sequence according to the timestamps of each frame image in the buffered video image frame set, so that the video playback end plays them in sequence according to the timestamps of each frame image, thereby improving the user viewing experience.

[0079] This embodiment obtains a buffered video image frame set by caching the received video data, so as to adjust the image frames in the video before the video is displayed to enhance the smoothness of subsequent playback, and then extract the key frame images in the buffered video image frame set to reduce the workload of video image frame processing. When the key frame image is the first screen frame, the key frame image is restored by a preset image restoration model to obtain a target key frame image; finally, the buffered video image frame set is updated according to the target key frame image, and the updated buffered video frame set is sent to the video playback end for display, thereby reducing the impact of blurred periods in the video on user viewing, avoiding the technical problem in the prior art that dynamic blurred video affects the user's normal viewing when displayed, and improving the user's viewing experience.

[0080] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , step S30, comprising:

[0081] Step S301: split the key frame image to obtain a plurality of sub-key frame images.

[0082] It should be noted that the reference Figure 3 Splitting a key frame image refers to splitting a key frame image into multiple sub-key frame images of the same size. In the specific splitting process, the key frame image can be preliminarily split first, and the grayscale value of each sub-key frame image can be calculated to adjust the size of each sub-key frame image. The key frame image can also be split according to a set number or size. This embodiment does not impose any specific restrictions on this.

[0083] In a specific implementation, since the entire image in a key frame image is rarely in a blurred state, most of the time, some areas or boundaries of some objects are blurred. In order to improve the image restoration effect, this embodiment can perform adaptive adjustments to the blurred sub-areas. It should be clear that the image restoration process will have a certain impact on the clarity of the clear image or area. This embodiment adjusts all the key frame images so that the clarity of each key frame image in the video data tends to be consistent, thereby avoiding the user's sense of abruptness for the pre-processed video of some time periods, affecting the normal viewing experience.

[0084] In addition, during the video shooting process, if the blur is caused by lens movement, the blurred image range will be larger. By determining the representative sub-image blocks of the entire image by dividing the image into blocks, the image restoration effect can be improved. If the blur is caused by the movement of the photographed object, in the entire image, only some areas may be blurred. At this time, restoring this area can reduce the workload and will not affect the clarity of other areas.

[0085] Step S302: Determine a first key frame image representing global blur, a second key frame image representing motion blur, and remaining sub-key frame images among the plurality of sub-key frame images.

[0086] In this embodiment, since both the shooting lens and the shooting object may be in motion during the shooting process of the video data, causing the video to be blurred, when restoring the key frame image, the image restoration is generally performed separately for both situations to improve the image quality and thus improve the display quality of the video.

[0087] The first key frame image representing global blur generally refers to a sub-image area caused by the motion of the shooting lens, and the second key frame image representing motion blur generally refers to a sub-image area caused by the motion of the shooting object.

[0088] Step S303: respectively determining a first degraded key frame image corresponding to the first key frame image and a second degraded key frame image corresponding to the second key frame image.

[0089] Since the present application adjusts the video display by restoring the key frame images, in this process, restoring the key frame images mainly involves first determining the degraded key frame images of the key frame images, and then restoring the degraded key frame images to improve the clarity of the key frame images, thereby improving the clarity of the video.

[0090] Further, the respectively determining a first degraded key frame image corresponding to the first key frame image and a second degraded key frame image corresponding to the second key frame image includes:

[0091] Performing Fourier transformation on the first key frame image and the second key frame image respectively to obtain first image transformation data and second key frame image transformation data;

[0092] Performing frequency domain convolution on the first image conversion data and the second key frame image conversion data with a preset degradation function respectively to obtain first convolution data and second convolution data;

[0093] The first convolution data and the second convolution data are inversely transformed to obtain a first degraded key frame image corresponding to the first image conversion data and a second degraded key frame image corresponding to the second image conversion data.

[0094] In this embodiment, the preset degradation function refers to:

[0095]

[0096] Among them, H (u, v) refers to the preset degradation function, a refers to the displacement in the vertical direction, b refers to the displacement in the horizontal direction, T refers to the exposure time, and u and v refer to the horizontal and vertical axis parameters of the image after Fourier transformation.

[0097] In the specific implementation, the specific calculation formula for frequency domain convolution of the image conversion data and the preset degradation function is:

[0098]

[0099] Among them, G(u,v) refers to convolution data, H(u,v) refers to the preset degradation function, and F(u,v) refers to the image conversion data obtained by Fourier transforming the key frame image.

[0100] Inversely transforming the first convolution data and the second convolution data refers to performing an inverse Fourier transform on the above G(u, v), so as to obtain a degraded key frame image corresponding to the key frame image.

[0101] In this embodiment, random noise gain may be added on the basis of the above formula to improve the image restoration effect.

[0102] Step S304: performing image combination according to the first degraded key frame image, the second degraded key frame image and the remaining sub-key frame images to obtain a target key frame image.

[0103] Since the degraded key frame images have lower definition than the original key frame images, in order to improve the definition of the video key frame images, this embodiment also requires blurring and restoring the degraded key frame images before combining and restoring the key frame images.

[0104] Further, the performing image combination according to the first degraded key frame image, the second degraded key frame image and the remaining sub-key frame images to obtain a target key frame image includes:

[0105] Restoring the first degraded key frame image and the second degraded key frame image to obtain a first restored key frame image and a second restored key frame image;

[0106] The first restored key frame image, the second restored key frame image and the remaining sub-key frame images are combined according to a preset image position relationship to obtain a target key frame image, wherein the preset image position relationship is the position relationship between the first key frame image, the second key frame image and the remaining sub-key frame images.

[0107] Further, restoring the degraded key frame image to obtain a restored key frame image includes:

[0108] Calculating convolution data corresponding to the degraded key frame image;

[0109] Determining fuzzy parameters of a preset degradation function and calculating the power spectral density of the convolution data;

[0110] Constructing a restoration filter according to the fuzzy parameter and the power spectrum density;

[0111] The convolution data is restored by the restoration filter to obtain a restored key frame image.

[0112] Restoring the degraded key frame image to obtain a restored key frame image also includes:

[0113] Calculating convolution data corresponding to the degraded key frame image;

[0114] Iteratively optimize the model parameters of the preset filtering model and determine the optimization constraints;

[0115] Based on the optimization constraint condition, the convolution data is iteratively calculated through a preset filtering model to obtain a restored key frame image.

[0116] In a specific implementation, the process of restoring the degraded key frame image provides two methods in this embodiment. One is to construct a restoration filter according to the fuzzy parameters when the key frame image is degraded in the preset degradation function mentioned above and the power spectral density of the convolution data. The degraded key frame image is restored by the restoration filter. The image is restored with the parameters at the time of degradation, which can minimize the image distortion. Specifically, the fuzzy parameters of the preset degradation function are determined, and the power spectral densities of the first convolution data and the second convolution data are calculated; a restoration filter is constructed according to the fuzzy parameters and the power spectral density; and the first convolution data and the second convolution data are restored by the restoration filter respectively to obtain a first restored key frame image and a second restored key frame image.

[0117] The second is to iteratively optimize the model parameters of the preset filtering model, iteratively calculate the convolution data through the preset filtering model according to the optimization constraints, and obtain the restored key frame image, which is specifically manifested in iteratively optimizing the model parameters of the preset filtering model and determining the optimization constraints; based on the optimization constraints, iteratively calculating the first convolution data and the second convolution data through the preset filtering model respectively, to obtain the first restored key frame image corresponding to the first convolution data and the second restored key frame image corresponding to the second convolution data.

[0118] The preset filtering model refers to:

[0119]

[0120] Among them, P (u, v) is the preset coefficient matrix, and r is the model parameter.

[0121] This embodiment restores the first key frame image representing global blur and the second key frame image representing motion blur respectively, and after restoration, combines the first degraded key frame image, the second degraded key frame image and the remaining sub-key frame images to obtain a target key frame image, thereby restoring the key frame image and reducing the influence of key frame blur on video playback clarity during video display.

[0122] This application also provides a video display adjustment device, please refer to Figure 4 , the video display adjustment device comprises:

[0123] The buffer module 10 is used to buffer the received video data to obtain a buffered video image frame set.

[0124] The extraction module 20 is used to extract key frame images from the buffered video image frame set.

[0125] The restoration module 30 is used to restore the key frame image by using a preset image restoration model to obtain a target key frame image when the key frame image is the first screen frame.

[0126] The display module 40 is used to update the buffered video image frame set according to the target key frame image, and send the updated buffered video frame set to the video playback end for display.

[0127] In one embodiment, the restoration module 30 is further used to split the key frame image to obtain multiple sub-key frame images; determine a first key frame image representing global blur, a second key frame image representing motion blur, and remaining sub-key frame images among the multiple sub-key frame images; respectively determine a first degraded key frame image corresponding to the first key frame image and a second degraded key frame image corresponding to the second key frame image; and perform image combination based on the first degraded key frame image, the second degraded key frame image, and the remaining sub-key frame images to obtain a target key frame image.

[0128] In one embodiment, the restoration module 30 is further used to perform Fourier transform on the first key frame image and the second key frame image, respectively, to obtain first image conversion data and second key frame image conversion data; perform frequency domain convolution on the first image conversion data and the second key frame image conversion data with a preset degradation function, respectively, to obtain first convolution data and second convolution data; and perform inverse transformation on the first convolution data and the second convolution data to obtain a first degraded key frame image corresponding to the first image conversion data and a second degraded key frame image corresponding to the second image conversion data.

[0129] In one embodiment, the restoration module 30 is further used to restore the first degraded key frame image and the second degraded key frame image to obtain a first restored key frame image and a second restored key frame image; the first restored key frame image, the second restored key frame image and the remaining sub-key frame images are combined according to a preset image position relationship to obtain a target key frame image, and the preset image position relationship is the position relationship between the first key frame image, the second key frame image and the remaining sub-key frame images.

[0130] In one embodiment, the restoration module 30 is also used to calculate the convolution data corresponding to the degraded key frame image; determine the fuzzy parameters of the preset degradation function, and calculate the power spectral density of the convolution data; construct a restoration filter based on the fuzzy parameters and the power spectral density; and restore the convolution data through the restoration filter to obtain a restored key frame image.

[0131] In one embodiment, the restoration module 30 is also used to calculate the convolution data corresponding to the degraded key frame image; iteratively optimize the model parameters of the preset filtering model and determine the optimization constraints; based on the optimization constraints, the convolution data is iteratively calculated through the preset filtering model to obtain a restored key frame image.

[0132] In one embodiment, the display module 40 is also used to replace the key frame image in the buffered video image frame set with the target key frame image to obtain an updated buffered video frame set; obtain the timestamp of each frame image in the buffered video image frame set; and send each image frame in the buffered video image frame set to the video playback end for display according to the timestamp of each frame image.

[0133] This embodiment obtains a buffered video image frame set by caching the received video data, so as to adjust the image frames in the video before the video is displayed to enhance the smoothness of subsequent playback, and then extract the key frame images in the buffered video image frame set to reduce the workload of video image frame processing. When the key frame image is the first screen frame, the key frame image is restored by a preset image restoration model to obtain a target key frame image; finally, the buffered video image frame set is updated according to the target key frame image, and the updated buffered video frame set is sent to the video playback end for display, thereby reducing the impact of blurred periods in the video on user viewing, avoiding the technical problem in the prior art that dynamic blurred video affects the user's normal viewing when displayed, and improving the user's viewing experience.

[0134] The video display adjustment device provided by the present application adopts the video display adjustment method in the above embodiment, which can solve the technical problem of video display adjustment. Compared with the prior art, the beneficial effects of the video display adjustment device provided by the present application are the same as the beneficial effects of the video display adjustment method provided by the above embodiment, and other technical features in the video display adjustment device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0135] The present application provides a video display adjustment device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the video display adjustment method in the above-mentioned embodiment 1.

[0136] Reference below Figure 5 , which shows a schematic diagram of the structure of a video display adjustment device suitable for implementing an embodiment of the present application. The video display adjustment device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The video display adjustment device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0137] like Figure 5As shown, the video display adjustment device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the video display adjustment device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the video display adjustment device to communicate with other devices wirelessly or by wire to exchange data. Although the video display adjustment device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0138] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0139] The video display adjustment device provided by the present application adopts the video display adjustment method in the above embodiment to solve the technical problem of video display adjustment. Compared with the prior art, the beneficial effects of the video display adjustment device provided by the present application are the same as the beneficial effects of the video display adjustment method provided by the above embodiment, and other technical features in the video display adjustment device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0140] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0141] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0142] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the video display adjustment method in the above-mentioned embodiment.

[0143] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0144] The computer-readable storage medium may be included in the video display adjustment device; or may exist independently without being assembled into the video display adjustment device.

[0145] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the video display adjustment device, the video display adjustment device can run the video display adjustment method described above.

[0146] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0148] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0149] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned video display adjustment method, and can solve the technical problem of video display adjustment. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the video display adjustment method provided in the above-mentioned embodiment, and will not be described in detail here.

[0150] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned video display adjustment method when executed by a processor.

[0151] The computer program product provided by the present application can solve the technical problem of video display adjustment. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the video display adjustment method provided by the above embodiment, which will not be repeated here.

[0152] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A video display adjustment method, characterized in that: The video display adjustment method comprises: Buffering the received video data to obtain a buffered video image frame set, wherein the buffered received video data refers to video frame data of multiple GOPs of the video data stream received by the cache display screen, wherein a video frame of a GOP includes an I frame and multiple P frames or B frames; Extracting key frame images from the buffered video image frame set; When the key frame image is the first screen frame, the key frame image is restored by a preset image restoration model to obtain a target key frame image, and the first screen frame is an I frame in the GOP video frame data; Updating the buffered video image frame set according to the target key frame image, and sending the updated buffered video frame set to a display screen for display; The step of restoring the key frame image through a preset image restoration model to obtain a target key frame image includes: Segmenting the key frame image to obtain a plurality of sub-key frame images; Determine a first key frame image representing global blur, a second key frame image representing motion blur, and remaining sub-key frame images among the plurality of sub-key frame images; respectively determining a first degraded key frame image corresponding to the first key frame image and a second degraded key frame image corresponding to the second key frame image; Image combination is performed according to the first degraded key frame image, the second degraded key frame image and the remaining sub-key frame images to obtain a target key frame image.

2. The video display adjustment method according to claim 1, characterized in that: The step of respectively determining a first degraded key frame image corresponding to the first key frame image and a second degraded key frame image corresponding to the second key frame image comprises: Performing Fourier transformation on the first key frame image and the second key frame image respectively to obtain first image transformation data and second image transformation data; Performing frequency domain convolution on the first image conversion data and the second image conversion data with a preset degradation function respectively to obtain first convolution data and second convolution data; The first convolution data and the second convolution data are inversely transformed to obtain a first degraded key frame image corresponding to the first image conversion data and a second degraded key frame image corresponding to the second image conversion data.

3. The video display adjustment method according to claim 1, characterized in that: The step of combining the first degraded key frame image, the second degraded key frame image, and the remaining sub-key frame images to obtain a target key frame image includes: Restoring the first degraded key frame image and the second degraded key frame image to obtain a first restored key frame image and a second restored key frame image; The first restored key frame image, the second restored key frame image and the remaining sub-key frame images are combined according to a preset image position relationship to obtain a target key frame image, wherein the preset image position relationship is the position relationship between the first key frame image, the second key frame image and the remaining sub-key frame images.

4. The video display adjustment method according to claim 3, characterized in that: Restoring the degraded key frame image to obtain a restored key frame image includes: Calculating convolution data corresponding to the degraded key frame image; Determining fuzzy parameters of a preset degradation function and calculating the power spectral density of the convolution data; Constructing a restoration filter according to the fuzzy parameter and the power spectrum density; The convolution data is restored by the restoration filter to obtain a restored key frame image.

5. The video display adjustment method according to claim 3, characterized in that: Restoring the degraded key frame image to obtain a restored key frame image includes: Calculating convolution data corresponding to the degraded key frame image; Iteratively optimize the model parameters of the preset filtering model and determine the optimization constraints; Based on the optimization constraint condition, the convolution data is iteratively calculated through a preset filtering model to obtain a restored key frame image.

6. The video display adjustment method according to any one of claims 1 to 5, characterized in that: The updating of the buffered video image frame set according to the target key frame image, and sending the updated buffered video frame set to a display screen for display, comprises: Replacing the key frame image in the buffered video image frame set with the target key frame image to obtain an updated buffered video frame set; Obtaining the timestamp of each frame image in the buffered video image frame set; Each image frame in the updated buffered video image frame set is sent to a display screen for display according to the timestamp of each frame of image.

7. A video display adjustment device, characterized in that: The video display adjustment device comprises: A cache module, used for caching received video data to obtain a set of buffered video image frames, wherein the cached received video data refers to video frame data of multiple GOPs of the video data stream received by the cache display screen, wherein a video frame of a GOP includes an I frame and multiple P frames or B frames; An extraction module, used for extracting key frame images from the buffered video image frame set; A restoration module, used for restoring the key frame image through a preset image restoration model to obtain a target key frame image when the key frame image is a first screen frame, wherein the first screen frame is an I frame in the GOP video frame data; A display module, configured to update the buffered video image frame set according to the target key frame image, and send the updated buffered video frame set to a display screen for display; The restoration module is further used to segment the key frame image to obtain a plurality of sub-key frame images; Determine a first key frame image representing global blur, a second key frame image representing motion blur, and remaining sub-key frame images among the plurality of sub-key frame images; respectively determining a first degraded key frame image corresponding to the first key frame image and a second degraded key frame image corresponding to the second key frame image; Image combination is performed according to the first degraded key frame image, the second degraded key frame image and the remaining sub-key frame images to obtain a target key frame image.

8. A video display adjustment device, characterized in that: The video display adjustment device comprises: a memory, a processor, and a video display adjustment program stored in the memory and executable on the processor, wherein the video display adjustment program is configured to implement the video display adjustment method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores a video display adjustment program, and when the video display adjustment program is executed by the processor, the video display adjustment method according to any one of claims 1 to 6 is implemented.