Video deblurring method, apparatus and device, and storage medium

Abstract

The present application discloses a video deblurring method, apparatus and device, and a storage medium. The method comprises: extracting a current blurred frame from a preset blurred video; processing the current blurred frame and preset reference frames in blocks to obtain current blurred blocks and reference frame blocks, the preset reference frames including a high-definition video frame, a preset frame previous to the current blurred frame, and a preset frame following the current blurred frame; on the basis of a similarity analysis method, selecting most similar frame blocks from among the reference frame blocks for the current blurred blocks, and performing feature fusion processing on said most similar frame blocks to obtain a feature-fused frame; on the basis of an optical flow estimation algorithm, performing edge optical flow compensation on an optical flow field missing area between adjacent frames in the preset blurred video to obtain an optical flow-compensated frame; and by means of a preset generative adversarial network, performing deblurring processing on the preset blurred video on the basis of the feature-fused frame and the optical flow-compensated frame to obtain an optimized video. The present application can solve the technical problem in the prior art of poor deblurring effect caused by single consideration factor and no specificity.

Description

Video deblurring method, device and equipment and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a video deblurring method, device and equipment and storage medium. BACKGROUND

[0002] With the continuous popularity of video technology, a large number of video monitoring systems have been built in major cities. At present, monitoring systems have been widely used in banks, shopping malls, stations and traffic intersections and other public places. However, there are problems of unclear images and incomplete data in image playback. For example, it is often difficult to identify human facial features and recognize blurred license plates of passing vehicles, which cannot provide users with high-definition and lossless playback videos. With the continuous promotion of urban video monitoring and the further promotion of various types of monitoring system construction, such problems will become more and more prominent.

[0003] At present, it is an important direction to optimize and adjust the blurred video through image processing technology and deep learning technology. However, the existing technology considers too single factors and lacks pertinence in the deblurring process, resulting in that the actual deblurring effect is difficult to achieve satisfactory results. SUMMARY

[0004] The present application provides a video deblurring method, device and equipment and storage medium, which is used to solve the technical problem of poor deblurring effect caused by single consideration of factors and lack of pertinence in the prior art.

[0005] Therefore, the first aspect of the present application provides a video deblurring method, comprising:

[0006] extracting a blurred frame from a preset blurred video to obtain a current blurred frame;

[0007] performing block processing on the current blurred frame and a preset reference frame respectively to obtain a current blurred block and a reference frame block, wherein the preset reference frame includes a high-definition video frame, a front preset frame of the current blurred frame and a rear preset frame of the current blurred frame;

[0008] selecting a maximum similar frame block for the current blurred block from the reference frame block based on a similarity analysis method, and performing feature fusion processing on the maximum similar frame block to obtain a feature fusion frame;

[0009] performing edge optical flow compensation on an optical flow field missing area between adjacent frames in the preset blurred video based on an optical flow estimation algorithm to obtain an optical flow compensation frame;

[0010] performing deblurring processing on the preset blurred video according to the feature fusion frame and the optical flow compensation frame through a preset generative adversarial network to obtain an optimized video.

[0011] Preferably, the extracting a blur frame from the preset blur video to obtain the current blur frame further comprises:

[0012] The preset high-definition video is blurred by using a video degradation algorithm to obtain a preset blur video.

[0013] Preferably, the selecting the maximum similar frame block for the current blur block from the reference frame blocks based on the similarity analysis method and performing feature fusion processing on the maximum similar frame block to obtain a feature fusion frame comprises:

[0014] The cosine similarity between the current blur block and the reference frame blocks is calculated, and the reference frame block corresponding to the maximum cosine similarity is selected as the maximum similar frame block of the current blur block.

[0015] The maximum similar frame block is subjected to spatio-temporal feature extraction and feature depth fusion operations by a preset feature fusion network to obtain a feature fusion frame.

[0016] Preferably, the edge optical flow compensation on the missing area of the optical flow field between adjacent frames in the preset blur video based on the optical flow estimation algorithm to obtain an optical flow compensation frame comprises:

[0017] The optical flow between adjacent frames in the preset blur video is calculated based on the optical flow estimation algorithm to obtain adjacent frame optical flow.

[0018] The adjacent frame optical flow is subjected to an edge optical flow completion operation by using a Canny edge detection algorithm to obtain a completed optical flow.

[0019] The completed optical flow is compensated into the current blur frame to obtain an optical flow compensation frame.

[0020] The second aspect of the application provides a video deblurring device, comprising:

[0021] A frame extraction unit is configured to extract a blur frame from a preset blur video to obtain a current blur frame.

[0022] A frame blocking unit is configured to block the current blur frame and a preset reference frame respectively to obtain a current blur block and a reference frame block, wherein the preset reference frame comprises a high-definition video frame, a front preset frame of the current blur frame, and a rear preset frame of the current blur frame.

[0023] A feature fusion unit is configured to select a maximum similar frame block for the current blur block from the reference frame blocks based on a similarity analysis method, and perform feature fusion processing on the maximum similar frame block to obtain a feature fusion frame.

[0024] an optical flow compensation unit configured to perform edge optical flow compensation on a missing area of an optical flow field between adjacent frames in the preset blurred video based on an optical flow estimation algorithm to obtain an optical flow compensation frame;

[0025] a deblurring unit configured to perform deblurring processing on the preset blurred video based on the feature fusion frame and the optical flow compensation frame by using a preset generative adversarial network to obtain an optimized video.

[0026] Preferably, the method further comprises:

[0027] a video generation unit configured to perform blurring processing on a preset high-definition video by using a video degradation algorithm to obtain a preset blurred video.

[0028] Preferably, the feature fusion unit is specifically configured to:

[0029] calculate a cosine similarity between the current blurred block and the reference frame block, and select the reference frame block corresponding to the maximum cosine similarity as a maximum similar frame block of the current blurred block;

[0030] perform spatio-temporal feature extraction and feature depth fusion operations on the maximum similar frame block by using a preset feature fusion network to obtain a feature fusion frame.

[0031] Preferably, the optical flow compensation unit is specifically configured to:

[0032] calculate an optical flow between adjacent frames in the preset blurred video based on an optical flow estimation algorithm to obtain adjacent frame optical flow;

[0033] perform edge optical flow completion operations on the adjacent frame optical flow by using a Canny edge detection algorithm to obtain completed optical flow;

[0034] compensate the completed optical flow into the current blurred frame to obtain an optical flow compensation frame.

[0035] The third aspect of the present application provides a video deblurring device, the device comprising a processor and a memory;

[0036] the memory is configured to store program code and transmit the program code to the processor;

[0037] the processor is configured to execute the video deblurring method according to the instructions in the program code.

[0038] The fourth aspect of the present application provides a computer readable storage medium for storing program code, the program code being used to execute the video deblurring method of the first aspect.

[0039] As can be seen from the above technical solutions, the embodiments of the present application have the following advantages:

[0040] In the present application, a video deblurring method is provided, comprising: extracting a blurred frame from a preset blurred video to obtain a current blurred frame; performing block processing on the current blurred frame and a preset reference frame respectively to obtain a current blurred block and a reference frame block, the preset reference frame including a high-definition video frame, a front preset frame of the current blurred frame and a rear preset frame of the current blurred frame; selecting a maximum similar frame block for the current blurred block from the reference frame block based on a similarity analysis method, and performing feature fusion processing on the maximum similar frame block to obtain a feature fusion frame; performing edge optical flow compensation on an optical flow field missing area between adjacent frames in the preset blurred video based on an optical flow estimation algorithm to obtain an optical flow compensation frame; and performing deblurring processing on the preset blurred video according to the feature fusion frame and the optical flow compensation frame through a preset generative adversarial network to obtain an optimized video.

[0041] The video deblurring method provided in the present application optimizes the current blurred frame by taking the high-definition video frame, the front preset frame and the rear preset frame of the current blurred frame as the preset reference frame, and selects the most similar frame block for feature fusion based on similarity, and the feature fusion frame obtained based thereon can contain more detailed features and time sequence features, which is beneficial to blurred detail repair. In addition, the edge optical flow compensation performed by the optical flow estimation algorithm can obtain more real optical flow values and clearer optical flow compensation frames. In this process, not only the change characteristics of the blurred frame are analyzed from the feature perspective, but also the motion characteristics of the blurred frame are analyzed based on the optical flow estimation algorithm. Considering the changeable angle and the pertinence, it can be ensured that the deblurring operation based on the feature fusion frame and the optical flow compensation frame achieves good results. Therefore, the present application can solve the technical problem that the prior art considers only a single factor and has no pertinence, resulting in poor deblurring effect. BRIEF DESCRIPTION OF DRAWINGS

[0042] FIG. 1 is a flow diagram of a video deblurring method provided by an embodiment of the present application;

[0043] FIG. 2 is a structural diagram of a video deblurring device provided by an embodiment of the present application;

[0044] FIG. 3 is a schematic diagram of the overall process of video deblurring provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0046] For ease of understanding, please refer to FIG. 1, an embodiment of a video deblurring method provided by the present application comprises the following steps:

[0047] Step 101, extracting a blurred frame from a preset blurred video to obtain a current blurred frame.

[0048] Further, step 101 further comprises the following steps:

[0049] The preset blurred video is prepared by blurring the preset high-definition video based on a video degradation algorithm.

[0050] It should be noted that the preset blurred video can be prepared by blurring the preset high-definition video based on the video degradation algorithm, and then the blurred frame in time sequence can be extracted from the preset blurred video for subsequent processing to remove the blur. It can be understood that the current blurred frame refers to the image frame corresponding to the current research time, which is not limited to a certain time, as long as it is extracted from the blurred video.

[0051] Step 102, performing block processing on the current blurred frame and a preset reference frame respectively to obtain a current blurred block and a reference frame block, and the preset reference frame comprises a high-definition video frame, a front preset frame of the current blurred frame and a rear preset frame of the current blurred frame.

[0052] It should be noted that the front preset frame of the current blurred frame refers to several frames corresponding to the previous several time points of the time point corresponding to the current blurred frame, and the preset frame is a preset number of frames, which can be denoted as N; similarly, the rear preset frame of the current blurred frame refers to several frames corresponding to the next several time points of the time point corresponding to the current blurred frame, which can also be denoted as N. The same number of frames are extracted before and after the current blurred frame to optimize the current blurred frame. Considering the characteristics of the adjacent frames and the high-definition frames in the video, the high-definition video frame, the front preset frame of the current blurred frame and the rear preset frame of the current blurred frame are used as the reference frame to optimize the current blurred frame, which can ensure that more image details are covered in the analysis process, and is more conducive to blur recovery.

[0053] The current blurred frame and the preset reference frame are processed by the same block processing method, and it is assumed that each of them can be divided into K non-overlapping blocks, which can be expressed as It can be understood that the current blurred frame and the preset reference frame have the same size, and the K non-overlapping blocks divided have the same size. The current blurred frame is processed by block processing to obtain K current blurred blocks, and the preset reference frame is processed by block processing to obtain K reference frame blocks.

[0054] Step 103, selecting a maximum similar frame block for the current blurred block from the reference frame blocks based on a similarity analysis method, and performing feature fusion processing on the maximum similar frame block to obtain a feature fusion frame.

[0055] Further, step 103 comprises the following steps:

[0056] calculate the cosine similarity between the current blur block and the reference frame block, and select the reference frame block corresponding to the maximum cosine similarity as the maximum similar frame block of the current blur block;

[0057] extract the spatio-temporal features and perform the feature depth fusion operation on the maximum similar frame block through the preset feature fusion network to obtain a feature fusion frame.

[0058] It should be noted that the reference frame block can be divided into three categories or two categories according to the high-definition video frame, the pre-set frame before the current blur frame and the pre-set frame after the current blur frame. The cosine similarity is calculated between the current blur block and the reference frame block corresponding to the image frame of the three categories, and the maximum cosine similarity value is selected. The high-definition video frame block, the pre-set frame before the current blur frame block and the pre-set frame after the current blur frame block corresponding to the value are the maximum similar frame block, which can be expressed as wherein, respectively represent the pre-set n frame blocks and the pre-set n frame blocks at the current time t, is the block corresponding to the high-definition video frame. In addition, the cosine similarity calculation process is:

[0059] wherein, represents the current blur block, q j represents the reference frame block corresponding to the preset reference frame, is the cosine similarity value.

[0060] The preset feature fusion network includes a spatial feature extractor, a spatio-temporal feature fusioner and a depth integrator. Therefore, the frame image in the maximum similar frame block can be sequentially subjected to spatio-temporal feature extraction and feature depth fusion operation in combination with the preset feature fusion network, and then a feature fusion frame is obtained. For details, please refer to FIG. 3.

[0061] In this embodiment, each unit of the spatial feature extractor is composed of a 3x3 convolution layer, a batch normalization BN layer and an activation function ReLU. The spatio-temporal feature fusioner uses a three-dimensional convolution layer with a 3x3x3 kernel to perform spatio-temporal feature fusion. The depth integrator deep integrates the information within each group by applying 18 2D units in the 2D dense block to generate group-level features. In order to improve efficiency, the weights of the intra-group fusion module are shared for each group.

[0062] In step 104, the edge optical flow compensation is performed on the missing area of the optical flow field between adjacent frames in the pre-set blur video based on the optical flow estimation algorithm to obtain an optical flow compensated frame.

[0063] Further, step 104 includes:

[0064] An optical flow estimation algorithm is used to calculate the optical flow between adjacent frames in the preset blurred video, to obtain adjacent frame optical flow;

[0065] A Canny edge detection algorithm is used to perform edge optical flow completion on the adjacent frame optical flow, to obtain completed optical flow;

[0066] The completed optical flow is compensated into the current blurred frame, to obtain an optical flow compensation frame.

[0067] It should be noted that the optical flow estimation algorithm is used to estimate the motion of each pixel in the image between consecutive frames; the goal is to find the displacement vector of each pixel in the image moving from one position to another between two adjacent frames; the optical flow algorithm is based on an assumption that the motion of the same object in the adjacent frames in the image should be continuous. The optical flow estimation algorithm includes Lucas-Kanade optical flow algorithm, sparse optical flow algorithm, dense optical flow algorithm, and deep learning-based optical flow estimation algorithm.

[0068] Since the optical flow model based on the pretrained LiteFlowNet3 network has a very obvious degradation phenomenon and may even fail to estimate when estimating a large motion, the embodiment uses a transformation H j →i to compensate for the large motion between frame i and frame j before estimating the optical flow, and then applies an inverse transformation to map the optical flow back to the original frame: F i→j =F(I i ,H j→i (I j ))+H i→j ,|i-j|>1

[0069] An optical flow between adjacent frames i and j is calculated using a pretrained LiteFlowNet3 network F: F i→j =f(I i ,I j ),|i-j|=1

[0070] Where F i→j represents the adjacent frame optical flow, H i→j and H j→i represent the forward and inverse transformations, I i and I j represent the pixel values of the adjacent two frames, and f(I i , I j ) represents the optical flow calculation result in the region.

[0071] Since the optical flow estimation algorithm cannot maintain clear boundaries, the embodiment also uses a Canny edge detection algorithm to perform edge optical flow completion on the adjacent frame optical flow. Specifically, the Canny edge detection algorithm is used to extract the optical flow edge map E i→jThen the optical flow is completed by using the optical flow edge map as a guide:

[0072] where, Δ x , Δ y respectively represent the horizontal and vertical finite forward difference operators, P represents the edge detection map, represents the non-edge pixel value, represents the optical flow map, represents the non-edge optical flow value. The above sum covers all non-edge pixels, and the boundary condition ensures the smooth continuation of the optical flow, so that the completed optical flow can be obtained; and the completed optical flow is compensated into the current blurred frame, so that the optical flow compensation frame can be obtained.

[0073] Step 105, deblurring the preset blurred video according to the feature fusion frame and the optical flow compensation frame by using the preset generative adversarial network to obtain an optimized video.

[0074] Referring to FIG. 3, the preset generative adversarial network constitutes a deblurring model. According to the feature fusion frame and the optical flow compensation frame obtained by the above analysis, the deblurring of the preset blurred video is performed, so that a high-quality video after optimization, i.e., an optimized video, can be obtained. In this process, the introduction of adjacent frames and high-definition frames can simultaneously consider the changes of features in time and space during the motion, which is beneficial to the repair of texture details. The introduction of the optical flow completion model can obtain more real optical flow values, and can obtain clearer deblurring video frames. In addition, the method of the embodiment can greatly reduce the training sample demand of the deblurring model, and the occupation of GPU resources is also reduced, which has certain economy.

[0075] The video deblurring method provided by the embodiment of the application uses the high-definition video frame, the front preset frame and the rear preset frame of the current blurred frame as the preset reference frame to optimize the current blurred frame, and selects the most similar frame block for feature fusion according to the similarity. The feature fusion frame obtained based on this can contain more detailed features and time sequence features, which is beneficial to the repair of blurred details. In addition, the edge optical flow compensation by the optical flow estimation method can obtain more real optical flow values and clearer optical flow compensation frames. In this process, not only the change characteristics of the blurred frame are analyzed from the feature perspective, but also the motion characteristics of the blurred frame are analyzed based on the optical flow estimation method. Considering the changeable angle and the pertinence, it can be ensured that the deblurring operation based on the feature fusion frame and the optical flow compensation frame achieves good results. Therefore, the embodiment of the application can solve the technical problem that the prior art considers only a single factor and has no pertinence, resulting in poor deblurring effect.

[0076] For ease of understanding, please refer to FIG. 2, the embodiment of the application provides a video deblurring device, which comprises:

[0077] The frame extraction unit 201 is configured to extract a blurred frame from the preset blurred video to obtain a current blurred frame.

[0078] a frame blocking unit 202, configured to perform blocking processing on the current blur frame and preset reference frames respectively to obtain current blur blocks and reference frame blocks, the preset reference frames including a high-definition video frame, a front preset frame of the current blur frame, and a rear preset frame of the current blur frame;

[0079] a feature fusion unit 203, configured to select a maximum similar frame block for the current blur block from the reference frame blocks based on a similarity analysis method, and perform feature fusion processing on the maximum similar frame block to obtain a feature fusion frame;

[0080] an optical flow compensation unit 204, configured to perform edge optical flow compensation on an optical flow field missing area between adjacent frames in the preset blur video based on an optical flow estimation algorithm to obtain an optical flow compensation frame;

[0081] a deblurring unit 205, configured to perform deblurring processing on the preset blur video according to the feature fusion frame and the optical flow compensation frame by using a preset generative adversarial network to obtain an optimized video.

[0082] Further, the device further comprises:

[0083] a video generation unit 206, configured to perform blur processing on the preset high-definition video by using a video degradation algorithm to obtain a preset blur video.

[0084] Further, the feature fusion unit 203 is specifically configured to:

[0085] calculate a cosine similarity between the current blur block and the reference frame blocks, and select a reference frame block corresponding to a maximum cosine similarity as a maximum similar frame block of the current blur block;

[0086] perform spatio-temporal feature extraction and feature depth fusion operations on the maximum similar frame block by using a preset feature fusion network to obtain the feature fusion frame.

[0087] Further, the optical flow compensation unit 204 is specifically configured to:

[0088] calculate optical flow between adjacent frames in the preset blur video based on an optical flow estimation algorithm to obtain adjacent frame optical flow;

[0089] perform edge optical flow completion operation on the adjacent frame optical flow by using a Canny edge detection algorithm to obtain completed optical flow;

[0090] compensate the completed optical flow into the current blur frame to obtain the optical flow compensation frame.

[0091] The application further provides a video deblurring device, the device comprising a processor and a memory;

[0092] The memory is configured to store program code and transmit the program code to the processor;

[0093] The processor is configured to perform the video deblurring method in the method embodiments above according to instructions in the program code.

[0094] The application further provides a computer-readable storage medium for storing program code, the program code being used to perform the video deblurring method in the method embodiments above.

[0095] In several embodiments provided in the application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. The division of the units is merely a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0096] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0097] In addition, each functional unit in the embodiments of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware or software function unit.

[0098] When the integrated unit is implemented in the form of software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the application essentially or the part that makes a contribution to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes several instructions for performing all or part of the steps of the methods described in the embodiments of the application by a computer device (which can be a personal computer, a server, or a network device, etc.). The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), a random access memory (English full name: Random Access Memory, English abbreviation: RAM), a magnetic disk or an optical disk, and various media that can store program codes.​

[0099] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of video deblurring, characterized by, The method comprises the steps of: extracting a blur frame from a preset blur video to obtain a current blur frame; performing block processing on the current blur frame and a preset reference frame respectively to obtain a current blur block and a reference frame block, wherein the preset reference frame comprises a high-definition video frame, a front preset frame of the current blur frame and a rear preset frame of the current blur frame; selecting a maximum similar frame block for the current blur block from the reference frame block based on a similarity analysis method, and performing feature fusion processing on the maximum similar frame block to obtain a feature fusion frame; performing edge optical flow compensation on an optical flow field missing area between adjacent frames in the preset blur video based on an optical flow estimation algorithm to obtain an optical flow compensation frame; performing deblurring processing on the preset blur video according to the feature fusion frame and the optical flow compensation frame through a preset generative adversarial network to obtain an optimized video.

2. The video deblurring method of claim 1, wherein, The step of extracting a blur frame from a preset blur video to obtain a current blur frame further comprises the steps of: performing blur processing on a preset high-definition video by using a video degradation algorithm to obtain a preset blur video.

3. The video deblurring method of claim 1, wherein, The step of selecting a maximum similar frame block for the current blur block from the reference frame block based on a similarity analysis method, and performing feature fusion processing on the maximum similar frame block to obtain a feature fusion frame comprises the steps of: calculating a cosine similarity between the current blur block and the reference frame block, and selecting the reference frame block corresponding to the maximum cosine similarity as the maximum similar frame block of the current blur block; performing spatio-temporal feature extraction and feature depth fusion operation on the maximum similar frame block through a preset feature fusion network to obtain a feature fusion frame.

4. The video deblurring method of claim 1, wherein, The step of performing edge optical flow compensation on an optical flow field missing area between adjacent frames in the preset blur video based on an optical flow estimation algorithm to obtain an optical flow compensation frame comprises the steps of: calculating an optical flow between adjacent frames in the preset blur video based on an optical flow estimation algorithm to obtain adjacent frame optical flow; performing edge optical flow completion operation on the adjacent frame optical flow by using a Canny edge detection algorithm to obtain a completed optical flow; compensating the completed optical flow into the current blur frame to obtain an optical flow compensation frame.

5. A video deblurring apparatus characterized by comprising: The method comprises the steps of: a frame extraction unit configured to extract a blur frame from a preset blur video to obtain a current blur frame; a frame block unit configured to perform block processing on the current blur frame and a preset reference frame respectively to obtain a current blur block and a reference frame block, wherein the preset reference frame comprises a high-definition video frame, a front preset frame of the current blur frame and a rear preset frame of the current blur frame; a feature fusion unit configured to select a maximum similar frame block for the current blur block from the reference frame block based on a similarity analysis method, and perform feature fusion processing on the maximum similar frame block to obtain a feature fusion frame; an optical flow compensation unit configured to perform edge optical flow compensation on an optical flow field missing area between adjacent frames in the preset blur video based on an optical flow estimation algorithm to obtain an optical flow compensation frame; a deblurring unit configured to perform deblurring processing on the preset blur video according to the feature fusion frame and the optical flow compensation frame through a preset generative adversarial network to obtain an optimized video.

6. The video deblurring apparatus of claim 5, wherein, The method further comprises the step of: a video generation unit configured to perform blur processing on a preset high-definition video by using a video degradation algorithm to obtain a preset blur video.

7. The video deblurring apparatus of claim 5, wherein, The feature fusion unit is specifically used for: calculating the cosine similarity between the current blur block and the reference frame block, and selecting the reference frame block corresponding to the maximum cosine similarity as the maximum similar frame block of the current blur block; performing spatio-temporal feature extraction and feature depth fusion operations on the maximum similar frame block through a preset feature fusion network to obtain a feature fusion frame.

8. The video deblurring apparatus of claim 5, wherein, The optical flow compensation unit is specifically used for: calculating the optical flow between adjacent frames in the preset blur video based on an optical flow estimation algorithm to obtain adjacent frame optical flow; performing edge optical flow completion operations on the adjacent frame optical flow using a Canny edge detection algorithm to obtain completed optical flow; compensating the completed optical flow into the current blur frame to obtain an optical flow compensation frame.

9. A video deblurring device, characterized by The device comprises a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the video deblurring method according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium is used to store program code, and the program code is used to execute the video deblurring method.

Images