Video deblurring method, medium and device based on 3D spatiotemporal grid perception

Abstract

The present invention discloses a video deblurring method, medium and device based on 3D spatiotemporal grid perception. The method first receives a sequence of N consecutive video frames, with [N, H, W, 3] as the input dimension vector; calculates the shaking intensity through the amplitude of the optical flow of adjacent frames, adaptively divides each frame of the image into S×S overlapping grid blocks, extracts the grid blocks corresponding to the pixel positions to form a [N, S, S, 3] sequence. The grid block sequence is input into a 3D spatiotemporal network model for processing, and a deblurred grid block is output; then, the fusion weight is calculated based on the distance between the pixel and the center of the adjacent grid block, the grid blocks are weighted and reorganized, and a clear current frame image is output. The method utilizes 3D spatiotemporal network and grid block processing, combined with the scene shaking characteristics, to achieve efficient real-time deblurring, which can effectively solve the problem of blurred images in port container monitoring and improve operational safety.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a video deblurring method, medium and device based on 3D spatiotemporal grid perception. Background Art

[0002] In port container transportation and logistics operations, cameras mounted on container trolleys shoulder the important mission of real-time monitoring and recording the entire process of container loading, unloading, stacking, and transshipment. Compared with conventional monitoring scenarios, port container monitoring presents significant special features, which are specifically reflected in the following aspects:

[0003] First, the scenes are highly fixed and repetitive. Surveillance cameras in port container yards are typically fixed, focused on the same area for extended periods, and the content of their footage is highly repetitive and predictable. This characteristic enables the system to memorize, learn, and predict historical scenes.

[0004] Secondly, unique shaking patterns pose a monitoring challenge. During container operations, the operation of heavy equipment such as cranes and gantry cranes can cause sudden, large-scale shaking. This shaking is extremely short-lived, typically less than a second, but its intensity far exceeds the more common, continuous, mild vibrations, instantly blurring the monitoring footage.

[0005] Furthermore, real-time performance is crucial. The high demands on operational safety and efficiency in port container operations dictate that monitoring systems must process video streams instantly. Traditional offline processing methods or high-latency solutions simply cannot meet these requirements.

[0006] However, current monitoring technologies struggle to meet these challenges. Traditional anti-shake algorithms, often designed for scenarios involving continuous or slow shaking, are inadequate for dealing with the transient blurring caused by short bursts of intense shaking in port container operations. Existing general-purpose deblurring technologies not only lack the ability to specifically address the periodic, short bursts of intense shaking found in port container yards, but also fail to fully leverage the advantages of scene stability and predictability. Furthermore, they suffer from real-time limitations, resulting in high processing delays, which severely impact the effectiveness of operational safety monitoring. Summary of the Invention

[0007] In view of the above problems, the present invention provides a video deblurring method, medium and device based on 3D spatiotemporal grid perception, which is used to solve the technical problems that the existing video stabilization algorithm is not applicable to the stabilization processing of video images with severe shaking, cannot meet the application of port container monitoring scenarios, and is prone to missing monitoring images, affecting safe operations.

[0008] To achieve the above objectives, in a first aspect, the present application provides a video deblurring method based on 3D spatiotemporal grid perception, the method comprising the following steps:

[0009] Receive a video image sequence of N consecutive frames, and the input dimension vector is [N,H,W,3], where N represents the number of input video image frames, H and W represent the height and width respectively, and 3 represents the RGB channel;

[0010] The shake intensity is calculated by the optical flow amplitude of adjacent frames. Each video frame is adaptively divided into overlapping grid blocks of size S×S. For each pixel position (i, j) in the video image, the grid block corresponding to that pixel position is extracted from N video frames, forming a grid block sequence of shape [N, S, S, 3], where N is a positive integer greater than 1, the N-1 frame is the clear frame image, and the Nth frame image is the current frame image to be deblurred.

[0011] Input the grid block sequence into the trained 3D spatiotemporal network model for block processing, and output the deblurred grid blocks;

[0012] The fusion weight is calculated based on the distance between the pixel position and the center of the adjacent grid block, and the deblurred grid blocks are weightedly fused and reorganized to output a clear current frame image.

[0013] Furthermore, the 3D spatiotemporal network model includes a model generator, and the model generator includes:

[0014] 3D spatiotemporal feature extraction module, which converts the input [N, S, S, 3] grid block sequence into [1, 3, N, S, S] dimensions to adapt to 3D convolution input;

[0015] The shake-aware encoder extracts multi-scale features through three layers of downsampling convolution and stores the output in a skip connection layer.

[0016] Residual block module, which consists of 6 residual blocks connected in series. Each residual block contains two 3×3 convolutional layers and an identity skip connection.

[0017] The spatiotemporal context decoder, which upsamples through deconvolution and uses skip connections to obtain multi-scale features of the shake-aware encoder;

[0018] The detail enhancement output layer reduces the feature channels from 64 to 3 through 3×3 convolution and outputs an RGB image in the range of [-1, 1] through the Tanh activation function.

[0019] Furthermore, the shake-aware encoder includes a first encoding layer, a second encoding layer, and a third encoding layer. The input of the first encoding layer is a feature map with original resolution, the input of the second encoding layer is a feature map downsampled by 2 times, and the input of the third encoding layer is a feature map downsampled by 4 times.

[0020] The output results of the first coding layer and the second coding layer are passed to the spatiotemporal context decoder through a skip connection layer, and the output result of the third coding layer is resized by bilinear interpolation and then passed to the residual block module for processing.

[0021] Furthermore, the format of the RGB image output by the detail enhancement output layer is [batch, S, S, 3], where batch represents the output block batch, S represents the block size, and 3 represents the RGB channel.

[0022] Furthermore, the 3D spatiotemporal network model includes a model discriminator, and the model discriminator includes:

[0023] The 3D spatiotemporal feature perception module combines the grid block sequence and the generator output image into a tensor of shape [1, 3, N+1, S, S], applies 3D convolution with kernel size (N+1, 4, 4) and stride (1, 2, 2), and outputs a feature map of shape [1, 64, S / 2, S / 2], where S is the grid block size and is an even number.

[0024] The 2D block GAN discriminant chain performs multi-layer convolution downsampling on the feature map and outputs a true or false feature map.

[0025] Furthermore, the activation function of the 3D convolution is LeakyReLU.

[0026] Furthermore, the 3D spatiotemporal network model is trained according to the following method:

[0027] A sample image set is obtained, a sample grid block sequence is obtained based on the sample image set, the sample grid block sequence is input into a 3D spatiotemporal network model to be trained, and iterative training is performed to obtain a trained 3D spatiotemporal network model.

[0028] Further, the sample image set includes a first sample image set, a second sample image set and a third sample image set;

[0029] The first sample image set is obtained according to the following method:

[0030] Collecting actual surveillance video streams from multiple port container yards, extracting all video frame images from each video stream, calculating the clarity score of each video frame image, screening out video frame images with an average clarity score higher than a preset clarity threshold, and extracting a continuous video frame sequence from the screened video frame images as a first sample image set;

[0031] The second sample image set is obtained according to the following method:

[0032] Setting a linear shake model, a jitter shake model, and a combined shake model, wherein the linear shake model includes a linear shake blur kernel, the jitter shake model includes a jitter shake blur kernel, and the combined shake model includes a linear shake blur kernel and a jitter shake blur kernel;

[0033] Inputting the first sample image set into a linear shake model, a jitter shake model, and a combined shake model respectively to obtain a second sample image set;

[0034] The third sample image set is obtained according to the following method:

[0035] Transformation processing is performed on the images in the first sample image set and the second sample image set to obtain a third sample image set, where the transformation processing includes geometric transformation, illumination transformation, and noise addition.

[0036] In a second aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video deblurring method based on 3D spatiotemporal grid perception as described in the first aspect of the present application.

[0037] In a third aspect, the present application provides an electronic device having a computer program stored thereon, comprising a processor and a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by the processor, the video deblurring method based on 3D spatiotemporal grid perception as described in the first aspect of the present application is implemented.

[0038] Different from existing technologies, the above-mentioned technical solution provides a video deblurring method, medium, and device based on 3D spatiotemporal grid perception. This method first receives a sequence of N consecutive video frames, with the input dimension vector [N, H, W, 3]. The shake intensity is calculated using the optical flow amplitude of adjacent frames, and each frame is adaptively divided into S×S overlapping grid blocks. The grid blocks corresponding to the pixel positions are extracted to form a [N, S, S, 3] sequence. The grid block sequence is input into a 3D spatiotemporal network model for processing, and a deblurred grid block is output. Fusion weights are then calculated based on the distance between the pixel and the center of the adjacent grid block, and the grid blocks are weighted and reassembled to output a clear image of the current frame. This method utilizes a 3D spatiotemporal network and grid block processing, combined with scene shake characteristics, to achieve efficient real-time deblurring. This method can effectively solve the problem of blurred images in port container monitoring and improve operational safety.

[0039] The above-mentioned description of the invention content is only an overview of the technical solution of the present invention. In order to enable ordinary technicians in this field to more clearly understand the technical solution of the present invention, and then implement it according to the text of the specification and the contents recorded in the drawings, and to make the above-mentioned objects and other objects, features and advantages of the present invention easier to understand, the following is an explanation in combination with the specific implementation methods and drawings of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are only used to illustrate the principles, implementations, applications, features, and effects of the specific embodiments of the present invention and other related contents, and are not to be considered as limiting the present invention.

[0041] In the drawings of the specification:

[0042] Figure 1 This is a first flow chart of a video deblurring method based on 3D spatiotemporal grid perception according to a specific embodiment;

[0043] Figure 2 A schematic diagram of a spatiotemporal grid perception architecture according to a specific embodiment;

[0044] Figure 3 A second flow chart of a video deblurring method based on 3D spatiotemporal grid perception according to a specific embodiment;

[0045] Figure 4 Schematic diagram of the distribution of grid blocks identified in the port monitoring screen involved in the specific implementation method;

[0046] Figure 5 A schematic diagram of a 3D spatiotemporal grid perception and processing grid block according to a specific embodiment;

[0047] Figure 6 A workflow diagram of a model generator involved in a specific implementation manner;

[0048] Figure 7 This is a workflow diagram of a 3D spatiotemporal feature extraction module according to a specific embodiment;

[0049] Figure 8 This is a flowchart of a shake-aware encoder according to a specific embodiment;

[0050] Figure 9 This is a workflow diagram of the residual block module involved in the specific implementation method;

[0051] Figure 10 This is a workflow diagram of a single residual block involved in a specific implementation;

[0052] Figure 11 This is a workflow diagram of a spatiotemporal context decoder according to a specific embodiment;

[0053] Figure 12 Enhance the output layer workflow diagram with details related to specific implementations;

[0054] Figure 13 This is a workflow diagram of a model discriminator involved in a specific implementation method;

[0055] Figure 14 A flowchart of self-supervisory data generation according to a specific embodiment;

[0056] Figure 15 A flowchart for constructing a port sloshing pattern library involved in a specific implementation method;

[0057] Figure 16 A flowchart of a training process of a model generator according to a specific embodiment;

[0058] Figure 17 Flowchart of the training process of the model discriminator involved in the specific implementation method;

[0059] Figure 18 A schematic diagram of a module of an electronic device according to a specific embodiment;

[0060] The reference numerals in the above drawings are described as follows:

[0061] 10. Electronic equipment;

[0062] 101. Processor;

[0063] 102. Storage medium. DETAILED DESCRIPTION

[0064] In order to explain in detail the possible application scenarios, technical principles, specific solutions that can be implemented, and the purpose and effects of the present invention, the following is a detailed description of the specific embodiments listed in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of the present invention and are therefore only examples and are not intended to limit the scope of protection of the present invention.

[0065] References to "embodiments" herein mean that the specific features, structures, or characteristics described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the term "embodiment" in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or relevance to other embodiments. In principle, in the present invention, as long as there are no technical contradictions or conflicts, the various technical features mentioned in the embodiments can be combined in any manner to form a corresponding implementable technical solution.

[0066] Unless otherwise defined, the technical terms used herein have the same meanings as those generally understood by those skilled in the art to which the present invention belongs. The use of relevant terms herein is only for describing specific embodiments and is not intended to limit the present invention.

[0067] In the description of the present invention, the term "and / or" is used to describe a logical relationship between objects, indicating that three possible relationships exist. For example, A and / or B means: A exists, B exists, and both A and B exist. Furthermore, the character " / " generally indicates that the objects are in a logical "or" relationship.

[0068] In the present invention, terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.

[0069] Without further restrictions, in the present invention, the words "include", "comprise", "have" or other similar expressions used in sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0070] In the present invention, expressions such as "greater than," "less than," and "exceed" are understood to exclude the number itself; expressions such as "above," "below," and "within" are understood to include the number itself. Furthermore, in the description of the embodiments of the present invention, "multiple" means two or more (including two), and similar expressions related to "multiple," such as "multiple groups" and "multiple times," are also understood in this manner, unless otherwise specifically defined.

[0071] In the description of the embodiments of the present invention, the space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present invention or facilitating the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present invention.

[0072] Unless otherwise expressly specified or limited, in the description of the embodiments of the present invention, the terms "installed", "connected", "connected", "fixed", "set", etc. used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection between two elements or the interaction relationship between two elements. For those skilled in the art of the technology to which the present invention belongs, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0073] like Figures 1-17 As shown, in a first aspect, the present application provides a video deblurring method based on 3D spatiotemporal grid perception, the method comprising the following steps:

[0074] Step S1: Receive a video image sequence of N consecutive frames, and the input dimension vector is [N, H, W, 3], where N represents the number of input video image frames, H and W represent height and width respectively, and 3 represents RGB channels;

[0075] Step S2: Calculate the shake intensity using the optical flow amplitude of adjacent frames. Adaptively divide each video frame into overlapping grid blocks of size S×S. For each pixel position (i, j) in the video image, extract the grid block corresponding to that pixel position from N frames of video images, forming a grid block sequence of shape [N, S, S, 3], where N is a positive integer greater than 1, the N-1 frame is a clear frame image, and the Nth frame image is the current frame image to be deblurred.

[0076] Step S3: input the grid block sequence into the trained 3D spatiotemporal network model for block processing, and output the deblurred grid blocks;

[0077] Step S4: Calculate the fusion weight based on the pixel position and the distance between the center of the adjacent grid blocks, perform weighted fusion and reorganization on the deblurred grid blocks, and output a clear current frame image.

[0078] The above method uses 3D spatiotemporal network and grid block processing, combined with the scene shaking characteristics, to achieve efficient real-time deblurring, which can effectively solve the problem of blurred images in port container monitoring and improve operational safety.

[0079] like Figure 2 As shown, taking the monitoring video stream for port container yard operations as an example, the spatiotemporal grid perception of this application includes an input layer, a processing layer, an output layer and a training auxiliary module, wherein the input information of the input layer is a multi-frame input monitoring video sequence of the port, the processing layer includes a preprocessing module and a generator network, the preprocessing module is used for frame alignment and grid segmentation, the generator network adopts a 3D-TSPatchNet architecture, when the network model is trained, sample images can be obtained from the port scene video library, the discriminator network is a 3D-TSPatchGAN architecture, and the output of the model is a clear port image after deblurring.

[0080] like Figure 3 As shown, when the model receives a video image sequence of N consecutive frames, it first uses the shake warning and detection module to determine whether there are video frames with large shake intensity in the current video image sequence. If not, it directly continues to output the original monitoring picture. If so, it performs preprocessing and deblurring processing through steps S1-S4, and then restores a clear current frame image. After that, the clear current frame image replaces the original current frame image and outputs the video monitoring picture.

[0081] In this embodiment, before deblurring the current frame, it is necessary to complete input data preprocessing and grid division. Taking port surveillance video as an example, the input data processing flow is as follows:

[0082] (1) Original video input: First, a complete port surveillance video sequence is received. The sequence contains N consecutive frames (the value of N is preferably 3). The original input dimension can be expressed as [N, H, W], where H and W represent the height and width of the original frame, respectively. The interval between each frame is about 33ms (based on 30fps video).

[0083] (2) Adaptive grid division: Each frame image is divided into a series of overlapping grid blocks according to the detected shake intensity. The standard grid block size is S×S (such as 64×64 or 128×128 pixels). The number of overlapping pixels of adjacent grid blocks is dynamically adjusted according to the shake intensity (the standard is S / 4, such as 16 or 32 pixels).

[0084] (3) Grid block extraction and organization: For each position (i, j) in the video sequence, extract the grid block at the corresponding position from N frames to form a grid block sequence: a tensor of shape [N, S, S, 3]. For example, assuming S = 64, the input for a single position is [3, 64, 64, 3], representing a sequence of 3 frames of 64×64 pixel RGB images.

[0085] (4) Grid block processing characteristics: In each grid block sequence, the first N-1 frames (e.g., the first two frames) are usually relatively clear frames, and the last frame is the current frame that needs to be deblurred. There is an overlapping area between adjacent grid blocks to ensure smooth fusion of the processing results. All grid blocks can be input into the 3D-TSPatchNet model in parallel for processing.

[0086] (5) Grid block preprocessing, which includes normalization, data format adjustment, batch organization, and other operations. Normalization refers to mapping the pixel values ​​in the grid from [0, 255] to the range of [-1, 1]. Data format adjustment is to adjust [N, S, S, 3] to the input format required by the model network. Batch organization is to organize grid blocks at multiple locations into batches and synchronously input them into the model network to improve processing efficiency.

[0087] Through this grid division and processing method, the 3D-TSPatchNet model can efficiently process the blurred images of short-term violent shaking in the port surveillance video and restore clear surveillance images while keeping the computing resource requirements within a reasonable range. Figure 4 shown.

[0088] In some embodiments, the 3D spatiotemporal network model includes a model generator, wherein the model generator includes:

[0089] 3D spatiotemporal feature extraction module, which converts the input [N, S, S, 3] grid block sequence into [1, 3, N, S, S] dimensions to adapt to 3D convolution input;

[0090] The shake-aware encoder extracts multi-scale features through three layers of downsampling convolution and stores the output in a skip connection layer.

[0091] Residual block module, which consists of 6 residual blocks connected in series. Each residual block contains two 3×3 convolutional layers and an identity skip connection.

[0092] The spatiotemporal context decoder, which upsamples through deconvolution and uses skip connections to obtain multi-scale features of the shake-aware encoder;

[0093] The detail enhancement output layer reduces the feature channels from 64 to 3 through 3×3 convolution and outputs an RGB image in the range of [-1, 1] through the Tanh activation function.

[0094] In this embodiment, the model generator receives a multi-frame sequence of a single grid block Patch, processes the spatiotemporal information, and outputs the corresponding deblurred clear image. The shape of the multi-frame sequence of a single grid block is expressed as [N, S, S, 3], and the value range is [-1, 1]. For example, [3, 64, 64, 3] represents a sequence of 3 frames of 64×64 RGB images, of which the first N-1 frames are clear frame images and the last frame is the current frame image that needs to be deblurred. The output of the model is a deblurred image, the shape is expressed as [S, S, 3], the value range is consistent with the input, and the output is the clear version of the grid block corresponding to the last frame (current frame image). Each grid block sequence input to 3D-TSPatchNet comes from the previous and next frame videos, in the form of Figure 5 shown.

[0095] like Figure 6 As shown in the figure, this paper innovatively designs a 3D spatiotemporal perception network (3D-TSPatchNet), which is completely different from the traditional 2D U-Net architecture and implements a series of key optimizations and innovations specifically for the task of deblurring short-term and violent sloshing of port containers. This generator network simultaneously captures information in both temporal and spatial dimensions through 3D convolution and combines it with a grid block processing approach to achieve efficient spatiotemporal feature extraction, multi-scale feature learning, residual learning, and spatiotemporal skip connections. The overall process is as follows:

[0096] (1) 3D spatiotemporal feature fusion design: Traditional U-Net is mainly used for image segmentation tasks and only processes the 2D information of a single image. This invention has made a fundamental innovation by creating a new 3D-TSPatchNet architecture, which enables it to process features in three dimensions of time and space at the same time, forming a true spatiotemporal fusion architecture. Specifically, a special 3D-2D transition layer is designed to retain the temporal information while being compatible with the U-Net structure; a transmission mechanism of temporal features in jump connections is introduced to ensure that temporal information is not lost in the deep network; and the downsampling ratio in the encoder-decoder structure is adjusted to adapt to the scale characteristics of shaking and blurring during port container loading and unloading.

[0097] (2) Sway characteristic perception mechanism: Traditional U-Net lacks the ability to model motion characteristics. This paper designs a sway characteristic perception component specifically for port container operations. Specifically, it introduces a motion characteristic analysis branch at the encoder stage to extract the direction, intensity, and pattern of port container handling sway; designs a fusion mechanism for sway characteristics and image features, enabling the network to adjust the deblurring strategy based on the port's unique sway characteristics; and retains sway information during the downsampling process to ensure that deep features still contain motion clues caused by port equipment.

[0098] (3) Multi-scale residual learning: The innovative combination of residual learning and U-Net architecture enables the network to more effectively learn the blur transformation caused by the shaking of port containers. Specifically, a special multi-scale residual block is designed to simultaneously handle the blur of different scales of port container shaking; the residual connection structure is optimized to make it more suitable for the short-term and violent shaking characteristics of the port environment; and a channel attention mechanism is introduced to enhance the ability to extract key features of port containers.

[0099] (4) Spatiotemporal context-aware skip connections: Traditional U-Net skip connections only transmit spatial information. This paper designs enhanced skip connections that can transmit spatiotemporal context information. Specifically, a special spatiotemporal feature fusion module is designed to simultaneously transmit spatial and temporal information of the port scene in the skip connection; a learnable spatiotemporal feature selection mechanism is introduced to dynamically determine which features are more helpful for deblurring the port container monitoring images; and the feature fusion algorithm is optimized to reduce computational overhead while improving information transmission efficiency.

[0100] (5) Lightweight and efficient design: To achieve real-time processing in a port environment, a series of lightweight innovations have been made to the architecture, specifically: depthwise separable convolutions are used to replace some standard convolutions, significantly reducing the number of parameters and computational complexity; a mechanism for balancing computational cost and effect is designed, allocating computing resources according to the importance of different layers; through the optimization of network depth and width, the computational complexity is minimized while ensuring the deblurring effect; all grid blocks share the same network weights, significantly reducing the number of parameters and memory usage.

[0101] The following is a detailed description of each module generated by the model:

[0102] The first is the 3D spatiotemporal feature extraction module, such as Figure 7As shown, this module is used to capture the spatiotemporal relationships and sway pattern characteristics of short-term, violent sway sequences in port container yard monitoring. In this application, the input multi-frame sequence is arranged in a special dimension, converting the input shape of [N, S, S, 3] into the form of [1, 3, N, S, S], so that 3D convolution can simultaneously process temporal and spatial information, where N represents the number of frames (preferably 3 frames), S represents the grid block size, 3 represents the RGB channels, and 1 is the batch size, representing the current sample (a multi-frame sequence). This arrangement enables the network to better capture the characteristics of short-term, violent sway in port container operations.

[0103] In addition, this application also designs a special 3D convolution kernel with a shape of (n_frames,3,3). Unlike traditional 2D convolution or sequential scanning methods, this design can simultaneously extract features in both temporal and spatial dimensions. The number of input channels is 3 (RGB) and the number of output channels is 64, achieving effective information compression and feature enhancement. The relevant parameters are set as follows: the stride is 1, the padding in the temporal dimension is 0 (to capture complete temporal information), and the spatial dimension is 1 (to preserve edge features).

[0104] In this application, the blurriness of video frames can be determined using a sharp-blur information fusion module, a submodule of the 3D spatiotemporal feature extraction module. This module effectively distinguishes and utilizes the sharp and blurred frames in the input sequence. Through feature interaction in the temporal dimension, the port container structure information in the sharp frames can guide the recovery process of the blurred frames. This sharp-blur information fusion is a key advantage of this invention over traditional single-frame deblurring methods.

[0105] The second is the shake-aware encoder, such as Figure 8 As shown in the figure, the encoder extracts deeper features layer by layer through a multi-layer downsampling convolutional network, while reducing the spatial resolution and providing multi-scale feature representation for subsequent processing.

[0106] Preferably, the shake perception encoder includes a first encoding layer, a second encoding layer and a third encoding layer, the input of the first encoding layer is a feature map maintaining the original resolution, the input of the second encoding layer is a feature map downsampled by 2 times, and the input of the third encoding layer is a feature map downsampled by 4 times.

[0107] The output results of the first coding layer and the second coding layer are passed to the spatiotemporal context decoder through a skip connection layer, and the output result of the third coding layer is resized by bilinear interpolation and then passed to the residual block module for processing.

[0108] Specifically, the first coding layer (i.e. Figure 8The encoding layer 1 in [1] takes as input a feature map of size [batch, 64, S, S]. A 3×3 convolution kernel (stride 1, padding 1) is used to expand the number of channels from 64 to 128. Batch normalization is used to standardize the feature distribution and accelerate training. A ReLU activation function is then used to introduce nonlinear transformation capabilities. The output is a feature map of size [batch, 128, S, S] with the same resolution. This map is then saved to the decoder layer via skip connections for feature reuse.

[0109] The second coding layer (i.e. Figure 8 The input of the encoding layer 2 in [1] is the feature map of [batch, 128, S, S]. The number of channels is expanded from 128 to 256 through a 3×3 convolution kernel (stride 2, padding 1). Batch normalization standardizes the feature distribution and accelerates training. The ReLU activation function introduces nonlinear transformation capability and outputs a feature map of [batch, 256, S / 2, S / 2] with half the spatial size. This feature map is saved to the decoder layer via skip connections to achieve cross-level feature fusion and enhance detail restoration capabilities.

[0110] The third coding layer (i.e. Figure 8 The input of the encoding layer 3 in the [batch, 256, S / 2, S / 2] feature map is spatially downsampled through a 3×3 convolution kernel (stride 2, padding 1), expanding the number of channels from 256 to 512 to enhance feature expression capabilities. The batch normalization layer standardizes the data distribution and accelerates model convergence. The ReLU activation function introduces nonlinear transformation capabilities, and the output spatial size is halved to a feature map of [batch, 512, S / 4, S / 4]. This feature map is passed to the subsequent residual block processing stage for deep feature extraction and fusion, improving the model's ability to model complex patterns.

[0111] like Figure 9 As shown in the figure, the residual block module is set between the encoder and decoder. Through multi-layer residual learning, the network's ability to express complex shaking patterns in port container operations is enhanced, and at the same time, the residual connection is used to effectively alleviate the gradient vanishing problem of the deep network.

[0112] Specifically, when configuring the residual block, in the deep feature extraction stage, the optimal balance between complexity and performance is achieved by configuring 6 residual blocks in series, where Figure 10 As shown in the figure, each residual block contains a 3×3 convolution layer, batch normalization, and ReLU activation function. With the above settings, the following beneficial effects can be achieved:

[0113] (1) Complexity balance: Experiments have shown that when the number of residual blocks is less than 4, the modeling capability of complex swaying patterns in port container operations is insufficient. When the number of residual blocks exceeds 8, the computational overhead increases significantly while the performance improvement is less than 5%. Six residual blocks achieve optimal feature expression capability while maintaining computational efficiency.

[0114] (2) Receptive field matching: The series connection of six 3×3 convolutional layers expands the theoretical receptive field of the network to 13×13 pixels, which accurately matches the typical blur kernel size (10-15 pixels) caused by short-term violent shaking caused by port container loading and unloading equipment, thereby improving the targetedness of motion blur removal.

[0115] (3) Gradient stability: The residual connection structure allows the gradient to propagate stably in deep networks. Tests show that the variance of the training loss convergence curve is reduced by 40% without introducing additional regularization measures at a depth of 6 layers, effectively suppressing the gradient vanishing problem.

[0116] Residual blocks provide a shortcut for gradient backpropagation through identity mapping, enabling effective training even with a depth of six residual blocks. The network model learns the residual portion rather than the complete mapping, making it easier for the model to approximate the ideal deblurring function. Special optimizations for the swaying scenes of port container operations enable the model to more effectively handle different types of camera shake, including random jitter, linear motion, and rotational motion.

[0117] like Figure 11 As shown, the decoder module gradually restores spatial resolution through transposed convolution (deconvolution) operations. It also utilizes spatiotemporal context-aware skip connections to obtain multi-scale feature information from the encoder, achieving high-quality deblurred reconstruction. Traditional U-Net skip connections only transmit spatial information. This paper designs enhanced skip connections that can transmit spatiotemporal context information. Information about the port container scene can be effectively transferred between different layers through these skip connections, ensuring that detailed information is not lost in deep network processing. Each skip connection incorporates a feature channel attention mechanism that intelligently selects and weights encoder features, prioritizing the information most useful for recovering port scene details.

[0118] like Figure 12 As shown in the figure, the detail enhancement output layer converts the feature map output by the decoder into the final RGB image, focusing on restoring the edges and texture details of the port containers. The format of the RGB image output by the detail enhancement output layer is [batch, S, S, 3], where batch represents the output block batch, S represents the block size, and 3 represents the RGB channels.

[0119] The detail enhancement output layer uses a 3×3 convolution kernel to reduce the number of feature channels from 64 to 3 (corresponding to the three RGB channels). The Tanh activation function constrains the output value range to [-1, 1] to achieve dynamic color balance. The feature map is then reconstructed into the standard RGB format [batch, S, S, 3] through a dimensionality adjustment operation. This design significantly improves the clarity and structural integrity of port container images by preserving high-frequency edge details and texture features.

[0120] In some embodiments, as Figure 13 As shown, the 3D spatiotemporal network model includes a model discriminator, and the model discriminator includes:

[0121] The 3D spatiotemporal feature perception module combines the grid block sequence and the generator output image into a tensor of shape [1, 3, N+1, S, S], applies 3D convolution with kernel size (N+1, 4, 4) and stride (1, 2, 2), and outputs a feature map of shape [1, 64, S / 2, S / 2], where S is the grid block size and is an even number.

[0122] The 2D block GAN discriminant chain performs multi-layer convolution downsampling on the feature map and outputs a true or false feature map.

[0123] To address the key flaw of traditional 2D PatchGAN, which cannot model temporal dimension information, this application designs a 3D spatiotemporal feature perception module, which is used to capture the temporal correlation features between the input sequence (N frames) and the output image of port container monitoring. The specific implementation process includes:

[0124] (1) Combine the input sequence and the output image into a tensor of dimension [1,3,N+1,S,S];

[0125] (2) Apply a 3D convolution operation with a kernel size of (N+1,4,4) and a stride of (1,2,2);

[0126] (3) Through full compression of the time dimension and spatial downsampling (step 2), the output dimension is converted to [1, 64, S / 2, S / 2];

[0127] (4) The LeakyReLU activation function with a negative slope coefficient α=0.2 is used to significantly enhance the network's sensitivity to the swaying characteristics of the port scene and the stability of gradient propagation.

[0128] The 2D block GAN discriminant chain generates feature maps with decreasing spatial resolution through multi-layer convolution downsampling to quantify the authenticity probability of the port scene image. The specific hierarchical structure is as follows:

[0129] The first layer (spatial downsampling): The input dimension is [batch, 64, S / 2, S / 2]. A 4×4 convolution kernel with a stride of 2 is used to expand the number of channels from 64 to 128, reducing the output spatial resolution to 1 / 4 of the original size (i.e., [batch, 128, S / 4, S / 4]). This layer integrates batch normalization and a LeakyReLU activation function with a negative slope coefficient α=0.2 to mitigate the vanishing gradient problem in low-light port scenes.

[0130] The second layer (deep feature extraction) uses a 4×4 convolution kernel with a stride of 2 to increase the number of channels from 128 to 256, with the input dimension [batch, 128, S / 4, S / 4]. The output resolution is further reduced to 1 / 8 ([batch, 256, S / 8, S / 8]). The batch normalization layer mitigates feature distribution shifts caused by port rain and fog during training, and the LeakyReLU layer enhances sensitivity to container displacement features.

[0131] The third layer (channel expansion and feature preservation): The input dimensions are [batch, 256, S / 8, S / 8]. A 4×4 convolution kernel with a stride of 1 and padding of 1 is used to expand the number of channels from 256 to 512, while the output maintains the same resolution of [batch, 512, S / 8, S / 8]. This design avoids excessive loss of spatial information and preserves high-frequency texture information for subsequent local detail discrimination.

[0132] Output layer (discriminant score generation): Input dimensions are [batch, 512, S / 8, S / 8]. A 4×4 convolution kernel with a stride of 1 and padding of 1 is used to reduce the number of channels from 512 to 1. The output is a raw logits matrix of dimensions [batch, 1, S / 16, S / 16]. The absence of an activation function ensures an unrestricted output range. Each spatial point corresponds to the authenticity probability of a local area (such as a container corner fitting or door edge), enabling pixel-level quality assessment.

[0133] The proposed discriminator, a 3D-TSPatchGAN, is specifically designed for port monitoring. It achieves joint discrimination between input sequences and output images through a hybrid 3D+2D architecture (3D convolution compresses the temporal dimension + 2D convolution processes spatial features). This design effectively captures dynamic temporal correlations in container operations (such as the continuity of lifting trajectories), identifies temporal inconsistencies between generated frames and monitored sequences, and significantly improves temporal coherence. Furthermore, a local grid-based discriminator (output resolution S / 16×S / 16) focuses on container edge textures and reflective details on metal surfaces, enhancing detection sensitivity to blur and artifacts and improving container number recognition accuracy. Its lightweight design significantly reduces the number of parameters compared to a global discriminator, meeting the latency requirements of real-time inspection at port gates. Training stability is enhanced through batch normalization and a local discriminant strategy, effectively suppressing the risk of mode collapse. Dynamic input resizing (e.g., S=512 / 1024) is supported, resulting in minimal performance fluctuation across resolutions.

[0134] In some embodiments, the 3D spatiotemporal network model is trained according to the following method:

[0135] A sample image set is obtained, a sample grid block sequence is obtained based on the sample image set, the sample grid block sequence is input into a 3D spatiotemporal network model to be trained, and iterative training is performed to obtain a trained 3D spatiotemporal network model.

[0136] Preferably, the sample image set includes a first sample image set, a second sample image set and a third sample image set;

[0137] The first sample image set is obtained according to the following method:

[0138] Collecting actual surveillance video streams from multiple port container yards, extracting all video frame images from each video stream, calculating the clarity score of each video frame image, screening out video frame images with an average clarity score higher than a preset clarity threshold, and extracting a continuous video frame sequence from the screened video frame images as a first sample image set;

[0139] The second sample image set is obtained according to the following method:

[0140] Setting a linear shake model, a jitter shake model, and a combined shake model, wherein the linear shake model includes a linear shake blur kernel, the jitter shake model includes a jitter shake blur kernel, and the combined shake model includes a linear shake blur kernel and a jitter shake blur kernel;

[0141] Inputting the first sample image set into a linear shake model, a jitter shake model, and a combined shake model respectively to obtain a second sample image set;

[0142] The third sample image set is obtained according to the following method:

[0143] Transformation processing is performed on the images in the first sample image set and the second sample image set to obtain a third sample image set, where the transformation processing includes geometric transformation, illumination transformation, and noise addition.

[0144] This solution uses a dual-path dataset construction method to collect samples required for model training:

[0145] (1) Simulation data expansion: extract key frames from clear port surveillance videos, generate noisy samples by synthesizing multiple motion blur models (including typical interference such as ship sway and crane displacement), and construct time-matched blur-clear frame pairs;

[0146] (2) Physical anti-shake comparison acquisition: Deploy a camera array equipped with mechanical optical image stabilization (OIS) to simultaneously capture the clear image after anti-shake processing and the original shaken image in the port operation scene, forming an accurate blur-clear pairing sample verified by the physical mechanism.

[0147] The process of collecting real-scene data of port container yards is as follows: actual surveillance videos are collected from multiple port container yards, the port surveillance video files in the dataset are traversed, all frame images of each video are extracted, the clarity score of each frame is calculated (using gradient amplitude, frequency domain analysis and other methods), and the videos whose average clarity score of each frame image is higher than the preset clarity threshold are screened out. Continuous video sequences are extracted from the screened videos as basic training data.

[0148] The clarity score of a video frame is calculated as follows:

[0149] First, the gradient magnitude is evaluated , the formula is as follows:

[0150] ;

[0151] in, is the height of the image (number of pixel rows), is the width of the image (number of pixel columns), For the Row, No. The grayscale value (or brightness) of the column pixels, Indicates that the image is in the horizontal direction ( axis), Indicates that the image is in the vertical direction ( axis).

[0152] Secondly, high frequency energy ratio The calculation formula is as follows:

[0153] ;

[0154] in, is the frequency domain representation (complex value) of the image after Fourier transform, It is the coordinate in the frequency domain. High-frequency area: usually refers to the area close to the edge of the spectrum, corresponding to the detailed information of the image.

[0155] Then proceed to contrast evaluation , the formula is as follows:

[0156] ;

[0157] in, is the mean gray value of the image, is the standard deviation of the image grayscale value, is a grayscale image.

[0158] Finally, a comprehensive clarity score is given. The calculation formula is as follows:

[0159] ,

[0160] in, They are the weight coefficients of the three evaluation indicators, satisfying (usually set by experience or optimized through training); 、 、 They are the gradient amplitude evaluation, high frequency energy ratio and contrast evaluation indicators calculated previously; Provides a comprehensive score for the final image clarity.

[0161] Of course, the model can also use the real scene data of the port container yard to complete the self-supervision generation of data. The generation method is as follows: Figure 14 And based on the real scene data and the data generated by the model self-supervision, transformation processing can be performed to obtain a new sample image set, and the transformation processing includes geometric transformation, illumination transformation, and adding noise.

[0162] This application specifically designs two dedicated fuzzy kernels to simulate the sudden shaking of the port container monitoring system:

[0163] Linear Motion Blur Kernel: Given a blur length parameter L and an angle parameter θ, a kernel matrix of size (2L+1) × (2L+1) is constructed. With the center of the matrix as the origin, L pixels are extended on either side of the angle θ. The pixel values ​​covered by the trajectory are set to 1, and the kernel matrix is ​​normalized to simulate the uniform linear motion caused by the translation of the port crane.

[0164] Jitter blur kernel: Given a jitter intensity I and a complexity parameter C, a jitter trajectory is formed by generating C randomly connected points from the center point of the kernel matrix, and the pixel value covered by the trajectory is set to 1. A Gaussian filter with a standard deviation of I is applied to the trajectory matrix to simulate the irregular optical defocus blur caused by ship surges or mechanical vibrations.

[0165] This design can cover the main disturbance types in ports and adapt to different operating environments by adjusting the L / θ / I / C parameters to meet the needs of real-time data enhancement. This fuzzy kernel generation method, as a training data enhancement module, works in conjunction with the 3D-TSPatchGAN discriminator to improve the model's anti-disturbance capabilities.

[0166] like Figure 15 As shown, when constructing a port-specific sway pattern library, the present application divides the core sway types into linear sway, jitter sway and compound sway, and generates a parameterized fuzzy template library based on these three core types. Through multi-dimensional physical modeling, the sudden disturbance characteristics in port loading and unloading operations (such as crane translation, ship bumps and compound mechanical vibrations) are accurately restored. Finally, the template library is applied to clear video frames to generate simulated sway degradation effects, providing customized training data covering the entire scene for the deblurring algorithm.

[0167] The training processes of the model generator and model discriminator involved in this application are as follows: Figure 16 and Figure 17 shown.

[0168] This application innovatively proposes a 3D-TSPatchNet architecture, breaking through the limitations of traditional 2D U-Net and enabling the network to have the capabilities of three-dimensional spatiotemporal information processing, efficient grid block calculation and multi-scale feature extraction; on this basis, this application designs a 3D spatiotemporal feature extraction front-end specifically for the short-term sway characteristics of ports, effectively capturing and utilizing the port's unique sway pattern information; at the same time, it introduces spatiotemporal context-aware jump connections to improve the deblurring quality and port container detail restoration capabilities by transmitting spatiotemporal context information; develops a sway characteristic perception mechanism, enabling the network to dynamically adjust the deblurring strategy according to the sway characteristics of port container operations; and innovatively designs a time-series-aware 3D-TSPatchGAN discriminator, which adopts a 3D-2D hybrid architecture to achieve effective discrimination of the temporal continuity of port scenes, fundamentally solving the defect that traditional discriminators cannot process time dimension information, thereby comprehensively improving the robustness and detail restoration accuracy of port video deblurring.

[0169] In a second aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video deblurring method based on 3D spatiotemporal grid perception as described in the first aspect of the present invention.

[0170] The computer-readable storage medium may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories.

[0171] The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface storage, an optical disc, or a compact disc read-only memory (CD ROM); the magnetic surface storage may be a magnetic disk storage or a magnetic tape storage.

[0172] The volatile memory may be a random access memory (RAM) that is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronized dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). The computer-readable storage medium described in the embodiments of the present invention is intended to include these and any other suitable types of memory.

[0173] like Figure 18 As shown, in a third aspect, the present invention provides an electronic device 10, comprising a processor 101 and a storage medium 102, on which a computer program is stored, and when the computer program is executed by the processor, the video deblurring method based on 3D spatiotemporal grid perception as described in the first aspect of the present invention is implemented.

[0174] In some embodiments, the processor can be implemented by software, hardware, firmware or a combination thereof, and can use at least one of a circuit, a single or multiple application-specific integrated circuits (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor, so that the processor can execute some or all of the steps or any combination of the steps in the video deblurring method based on 3D spatiotemporal grid perception described in various embodiments of the present application.

[0175] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of this application, this does not limit the scope of patent protection of this application. All technical solutions generated by replacing or modifying equivalent structures or equivalent processes based on the essential concepts of this application using the contents recorded in the specification and drawings of this application, as well as directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are included in the scope of patent protection of this application.

Claims

1. A video deblurring method based on 3D spatiotemporal grid perception, characterized in that: The following steps are involved: Receive a video image sequence of N consecutive frames, and the input dimension vector is [N,H,W,3], where N represents the number of input video image frames, H and W represent the height and width respectively, and 3 represents the RGB channel; The shake intensity is calculated by the optical flow amplitude of adjacent frames. Each video frame is adaptively divided into overlapping grid blocks of size S×S. For each pixel position (i, j) in the video image, the grid block corresponding to that pixel position is extracted from N video frames, forming a grid block sequence of shape [N, S, S, 3], where N is a positive integer greater than 1, the N-1 frame is the clear frame image, and the Nth frame image is the current frame image to be deblurred. Input the grid block sequence into the trained 3D spatiotemporal network model for block processing, and output the deblurred grid blocks; Calculating a fusion weight based on the distance between the pixel position and the center of the adjacent grid block, performing weighted fusion and reorganization on the deblurred grid blocks, and outputting a clear current frame image; The 3D spatiotemporal network model includes a model generator, and the model generator includes: The 3D spatiotemporal feature extraction module converts the input [N, S, S, 3] grid block sequence into [1, 3, N, S, S] dimensions to adapt to the 3D convolution input, where 1 represents the batch size, indicating that the input is a multi-frame sequence; The shake-aware encoder extracts multi-scale features through three layers of downsampling convolution and stores the output in a skip connection layer. Residual block module, which consists of 6 residual blocks connected in series. Each residual block contains two 3×3 convolutional layers and an identity skip connection. The spatiotemporal context decoder, which upsamples through deconvolution and uses skip connections to obtain multi-scale features of the shake-aware encoder; The detail enhancement output layer reduces the feature channels from 64 to 3 through 3×3 convolution and outputs an RGB image in the range [-1, 1] through the Tanh activation function; The 3D spatiotemporal network model includes a model discriminator, and the model discriminator includes: The 3D spatiotemporal feature perception module combines the grid block sequence and the generator output image into a tensor of shape [1, 3, N+1, S, S], applies 3D convolution with kernel size (N+1, 4, 4) and stride (1, 2, 2), and outputs a feature map of shape [1, 64, S / 2, S / 2], where S is the grid block size and is an even number. The 2D block GAN discriminant chain performs multi-layer convolution downsampling on the feature map and outputs a true or false feature map.

2. The video deblurring method based on 3D spatiotemporal grid perception according to claim 1, characterized in that: The shake-aware encoder includes a first coding layer, a second coding layer, and a third coding layer. The input of the first coding layer is a feature map with original resolution, the input of the second coding layer is a feature map downsampled by 2 times, and the input of the third coding layer is a feature map downsampled by 4 times. The output results of the first coding layer and the second coding layer are passed to the spatiotemporal context decoder through a skip connection layer, and the output result of the third coding layer is resized by bilinear interpolation and then passed to the residual block module for processing.

3. The video deblurring method based on 3D spatiotemporal grid perception according to claim 1, wherein: The format of the RGB image output by the detail enhancement output layer is [batch, S, S, 3], where batch represents the output block batch, S represents the block size, and 3 represents the RGB channel.

4. The video deblurring method based on 3D spatiotemporal grid perception according to claim 1, wherein: The activation function of the 3D convolution is LeakyReLU.

5. The video deblurring method based on 3D spatiotemporal grid perception according to claim 1, wherein: The 3D spatiotemporal network model is trained according to the following method: A sample image set is obtained, a sample grid block sequence is obtained based on the sample image set, the sample grid block sequence is input into a 3D spatiotemporal network model to be trained, and iterative training is performed to obtain a trained 3D spatiotemporal network model.

6. The video deblurring method based on 3D spatiotemporal grid perception according to claim 5, characterized in that: The sample image set includes a first sample image set, a second sample image set and a third sample image set; The first sample image set is obtained according to the following method: Collecting actual surveillance video streams from multiple port container yards, extracting all video frame images from each video stream, calculating the clarity score of each video frame image, screening out video frame images with an average clarity score higher than a preset clarity threshold, and extracting a continuous video frame sequence from the screened video frame images as a first sample image set; The second sample image set is obtained according to the following method: Setting a linear shake model, a jitter shake model, and a combined shake model, wherein the linear shake model includes a linear shake blur kernel, the jitter shake model includes a jitter shake blur kernel, and the combined shake model includes a linear shake blur kernel and a jitter shake blur kernel; Inputting the first sample image set into a linear shake model, a jitter shake model, and a combined shake model respectively to obtain a second sample image set; The third sample image set is obtained according to the following method: Transformation processing is performed on the images in the first sample image set and the second sample image set to obtain a third sample image set, where the transformation processing includes geometric transformation, illumination transformation, and noise addition.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the video deblurring method based on 3D spatiotemporal grid perception is implemented as described in any one of claims 1 to 6.

8. An electronic device having a computer program stored thereon, characterized in that: The method comprises a processor and a storage medium, wherein a computer program is stored on the storage medium, and when the computer program is executed by the processor, the video deblurring method based on 3D spatiotemporal grid perception is implemented as claimed in any one of claims 1 to 6.

Images