An image motion deblurring method based on event-guided attention network

Abstract

An event-guided lightweight attention network for efficiently removing image motion blur belongs to the field of image processing technology. The attention network includes an image feature extraction module, an event feature extraction module, a cross-modal attention feature fusion module, and an image reconstruction module. First, the image coding features and event coding features are extracted from the motion blurred image and the event voxel grid respectively through the image feature extraction module and the event feature extraction module. Then, the image coding features and the event coding features are fused through the cross-modal attention feature fusion module to obtain a fused feature. Finally, the fused feature is passed through the image reconstruction module to obtain a clear image. The present application realizes the removal of image motion blur, and the obtained image has higher clarity, saves image processing time, and has higher image processing efficiency. The attention network of the present invention is a single-stage network with lightweight characteristics.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image motion deblurring method based on an event-guided attention network. Background Art

[0002] Image motion deblurring is a fundamental computer vision task that involves removing blurring artifacts from images to restore the original, clear content. Motion blur can be caused by a variety of factors, such as camera shake and rapid object movement, resulting in a loss of detail and quality in the captured image. The goal of image motion deblurring is to produce a clear, high-quality image that accurately represents the original scene.

[0003] Motion blur is caused by the long exposure time and low frame rate of ordinary cameras, which prevent the lens from accurately focusing light onto the same position on the photosensitive element. A blurred image can be viewed as the integral of a series of potentially sharp images. The final imaging result is that a single image frame incorporates information from multiple frames of the original scene, resulting in a blurred state. Recovering a sharp image from a blurred image is a highly ill-posed problem due to the infinite number of feasible solutions. While image-only methods based on convolutional neural networks have achieved promising results in recent years, these methods rely on supervised training on simulated datasets. In real-world scenarios, the inconsistency between the distribution of real-world and simulated data leads to significant performance degradation for these methods. Furthermore, since image-only methods cannot explicitly capture motion, their performance is inherently limited when dealing with real-world blurred images, especially under extreme blur conditions.

[0004] An event camera is a novel neuromorphic visual sensor inspired by key features of the human visual system. Compared to traditional cameras, it offers the advantages of high frame rate (equivalent to 10k+fps), extremely low latency (<1ms), high dynamic range (over 120dB), extremely low power consumption (100mW or less), and low data rate. The high frame rate allows event cameras to record scene motion information that traditional cameras cannot. Therefore, this paper applies event cameras to image motion deblurring to address the motion blur problem faced by cameras in high-speed motion scenes.

[0005] To address this issue, the present invention introduces an event camera to capture more motion information. Furthermore, existing methods suffer from large model size and parameter count, making them difficult to deploy on embedded devices and limiting their application. To address this issue, the present invention designs a lightweight single-stage network consisting of a lightweight image feature extraction module, a lightweight event feature extraction module, a lightweight feature fusion module, and a lightweight image reconstruction module. Summary of the Invention

[0006] In order to overcome the deficiencies of the prior art, the present invention provides an image de-motion blurring method based on an event-guided attention network, wherein the attention network includes an image feature extraction module, an event feature extraction module, a cross-modal attention feature fusion module, and an image reconstruction module; first, image coding features and event coding features are extracted from the motion-blurred image and the event voxel grid respectively through the image feature extraction module and the event feature extraction module, then, the image coding features and the event coding features are fused through the cross-modal attention feature fusion module to obtain a fusion feature, and finally, the fusion feature is passed through the image reconstruction module to obtain a clear image. The present application realizes the removal of motion blur of an image, and the obtained image has higher clarity, saves image processing time, and has higher image processing efficiency; the attention network of the present invention is a single-stage network with lightweight characteristics.

[0007] The steps of the specific technical solution of the present invention are as follows:

[0008] Step 1: Build a dataset: Based on the existing GoPro dataset, use the ESIM simulator to generate event streams for motion-blurred images. The GoPro dataset contains 3,214 motion-blurred images with a resolution of 1,280 × 720, including 2,103 training images and 1,111 test images.

[0009] Step 2, motion blurred image preprocessing: crop the motion blurred image into 256×256 image blocks according to the resolution;

[0010] Step 3, event stream preprocessing: Integrate the event stream along the time dimension to obtain an event voxel grid, and crop it according to the cropping method in step 2 to obtain an event voxel grid corresponding to the motion blurred image block space;

[0011] Step 4: Build an attention network: The attention network consists of four core modules, namely the image feature extraction module (Gimg module), the event feature extraction module (Gevent module), the multimodal attention feature fusion module (Gattention module), and the image reconstruction module (Grestoration module). The motion blurred image preprocessed in step 2 is input into the Gimg module to obtain image coding features, and the event voxel grid preprocessed in step 3 is input into the Gevent module to obtain event coding features. The image coding features and event coding features are passed through the Gattention module to obtain fused features. The fused features are passed through the Grestoration module to reconstruct the image.

[0012] Step 5: Train the attention network: Construct a loss function and update the attention network parameters by calculating the value of the loss function until the attention network converges.

[0013] S5.1, determine the loss function PSNRLoss: The loss function PSNRLoss formula is as follows:

[0014]

[0015] Where MAX_I is the maximum value of the pixel in the training image block, and MSE is the mean square error between the predicted image and the true image;

[0016] S5.2, training on the training set: two NVIDA 1080Ti GPUs are used for training; the learning rate of the attention network is set to 0.0002, and a cosine decay strategy is used; the batch_size of each NVIDA 1080Ti GPU is set to 4; the number of iterations is 200,000; after the attention network converges, the model weights are saved;

[0017] S5.3, tested on the test set: In the test results, the peak signal-to-noise ratio (PSNR) index is 34.57, and the structural similarity (SSIM) index is 0.9660;

[0018] Step 6, perform model inference: First, preprocess the motion blurred image according to steps 2 to 3, and then input the preprocessed motion blurred image into the trained attention network model to obtain a clear image without motion blur.

[0019] Furthermore, the formula used in the integration method in step 3 is as follows:

[0020]

[0021] Among them, e(t) represents the event triggered at time t; ei represents a single event;

[0022] Each motion blurred image is obtained by weighting 7 consecutive clear images. The capture time of the 7 consecutive clear images is t i , the value range of i is [1,7]; the time interval between adjacent clear images is t; according to the above integration formula, through 6 integration operations, an event voxel grid with 6 channels is obtained, and each blurred image corresponds to an event voxel grid.

[0023] Furthermore, the Gimg module in step 4 includes three identical image convolution modules (ImgConvBlock modules) and two downsampling layers; the three identical image convolution modules are image convolution module 1, image convolution module 2 and image convolution module 3, each image convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers and a 1×1 convolution layer for residual connection; the residual connection helps to improve the attention network's ability to express and learn image features, the activation layer uses a RELU activation function, and the downsampling layer consists of a convolution layer with a kernel size of 4×4 and a stride of 2; the execution steps of the Gimg module are as follows:

[0024] S4.1.1, convolve the motion blurred image to generate image feature f1, and the number of its channels increases to 64;

[0025] S4.1.2, input feature f1 into image convolution module 1 for feature extraction and perform downsampling once, obtaining image encoding feature img_encs[1] and downsampled feature img_down1. The image size of downsampled feature img_down1 is reduced from 256×256 to 128×128.

[0026] S4.1.3, the downsampled feature img_down1 passes through the image convolution module 2 to generate the image encoding feature img_encs[2] and the downsampled feature img_down2. The image size of the downsampled feature img_down2 is reduced from 128×128 to 64×64.

[0027] S4.1.4, the downsampled feature img_down2 is processed by the image convolution module 3 to obtain the image encoding feature img_encs[3];

[0028] Finally, after the blurred image is extracted by the Gimg module, three multi-level image encoding features img_encs[1], img_encs[2] and img_encs[3] are obtained.

[0029] Furthermore, the Gevent module in step 4 includes three identical event convolution modules (EventConvBlock modules) and two downsampling layers; the three identical event convolution modules are event convolution module 1, event convolution module 2, and event convolution module 3, each event convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers, and a 1×1 convolution layer for residual connection; the residual connection helps to improve the network's ability to express and learn event features, and the activation layer uses a RELU activation function; the execution steps of the Gevent module are as follows:

[0030] S4.2.1, convolve the event voxel grid to generate event feature e1, and its channel number is increased to 64;

[0031] S4.2.2, input the event feature e1 into the event convolution module 1 for feature extraction and perform downsampling once, obtaining the event encoding feature event_encs[1] and the downsampled feature event_down1, and reducing the image size from 256×256 to 128×128;

[0032] S4.2.3, the downsampled feature event_down1 passes through the event convolution module 2 to generate the event encoding feature event_encs[2] and the downsampled feature event_down2. The image size of the downsampled feature event_down2 is reduced from 128×128 to 64×64.

[0033] S4.2.4, the downsampled feature event_down2 is processed by the event convolution module 3 and finally obtains the event encoding feature event_encs[3];

[0034] Finally, after the event voxel grid passes through the Gevent module, three multi-level event encoding features event_encs[1], event_encs[2] and event_encs[3] are obtained.

[0035] Furthermore, the Gattention module in step 4 includes three fusion modules (Fused-Moudle modules), namely Fused-Moudle module 1, Fused-Moudle module 2 and Fused-Moudle module 3. Each Fused-Moudle module includes two custom normalization layers (LayerNorm), namely image normalization layer and event normalization layer. The normalization layer is used to normalize the image coding features and event coding features before feature fusion; an attention layer for cross-modal feature fusion; a multi-layer perceptron (MLP network) for feature mapping and feature dimension processing; the execution steps of the Gattention module are as follows:

[0036] S4.3.1, input the event encoding feature event_encs[1] and the image encoding feature img_encs[1] into the Fused-Moudle module 1 to determine whether the dimensions of the two are consistent. If the dimensions are inconsistent, fusion cannot be performed. If the dimensions are consistent, continue to execute as follows;

[0037] S4.3.2, the image encoding feature img_encs[1] passes through the image normalization layer to obtain the normalized image feature 1; similarly, the event encoding feature event_encs[1] passes through the event normalization layer to obtain the normalized event feature 1;

[0038] S4.3.3, input the image feature 1 and event feature 1 obtained in S4.3.2 into the attention layer for feature fusion to obtain the fused feature, input the fused feature into the MLP network for feature mapping and size adjustment, and finally output the fused feature fused[1] with the same dimension as event feature 1 and image feature 1;

[0039] S4.3.4, input the event encoding feature event_encs[2] and the image encoding feature img_encs[2] into Fused-Moudle module 2 respectively, obtain event feature 2 and image feature 2 after event normalization and image normalization, and then obtain the fused feature fused[2] with the same dimension as event feature 2 and image feature 2 according to S4.3.3;

[0040] S4.3.5, input the event encoding feature event_encs[3] and the image encoding feature img_encs[3] into the Fused-Moudle module 3 respectively, and obtain the event feature 3 and image feature 3 after event normalization and image normalization. Then, according to S4.3.3, obtain the fused feature fused[3] with the same dimension as the event feature 3 and image feature 3;

[0041] Finally, the event coding features and image coding features at three different levels are fused through the Gattention module to obtain three fused features fused[1], fused[2] and fused[3]. The three fused features fused[1], fused[2] and fused[3] form a fused feature pyramid with three layers. The three fused features fused[1], fused[2] and fused[3] serve as the input of the subsequent image reconstruction module.

[0042] Furthermore, the Grestauration module described in step 4 is used to restore a clear image from the fused features. The Grestauration module includes two upsampling modules (UpBlock) and a convolution activation module (ConvSAM); the two upsampling modules include upsampling module 1 and upsampling module 2, each upsampling module includes a deconvolution layer and an image convolution module, the convolution kernel size of the deconvolution layer is 2×2, stride=2, padding=1, and the convolution activation module includes 3 convolution layers and a RELU activation layer; after the fused features fused[1], fused[2] and fused[3] pass through the Grestauration module, a clear image is output; the execution steps of the Grestauration module are as follows:

[0043] S4.4.1, input the fused features fused[2] and fused[3] into upsampling module 1, and after deconvolution upsampling and convolution, activation, and residual connection, the decoded feature decs[0] is obtained;

[0044] S4.4.2, the decoded feature decs[0] and the fused feature fused[1] are input into upsampling module 2, and after deconvolution upsampling and convolution, activation, and residual connection, the decoded feature decs[1] is obtained;

[0045] S4.4.3, the decoded features decs[1] and the motion blurred image are input into the convolution activation module, and the clear image is finally output;

[0046] Finally, the fused features fused[1], fused[2] and fused[3] are processed by the Grestoration module, which can effectively remove motion blur and output a clear image.

[0047] Furthermore, in the image convolution module 1, the two convolution layers are convolution layer 1 and convolution layer 2, each convolution layer has 64 input channels, 64 output channels, a convolution kernel size of 3×3, 64 convolution kernels, stride=1, padding=1, and includes a bias term; the two activation layers include activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use a RELU activation function; the residual connection layer has 64 input channels, 64 output channels, a convolution kernel size of 1×1, 64 convolution kernels, stride=1, padding=0, and includes a bias term;

[0048] In the image convolution module 2, the two convolution layers are convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 128, the output channel is 128, the convolution kernel size is 1×1, the number of convolution kernels is 128, the step length stride=1, padding=0, and the bias term is included;

[0049] In the image convolution module 3, the two convolution layers are convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 256, the output channel is 256, the convolution kernel size is 1×1, the number of convolution kernels is 256, the step length stride=1, padding=0, and the bias term is included.

[0050] Furthermore, in the event convolution module 1, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3, the input channels of convolution layer 1 and convolution layer 2 are 64, the output channels are 64, the convolution kernel size is 3×3, the number of convolution kernels is 64, the step length stride=1, padding=1, and the bias term is included; the input channels of convolution layer 3 are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included; the two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channels of the residual connection layer are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included;

[0051] In the event convolution module 2, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the stride is 1, the padding is 1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the stride is 1, the padding is 1, and the bias term is included; 3 has 128 input channels, 128 output channels, a convolution kernel size of 1×1, 128 convolution kernels, stride=1, padding=0, and includes a bias term. The two activation layers are activation layer 1 and activation layer 2, and both activation layers 1 and 2 use the RELU activation function. The residual connection layer has 128 input channels, 128 output channels, a convolution kernel size of 1×1, 128 convolution kernels, stride=1, padding=0, and includes a bias term.

[0052] In the event convolution module 3, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; 3 has 256 input channels, 256 output channels, a 1×1 convolution kernel size, 256 convolution kernels, stride=1, padding=0, and includes a bias term. The two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function. The residual connection layer has 256 input channels, 256 output channels, a 1×1 convolution kernel size, 256 convolution kernels, stride=1, padding=0, and includes a bias term.

[0053] Furthermore, in the Fused-Moudle module 1, the input dimension of the image normalization layer 1 and the event normalization layer 1 is 64, with a bias term; the input dimension of the attention layer is 64, without a bias term; the MLP network includes an input layer, a hidden layer, and an output layer, the input layer contains 64 linear units, the hidden layer contains 128 linear units, and the output layer contains 64 linear units;

[0054] In the Fused-Moudle module 2, the input dimension of the image normalization layer 2 and the event normalization layer 2 is 64, with a bias term; the input dimension of the attention layer is 128, without a bias term; the MLP network includes an input layer, a hidden layer, and an output layer. The input layer contains 128 linear units, the hidden layer contains 256 linear units, and the output layer contains 128 linear units.

[0055] In the Fused-Moudle module 3, the input dimension of the image normalization layer 3 and the event normalization layer 3 is 64, with a bias term; the input dimension of the attention layer is 256, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer contains 256 linear units, the hidden layer contains 512 linear units, and the output layer contains 512 linear units.

[0056] Beneficial effects of the present invention:

[0057] 1. The lightweight single-stage attention network structure provided by this invention has lightweight characteristics after training. The attention network model size is only 20431KB (19.95MB), which can be easily deployed on a single NVIDA 1080Ti GPU server and perform inference at a speed of 53ms per image, with a frame rate of up to 20FPS;

[0058] 2. The present invention has an excellent deblurring effect on images, and its PSNR evaluation index is ahead of most existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the present invention and show the differences from existing inventions, the following is an illustration of the various technical solutions of the present invention.

[0060] Figure 1 The overall flow chart of the present invention;

[0061] Figure 2 The Gimg module of the present invention;

[0062] Figure 3 The Gevent module of the present invention;

[0063] Figure 4 The Gattention module of the present invention;

[0064] Figure 5 Grestoration module of the present invention;

[0065] Figure 6 The image de-motion blur effect of the present invention is demonstrated;

[0066] Figure 6In the figure, (a) is the input blurred image, and (b) is the output clear image.

[0067] Figure 7 The attention network model size and quantitative evaluation performance of this invention. DETAILED DESCRIPTION

[0068] The present invention will be further described below with reference to the accompanying drawings and examples.

[0069] like Figure 1 As shown, the present invention provides an image de-motion blurring method based on an event-guided attention network, wherein the attention network includes an image feature extraction module, an event feature extraction module, a cross-modal attention feature fusion module, and an image reconstruction module; first, image coding features and event coding features are extracted from the motion-blurred image and the event voxel grid respectively through the image feature extraction module and the event feature extraction module, then, the image coding features and the event coding features are fused through the cross-modal attention feature fusion module to obtain a fusion feature, and finally, the fusion feature is passed through the image reconstruction module to obtain a clear image. The present application realizes the removal of motion blur of an image, and the obtained image has higher clarity, saves image processing time, and has higher image processing efficiency; the attention network of the present invention is a single-stage network with lightweight characteristics.

[0070] The steps of the specific technical solution of the present invention are as follows:

[0071] Step 1: Build a dataset: Based on the existing GoPro dataset, use the ESIM simulator to generate event streams for motion-blurred images. The GoPro dataset contains 3,214 motion-blurred images with a resolution of 1,280 × 720, including 2,103 training images and 1,111 test images.

[0072] Step 2, motion blurred image preprocessing: crop the motion blurred image into 256×256 image blocks according to the resolution;

[0073] Step 3, event stream preprocessing: Integrate the event stream along the time dimension to obtain an event voxel grid, and crop it according to the cropping method in step 2 to obtain an event voxel grid corresponding to the motion blurred image block space;

[0074] Step 4: Build an attention network: The attention network consists of four core modules, namely the image feature extraction module (Gimg module), the event feature extraction module (Gevent module), the multimodal attention feature fusion module (Gattention module), and the image reconstruction module (Grestoration module). The motion blurred image preprocessed in step 2 is input into the Gimg module to obtain image coding features, and the event voxel grid preprocessed in step 3 is input into the Gevent module to obtain event coding features. The image coding features and event coding features are passed through the Gattention module to obtain fused features. The fused features are passed through the Grestoration module to reconstruct the image.

[0075] Step 5: Train the attention network: Construct a loss function and update the attention network parameters by calculating the value of the loss function until the attention network converges.

[0076] S5.1, determine the loss function PSNRLoss: The loss function PSNRLoss formula is as follows:

[0077]

[0078] Where MAX_I is the maximum value of the pixel in the training image block, and MSE is the mean square error between the predicted image and the true image;

[0079] S5.2, training on the training set: two NVIDA 1080Ti GPUs are used for training; the learning rate of the attention network is set to 0.0002, and a cosine decay strategy is used; the batch_size of each NVIDA 1080Ti GPU is set to 4; the number of iterations is 200,000; after the attention network converges, the model weights are saved;

[0080] S5.3, tested on the test set: In the test results, the peak signal-to-noise ratio (PSNR) index is 34.57, and the structural similarity (SSIM) index is 0.9660;

[0081] Step 6, perform model inference: First, preprocess the motion blurred image according to steps 2 to 3, and then input the preprocessed motion blurred image into the trained attention network model to obtain a clear image without motion blur.

[0082] Furthermore, the formula used in the integration method in step 3 is as follows:

[0083]

[0084] Among them, e(t) represents the event triggered at time t; ei represents a single event;

[0085] Each motion blurred image is obtained by weighting 7 consecutive clear images. The capture time of the 7 consecutive clear images is t i , the value range of i is [1,7]; the time interval between adjacent clear images is t; according to the above integration formula, through 6 integration operations, an event voxel grid with 6 channels is obtained, and each blurred image corresponds to an event voxel grid.

[0086] Furthermore, the Gimg module in step 4 includes three identical image convolution modules (ImgConvBlock modules) and two downsampling layers; the three identical image convolution modules are image convolution module 1, image convolution module 2 and image convolution module 3, each image convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers and a 1×1 convolution layer for residual connection; the residual connection helps to improve the attention network's ability to express and learn image features, the activation layer uses the RELU activation function, and the downsampling layer consists of a convolution layer with a kernel size of 4×4 and a stride of 2; Figure 2 As shown, the execution steps of the Gimg module are as follows:

[0087] S4.1.1, convolution is performed on the motion blurred image with 64 convolution kernels to generate image feature f1, and the number of its channels increases to 64;

[0088] S4.1.2, input feature f1 into image convolution module 1 for feature extraction and perform downsampling once, obtaining image encoding feature img_encs[1] and downsampled feature img_down1. The image size of downsampled feature img_down1 is reduced from 256×256 to 128×128.

[0089] S4.1.3, the downsampled feature img_down1 passes through the image convolution module 2 to generate the image encoding feature img_encs[2] and the downsampled feature img_down2. The image size of the downsampled feature img_down2 is reduced from 128×128 to 64×64.

[0090] S4.1.4, the downsampled feature img_down2 is processed by the image convolution module 3 to obtain the image encoding feature img_encs[3];

[0091] Finally, after the blurred image is extracted by the Gimg module, three multi-level image encoding features img_encs[1], img_encs[2] and img_encs[3] are obtained.

[0092] Furthermore, the Gevent module in step 4 includes three identical event convolution modules (EventConvBlock modules) and two downsampling layers; the three identical event convolution modules are event convolution module 1, event convolution module 2 and event convolution module 3, each event convolution module consists of two convolution layers, the convolution kernel size of each convolution layer is 3×3, two activation layers and a 1×1 convolution layer for residual connection; the residual connection helps to improve the network's ability to express and learn event features, and the activation layer uses the RELU activation function; Figure 3 As shown, the execution steps of the Gevent module are as follows:

[0093] S4.2.1, convolution is performed on the event voxel grid, the number of convolution kernels is 64, and event feature e1 is generated, and the number of channels thereof is increased to 64;

[0094] S4.2.2, input the event feature e1 into the event convolution module 1 for feature extraction and perform downsampling once, obtaining the event encoding feature event_encs[1] and the downsampled feature event_down1, and reducing the image size from 256×256 to 128×128;

[0095] S4.2.3, the downsampled feature event_down1 passes through the event convolution module 2 to generate the event encoding feature event_encs[2] and the downsampled feature event_down2. The image size of the downsampled feature event_down2 is reduced from 128×128 to 64×64.

[0096] S4.2.4, the downsampled feature event_down2 is processed by the event convolution module 3 and finally obtains the event encoding feature event_encs[3];

[0097] Finally, after the event voxel grid passes through the Gevent module, three multi-level event encoding features event_encs[1], event_encs[2] and event_encs[3] are obtained.

[0098] Furthermore, the Gattention module in step 4 includes three fusion modules (Fused-Moudle modules), which are Fused-Moudle module 1, Fused-Moudle module 2 and Fused-Moudle module 3. Each Fused-Moudle module includes two custom normalization layers (LayerNorm), which are image normalization layer and event normalization layer. The normalization layer is used to normalize the image coding features and event coding features before feature fusion; an attention layer for cross-modal feature fusion; and a multi-layer perceptron (MLP network) for feature mapping and feature dimension processing. Figure 4 As shown in Figure 2, the execution steps of the Gattention module are as follows:

[0099] S4.3.1, input the event encoding feature event_encs[1] and the image encoding feature img_encs[1] into the Fused-Moudle module 1 to determine whether the dimensions of the two are consistent. If the dimensions are inconsistent, fusion cannot be performed. If the dimensions are consistent, continue to execute as follows;

[0100] S4.3.2, the image encoding feature img_encs[1] passes through the image normalization layer to obtain the normalized image feature 1; similarly, the event encoding feature event_encs[1] passes through the event normalization layer to obtain the normalized event feature 1;

[0101] S4.3.3, input the image feature 1 and event feature 1 obtained in S4.3.2 into the attention layer for feature fusion to obtain the fused feature, input the fused feature into the MLP network for feature mapping and size adjustment, and finally output the fused feature fused[1] with the same dimension as event feature 1 and image feature 1;

[0102] S4.3.4, input the event encoding feature event_encs[2] and the image encoding feature img_encs[2] into Fused-Moudle module 2 respectively, obtain event feature 2 and image feature 2 after event normalization and image normalization, and then obtain the fused feature fused[2] with the same dimension as event feature 2 and image feature 2 according to S4.3.3;

[0103] S4.3.5, input the event encoding feature event_encs[3] and the image encoding feature img_encs[3] into the Fused-Moudle module 3 respectively, and obtain the event feature 3 and image feature 3 after event normalization and image normalization. Then, according to S4.3.3, obtain the fused feature fused[3] with the same dimension as the event feature 3 and image feature 3;

[0104] Finally, the event coding features and image coding features at three different levels are fused through the Gattention module to obtain three fused features fused[1], fused[2] and fused[3]. The three fused features fused[1], fused[2] and fused[3] form a fused feature pyramid with three layers. The three fused features fused[1], fused[2] and fused[3] serve as the input of the subsequent image reconstruction module.

[0105] Furthermore, the Grestauration module described in step 4 is used to restore a clear image from the fused features. The Grestauration module includes two upsampling modules (UpBlock) and a convolution activation module (ConvSAM); the two upsampling modules include upsampling module 1 and upsampling module 2, each upsampling module includes a deconvolution layer and an image convolution module, the convolution kernel size of the deconvolution layer is 2×2, stride=2, padding=1, and the convolution activation module includes 3 convolution layers and a RELU activation layer; after the fused features fused[1], fused[2] and fused[3] pass through the Grestauration module, a clear image is output; as shown Figure 5 As shown, the execution steps of the Grestoration module are as follows:

[0106] S4.4.1, input the fused features fused[2] and fused[3] into upsampling module 1, and after deconvolution upsampling and convolution, activation, and residual connection, the decoded feature decs[0] is obtained;

[0107] S4.4.2, the decoded feature decs[0] and the fused feature fused[1] are input into upsampling module 2, and after deconvolution upsampling and convolution, activation, and residual connection, the decoded feature decs[1] is obtained;

[0108] S4.4.3, the decoded features decs[1] and the motion blurred image are input into the convolution activation module, and the clear image is finally output;

[0109] Finally, the fused features fused[1], fused[2] and fused[3] are processed by the Grestoration module, which can effectively remove motion blur and output a clear image. The comparison between the input blurred image and the output clear image is shown in Figure 6 ; Figure 7 This is a comparison chart of the effects of the present invention, the pure image method, and the fusion time method. The orange dots represent the effects of the pure image method, the green dots represent the effects of the fusion event method, and the red five-pointed star represents the effects of the present invention.

[0110] Furthermore, in the image convolution module 1, the two convolution layers are convolution layer 1 and convolution layer 2, each convolution layer has 64 input channels, 64 output channels, a convolution kernel size of 3×3, 64 convolution kernels, stride=1, padding=1, and includes a bias term; the two activation layers include activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use a RELU activation function; the residual connection layer has 64 input channels, 64 output channels, a convolution kernel size of 1×1, 64 convolution kernels, stride=1, padding=0, and includes a bias term;

[0111] In the image convolution module 2, the two convolution layers are convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 128, the output channel is 128, the convolution kernel size is 1×1, the number of convolution kernels is 128, the step length stride=1, padding=0, and the bias term is included;

[0112] In the image convolution module 3, the two convolution layers are convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 256, the output channel is 256, the convolution kernel size is 1×1, the number of convolution kernels is 256, the step length stride=1, padding=0, and the bias term is included.

[0113] Furthermore, in the event convolution module 1, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3, the input channels of convolution layer 1 and convolution layer 2 are 64, the output channels are 64, the convolution kernel size is 3×3, the number of convolution kernels is 64, the step length stride=1, padding=1, and the bias term is included; the input channels of convolution layer 3 are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included; the two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channels of the residual connection layer are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included;

[0114] In the event convolution module 2, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the stride is 1, the padding is 1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the stride is 1, the padding is 1, and the bias term is included; 3 has 128 input channels, 128 output channels, a convolution kernel size of 1×1, 128 convolution kernels, stride=1, padding=0, and includes a bias term. The two activation layers are activation layer 1 and activation layer 2, and both activation layers 1 and 2 use the RELU activation function. The residual connection layer has 128 input channels, 128 output channels, a convolution kernel size of 1×1, 128 convolution kernels, stride=1, padding=0, and includes a bias term.

[0115] In the event convolution module 3, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; 3 has 256 input channels, 256 output channels, a 1×1 convolution kernel size, 256 convolution kernels, stride=1, padding=0, and includes a bias term. The two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function. The residual connection layer has 256 input channels, 256 output channels, a 1×1 convolution kernel size, 256 convolution kernels, stride=1, padding=0, and includes a bias term.

[0116] Furthermore, in the Fused-Moudle module 1, the input dimension of the image normalization layer 1 and the event normalization layer 1 is 64, with a bias term; the input dimension of the attention layer is 64, without a bias term; the MLP network includes an input layer, a hidden layer, and an output layer, the input layer contains 64 linear units, the hidden layer contains 128 linear units, and the output layer contains 64 linear units;

[0117] In the Fused-Moudle module 2, the input dimension of the image normalization layer 2 and the event normalization layer 2 is 64, with a bias term; the input dimension of the attention layer is 128, without a bias term; the MLP network includes an input layer, a hidden layer, and an output layer. The input layer contains 128 linear units, the hidden layer contains 256 linear units, and the output layer contains 128 linear units.

[0118] In the Fused-Moudle module 3, the input dimension of the image normalization layer 3 and the event normalization layer 3 is 64, with a bias term; the input dimension of the attention layer is 256, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer contains 256 linear units, the hidden layer contains 512 linear units, and the output layer contains 512 linear units.

[0119] The orange dots are pure image methods, the green dots are methods that integrate events, and the red five-pointed stars are the methods of the present invention.

Claims

1. A method for image motion blurring based on event-guided attention network, characterized in that: The steps of the image de-motion blurring method are as follows: Step 1: Build a dataset: Based on the existing GoPro dataset, use the ESIM simulator to generate event streams for motion-blurred images. The GoPro dataset contains 3,214 motion-blurred images with a resolution of 1,280 × 720, including 2,103 training images and 1,111 test images. Step 2, motion blurred image preprocessing: crop the motion blurred image into 256×256 image blocks according to the resolution; Step 3, event stream preprocessing: Integrate the event stream along the time dimension using an integration method to obtain an event voxel grid, and crop it according to the cropping method in step 2 to obtain an event voxel grid corresponding to the motion blurred image block space; Step 4: Build an attention network. The attention network consists of four core modules: Gimg module, Gevent module, Gattention module, and Grestoration module. Input the motion blurred image preprocessed in step 2 into the Gimg module to obtain image coding features. Input the event voxel grid preprocessed in step 3 into the Gevent module to obtain event coding features. The image coding features and event coding features are fused by the Gattention module. The fused features are used for image reconstruction through the Grestoration module; The Gimg module includes three identical image convolution modules and two downsampling layers; the three identical image convolution modules are image convolution module 1, image convolution module 2 and image convolution module 3, each image convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers and a 1×1 convolution layer for residual connection, the activation layer uses the RELU activation function, and the downsampling layer consists of a convolution layer with a kernel size of 4×4 and a stride of 2; the execution steps of the Gimg module are as follows: S4.1.1, convolve the motion blurred image to generate image feature f1, and the number of convolution channels is increased to 64; S4.1.2, input feature f1 into image convolution module 1 for feature extraction and perform downsampling once, obtaining image encoding feature img_encs[1] and downsampled feature img_down1. The image size of downsampled feature img_down1 is reduced from 256×256 to 128×128. S4.1.3, the downsampled feature img_down1 passes through the image convolution module 2 to generate the image encoding feature img_encs[2] and the downsampled feature img_down2. The image size of the downsampled feature img_down2 is reduced from 128×128 to 64×64. S4.1.4, the downsampled feature img_down2 is processed by the image convolution module 3 to obtain the image encoding feature img_encs[3]; Finally, after the blurred image is extracted by the Gimg module, three multi-level image encoding features img_encs[1], img_encs[2] and img_encs[3] are obtained; The Gevent module contains three identical event convolution modules and two downsampling layers; the three identical event convolution modules are event convolution module 1, event convolution module 2 and event convolution module 3, each event convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers and a 1×1 convolution layer for residual connection; the activation layer uses the RELU activation function; the execution steps of the Gevent module are as follows: S4.2.1, convolve the event voxel grid to generate event feature e1, and its channel number is increased to 64; S4.2.2, input the event feature e1 into the event convolution module 1 for feature extraction and perform downsampling once, obtaining the event encoding feature event_encs[1] and the downsampled feature event_down1, and reducing the image size from 256×256 to 128×128; S4.2.3, the downsampled feature event_down1 passes through the event convolution module 2 to generate the event encoding feature event_encs[2] and the downsampled feature event_down2. The image size of the downsampled feature event_down2 is reduced from 128×128 to 64×64. S4.2.4, the downsampled feature event_down2 is processed by the event convolution module 3 and finally obtains the event encoding feature event_encs[3]; Finally, after the event voxel grid passes through the Gevent module, three multi-level event encoding features event_encs[1], event_encs[2] and event_encs[3] are obtained; The Gattention module includes three fusion modules, namely, Fused-Moudle module 1, Fused-Moudle module 2, and Fused-Moudle module 3. Each Fused-Moudle module includes two custom normalization layers, namely, an image normalization layer and an event normalization layer. The normalization layer is used to normalize the image coding features and the event coding features before feature fusion. An attention layer for cross-modal feature fusion; a multi-layer perceptron for feature mapping and feature dimension processing; the execution steps of the Gattention module are as follows: S4.3.1, input the event encoding feature event_encs[1] and the image encoding feature img_encs[1] into the Fused-Moudle module 1 to determine whether the dimensions of the two are consistent. If the dimensions are inconsistent, fusion cannot be performed. If the dimensions are consistent, continue to execute as follows; S4.3.2, the image encoding feature img_encs[1] passes through the image normalization layer to obtain the normalized image feature 1; similarly, the event encoding feature event_encs[1] passes through the event normalization layer to obtain the normalized event feature 1; S4.3.3, input the image feature 1 and event feature 1 obtained in S4.3.2 into the attention layer for feature fusion to obtain the fused feature, input the fused feature into the MLP network for feature mapping and size adjustment, and finally output the fused feature fused[1] with the same dimension as event feature 1 and image feature 1; S4.3.4, input the event encoding feature event_encs[2] and the image encoding feature img_encs[2] into Fused-Moudle module 2 respectively, obtain event feature 2 and image feature 2 after event normalization and image normalization, and then obtain the fused feature fused[2] with the same dimension as event feature 2 and image feature 2 according to S4.3.3; S4.3.5, input the event encoding feature event_encs[3] and the image encoding feature img_encs[3] into the Fused-Moudle module 3 respectively, and obtain the event feature 3 and image feature 3 after event normalization and image normalization. Then, according to S4.3.3, obtain the fused feature fused[3] with the same dimension as the event feature 3 and image feature 3; Finally, the event coding features and image coding features at three different levels are fused through the Gattention module to obtain three fused features fused[1], fused[2], and fused[3]. The three fused features fused[1], fused[2], and fused[3] form a fused feature pyramid with three layers. The three fused features fused[1], fused[2], and fused[3] serve as the input of the subsequent image reconstruction module. The Grestauration module is used to restore a clear image from the fused features. The Grestauration module includes two upsampling modules and a convolution activation module. The two upsampling modules include upsampling module 1 and upsampling module 2. Each upsampling module includes a deconvolution layer and an image convolution module. The convolution kernel size of the deconvolution layer is 2×2, stride=2, padding=1, and the convolution activation module includes 3 convolution layers and a RELU activation layer. After the fused features fused[1], fused[2] and fused[3] pass through the Grestauration module, a clear image is output. The execution steps of the Grestauration module are as follows: S4.4.1, input the fused features fused[2] and fused[3] into upsampling module 1, and after deconvolution upsampling and convolution, activation, and residual connection, the decoded feature decs[0] is obtained; S4.4.2, the decoded feature decs[0] and the fused feature fused[1] are input into upsampling module 2, and after deconvolution upsampling and convolution, activation, and residual connection, the decoded feature decs[1] is obtained; S4.4.3, the decoded features decs[1] and the motion blurred image are input into the convolution activation module, and the clear image is finally output; Finally, the fused features fused[1], fused[2] and fused[3] are processed by the Grestoration module, which can effectively remove motion blur and output a clear image; Step 5: Train the attention network: Construct a loss function and update the attention network parameters by calculating the value of the loss function until the attention network converges. S5.1, determine the loss function PSNRLoss: The loss function PSNRLoss formula is as follows: Where MAX_I is the maximum value of the pixel in the training image block, and MSE is the mean square error between the predicted image and the true image; S5.2, training on the training set: two NVIDA 1080Ti GPUs are used for training; the learning rate of the attention network is set to 0.0002, and a cosine decay strategy is used; the batch_size of each NVIDA 1080Ti GPU is set to 4; the number of iterations is 200,000; after the attention network converges, the model weights are saved; S5.3, tested on the test set: In the test results, the peak signal-to-noise ratio (PSNR) index is 34.57, and the structural similarity (SSIM) index is 0.9660; Step 6, perform model inference: First, preprocess the motion blurred image according to steps 2 to 3, and then input the preprocessed motion blurred image into the trained attention network model to obtain a clear image without motion blur.

2. The image de-motion blurring method based on event-guided attention network according to claim 1, characterized in that: In step 3, the formula of the integration method is as follows: Among them, e(t) represents the event triggered at time t; ei represents a single event; Each motion blurred image is obtained by weighting 7 consecutive clear images. The capture time of the 7 consecutive clear images is t i , the value range of i is [1,7]; the time interval between adjacent clear images is t; according to the formula of the integration method, through 6 integration operations, an event voxel grid with 6 channels is obtained, and each blurred image corresponds to an event voxel grid.

3. The image de-motion blurring method based on event-guided attention network according to claim 1, characterized in that: In the image convolution module 1, the two convolution layers are convolution layer 1 and convolution layer 2, each convolution layer has 64 input channels, 64 output channels, a convolution kernel size of 3×3, 64 convolution kernels, stride=1, padding=1, and includes a bias term; the two activation layers include activation layer 1 and activation layer 2, both activation layer 1 and activation layer 2 use the RELU activation function; the residual connection layer has 64 input channels, 64 output channels, a convolution kernel size of 1×1, 64 convolution kernels, stride=1, padding=0, and includes a bias term; In the image convolution module 2, the two convolution layers are convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 128, the output channel is 128, the convolution kernel size is 1×1, the number of convolution kernels is 128, the step length stride=1, padding=0, and the bias term is included; In the image convolution module 3, the two convolution layers are convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 256, the output channel is 256, the convolution kernel size is 1×1, the number of convolution kernels is 256, the step length stride=1, padding=0, and the bias term is included.

4. The image de-motion blurring method based on event-guided attention network according to claim 1, characterized in that: In the event convolution module 1, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3, the input channels of convolution layer 1 and convolution layer 2 are 64, the output channels are 64, the convolution kernel size is 3×3, the number of convolution kernels is 64, the step length stride=1, padding=1, and the bias term is included; the input channels of convolution layer 3 are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included; the two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channels of the residual connection layer are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included; In the event convolution module 2, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the stride is 1, the padding is 1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the stride is 1, the padding is 1, and the bias term is included; 3 has 128 input channels, 128 output channels, a convolution kernel size of 1×1, 128 convolution kernels, stride=1, padding=0, and includes a bias term. The two activation layers are activation layer 1 and activation layer 2, and both activation layers 1 and 2 use the RELU activation function. The residual connection layer has 128 input channels, 128 output channels, a convolution kernel size of 1×1, 128 convolution kernels, stride=1, padding=0, and includes a bias term. In the event convolution module 3, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; 3 has 256 input channels, 256 output channels, a 1×1 convolution kernel size, 256 convolution kernels, stride=1, padding=0, and includes a bias term. The two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function. The residual connection layer has 256 input channels, 256 output channels, a 1×1 convolution kernel size, 256 convolution kernels, stride=1, padding=0, and includes a bias term.

5. The image de-motion blurring method based on event-guided attention network according to claim 1, characterized in that: In the Fused-Moudle module 1, the input dimension of the image normalization layer 1 and the event normalization layer 1 is 64, with a bias term; the input dimension of the attention layer is 64, without a bias term; the MLP network includes an input layer, a hidden layer, and an output layer. The input layer contains 64 linear units, the hidden layer contains 128 linear units, and the output layer contains 64 linear units. In the Fused-Moudle module 2, the input dimension of the image normalization layer 2 and the event normalization layer 2 is 64, with a bias term; the input dimension of the attention layer is 128, without a bias term; the MLP network includes an input layer, a hidden layer, and an output layer. The input layer contains 128 linear units, the hidden layer contains 256 linear units, and the output layer contains 128 linear units. In the Fused-Moudle module 3, the input dimension of the image normalization layer 3 and the event normalization layer 3 is 64, with a bias term; the input dimension of the attention layer is 256, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer contains 256 linear units, the hidden layer contains 512 linear units, and the output layer contains 512 linear units.

6. An attention network for an image de-motion blurring method, characterized in that: The attention network is applicable to the image demotion blurring method according to claims 1 to 5.

Images