Image motion blur removing method based on event-guided attention network

Abstract

An event-guided lightweight attention network for efficiently removing image motion blur belongs to the technical field of image processing and comprises an image feature extraction module, an event feature extraction module, a cross-modal attention feature fusion module and an image reconstruction module. Firstly, image coding features and event coding features are extracted from a motion blurred image and an event voxel grid through an image feature extraction module and an event feature extraction module respectively, then, the image coding features and the event coding features are fused through a cross-modal attention feature fusion module to obtain fusion features, and finally, the motion blurred image and the event voxel grid are subjected to cross-modal attention feature fusion. The fused features are processed by an image reconstruction module to obtain a clear image. According to the invention, the motion blur removal of the image is realized, the obtained image is higher in definition, the image processing time is saved, and the image processing efficiency is higher; the attention network is a single-stage network and has the characteristic of light weight.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image motion blur removal method based on an event-guided attention network. Background Art

[0002] Image deblurring is a fundamental computer vision task that involves removing blur artifacts from an image to restore the original, clear content. Motion blur can be caused by a variety of factors, such as camera shake, rapid motion of objects, etc., resulting in loss of detail and quality of the captured image. The goal of image deblurring is to generate a clear, high-quality image that accurately represents the original scene.

[0003] Motion blur is caused by the long exposure time and low acquisition frame rate of ordinary cameras, which makes the lens unable to accurately focus the light on the same position of the photosensitive element. A frame of blurred image can be regarded as the integral of a series of potential clear images. From the final imaging result, one frame of image fuses multiple frames of original scene information, so it appears blurred. Restoring a clear image from a blurred image is a highly ill-posed problem because there are infinitely many feasible solutions. In recent years, although pure image methods based on convolutional neural networks have achieved good results, this relies on supervised training on simulation data sets. In real scenes, due to the inconsistency of the distribution of real scene data and simulation data, the pure image method has a great performance degradation. At the same time, since the pure image mode cannot explicitly capture motion, it inherently limits its performance when facing blurry images in the real world, especially under extreme blur conditions.

[0004] The event camera is a new type of neuromorphic visual sensor inspired by the key features of the human visual system. Compared with traditional cameras, it has the advantages of high frame rate (equivalent to 10k+fps), extremely low latency (<1ms), high dynamic range (over 120dB), extremely low power consumption (100mW or less), and low data rate. The advantage of high frame rate enables the event camera to record scene motion information that traditional cameras cannot record. Therefore, the present invention applies the event camera to the field of image de-motion blur to solve the motion blur problem faced by the camera in high-speed motion scenes.

[0005] In order to solve this problem, the present invention introduces an event camera to capture more motion information. In addition, the existing methods all face the problems of large model size and large number of parameters, which makes it difficult to deploy on embedded devices and limits their application scope. In order to solve this problem, the present invention designs a lightweight single-stage network, including a lightweight image feature extraction module, a lightweight event feature extraction module, a lightweight feature fusion module, and a lightweight image reconstruction module. Summary of the invention

[0006] In order to overcome the shortcomings of the prior art, the present invention provides an image de-motion blurring method based on an event-guided attention network, wherein the attention network includes an image feature extraction module, an event feature extraction module, a cross-modal attention feature fusion module, and an image reconstruction module; first, image coding features and event coding features are extracted from the motion blurred image and the event voxel grid through the image feature extraction module and the event feature extraction module, respectively, and then, the image coding features and the event coding features are fused through the cross-modal attention feature fusion module to obtain fused features, and finally, the fused features are passed through the image reconstruction module to obtain a clear image. The present application realizes the removal of motion blur of an image, and the obtained image has higher clarity, saves image processing time, and has higher image processing efficiency; the attention network of the present invention is a single-stage network with lightweight characteristics.

[0007] The steps of the specific technical solution of the present invention are as follows:

[0008] Step 1, build a dataset: Based on the existing GoPro dataset, use the ESIM simulator to generate event streams for motion blurred images; the GoPro dataset contains 3214 motion blurred images with a resolution of 1280×720, including 2103 training images and 1111 test images;

[0009] Step 2, motion blurred image preprocessing: the motion blurred image is cropped into 256×256 image blocks according to the resolution;

[0010] Step 3, event stream preprocessing: integrate the event stream along the time dimension to obtain an event voxel grid, and crop it according to the cropping method of step 2 to obtain an event voxel grid corresponding to the motion blurred image block space;

[0011] Step 4, build an attention network: the attention network consists of four core modules, namely, image feature extraction module (Gimg module), event feature extraction module (Gevent module), multimodal attention feature fusion module (Gattention module), and image reconstruction module (Grestoration module); the motion blurred image preprocessed in step 2 is input into the Gimg module to obtain image coding features, and the event voxel grid preprocessed in step 3 is input into the Gevent module to obtain event coding features. The image coding features and event coding features are passed through the Gattention module to obtain fused features; the fused features are passed through the Grestoration module to reconstruct the image;

[0012] Step 5, training the attention network: construct a loss function, and update the attention network parameters by calculating the value of the loss function until the attention network converges;

[0013] S5.1, determine the loss function PSNRLoss: The loss function PSNRLoss formula is as follows:

[0014]

[0015] Among them, MAX_I is the maximum value of the pixel in the training image block, and MSE is the mean square error between the predicted image and the true image;

[0016] S5.2, training on the training set: two NVIDA 1080Ti GPU graphics cards are used for training; the learning rate of the attention network is set to 0.0002, and the cosine decay strategy is adopted; the batch_size of each NVIDA 1080Ti GPU graphics card is set to 4; the number of iterations is 200,000; the model weights are saved after the attention network converges;

[0017] S5.3, tested on the test set: In the test results, the peak signal-to-noise ratio PSNR index is 34.57, and the structural similarity SSIM index is 0.9660;

[0018] Step 6, perform model inference: first preprocess the motion blurred image according to steps 2 to 3, and then input the preprocessed motion blurred image into the trained attention network model to obtain a clear image without motion blur.

[0019] Furthermore, the formula used in the integration method in step 3 is as follows:

[0020]

[0021] Among them, e(t) represents the event triggered at time t; ei represents a single event;

[0022] Each motion blurred image is obtained by weighting 7 consecutive clear images. The capture time of the 7 consecutive clear images is denoted as t i , the value range of i is [1,7]; the time interval between adjacent clear images is t; according to the above integral formula, through 6 integral operations, an event voxel grid with 6 channels is obtained, and each blurred image corresponds to an event voxel grid.

[0023] Furthermore, the Gimg module in step 4 includes three identical image convolution modules (ImgConvBlock modules) and two downsampling layers; the three identical image convolution modules are image convolution module 1, image convolution module 2 and image convolution module 3, each image convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers and a 1×1 convolution layer for residual connection; the residual connection helps to improve the attention network's ability to express and learn image features, the activation layer uses a RELU activation function, and the downsampling layer consists of a convolution layer with a kernel size of 4×4 and a step size of 2; the execution steps of the Gimg module are as follows:

[0024] S4.1.1, convolve the motion blurred image to generate image feature f1, and the number of its channels increases to 64;

[0025] S4.1.2, input feature f1 to image convolution module 1 for feature extraction and perform downsampling once, to obtain image encoding feature img_encs[1] and downsampled feature img_down1, and the image size of downsampled feature img_down1 is reduced from 256×256 to 128×128;

[0026] S4.1.3, the downsampled feature img_down1 passes through the image convolution module 2 to generate the image encoding feature img_encs[2] and the downsampled feature img_down2. The image size of the downsampled feature img_down2 is reduced from 128×128 to 64×64.

[0027] S4.1.4, the downsampled feature img_down2 is processed by the image convolution module 3 to obtain the image encoding feature img_encs[3];

[0028] Finally, after the blurred image is extracted by the Gimg module, three multi-level image encoding features img_encs[1], img_encs[2] and img_encs[3] are obtained.

[0029] Furthermore, the Gevent module in step 4 includes three identical event convolution modules (EventConvBlock modules) and two downsampling layers; the three identical event convolution modules are event convolution module 1, event convolution module 2 and event convolution module 3, each event convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers and a 1×1 convolution layer for residual connection; the residual connection helps to improve the network's ability to express and learn event features, and the activation layer uses a RELU activation function; the execution steps of the Gevent module are as follows:

[0030] S4.2.1, convolve the event voxel grid to generate event feature e1, and its channel number is increased to 64;

[0031] S4.2.2, input the event feature e1 into the event convolution module 1 for feature extraction and perform downsampling once, obtain the event encoding feature event_encs[1] and the downsampled feature event_down1, and reduce the image size from 256×256 to 128×128;

[0032] S4.2.3, the downsampled feature event_down1 passes through the event convolution module 2 to generate the event encoding feature event_encs[2] and the downsampled feature event_down2. The image size of the downsampled feature event_down2 is reduced from 128×128 to 64×64.

[0033] S4.2.4, the downsampled feature event_down2 is processed by the event convolution module 3 to obtain the event encoding feature event_encs[3];

[0034] Finally, after the event voxel grid passes through the Gevent module, three multi-level event encoding features event_encs[1], event_encs[2] and event_encs[3] are obtained.

[0035] Furthermore, the Gattention module in step 4 includes three fusion modules (Fused-Moudle modules), the three Fused-Moudle modules are Fused-Moudle module 1, Fused-Moudle module 2 and Fused-Moudle module 3, each Fused-Moudle module includes two custom normalization layers (LayerNorm), the two custom normalization layers are image normalization layer and event normalization layer, the normalization layer is used to normalize the image coding features and event coding features before feature fusion; an attention layer is used for cross-modal feature fusion; a multi-layer perceptron (MLP network) is used for feature mapping and feature dimension processing; the execution steps of the Gattention module are as follows:

[0036] S4.3.1, input the event encoding feature event_encs[1] and the image encoding feature img_encs[1] into the Fused-Moudle module 1, and determine whether the dimensions of the two are consistent. If the dimensions are inconsistent, they cannot be fused. If the dimensions are consistent, continue to execute as follows;

[0037] S4.3.2, the image coding feature img_encs[1] passes through the image normalization layer to obtain the normalized image feature 1; similarly, the event coding feature event_encs[1] passes through the event normalization layer to obtain the normalized event feature 1;

[0038] S4.3.3, input the image feature 1 and event feature 1 obtained in S4.3.2 into the attention layer for feature fusion to obtain the fused feature, input the fused feature into the MLP network for feature mapping and size adjustment, and finally output the fused feature fused[1] with the same dimension as event feature 1 and image feature 1;

[0039] S4.3.4, input the event coding feature event_encs[2] and the image coding feature img_encs[2] into Fused-Moudle module 2 respectively, obtain event feature 2 and image feature 2 after event normalization and image normalization, and then obtain the fused feature fused[2] with the same dimension as event feature 2 and image feature 2 according to S4.3.3;

[0040] S4.3.5, input the event encoding feature event_encs[3] and the image encoding feature img_encs[3] into the Fused-Moudle module 3 respectively, obtain the event feature 3 and the image feature 3 after the event normalization and image normalization, and then obtain the fused feature fused[3] with the same dimension as the event feature 3 and the image feature 3 according to S4.3.3;

[0041] Finally, the event coding features and image coding features at three different levels are fused through the Gattention module to obtain three fused features fused[1], fused[2] and fused[3]. The three fused features fused[1], fused[2] and fused[3] form a fused feature pyramid with three layers. The three fused features fused[1], fused[2] and fused[3] are used as the input of the subsequent image reconstruction module.

[0042] Furthermore, the Grerestoration module in step 4 is used to restore a clear image from the fused features. The Grerestoration module includes two upsampling modules (UpBlock) and a convolution activation module (ConvSAM); the two upsampling modules include upsampling module 1 and upsampling module 2, each upsampling module includes a deconvolution layer and an image convolution module, the convolution kernel size of the deconvolution layer is 2×2, stride=2, padding=1, and the convolution activation module includes 3 convolution layers and a RELU activation layer; after the fused features fused[1], fused[2] and fused[3] pass through the Grerestoration module, a clear image is output; the execution steps of the Grerestoration module are as follows:

[0043] S4.4.1, input the fused features fused[2] and fused[3] into upsampling module 1, and obtain the decoded feature decs[0] after deconvolution upsampling and convolution, activation, and residual connection;

[0044] S4.4.2, the decoded feature decs[0] and the fused feature fused[1] are input into upsampling module 2, and after deconvolution upsampling, convolution, activation, and residual connection, the decoded feature decs[1] is obtained;

[0045] S4.4.3, the decoded feature decs[1] and the motion blurred image are input into the convolution activation module, and finally a clear image is output;

[0046] Finally, the fused features fused[1], fused[2] and fused[3] are processed by the Grestoration module, which can effectively remove motion blur and output a clear image.

[0047] Furthermore, in the image convolution module 1, the two convolution layers are respectively convolution layer 1 and convolution layer 2, the input channel of each convolution layer is 64, the output channel is 64, the convolution kernel size is 3×3, the number of convolution kernels is 64, the step length stride=1, padding=1, and a bias term is included; the two activation layers include activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use a RELU activation function; the input channel of the residual connection layer is 64, the output channel is 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and a bias term is included;

[0048] In the image convolution module 2, the two convolution layers are respectively convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the two activation layers are respectively activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 128, the output channel is 128, the convolution kernel size is 1×1, the number of convolution kernels is 128, the step length stride=1, padding=0, and the bias term is included;

[0049] In the image convolution module 3, the two convolution layers are respectively convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and a bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and a bias term is included; the two activation layers are respectively activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 256, the output channel is 256, the convolution kernel size is 1×1, the number of convolution kernels is 256, the step length stride=1, padding=0, and a bias term is included.

[0050] Furthermore, in the event convolution module 1, the three convolution layers are respectively convolution layer 1, convolution layer 2 and convolution layer 3, the input channels of convolution layer 1 and convolution layer 2 are 64, the output channels are 64, the convolution kernel size is 3×3, the number of convolution kernels is 64, the step length stride=1, padding=1, and the bias term is included; the input channels of convolution layer 3 are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included; the two activation layers are respectively activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channels of the residual connection layer are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included;

[0051] In the event convolution module 2, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the convolution layer 3 has 128 input channels, 128 output channels, 1×1 convolution kernel size, 128 convolution kernel number, stride=1, padding=0, including bias term; the two activation layers are activation layer 1 and activation layer 2, both of which use RELU activation function; the residual connection layer has 128 input channels, 128 output channels, 1×1 convolution kernel size, 128 convolution kernel number, stride=1, padding=0, including bias term;

[0052] In the event convolution module 3, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the convolution layer 3 has 256 input channels, 256 output channels, a convolution kernel size of 1×1, a number of convolution kernels of 256, stride=1, padding=0, and includes a bias term. The two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function. The residual connection layer has 256 input channels, 256 output channels, a convolution kernel size of 1×1, a number of convolution kernels of 256, stride=1, padding=0, and includes a bias term.

[0053] Furthermore, in the Fused-Moudle module 1, the input dimension of the image normalization layer 1 and the event normalization layer 1 is 64, with a bias term; the input dimension of the attention layer is 64, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer includes 64 linear units, the hidden layer includes 128 linear units, and the output layer includes 64 linear units;

[0054] In the Fused-Moudle module 2, the input dimension of the image normalization layer 2 and the event normalization layer 2 is 64, with a bias term; the input dimension of the attention layer is 128, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer includes 128 linear units, the hidden layer includes 256 linear units, and the output layer includes 128 linear units;

[0055] In the Fused-Moudle module 3, the input dimension of the image normalization layer 3 and the event normalization layer 3 is 64, with a bias term; the input dimension of the attention layer is 256, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer contains 256 linear units, the hidden layer contains 512 linear units, and the output layer contains 512 linear units.

[0056] Beneficial effects of the present invention:

[0057] 1. The lightweight single-stage attention network structure provided by the present invention has lightweight characteristics after training. The size of the attention network model is only 20431KB (19.95MB), which can be easily deployed on a single NVIDA 1080Ti GPU server and perform reasoning at a speed of 53ms per picture, with a frame rate of up to 20FPS;

[0058] 2. The present invention has excellent deblurring effect on images, and its PSNR evaluation index is ahead of most existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the present invention and show the differences from the existing inventions, various technical solutions of the present invention will be described below with reference to the accompanying drawings.

[0060] Figure 1 The overall flow chart of the present invention;

[0061] Figure 2 The Gimg module of the present invention;

[0062] Figure 3 The Gevent module of the present invention;

[0063] Figure 4 The Gattention module of the present invention;

[0064] Figure 5 Grestoration module of the present invention;

[0065] Figure 6 The image de-motion blur effect of the present invention is demonstrated;

[0066] Figure 6In the figure, (a) is the input blurred image, and (b) is the output clear image.

[0067] Figure 7 The attention network model size and quantitative evaluation performance of this invention. DETAILED DESCRIPTION

[0068] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0069] like Figure 1 As shown, the present invention provides an image de-motion blurring method based on an event-guided attention network, wherein the attention network includes an image feature extraction module, an event feature extraction module, a cross-modal attention feature fusion module, and an image reconstruction module; first, image coding features and event coding features are extracted from the motion blurred image and the event voxel grid through the image feature extraction module and the event feature extraction module, respectively, and then, the image coding features and the event coding features are fused through the cross-modal attention feature fusion module to obtain fused features, and finally, the fused features are passed through the image reconstruction module to obtain a clear image. The present application realizes the removal of motion blur of an image, and the obtained image has higher clarity, saves image processing time, and has higher image processing efficiency; the attention network of the present invention is a single-stage network, which has the characteristics of being lightweight.

[0070] The steps of the specific technical solution of the present invention are as follows:

[0071] Step 1, build a dataset: Based on the existing GoPro dataset, use the ESIM simulator to generate event streams for motion blurred images; the GoPro dataset contains 3214 motion blurred images with a resolution of 1280×720, including 2103 training images and 1111 test images;

[0072] Step 2, motion blurred image preprocessing: the motion blurred image is cropped into 256×256 image blocks according to the resolution;

[0073] Step 3, event stream preprocessing: integrate the event stream along the time dimension to obtain an event voxel grid, and crop it according to the cropping method of step 2 to obtain an event voxel grid corresponding to the motion blurred image block space;

[0074] Step 4, build an attention network: the attention network consists of four core modules, namely, image feature extraction module (Gimg module), event feature extraction module (Gevent module), multimodal attention feature fusion module (Gattention module), and image reconstruction module (Grestoration module); the motion blurred image preprocessed in step 2 is input into the Gimg module to obtain image coding features, and the event voxel grid preprocessed in step 3 is input into the Gevent module to obtain event coding features. The image coding features and event coding features are passed through the Gattention module to obtain fused features; the fused features are passed through the Grestoration module to reconstruct the image;

[0075] Step 5, training the attention network: construct a loss function, and update the attention network parameters by calculating the value of the loss function until the attention network converges;

[0076] S5.1, determine the loss function PSNRLoss: The loss function PSNRLoss formula is as follows:

[0077]

[0078] Among them, MAX_I is the maximum value of the pixel in the training image block, and MSE is the mean square error between the predicted image and the true image;

[0079] S5.2, training on the training set: two NVIDA 1080Ti GPU graphics cards are used for training; the learning rate of the attention network is set to 0.0002, and the cosine decay strategy is adopted; the batch_size of each NVIDA 1080Ti GPU graphics card is set to 4; the number of iterations is 200,000; the model weights are saved after the attention network converges;

[0080] S5.3, tested on the test set: In the test results, the peak signal-to-noise ratio PSNR index is 34.57, and the structural similarity SSIM index is 0.9660;

[0081] Step 6, perform model inference: first preprocess the motion blurred image according to steps 2 to 3, and then input the preprocessed motion blurred image into the trained attention network model to obtain a clear image without motion blur.

[0082] Furthermore, the formula used in the integration method in step 3 is as follows:

[0083]

[0084] Among them, e(t) represents the event triggered at time t; ei represents a single event;

[0085] Each motion blurred image is obtained by weighting 7 consecutive clear images. The capture time of the 7 consecutive clear images is denoted as t i , the value range of i is [1,7]; the time interval between adjacent clear images is t; according to the above integral formula, through 6 integral operations, an event voxel grid with 6 channels is obtained, and each blurred image corresponds to an event voxel grid.

[0086] Furthermore, the Gimg module in step 4 includes three identical image convolution modules (ImgConvBlock modules) and two downsampling layers; the three identical image convolution modules are image convolution module 1, image convolution module 2 and image convolution module 3, each image convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers and a 1×1 convolution layer for residual connection; the residual connection helps to improve the attention network's ability to express and learn image features, the activation layer uses a RELU activation function, and the downsampling layer consists of a convolution layer with a kernel size of 4×4 and a step size of 2; Figure 2 As shown, the execution steps of the Gimg module are as follows:

[0087] S4.1.1, convolution is performed on the motion blurred image, the number of convolution kernels is 64, and the image feature f1 is generated, and the number of its channels increases to 64;

[0088] S4.1.2, input feature f1 to image convolution module 1 for feature extraction and perform downsampling once, to obtain image encoding feature img_encs[1] and downsampled feature img_down1, and the image size of downsampled feature img_down1 is reduced from 256×256 to 128×128;

[0089] S4.1.3, the downsampled feature img_down1 passes through the image convolution module 2 to generate the image encoding feature img_encs[2] and the downsampled feature img_down2. The image size of the downsampled feature img_down2 is reduced from 128×128 to 64×64.

[0090] S4.1.4, the downsampled feature img_down2 is processed by the image convolution module 3 to obtain the image encoding feature img_encs[3];

[0091] Finally, after the blurred image is extracted by the Gimg module, three multi-level image encoding features img_encs[1], img_encs[2] and img_encs[3] are obtained.

[0092] Furthermore, the Gevent module in step 4 includes three identical event convolution modules (EventConvBlock modules) and two downsampling layers; the three identical event convolution modules are event convolution module 1, event convolution module 2 and event convolution module 3, each event convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers and a 1×1 convolution layer for residual connection; the residual connection helps to improve the network's ability to express and learn event features, and the activation layer uses a RELU activation function; Figure 3 As shown, the execution steps of the Gevent module are as follows:

[0093] S4.2.1, convolving the event voxel grid, wherein the number of convolution kernels is 64, generating event feature e1, and the number of channels thereof is increased to 64;

[0094] S4.2.2, input the event feature e1 into the event convolution module 1 for feature extraction and perform downsampling once, obtain the event encoding feature event_encs[1] and the downsampled feature event_down1, and reduce the image size from 256×256 to 128×128;

[0095] S4.2.3, the downsampled feature event_down1 passes through the event convolution module 2 to generate the event encoding feature event_encs[2] and the downsampled feature event_down2. The image size of the downsampled feature event_down2 is reduced from 128×128 to 64×64.

[0096] S4.2.4, the downsampled feature event_down2 is processed by the event convolution module 3 to obtain the event encoding feature event_encs[3];

[0097] Finally, after the event voxel grid passes through the Gevent module, three multi-level event encoding features event_encs[1], event_encs[2] and event_encs[3] are obtained.

[0098] Furthermore, the Gattention module in step 4 includes three fusion modules (Fused-Moudle modules), the three Fused-Moudle modules are Fused-Moudle module 1, Fused-Moudle module 2 and Fused-Moudle module 3, each Fused-Moudle module includes two custom normalization layers (LayerNorm), the two custom normalization layers are image normalization layer and event normalization layer, the normalization layer is used to normalize the image coding features and event coding features before feature fusion; an attention layer is used for cross-modal feature fusion; a multi-layer perceptron (MLP network) is used for feature mapping and feature dimension processing; as shown in FIG. Figure 4 As shown, the execution steps of the Gattention module are as follows:

[0099] S4.3.1, input the event encoding feature event_encs[1] and the image encoding feature img_encs[1] into the Fused-Moudle module 1, and determine whether the dimensions of the two are consistent. If the dimensions are inconsistent, they cannot be fused. If the dimensions are consistent, continue to execute as follows;

[0100] S4.3.2, the image coding feature img_encs[1] passes through the image normalization layer to obtain the normalized image feature 1; similarly, the event coding feature event_encs[1] passes through the event normalization layer to obtain the normalized event feature 1;

[0101] S4.3.3, input the image feature 1 and event feature 1 obtained in S4.3.2 into the attention layer for feature fusion to obtain the fused feature, input the fused feature into the MLP network for feature mapping and size adjustment, and finally output the fused feature fused[1] with the same dimension as event feature 1 and image feature 1;

[0102] S4.3.4, input the event coding feature event_encs[2] and the image coding feature img_encs[2] into Fused-Moudle module 2 respectively, obtain event feature 2 and image feature 2 after event normalization and image normalization, and then obtain the fused feature fused[2] with the same dimension as event feature 2 and image feature 2 according to S4.3.3;

[0103] S4.3.5, input the event encoding feature event_encs[3] and the image encoding feature img_encs[3] into the Fused-Moudle module 3 respectively, obtain the event feature 3 and the image feature 3 after the event normalization and image normalization, and then obtain the fused feature fused[3] with the same dimension as the event feature 3 and the image feature 3 according to S4.3.3;

[0104] Finally, the event coding features and image coding features at three different levels are fused through the Gattention module to obtain three fused features fused[1], fused[2] and fused[3]. The three fused features fused[1], fused[2] and fused[3] form a fused feature pyramid with three layers. The three fused features fused[1], fused[2] and fused[3] are used as the input of the subsequent image reconstruction module.

[0105] Furthermore, the Grestauration module described in step 4 is used to restore a clear image from the fused features. The Grestauration module includes two upsampling modules (UpBlock) and a convolution activation module (ConvSAM); the two upsampling modules include upsampling module 1 and upsampling module 2, each upsampling module includes a deconvolution layer and an image convolution module, the convolution kernel size of the deconvolution layer is 2×2, the step size stride=2, padding=1, and the convolution activation module includes 3 convolution layers and a RELU activation layer; after the fused features fused[1], fused[2] and fused[3] pass through the Grestauration module, a clear image is output; as shown in Figure 5 As shown, the execution steps of the Grestoration module are as follows:

[0106] S4.4.1, input the fused features fused[2] and fused[3] into upsampling module 1, and obtain the decoded feature decs[0] after deconvolution upsampling and convolution, activation, and residual connection;

[0107] S4.4.2, the decoded feature decs[0] and the fused feature fused[1] are input into upsampling module 2, and after deconvolution upsampling, convolution, activation, and residual connection, the decoded feature decs[1] is obtained;

[0108] S4.4.3, the decoded feature decs[1] and the motion blurred image are input into the convolution activation module, and finally a clear image is output;

[0109] Finally, the fused features fused[1], fused[2] and fused[3] are processed by the Grestoration module to effectively remove motion blur and output a clear image. The comparison between the input blurred image and the output clear image is shown in Figure 6 ; Figure 7 This is a comparison chart of the effects of the present invention, the pure image method and the fusion time method, wherein the orange dots are the effects of the pure image method, the green dots are the effects of the fusion event method, and the red five-pointed star is the effect of the present invention.

[0110] Furthermore, in the image convolution module 1, the two convolution layers are respectively convolution layer 1 and convolution layer 2, the input channel of each convolution layer is 64, the output channel is 64, the convolution kernel size is 3×3, the number of convolution kernels is 64, the step length stride=1, padding=1, and a bias term is included; the two activation layers include activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use a RELU activation function; the input channel of the residual connection layer is 64, the output channel is 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and a bias term is included;

[0111] In the image convolution module 2, the two convolution layers are respectively convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the two activation layers are respectively activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 128, the output channel is 128, the convolution kernel size is 1×1, the number of convolution kernels is 128, the step length stride=1, padding=0, and the bias term is included;

[0112] In the image convolution module 3, the two convolution layers are respectively convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and a bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and a bias term is included; the two activation layers are respectively activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 256, the output channel is 256, the convolution kernel size is 1×1, the number of convolution kernels is 256, the step length stride=1, padding=0, and a bias term is included.

[0113] Furthermore, in the event convolution module 1, the three convolution layers are respectively convolution layer 1, convolution layer 2 and convolution layer 3, the input channels of convolution layer 1 and convolution layer 2 are 64, the output channels are 64, the convolution kernel size is 3×3, the number of convolution kernels is 64, the step length stride=1, padding=1, and the bias term is included; the input channels of convolution layer 3 are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included; the two activation layers are respectively activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channels of the residual connection layer are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included;

[0114] In the event convolution module 2, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the convolution layer 3 has 128 input channels, 128 output channels, 1×1 convolution kernel size, 128 convolution kernel number, stride=1, padding=0, including bias term; the two activation layers are activation layer 1 and activation layer 2, both of which use RELU activation function; the residual connection layer has 128 input channels, 128 output channels, 1×1 convolution kernel size, 128 convolution kernel number, stride=1, padding=0, including bias term;

[0115] In the event convolution module 3, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the convolution layer 3 has 256 input channels, 256 output channels, a convolution kernel size of 1×1, a number of convolution kernels of 256, stride=1, padding=0, and includes a bias term. The two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function. The residual connection layer has 256 input channels, 256 output channels, a convolution kernel size of 1×1, a number of convolution kernels of 256, stride=1, padding=0, and includes a bias term.

[0116] Furthermore, in the Fused-Moudle module 1, the input dimension of the image normalization layer 1 and the event normalization layer 1 is 64, with a bias term; the input dimension of the attention layer is 64, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer includes 64 linear units, the hidden layer includes 128 linear units, and the output layer includes 64 linear units;

[0117] In the Fused-Moudle module 2, the input dimension of the image normalization layer 2 and the event normalization layer 2 is 64, with a bias term; the input dimension of the attention layer is 128, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer includes 128 linear units, the hidden layer includes 256 linear units, and the output layer includes 128 linear units;

[0118] In the Fused-Moudle module 3, the input dimension of the image normalization layer 3 and the event normalization layer 3 is 64, with a bias term; the input dimension of the attention layer is 256, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer contains 256 linear units, the hidden layer contains 512 linear units, and the output layer contains 512 linear units.

[0119] The orange dots are pure image methods, the green dots are methods that integrate events, and the red five-pointed stars are the methods of the present invention.

Claims

1. A method for image motion blurring based on event-guided attention network, characterized in that: The steps of the image motion blur removal method are as follows: Step 1, build a dataset: Based on the existing GoPro dataset, use the ESIM simulator to generate event streams for motion blurred images; the GoPro dataset contains 3214 motion blurred images with a resolution of 1280×720, including 2103 training images and 1111 test images; Step 2, motion blurred image preprocessing: the motion blurred image is cropped into 256×256 image blocks according to the resolution; Step 3, event stream preprocessing: integrate the event stream along the time dimension to obtain an event voxel grid, and crop it according to the cropping method of step 2 to obtain an event voxel grid corresponding to the motion blurred image block space; Step 4, build an attention network: The attention network consists of four core modules, namely, image feature extraction module, event feature extraction module, multimodal attention feature fusion module, and image reconstruction module; the motion blurred image preprocessed in step 2 is input into the Gimg module to obtain image coding features, and the event voxel grid preprocessed in step 3 is input into the Gevent module to obtain event coding features. The image coding features and event coding features are fused through the Gattention module; The fused features are used for image reconstruction through the Grestoration module; Step 5, training the attention network: construct a loss function, and update the attention network parameters by calculating the value of the loss function until the attention network converges; S5.1, determine the loss function PSNRLoss: The loss function PSNRLoss formula is as follows: Among them, MAX_I is the maximum value of the pixel in the training image block, and MSE is the mean square error between the predicted image and the true image; S5.2, training on the training set: two NVIDA 1080TiGPU graphics cards are used for training; the learning rate of the attention network is set to 0.0002, and the cosine decay strategy is adopted; the batch_size of each NVIDA 1080TiGPU graphics card is set to 4; the number of iterations is 200,000; the model weights are saved after the attention network converges; S5.3, tested on the test set: In the test results, the peak signal-to-noise ratio PSNR index is 34.57, and the structural similarity SSIM index is 0.9660; Step 6, perform model inference: first preprocess the motion blurred image according to steps 2 to 3, and then input the preprocessed motion blurred image into the trained attention network model to obtain a clear image without motion blur.

2. According to claim 1, the image de-motion blurring method based on event-guided attention network is characterized in that: The formula used in the integration method described in step 3 is as follows: Among them, e(t) represents the event triggered at time t; ei represents a single event; Each motion blurred image is obtained by weighting 7 consecutive clear images. The capture time of the 7 consecutive clear images is denoted as t i , the value range of i is [1,7]; the time interval between adjacent clear images is t; according to the above integral formula, through 6 integral operations, an event voxel grid with 6 channels is obtained, and each blurred image corresponds to an event voxel grid.

3. The image de-motion blurring method based on event-guided attention network according to claim 1, characterized in that: The Gimg module in step 4 includes three identical image convolution modules (ImgConvBlock modules) and two downsampling layers; the three identical image convolution modules are image convolution module 1, image convolution module 2 and image convolution module 3, each image convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers and a 1×1 convolution layer for residual connection, the activation layer uses a RELU activation function, and the downsampling layer consists of a convolution layer with a kernel size of 4×4 and a step size of 2; the execution steps of the Gimg module are as follows: S4.1.1, convolve the motion blurred image to generate image feature f1, and the number of its channels increases to 64; S4.1.2, input feature f1 to image convolution module 1 for feature extraction and perform downsampling once, to obtain image encoding feature img_encs[1] and downsampled feature img_down1, and the image size of downsampled feature img_down1 is reduced from 256×256 to 128×128; S4.1.3, the downsampled feature img_down1 passes through the image convolution module 2 to generate the image encoding feature img_encs[2] and the downsampled feature img_down2. The image size of the downsampled feature img_down2 is reduced from 128×128 to 64×64. S4.1.4, the downsampled feature img_down2 is processed by the image convolution module 3 to obtain the image encoding feature img_encs[3]; Finally, after the blurred image is extracted by the Gimg module, three multi-level image encoding features img_encs[1], img_encs[2] and img_encs[3] are obtained.

4. The image de-motion blurring method based on event-guided attention network according to claim 1, characterized in that: The Gevent module in step 4 includes three identical event convolution modules (EventConvBlock modules) and two downsampling layers; the three identical event convolution modules are event convolution module 1, event convolution module 2 and event convolution module 3, each event convolution module consists of two convolution layers, each convolution layer has a convolution kernel size of 3×3, two activation layers and a 1×1 convolution layer for residual connection; the activation layer uses a RELU activation function; the execution steps of the Gevent module are as follows: S4.2.1, convolve the event voxel grid to generate event feature e1, and its channel number is increased to 64; S4.2.2, input the event feature e1 into the event convolution module 1 for feature extraction and perform downsampling once, obtain the event encoding feature event_encs[1] and the downsampled feature event_down1, and reduce the image size from 256×256 to 128×128; S4.2.3, the downsampled feature event_down1 passes through the event convolution module 2 to generate the event encoding feature event_encs[2] and the downsampled feature event_down2. The image size of the downsampled feature event_down2 is reduced from 128×128 to 64×64. S4.2.4, the downsampled feature event_down2 is processed by the event convolution module 3 to obtain the event encoding feature event_encs[3]; Finally, after the event voxel grid passes through the Gevent module, three multi-level event encoding features event_encs[1], event_encs[2] and event_encs[3] are obtained.

5. The image de-motion blurring method based on event-guided attention network according to claim 1, characterized in that: The Gattention module described in step 4 includes three fusion modules (Fused-Moudle modules), the three Fused-Moudle modules are Fused-Moudle module 1, Fused-Moudle module 2 and Fused-Moudle module 3, each Fused-Moudle module includes two custom normalization layers (LayerNorm), the two custom normalization layers are image normalization layer and event normalization layer, and the normalization layer is used to normalize the image coding features and event coding features before feature fusion; An attention layer for cross-modal feature fusion; a multi-layer perceptron (MLP network) for feature mapping and feature dimension processing; the execution steps of the Gattention module are as follows: S4.3.1, input the event encoding feature event_encs[1] and the image encoding feature img_encs[1] into the Fused-Moudle module 1, and determine whether the dimensions of the two are consistent. If the dimensions are inconsistent, they cannot be fused. If the dimensions are consistent, continue to execute as follows; S4.3.2, the image coding feature img_encs[1] passes through the image normalization layer to obtain the normalized image feature 1; similarly, the event coding feature event_encs[1] passes through the event normalization layer to obtain the normalized event feature 1; S4.3.3, input the image feature 1 and event feature 1 obtained in S4.3.2 into the attention layer for feature fusion to obtain the fused feature, input the fused feature into the MLP network for feature mapping and size adjustment, and finally output the fused feature fused[1] with the same dimension as event feature 1 and image feature 1; S4.3.4, input the event coding feature event_encs[2] and the image coding feature img_encs[2] into Fused-Moudle module 2 respectively, obtain event feature 2 and image feature 2 after event normalization and image normalization, and then obtain the fused feature fused[2] with the same dimension as event feature 2 and image feature 2 according to S4.3.3; S4.3.5, input the event encoding feature event_encs[3] and the image encoding feature img_encs[3] into the Fused-Moudle module 3 respectively, obtain the event feature 3 and the image feature 3 after the event normalization and image normalization, and then obtain the fused feature fused[3] with the same dimension as the event feature 3 and the image feature 3 according to S4.3.3; Finally, the event coding features and image coding features at three different levels are fused through the Gattention module to obtain three fused features fused[1], fused[2] and fused[3]. The three fused features fused[1], fused[2] and fused[3] form a fused feature pyramid with three layers. The three fused features fused[1], fused[2] and fused[3] are used as the input of the subsequent image reconstruction module.

6. The image de-motion blurring method based on event-guided attention network according to claim 1, characterized in that: The Grerestoration module described in step 4 is used to restore a clear image from the fused features. The Grerestoration module includes two upsampling modules (UpBlock) and a convolution activation module (ConvSAM); the two upsampling modules include upsampling module 1 and upsampling module 2, each upsampling module includes a deconvolution layer and an image convolution module, the convolution kernel size of the deconvolution layer is 2×2, the step size stride=2, padding=1, and the convolution activation module includes 3 convolution layers and a RELU activation layer; after the fused features fused[1], fused[2] and fused[3] pass through the Grerestoration module, a clear image is output; the execution steps of the Grerestoration module are as follows: S4.4.1, input the fused features fused[2] and fused[3] into upsampling module 1, and after deconvolution upsampling and convolution, activation, and residual connection, the decoded feature decs[0] is obtained; S4.4.2, the decoded feature decs[0] and the fused feature fused[1] are input into upsampling module 2, and after deconvolution upsampling, convolution, activation, and residual connection, the decoded feature decs[1] is obtained; S4.4.3, the decoded feature decs[1] and the motion blurred image are input into the convolution activation module, and finally a clear image is output; Finally, the fused features fused[1], fused[2] and fused[3] are processed by the Grestoration module, which can effectively remove motion blur and output a clear image.

7. The image de-motion blurring method based on event-guided attention network according to claim 3, characterized in that: In the image convolution module 1, the two convolution layers are respectively convolution layer 1 and convolution layer 2, each convolution layer has 64 input channels, 64 output channels, a convolution kernel size of 3×3, a number of convolution kernels of 64, a step length of stride=1, padding=1, and includes a bias term; the two activation layers include activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use a RELU activation function; the residual connection layer has 64 input channels, 64 output channels, a convolution kernel size of 1×1, a number of convolution kernels of 64, a step length of stride=1, padding=0, and includes a bias term; In the image convolution module 2, the two convolution layers are respectively convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the two activation layers are respectively activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 128, the output channel is 128, the convolution kernel size is 1×1, the number of convolution kernels is 128, the step length stride=1, padding=0, and the bias term is included; In the image convolution module 3, the two convolution layers are respectively convolution layer 1 and convolution layer 2, the input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and a bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and a bias term is included; the two activation layers are respectively activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channel of the residual connection layer is 256, the output channel is 256, the convolution kernel size is 1×1, the number of convolution kernels is 256, the step length stride=1, padding=0, and a bias term is included.

8. The image de-motion blurring method based on event-guided attention network according to claim 4, characterized in that: In the event convolution module 1, the three convolution layers are respectively convolution layer 1, convolution layer 2 and convolution layer 3, the input channels of convolution layer 1 and convolution layer 2 are 64, the output channels are 64, the convolution kernel size is 3×3, the number of convolution kernels is 64, the step length stride=1, padding=1, and the bias term is included; the input channels of convolution layer 3 are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included; the two activation layers are respectively activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function; the input channels of the residual connection layer are 64, the output channels are 64, the convolution kernel size is 1×1, the number of convolution kernels is 64, the step length stride=1, padding=0, and the bias term is included; In the event convolution module 2, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 64, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 128, the output channel is 128, the convolution kernel size is 3×3, the number of convolution kernels is 128, the step length stride=1, padding=1, and the bias term is included; the convolution layer 3 has 128 input channels, 128 output channels, 1×1 convolution kernel size, 128 convolution kernel number, stride=1, padding=0, including bias term; the two activation layers are activation layer 1 and activation layer 2, both of which use RELU activation function; the residual connection layer has 128 input channels, 128 output channels, 1×1 convolution kernel size, 128 convolution kernel number, stride=1, padding=0, including bias term; In the event convolution module 3, the three convolution layers are convolution layer 1, convolution layer 2 and convolution layer 3. The input channel of convolution layer 1 is 128, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the input channel of convolution layer 2 is 256, the output channel is 256, the convolution kernel size is 3×3, the number of convolution kernels is 256, the step length stride=1, padding=1, and the bias term is included; the convolution layer 3 has 256 input channels, 256 output channels, a convolution kernel size of 1×1, a number of convolution kernels of 256, stride=1, padding=0, and includes a bias term. The two activation layers are activation layer 1 and activation layer 2, and both activation layer 1 and activation layer 2 use the RELU activation function. The residual connection layer has 256 input channels, 256 output channels, a convolution kernel size of 1×1, a number of convolution kernels of 256, stride=1, padding=0, and includes a bias term.

9. The image de-motion blurring method based on event-guided attention network according to claim 5, characterized in that: In the Fused-Moudle module 1, the input dimension of the image normalization layer 1 and the event normalization layer 1 is 64, with a bias term; the input dimension of the attention layer is 64, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer includes 64 linear units, the hidden layer includes 128 linear units, and the output layer includes 64 linear units; In the Fused-Moudle module 2, the input dimension of the image normalization layer 2 and the event normalization layer 2 is 64, with a bias term; the input dimension of the attention layer is 128, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer includes 128 linear units, the hidden layer includes 256 linear units, and the output layer includes 128 linear units; In the Fused-Moudle module 3, the input dimension of the image normalization layer 3 and the event normalization layer 3 is 64, with a bias term; the input dimension of the attention layer is 256, without a bias term; the MLP network includes an input layer, a hidden layer and an output layer, the input layer contains 256 linear units, the hidden layer contains 512 linear units, and the output layer contains 512 linear units.

10. An attention network for an image de-motion blurring method, characterized in that: The attention network is applicable to the image demotion blur method described in claims 1-9.

Images