An event data driven based image deblurring method

Abstract

This invention discloses an event-data-driven image deblurring method, comprising the following steps: 1. acquiring blurred image data and its corresponding event data, as well as a set of clear images; 2. constructing an event-driven motion-adaptive image deblurring network and processing the blurred images to obtain predicted clear images; 3. constructing a loss function based on the clear images and the predicted clear images to train the deblurring network, thereby utilizing the trained optimal deblurring model to achieve the deblurring target of the blurred images. This invention, by utilizing an event-data-driven approach, can improve the deblurring effect of blurred images, thereby generating clear images.

Description

Technical Field

[0001] This invention relates to the field of image deblurring, specifically to an event data-driven image deblurring method. Background Technology

[0002] Motion blur, typically caused by the relative motion between the camera and the scene during exposure, is a common degradation phenomenon in frame-based camera imaging. Image deblurring, as a typical ill-posed inverse problem, aims to recover a sharp image from a blurred observation. Although deep learning-based approaches have significantly improved the performance of traditional deblurring, they are still limited by the inherent limitation of traditional cameras in capturing dynamic details during exposure. In contrast, event cameras asynchronously record pixel intensity changes with microsecond-level precision. Their high temporal resolution provides crucial motion cues for deblurring and has recently demonstrated the potential to enhance traditional methods.

[0003] Existing event-driven deblurring methods have the following key limitations: (1) They do not effectively explore the spatial correlation between events and blurred areas; (2) They lack motion perception capabilities. Existing CNN architectures are difficult to model global motion patterns and have limited ability to locate the motion regions hidden in events. Traditional attention mechanisms are easily interfered with by irrelevant tokens in high-density visual information, which reduces the quality of feature aggregation; (3) Existing schemes simply stitch together or linearly weighted fuse event and image features, ignoring the dynamic correlation between event intensity changes and image spatial consistency, resulting in insufficient detail recovery.

[0004] The aforementioned issues limit the effectiveness of event-driven deblurring methods in complex dynamic scenarios. There is an urgent need to design a novel attention mechanism that combines motion perception and computational efficiency in order to accurately utilize event priors to guide cross-modal feature fusion. Summary of the Invention

[0005] To overcome the shortcomings of existing methods, this invention provides an event data-driven image deblurring method, which aims to improve deblurring performance in complex dynamic scenes through accurate motion region localization and efficient cross-modal interaction.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] The present invention provides an event data-driven image deblurring method, characterized by the following steps:

[0008] Step 1: Obtain the blurred image set, denoted as... ,in, Indicates the first A blurry image, , The number of blurred images;

[0009] Obtain a blurred image set The corresponding event sequence is denoted as , Indicates the first Zhang blurry image The corresponding event sequence;

[0010] Obtain a clear image set, denoted as ,in, Indicates the first Zhang Qingquan image;

[0011] Step 2: Construct an event-driven motion-adaptive Transformer deblurring network, including: a feature extraction layer, Layer events - image encoder, image reconstruction decoder;

[0012] Step 2.1: The feature extraction layer consists of an image feature extraction layer and an event feature extraction layer, and performs feature extraction on the first and second parts of the image. Zhang blurry image and the A sequence of events Feature extraction is performed, and the corresponding result is obtained. Initial blurred image features and the Initial event characteristics ;

[0013] Step 2.2, the first The layer event-image encoder consists of an adaptive motion mask predictor, a motion sparse event block, and a motion-aware image block;

[0014] when When, the matrix consisting entirely of 1s is used as the first... Layer An initial motion mask and The input is processed in an adaptive motion mask predictor to obtain the first... Layer Predicted motion mask ;

[0015] The first Layer Predicted motion mask and The input of the motion sparse event block is processed to generate the first... Layer Event characteristics ;

[0016] The first Layer Predicted motion mask , and The motion-sensing image block is input for processing to generate the first... Layer A blurred image feature ;

[0017] when At that time, with the first Layer Predicted motion mask As the first Layer An initial motion mask and the Layer event - the first layer output of the image encoder Event characteristics The input is processed in an adaptive motion mask predictor to obtain the first... Layer Predicted motion mask ;

[0018] The first Layer Predicted motion mask With the Layer event - the first layer output of the image encoder Event characteristics The input of the motion sparse event block is processed to generate the first... Layer Event characteristics ;

[0019] The first Layer Predicted motion mask , No. Layer Event characteristics With the Layer A blurred image feature The motion-sensing image block is input for processing to generate the first... Layer A blurred image feature ; and thus by the first Layer Events - Image Encoder Output Layer A blurred image feature ;

[0020] Step 2.3, the image reconstruction decoder... After processing, the first result is obtained. A clear image of the prediction Thus, a clear set of predicted images is obtained. ;

[0021] Step 3: Construct the backpropagation loss function using equation (4). :

[0022] (4)

[0023] Step 4, based on , and The motion-adaptive Transformer deblurring network was trained, and the loss function was calculated. Simultaneously, an adaptive moment estimation optimization method is used with a learning rate. The network weights are updated. Training stops when the number of training iterations reaches a set number or the loss error is less than a set threshold, thus obtaining the optimal deblurring model, which is used to process blurred images to obtain corresponding clear images.

[0024] The image deblurring method based on event data driven by the present invention is characterized in that the adaptive motion mask predictor includes: a response calculation module, a response fusion unit, an adaptive adjustment unit, and a mask generation unit;

[0025] when At that time, the response calculation module... After linear projection, the input is processed through the GELU activation function to generate the first... Layer Local response At the same time, based on right Perform a normalized weighted average to generate the first... Layer A global response ;

[0026] The response fusion unit Expand the dimensions to make it compatible with After spatial alignment, the expanded global response is then combined with... Perform splicing to generate the first Layer The first fusion feature At the same time, the expanded global response and Perform element-wise multiplication to generate the first... Layer The second fusion feature ;

[0027] The adaptive adjustment unit according to Calculate the first Average sparsity Then set the first Layer An expansion factor to be learned Therefore, the first equation can be used to calculate the second equation. Layer High number of response tokens :

[0028] (1)

[0029] In equation (1), To train the stability coefficient, For the first Layer Total number of tokens in each space;

[0030] The mask generation unit utilizes a multilayer perceptron to... Processing is performed to obtain spatial projection features, and simultaneously... After performing max pooling and average pooling operations on the channel dimensions, the data is then input into a linear layer and processed using the GELU activation function to obtain channel projection features. These spatial projection features and channel projection features are then concatenated along the channels and input into another linear layer and GELU activation function for further processing to obtain the final channel projection feature. Layer Spatial motion score Finally, according to Before selection Generate a predictive motion mask at each high-response location. .

[0031] Furthermore, the motion sparse event block includes: a motion sparse attention unit and an extended control space gating unit;

[0032] when At that time, the motion sparse attention unit for After layer normalization, the data is processed sequentially through convolutional layers and depthwise separable convolutional layers to obtain event local projection features. Channel segmentation is then performed on these event local projection features to generate the first... Layer Event query matrix , No. Event key matrix , No. Event value matrix Therefore, the first equation is used to calculate the second equation. Layer Sparse Transposed Attention Graph of Event Motion ,Will and After multiplication, the first product is generated. Layer Updated event characteristics :

[0033] (2)

[0034] In equation (2), Let be the scaling factor to be learned, and Softmax represent the activation function. This indicates element-wise multiplication. Indicates transpose;

[0035] The extended control space gating unit will The input is processed in the GELU activation function to obtain the spatial modulation score, and the Sigmoid function is used to... Converted to a hybrid transmission rate, thereby combining spatial modulation fraction, hybrid transmission rate, and... After performing element-wise multiplication, then with... Add them together to get the first one. Layer Event characteristics ;

[0036] Furthermore, the motion-aware image block comprises a motion-aware attention unit and a cross-modal intensity gating unit;

[0037] when At that time, the motion-sensing attention unit... Layer normalization is performed, followed by processing through convolutional layers and depthwise separable convolutional layers to obtain local image projection features. Channel segmentation is then applied to these local image projection features to generate the next layer. Layer Image query matrix , No. Image key matrix , No. Image value matrix Therefore, the first equation (3) is used to calculate the second equation. Layer Image motion sparse transpose attention map , and After multiplication, the first product is generated. Layer Updated image features :

[0038] (3)

[0039] In equation (3), The motion factors to be learned;

[0040] The transmodal strength gating unit will The input layer is processed by the normalization layer to obtain normalized image features, and at the same time... The input layer normalization layer is processed with the GELU activation function to obtain event modulation features. The normalized image features and event modulation features are then multiplied element-wise, and then... Add them together to get the first one. Layer Image features after cross-modal fusion .

[0041] The present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the image deblurring method, and the processor is configured to execute the program stored in the memory.

[0042] The present invention discloses a computer-readable storage medium on which a computer program is stored, wherein the computer program is executed by a processor to perform the steps of the image deblurring method.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] 1. This invention utilizes an event-driven video deblurring task. Through motion-adaptive design, it enables the Transformer to achieve excellent end-to-end deblurring results. Compared to existing deblurring methods, it reduces the number of parameters and exhibits better robustness across different datasets. Experimental results show that the proposed method outperforms state-of-the-art methods on the GoPro, REVD, HS-ERGB, and REBlur datasets.

[0045] 2. This invention extracts event and image features separately by designing two specific attention mechanisms. First, leveraging the rich, high-temporal-resolution motion information in events, this invention acquires spatial motion information by designing an adaptive motion mask predictor. To address the sparse nature of event space, motion-sparse attention uses motion masks to shield the influence of irrelevant regions in the event; to address the dense nature of image space, motion-aware attention uses motion masks to highlight the influence of motion-related regions in the image, thus solving the problem of insufficient motion region perception capability of attention mechanisms.

[0046] 3. This invention designs two specific gating mechanisms. Extended control space gating enables gradient differentiability of the adaptive motion mask predictor, while also modulating the local dilation caused by convolutional operations in the network. Cross-modal intensity gating achieves efficient fusion of event features and image features. Through modeling using these gating mechanisms, the global information of the input features is deeply mined, thereby improving image deblurring performance and increasing the interpretability of the model.

[0047] 4. The motion-adaptive Transformer deblurring network designed in this invention is constructed based on motion-sparse event blocks and motion-aware image blocks. The motion-sparse event blocks consist of motion-sparse attention and extended control space gating; the motion-aware image blocks consist of motion-aware attention and cross-modal intensity gating. The network is trained end-to-end. This approach breaks through the limited receptive field of traditional CNN networks, and the Transformer constructed specifically for images and events makes fuller use of important motion information in events and images, thereby achieving more accurate motion blur modeling and better deblurring results. Attached Figure Description

[0048] Figure 1 A flowchart of the inventive method;

[0049] Figure 2 This is a structural diagram of the motion adaptive Transformer deblurring network method of the present invention;

[0050] Figure 3 This is a structural diagram of the adaptive mask predictor in this invention;

[0051] Figure 4 This is a structural diagram of the motion sparse event block in this invention;

[0052] Figure 5 This is a structural diagram of the motion-sensing image block in this invention. Detailed Implementation

[0053] In this embodiment, an event data-driven video deblurring method is described, the specific process of which can be found in [link to documentation]. Figure 1 This method comprehensively considers the characteristics of event sparsity and image density, and implements two specific feature extraction methods by designing a motion-adaptive attention mechanism. Then, it fuses the two features through a gating mechanism to achieve a deblurring effect. See the algorithm structure diagram for details. Figure 2 Specifically, the method is carried out in the following steps:

[0054] Step 1: Obtain the blurred image set, denoted as... ,in, Indicates the first A blurry image, , The number of blurred images;

[0055] Obtain a blurred image set The corresponding event sequence is denoted as , Indicates the first Zhang blurry image The corresponding event sequence;

[0056] Obtain a clear image set, denoted as ,in, Indicates the first Zhang Qingquan image;

[0057] In this embodiment, the model is trained and evaluated using the GoPro dataset, REVD dataset, HS-ERGB dataset, and REBlur dataset, respectively. The GoPro dataset and HS-ERGB dataset are synthetic datasets, while the REVD dataset and REBlur dataset are real-world datasets.

[0058] Step 2, in this embodiment, as follows: Figure 2 As shown, an event-driven motion adaptive Transformer deblurring network is constructed, including: a feature extraction layer, Layer events - image encoder, image reconstruction decoder; in this embodiment, ;

[0059] Step 2.1: The feature extraction layer consists of an image feature extraction layer and an event feature extraction layer, and performs feature extraction on the first and second parts of the image. Zhang blurry image and the A sequence of events Feature extraction is performed, and the corresponding result is obtained. Initial blurred image features and the Initial event characteristics .

[0060] Step 2.2, the first The layer event-image encoder consists of an adaptive motion mask predictor, a motion sparse event block, and a motion-aware image block;

[0061] when When, the matrix consisting entirely of 1s is used as the first... Layer An initial motion mask and The input is processed in an adaptive motion mask predictor to obtain the first... Layer Predicted motion mask ;

[0062] No. Layer Predicted motion mask and The input sparse event block is processed to generate the first... Layer Event characteristics ;

[0063] No. Layer Predicted motion mask , and The input motion-aware image patch is processed to generate the first... Layer A blurred image feature ;

[0064] when At that time, with the first Layer Predicted motion mask As the first Layer An initial motion mask and the Layer event - the first layer output of the image encoder Event characteristics The input is processed in an adaptive motion mask predictor to obtain the first... Layer Predicted motion mask ;

[0065] No. Layer Predicted motion mask With the Layer event - the first layer output of the image encoder Event characteristics The input sparse event block is processed to generate the first... Layer Event characteristics ;

[0066] No. Layer Predicted motion mask , No. Layer Event characteristics With the Layer A blurred image feature The input motion-aware image patch is processed to generate the first... Layer A blurred image feature ; and thus by the first Layer Events - Image Encoder Output Layer A blurred image feature .

[0067] Step 2.2.1: The adaptive motion mask predictor includes: a response calculation module, a response fusion unit, an adaptive adjustment unit, and a mask generation unit; the specific structure of the adaptive motion mask predictor is as follows... Figure 3 As shown;

[0068] when At that time, the response calculation module... After linear projection, the input is processed through the GELU activation function to generate the first... Layer Local response At the same time, based on right Perform a normalized weighted average to generate the first... Layer A global response ;

[0069] Response fusion unit Expand the dimensions to make it compatible with After spatial alignment, the expanded global response is then combined with... Perform splicing to generate the first Layer The first fusion feature At the same time, the expanded global response and Perform element-wise multiplication to generate the first... Layer The second fusion feature ;

[0070] The adaptive adjustment unit is based on Calculate the first Average sparsity Then set the first Layer An expansion factor to be learned Therefore, the first equation can be used to calculate the second equation. Layer High number of response tokens :

[0071] (1)

[0072] In equation (1), To train the stability coefficient, For the first Layer Total number of tokens in each space; in this embodiment... ;

[0073] The mask generation unit uses a multilayer perceptron to... Processing is performed to obtain spatial projection features, and simultaneously... After performing max pooling and average pooling operations on the channel dimensions, the data is then input into a linear layer and processed using the GELU activation function to obtain channel projection features. These spatial projection features and channel projection features are then concatenated along the channels and input into another linear layer and GELU activation function for further processing to obtain the final channel projection feature. Layer Spatial motion score Finally, according to Before selection Generate a predictive motion mask at each high-response location. .

[0074] Step 2.2.2: The motion sparse event block includes: a motion sparse attention unit and an extended control space gating unit; the specific structure of the motion sparse event block is as follows... Figure 4 As shown;

[0075] when At that time, the sparse attention unit of motion After layer normalization, the data is processed sequentially through convolutional layers and depthwise separable convolutional layers to obtain event local projection features. Channel segmentation is then performed on these event local projection features to generate the first... Layer Event query matrix , No. Event key matrix , No. Event value matrix Therefore, the first equation is used to calculate the second equation. Layer Sparse Transposed Attention Graph of Event Motion , and After multiplication, the first product is generated. Layer Updated event characteristics :

[0076] (2)

[0077] In equation (2), Let be the scaling factor to be learned, and Softmax represent the activation function. This indicates element-wise multiplication. Indicates transpose;

[0078] Extended control space gating unit will The input is processed in the GELU activation function to obtain the spatial modulation score, and the Sigmoid function is used to... Converted to a hybrid transmission rate, thereby combining spatial modulation fraction, hybrid transmission rate, and... After performing element-wise multiplication, then with... Add them together to get the first one. Layer Event characteristics .

[0079] Step 2.2.3: The motion-aware image block consists of a motion-aware attention unit and a cross-modal intensity gating unit; the specific structure of the motion-aware image block is as follows: Figure 5 As shown;

[0080] when At that time, the motion-sensing attention unit... Layer normalization is performed, followed by processing through convolutional layers and depthwise separable convolutional layers to obtain local image projection features. Channel segmentation is then applied to these local image projection features to generate the next layer. Layer Image query matrix , No. Image key matrix , No. Image value matrix Therefore, the first equation (3) is used to calculate the second equation. Layer Image motion sparse transpose attention map , and After multiplication, the first product is generated. Layer Updated image features :

[0081] (3)

[0082] In equation (2), The scaling factor to be learned. The learning process involves learning to dynamically adjust the contribution intensity of the motor region to the attention weight, using the motor factors to be learned.

[0083] The cross-modal strength gating unit will The input layer is processed by the normalization layer to obtain normalized image features, and at the same time... The input layer normalization layer is processed with the GELU activation function to obtain event modulation features. The normalized image features and the event modulation features are then multiplied element-wise, and then... Add them together to obtain the cross-modal fused image features. ;

[0084] Step 2.3, Image reconstruction decoder After processing, the first result is obtained. A clear image of the prediction Thus, a clear set of predicted images is obtained. .

[0085] Step 3: Construct the backpropagation loss function using equation (4). :

[0086] (4)

[0087] Step 4, based on , and The motion-adaptive Transformer deblurring network was trained, and the loss function was calculated. Simultaneously, an adaptive moment estimation optimization method is used with a learning rate. To update the network weights, in this example, the learning rate. The training is stopped when the number of training iterations reaches the set number or the loss error is less than the set threshold, thus obtaining the optimal deblurring model, which is used to process the blurred image to obtain the corresponding clear image.

[0088] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0089] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

[0090] Example

[0091] To verify the effectiveness of the method of the present invention, the commonly used GoPro dataset, HS-ERGB dataset, REBlur dataset and REVD dataset were selected for training and testing in this embodiment.

[0092] This method is trained on the training set of the GoPro dataset, fine-tuned on the training sets of three other datasets, and evaluated on the test sets of each dataset.

[0093] In this invention, structural similarity (PSNR) and peak signal-to-noise ratio (SSIM) are used as evaluation metrics.

[0094] In this embodiment, six methods were selected from the GoPro dataset, HS-ERGB dataset, and REBlur dataset to compare their performance with the method of this invention. The selected methods are UFPNet, EFNet, EIFNet, MAENet, Restormer, FFTFormer, and MAT, which are the methods of this invention. Six methods were selected from the REVD dataset to compare their performance with the method of this invention. The selected methods are REDNet, UEVD, FEVD, EFNet, EIFNet, MAENet, and MAT, which are the methods of this invention.

[0095] In Tables 1, 2, 3, and 4, EFNet refers to an event-driven image deblurring method based on cross-modal fusion, EIFNet refers to an event-driven image deblurring method based on modality awareness, and MAENet refers to an event-driven image deblurring method based on motion awareness.

[0096] In Tables 1, 3, and 4, UFPNet refers to the image deblurring method based on self-supervised kernel estimation, Restormer refers to the image deblurring method based on Transformer, and FFTFormer refers to the image deblurring method based on frequency domain-aware Transformer.

[0097] Table 2 shows that REDNet is an event-driven image deblurring method in a real-world environment, UEVD is an event-driven deblurring method for unknown image exposure duration, and FEVD is an event-driven deblurring method based on a frequency domain sensing network.

[0098] The experimental results are shown in Tables 1, 2, 3, and 4:

[0099] Table 1. Experimental results of the proposed method and six selected comparative methods for deblurring on the GoPro dataset.

[0100]

[0101] Table 2. Experimental results of the method of this invention and six selected comparison methods on the REVD dataset for deblurring.

[0102]

[0103] Table 3. Experimental results of deblurring using the method of this invention and six selected comparative methods on the HS-ERGB dataset.

[0104]

[0105] Table 4. Experimental results of the proposed method and six selected comparative methods for deblurring on the REBlur dataset.

[0106] Experimental results show that the proposed method outperforms other methods on four different datasets, thus proving its feasibility. The experiments demonstrate that the proposed method can achieve more efficient attention computation based on the sparsity of events and the density of images, and the gating mechanism enables efficient feature transfer, thereby achieving superior performance in the task of deblurring blurred images.

Claims

1. An event-data-driven image deblurring method, characterized in that, The procedure is as follows: Step 1: Obtain the blurred image set, denoted as... ,in, Indicates the first A blurry image, , The number of blurred images; Obtain a blurred image set The corresponding event sequence is denoted as , Indicates the first Zhang blurry image The corresponding event sequence; Obtain a clear image set, denoted as ,in, Indicates the first Zhang Qingquan image; Step 2: Construct an event-driven motion-adaptive Transformer deblurring network, including: a feature extraction layer, Layer events - image encoder, image reconstruction decoder; Step 2.1: The feature extraction layer consists of an image feature extraction layer and an event feature extraction layer, and performs feature extraction on the first and second parts of the image. Zhang blurry image and the A sequence of events Feature extraction is performed, and the corresponding result is obtained. Initial blurred image features and the Initial event characteristics ; Step 2.2, the The layer event-image encoder consists of an adaptive motion mask predictor, a motion sparse event block, and a motion-aware image block; when When, the matrix consisting entirely of 1s is used as the first... Layer An initial motion mask and The input is processed in an adaptive motion mask predictor to obtain the first... Layer Predicted motion mask ; The first Layer Predicted motion mask and The input of the motion sparse event block is processed to generate the first... Layer Event characteristics ; The first Layer Predicted motion mask , and The motion-sensing image block is input for processing to generate the first... Layer A blurred image feature ; when At that time, with the first Layer Predicted motion mask As the first Layer An initial motion mask and the Layer event - the first layer output of the image encoder Event characteristics The input is processed in an adaptive motion mask predictor to obtain the first... Layer Predicted motion mask ; The first Layer Predicted motion mask With the Layer event - the first layer output of the image encoder Event characteristics The input of the motion sparse event block is processed to generate the first... Layer Event characteristics ; The first Layer Predicted motion mask , No. Layer Event characteristics With the Layer A blurred image feature The motion-sensing image block is input for processing to generate the first... Layer A blurred image feature ; and thus by the first Layer Events - Image Encoder Output Layer A blurred image feature ; Step 2.3, the image reconstruction decoder... After processing, the first result is obtained. A clear image of the prediction Thus, a clear set of predicted images is obtained. ; Step 3: Construct the backpropagation loss function using equation (4). : (4) Step 4, based on , and The motion adaptive Transformer deblurring network is trained, and the loss function is calculated. Simultaneously, an adaptive moment estimation optimization method is used with a learning rate. The network weights are updated. Training stops when the number of training iterations reaches a set number or the loss error is less than a set threshold, thus obtaining the optimal deblurring model, which is used to process blurred images to obtain corresponding clear images.

2. The event-data-driven image deblurring method according to claim 1, characterized in that, The adaptive motion mask predictor includes: a response calculation module, a response fusion unit, an adaptive adjustment unit, and a mask generation unit; when At that time, the response calculation module... After linear projection, the input is processed through the GELU activation function to generate the first... Layer Local response At the same time, based on right Perform a normalized weighted average to generate the first... Layer A global response ; The response fusion unit Expand the dimensions to make it compatible with After spatial alignment, the expanded global response is then combined with... Perform splicing to generate the first Layer The first fusion feature At the same time, the expanded global response and Perform element-wise multiplication to generate the first... Layer The second fusion feature ; The adaptive adjustment unit according to Calculate the first Average sparsity Then set the first Layer An expansion factor to be learned Therefore, the first equation can be used to calculate the second equation. Layer High number of response tokens : (1) In equation (1), To train the stability coefficient, For the first Layer Total number of tokens in each space; The mask generation unit utilizes a multilayer perceptron to... Processing is performed to obtain spatial projection features, and simultaneously... After performing max pooling and average pooling operations on the channel dimensions, the data is then input into a linear layer and processed using the GELU activation function to obtain channel projection features. These spatial projection features and channel projection features are then concatenated along the channels and input into another linear layer and GELU activation function for further processing to obtain the final channel projection feature. Layer Spatial motion score Finally, according to Before selection Generate a predictive motion mask at each high-response location. .

3. The event-data-driven image deblurring method according to claim 2, characterized in that, The motion sparse event block includes: a motion sparse attention unit and an extended control space gating unit; when At that time, the motion sparse attention unit for After layer normalization, the data is processed sequentially through convolutional layers and depthwise separable convolutional layers to obtain event local projection features. Channel segmentation is then performed on these event local projection features to generate the first... Layer Event query matrix , No. Event key matrix , No. Event value matrix Therefore, the first equation is used to calculate the second equation. Layer Sparse Transposed Attention Graph of Event Motion ,Will and After multiplication, the first product is generated. Layer Updated event characteristics : (2) In equation (2), Let be the scaling factor to be learned, and Softmax represent the activation function. This indicates element-wise multiplication. Indicates transpose; The extended control space gating unit will The input is processed in the GELU activation function to obtain the spatial modulation score, and the Sigmoid function is used to... Converted to a hybrid transmission rate, thereby combining spatial modulation fraction, hybrid transmission rate, and... After performing element-wise multiplication, then with... Add them together to get the first one. Layer Event characteristics .

4. The event-data-driven image deblurring method according to claim 2, characterized in that, The motion-aware image block comprises a motion-aware attention unit and a cross-modal intensity gating unit. when At that time, the motion-sensing attention unit... Layer normalization is performed, followed by processing through convolutional layers and depthwise separable convolutional layers to obtain local image projection features. Channel segmentation is then applied to these local image projection features to generate the next layer. Layer Image query matrix , No. Image key matrix , No. Image value matrix Therefore, the first equation (3) is used to calculate the second equation. Layer Image motion sparse transpose attention map , and After multiplication, the first product is generated. Layer Updated image features : (3) In equation (3), The motion factors to be learned; The transmodal strength gating unit will The input layer is processed by the normalization layer to obtain normalized image features, and at the same time... The input layer normalization layer is processed with the GELU activation function to obtain event modulation features. The normalized image features and event modulation features are then multiplied element-wise, and then... Add them together to get the first one. Layer Image features after cross-modal fusion .

5. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing the image deblurring method of any one of claims 1-4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when run by a processor, performs the steps of the image deblurring method according to any one of claims 1-4.

Images