A lightweight multi-focus pulse fusion reconstruction method based on compressed sensing

Abstract

The application discloses a kind of light weight multi-focus pulse fusion reconstruction methods based on compressed sensing, constructs light weight multi-focus pulse fusion reconstruction network, contains multi-focus pulse input module, feature fusion module, sparse coding optimization module, image reconstruction module;Multi-focus pulse input module handles two multi-focus pulse flow data, generates single first fusion feature map and single second fusion feature map, feature fusion module extracts and fuses two fusion feature maps, obtains dictionary matrix and long exposure image, based on dictionary matrix and long exposure image, by iteratively sparse coding optimization module solution, obtain optimal sparse coding, image reconstruction module utilizes dictionary transpose matrix and optimal sparse coding, reconstructs output clear image richer in detail, overall loss function improves the approximation degree of reconstructed image and real image.The application effectively solves the problem of pulse camera imaging depth of field blur.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a lightweight multi-focus pulse fusion reconstruction method based on compressed sensing. Background Technology

[0002] Compared to traditional frame-based cameras and emerging Dynamic Vision Sensors (DVS), pulse cameras, with their unique bio-inspired working principle, achieve ultra-high temporal resolution by simulating the asynchronous triggering characteristics of retinal neurons, demonstrating significant advantages in high-speed motion scenes.

[0003] Currently, pulse cameras commonly use traditional optical lenses in practical applications. Due to the inherent limitations of optical lenses and image sensors in terms of depth of field, it is difficult to acquire a clear image of the entire scene in a single imaging session. This limitation results in only objects within the depth of field remaining sharp during imaging, while areas outside the depth of field exhibit defocusing and blurring, severely impacting subsequent image analysis and applications. Therefore, pulse cameras face similar challenges to traditional image sensors. Existing multi-focus fusion methods and theories are all based on traditional digital image processing and are not suitable for novel neuromorphic sensor data. Therefore, how to utilize multi-focus pulse data acquired by pulse cameras to achieve multi-focus fusion and reconstructed imaging is a problem that urgently needs to be researched and solved. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a lightweight multi-focus pulse fusion reconstruction method based on compressed sensing, which aims to solve the problems in the background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a lightweight multi-focus pulse fusion reconstruction method based on compressed sensing, comprising the following steps: Step S1: Construct a lightweight multi-focus pulse fusion reconstruction network, including a multi-focus pulse input module, a feature fusion module, a sparse coding optimization module, and an image reconstruction module; Step S2: The multi-focus pulse input module uses the pulse stream multiple exposure mechanism to process the acquired two segments of multi-focus pulse stream data respectively to obtain a single first fusion feature map and a single second fusion feature map; Step S3: Feature fusion module, which extracts and fuses features from a single first fused feature map and a single second fused feature map respectively, to obtain a dictionary matrix and a long exposure image; Step S4: Based on the dictionary matrix and the long exposure image, the optimal sparse coding is obtained by solving the problem through the sparse coding optimization module; Step S5: The image reconstruction module inputs the optimal sparse coding into the dictionary matrix to reconstruct and output a clearer image with richer details; Step S6: Set the overall loss function to optimize the performance of the lightweight multi-focus pulse fusion reconstruction network, and reconstruct a clearer image with richer details that approximates the real image.

[0006] Furthermore, the specific process of step S2 is as follows: Based on the pulse-stream multiple exposure mechanism, a pulse stream of continuous time T is used to... Divided into , , This is the first segment of multi-focus pulse flow data; This is the second segment of multi-focus pulse flow data; Following the rule that odd numbers increase from 3 to 41, for , Divide the exposure window and extract the locally sharp exposed image from the exposure window. , ;in, for Corresponding to the i-th time window The first set of locally clear, exposed images; for Corresponding to the i-th time window The second set of locally clear exposed images; Let be the set of real numbers. The height of the exposed image; The width of the exposed image; ;21 represents the total number of time windows; Will The image is compressed into a single first fusion feature map using a pulsed-flow multiple exposure mechanism. ; Will The image is compressed into a single second fusion feature map using a pulse-flow multiple exposure mechanism. .

[0007] Furthermore, in step S3, the dictionary matrix and the long exposure image are obtained. The specific process is as follows: Will , The exposure feature matrix is ​​extracted and fused into the multi-exposure feature matrix fusion module. ; Will , The input is fed into a lightweight pulse feature fusion module to extract and fuse fuzzy matrix features. ; Exposure feature matrix Features of fuzzy matrices Multiply the matrix features to obtain the matrix feature multiplication output. Then, apply average pooling to the matrix feature multiplication output to obtain the dictionary matrix D. Features of fuzzy matrix Long exposure images are obtained through convolution operations. ; This refers to long-exposure images within a continuous time period T.

[0008] Furthermore, the exposure feature matrix is ​​obtained. The specific process is as follows: Will and Enter the first 1 respectively 1 convolution and the second 1 1. Perform convolution processing to obtain the outputs of the first convolution and the second convolution, respectively; The outputs of the first convolution and the second convolution are added together to obtain the fused feature map. The fused feature map after addition is input into the attention feature fusion module for processing, and the output of the attention feature fusion module is obtained. The output of the attention feature fusion module, the output of the first convolution, and the output of the second convolution are input into the dynamic pixel-level attention mechanism for processing to obtain the output of the dynamic pixel-level attention mechanism. The output of the dynamic pixel-level attention mechanism is input into the first activation function for processing, resulting in weights of... W The first attention map and the weights are 1 -W The second attention map; W Attention weights; With weight W The first attention map and the output of the first convolution are multiplied together to obtain the first weighted feature map; With a weight of 1 -W The second attention map is multiplied by the output of the second convolution to obtain the second weighted feature map; Add the first weighted feature map and the second weighted feature map to obtain the first weighted feature map after addition; The weighted feature map after the first summation is added to the output of the first convolution and the output of the second convolution, respectively, to obtain the weighted feature map after the second summation. Input the weighted feature map after the second addition into the third... The exposure feature matrix is ​​obtained by processing the convolution. .

[0009] Furthermore, the specific process for obtaining the output of the attention feature fusion module is as follows: The fused feature map after addition is processed by local branching and global branching respectively; The fused feature maps after addition are input into the fourth [section / system / process]. 1 convolution and the fifth 1 The first convolution is processed to obtain the outputs of the fourth and fifth convolutions respectively; The process of handling local branches is as follows: The output of the fourth convolution is then channel-obfuscated to obtain the channel-obfuscated feature map. Input the feature map after channel confusion into the sixth 3rd section. 3. Processing is performed during convolution to obtain local features; The process of handling global branches is as follows: The output of the fifth convolution is input into the first 3. 3 deconvolutional layers, the second 3 3 deconvolutional layers, the third 3 The query vectors are processed in the 3rd deconvolution layer. Key vector Value vector ; Will and Multiply them to get the vector output; The output of the vector is input into the second activation function for processing to obtain the third attention map; Combine the third attention map with Multiplication Input 71 The first convolution is processed to obtain the output of the seventh convolution; The output of the seventh convolution is added to the fused feature map to obtain the global features; The global and local features are added together to obtain the output of the attention feature fusion module.

[0010] Furthermore, the fuzzy matrix features are obtained. The specific process is as follows: Will , The first 3×3 convolution kernel and the second 3×3 convolution kernel are input separately and processed to obtain the output of the first convolution kernel and the output of the second convolution kernel; The output of the first convolution kernel is input into the third activation function for processing, and the output of the third activation function is obtained. The output of the second convolution kernel is input into the fourth activation function for processing, and the output of the fourth activation function is obtained. The outputs of the third and fourth activation functions are concatenated by c to obtain the concatenated output of the activation functions. The outputs of the concatenated activation functions are sequentially input into the first residual block, the second residual block, the convolutional block attention module, the third 3×3 convolutional kernel, and the fifth activation function for processing to obtain the fuzzy matrix features. .

[0011] Furthermore, the specific process of obtaining the optimal sparse coding in step S4 is as follows: Create a clear image to be reconstructed. ; Using sparse representation and Both are represented as linear groups of sparse signals, indicating: (1); (2); In the formula, for The dictionary matrix; for sparse coding; for The dictionary matrix; for sparse coding; Solving for the optimal solution using formula (3) ,express: (3); In the formula, To minimize the solution operator; Represents the regularization coefficient; Use fuzzy kernel To establish and The mapping relationship between them is represented as: (4); In the formula, It is additive noise; This is a convolution operation; Will Decomposed into and Substituting the form into formula (4), we obtain formula (5), which means: (5); In the formula, For fuzzy kernel matrix; Substituting formula (5) into formula (3) yields formula (6), which means: (6); Based on formula (5), the continuous exposure time T is divided into the i-th time window. Therefore, the long exposure image of the i-th time window ,express: (7); In the formula, Let be the fuzzy kernel matrix for the i-th time window; The additive noise is for the i-th time window; Divided into the i-th time window back, The minimization optimization formula is expressed as: (8); exist Select Exposure Window back, The optimization formula is expressed as: (9); In the formula, For multi-window averaging operators; For window indexing; by sparse coding initial value Starting from this point, iterative processes are performed using the sparse coding optimization module. The following operations yielded... , ,…, ; For the first Sparse coding of the sharp image to be reconstructed from the output of -1 iterations; Based on formula (9), Iterate with the dictionary matrix input sparse coding optimization module This yields the optimal sparse coding. The specific process is as follows: (10); In the formula, For the first The sparse code after the next iteration is the optimal sparse code; It is the sixth activation function; For the objective function The gradient; It is the Lipschitz constant; This is the transpose of the fuzzy kernel matrix corresponding to the 0th time window; The image is a long-exposure image acquired within the 0th time window T0; This is the transpose of the fuzzy kernel matrix corresponding to the (N-1)th time window; For the (N-1)th time window T N-1 Long-exposure images captured internally; For a clear image to be reconstructed The transpose of the dictionary matrix.

[0012] Furthermore, the specific process of designing the overall loss function is as follows: (11); In the formula, The overall loss function; For hyperparameters; for L 1. Loss function; Let be the gradient loss function; The calculation formula is as follows: (12); In the formula, Label the real image; To utilize lightweight multi-focus pulse fusion reconstruction networks Output clearer images with richer details; The definition is as follows: (13); (14); (15); In the formula, For clearer images with richer details and pixels in real images , The horizontal gradient difference at that location; For clearer images with richer details and pixels in real images , The vertical gradient difference at that location; For clearer images with richer details, pixels , The horizontal gradient value; pixels in a real image , The horizontal gradient value; For clearer images with richer details, pixels , The gradient value in the vertical direction; pixels in a real image , The gradient value in the vertical direction; and These are the vertical and horizontal coordinates of the image pixels, respectively.

[0013] Compared with existing technologies, the present invention has the following advantages: This invention efficiently processes multi-focus pulse stream data through a pulse stream multiple exposure mechanism, accurately extracting and fusing features; it solves the optimal sparse coding by relying on an iterative sparse coding optimization module to reconstruct a clearer image with richer details; and it optimizes the lightweight multi-focus pulse fusion reconstruction network using an overall loss function to effectively solve the problem of depth-of-field blurring in pulse camera imaging. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of a Quantized Multi-Focused Pulse Fusion Reconstruction Network (MFSFRNCS) framework provided by the present invention.

[0015] Figure 2 This is a schematic diagram of a Multi Exposure Feature Matrix Fusion (MEEF) module provided by the present invention.

[0016] Figure 3 This is a schematic diagram of a multi-exposure feature matrix fusion module (LSEF) provided by the present invention. Detailed Implementation

[0017] like Figure 1 As shown, the present invention provides a technical solution: a lightweight multi-focus pulse fusion reconstruction method based on compressed sensing, comprising: Step S1: Construct a lightweight multi-focus pulse fusion reconstruction network, including a multi-focus pulse input module, a feature fusion module, a sparse coding optimization module, and an image reconstruction module; Step S2: The multi-focus pulse input module uses the pulse stream multiple exposure mechanism to process the acquired two segments of multi-focus pulse stream data respectively to obtain a single first fusion feature map and a single second fusion feature map; Step S3: Feature fusion module, which extracts and fuses features from a single first fused feature map and a single second fused feature map respectively, to obtain a dictionary matrix and a long exposure image; Step S4: Based on the dictionary matrix and the long exposure image, the optimal sparse coding is obtained by solving the problem through the sparse coding optimization module; Step S5: The image reconstruction module inputs the optimal sparse coding into the dictionary matrix to reconstruct and output a clearer image with richer details; Step S6: Set the overall loss function to optimize the performance of the lightweight multi-focus pulse fusion reconstruction network, and reconstruct a clearer image with richer details that approximates the real image.

[0018] The specific process of step S2 is as follows: Based on the pulse-stream multiple exposure mechanism, a pulse stream of continuous time T is used to... Divided into , The pulse length is set to 41. This is the first segment of multi-focus pulse flow data; This is the second segment of multi-focus pulse flow data; Following the rule that odd numbers increase from 3 to 41, for , Divide the exposure window and extract the locally sharp exposed image from the exposure window. , ;in, for Corresponding to the i-th time window The first set of locally clear, exposed images; for Corresponding to the i-th time window The second set of locally clear exposed images; Let be the set of real numbers. The height of the exposed image; The width of the exposed image; ;21 represents the total number of time windows; Will The image is compressed into a single first fusion feature map using a pulsed-flow multiple exposure mechanism. ; Will The image is compressed into a single second fusion feature map using a pulse-flow multiple exposure mechanism. .

[0019] In step S3, the dictionary matrix and the long exposure image are obtained. The specific process is as follows: Step S31: ... , The exposure feature matrix is ​​extracted and fused by the Multiple Exposure Feature Matrix Fusion (MEEF) module. ; Step S32: , The input is fed into the Lightweight Pulse Feature Fusion Module (LSEF) to extract and fuse fuzzy matrix features. ; Step S33: Exposure feature matrix Features of fuzzy matrices Multiply the matrix features to obtain the matrix feature multiplication output. Then, apply average pooling to the matrix feature multiplication output to obtain the dictionary matrix D. Features of fuzzy matrix Long exposure images are obtained through convolution operations. ; This refers to long-exposure images within a continuous time period T.

[0020] like Figure 2As shown, the exposure feature matrix is ​​obtained. The specific process is as follows: Will and Enter the first 1 respectively 1 convolution and the second 1 1. Perform convolution processing to obtain the outputs of the first convolution and the second convolution, respectively; The outputs of the first convolution and the second convolution are added together to obtain the fused feature map. The fused feature map after addition is input into the attention feature fusion module CAFM for processing, and the output of the attention feature fusion module is obtained. The output of the attention feature fusion module, the output of the first convolution, and the output of the second convolution are input into the dynamic pixel-level attention mechanism for processing to obtain the output of the dynamic pixel-level attention mechanism. The output of the dynamic pixel-level attention mechanism is processed by the first activation function (Sigmoid activation function) to obtain the weights. W The first attention map and the weights are 1 -W The second attention map; W Attention weights; With weight W The first attention map and the output of the first convolution are multiplied together to obtain the first weighted feature map; With a weight of 1 -W The second attention map is multiplied by the output of the second convolution to obtain the second weighted feature map; Add the first weighted feature map and the second weighted feature map to obtain the first weighted feature map after addition; The weighted feature map after the first summation is added to the output of the first convolution and the output of the second convolution, respectively, to obtain the weighted feature map after the second summation. Input the weighted feature map after the second addition into the third... The exposure feature matrix is ​​obtained by processing the convolution. .

[0021] The specific process for obtaining the output of the attention feature fusion module is as follows: The fused feature map after addition is processed by local branching and global branching respectively; The fused feature maps after addition are input into the fourth [section / system / process]. 1 convolution and the fifth 1 The first convolution is processed to obtain the outputs of the fourth and fifth convolutions respectively; The process of handling local branches is as follows: The output of the fourth convolution is then channel-obfuscated to obtain the channel-obfuscated feature map. Input the feature map after channel confusion into the sixth 3rd section. 3. Processing is performed during convolution to obtain local features; The process of handling global branches is as follows: The output of the fifth convolution is input into the first 3. 3 deconvolutional layers, the second 3 3 deconvolutional layers, the third 3 The query vectors are processed in the 3rd deconvolution layer. Key vector Value vector ; Will and Multiply them to get the vector output; The output of the vector is input into the second activation function (Softmax activation function) for processing to obtain the third attention map; Combine the third attention map with Multiplication Input 71 The first convolution is processed to obtain the output of the seventh convolution; The output of the seventh convolution is added to the fused feature map to obtain the global features; The global and local features are added together to obtain the output of the attention feature fusion module.

[0022] like Figure 3 As shown, the fuzzy matrix features are obtained. The specific process is as follows: Will , The first 3×3 convolution kernel and the second 3×3 convolution kernel are input separately and processed to obtain the output of the first convolution kernel and the output of the second convolution kernel; The output of the first convolution kernel is input into the third activation function (ReLU activation function) for processing, and the output of the third activation function is obtained. The output of the second convolution kernel is input into the fourth activation function (ReLU activation function) for processing, and the output of the fourth activation function is obtained. The outputs of the third and fourth activation functions are concatenated by c to obtain the concatenated output of the activation functions. The outputs of the concatenated activation functions are sequentially input into the first residual block, the second residual block, the convolutional block attention module, the third 3×3 convolutional kernel, and the fifth activation function for processing to obtain the fuzzy matrix features. .

[0023] The specific process for obtaining the optimal sparse coding in step S4 is as follows: Create a clear image to be reconstructed. ; Using sparse representation and Both are represented as linear groups of sparse signals, indicating: (1); (2); In the formula, for The dictionary matrix; for sparse coding; for The dictionary matrix; for sparse coding; Solving for the optimal solution using formula (3) ,express: (3); In the formula, To minimize the solution operator; Represents the regularization coefficient; Use fuzzy kernel To establish and The mapping relationship between them is represented as: (4); In the formula, It is additive noise; This is a convolution operation; Will Decomposed into and Substituting the form into formula (4), we obtain formula (5), which means: (5); In the formula, For fuzzy kernel matrix; Substituting formula (5) into formula (3) yields formula (6), which means: (6); Based on formula (5), the continuous exposure time T is divided into the i-th time window. Therefore, the long exposure image of the i-th time window ,express: (7); In the formula, Let be the fuzzy kernel matrix for the i-th time window; The additive noise is for the i-th time window; Divided into the i-th time window back, The minimization optimization formula is expressed as: (8); exist Select Exposure Window back, The optimization formula is expressed as: (9); In the formula, For multi-window averaging operators; For window indexing; by sparse coding initial value Starting from this point, iterative processes are performed using the sparse coding optimization module. The following operations yielded... , ,…, ; For the first Sparse coding of the sharp image to be reconstructed from the output of -1 iterations; Based on formula (9), Iterate with the dictionary matrix input sparse coding optimization module This yields the optimal sparse coding. The specific process is as follows: (10); In the formula, For the first The sparse code after the next iteration is the optimal sparse code; This is the sixth activation function (ReLU activation function); For the objective function The gradient; It is the Lipschitz constant; This is the transpose of the fuzzy kernel matrix corresponding to the 0th time window; The image is a long-exposure image acquired within the 0th time window T0; This is the transpose of the fuzzy kernel matrix corresponding to the (N-1)th time window; For the (N-1)th time window T N-1 Long-exposure images captured internally; For a clear image to be reconstructed The dictionary matrix transpose; where the minus sign in formula (10) is the -1 feedback unit, realizing the previous iteration. Feedback correction.

[0024] The specific process of designing the overall loss function is as follows: (11); In the formula, The overall loss function; For hyperparameters; for L 1. Loss function; Let be the gradient loss function; The calculation formula is as follows: (13); In the formula, Label the real image; To utilize lightweight multi-focus pulse fusion reconstruction networks Output clearer images with richer details; The definition is as follows: (12); (13); (14); In the formula, For clearer images with richer details and pixels in real images , The horizontal gradient difference at that location; For clearer images with richer details and pixels in real images , The vertical gradient difference at that location; For clearer images with richer details, pixels , The horizontal gradient value; pixels in a real image , The horizontal gradient value; For clearer images with richer details, pixels , The gradient value in the vertical direction; pixels in a real image , The gradient value in the vertical direction; and These are the vertical and horizontal coordinates of the image pixels, respectively.

[0025] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A lightweight multi-focus pulse fusion reconstruction method based on compressed sensing, characterized in that, Includes the following steps: Step S1: Construct a lightweight multi-focus pulse fusion reconstruction network, including a multi-focus pulse input module, a feature fusion module, a sparse coding optimization module, and an image reconstruction module; Step S2: The multi-focus pulse input module uses the pulse stream multiple exposure mechanism to process the acquired two segments of multi-focus pulse stream data respectively to obtain a single first fusion feature map and a single second fusion feature map; Step S3: Feature fusion module, which extracts and fuses features from a single first fused feature map and a single second fused feature map respectively, to obtain a dictionary matrix and a long exposure image; Step S4: Based on the dictionary matrix and the long exposure image, the optimal sparse coding is obtained by solving the problem through the sparse coding optimization module; Step S5: The image reconstruction module inputs the optimal sparse coding into the dictionary matrix to reconstruct and output a clearer image with richer details; Step S6: Set the overall loss function to optimize the performance of the lightweight multi-focus pulse fusion reconstruction network, and reconstruct a clearer image with richer details that approximates the real image.

2. The lightweight multi-focus pulse fusion reconstruction method based on compressed sensing according to claim 1, characterized in that: The specific process of step S2 is as follows: Based on the pulse-stream multiple exposure mechanism, a pulse stream of continuous time T is used to... Divided into , , This is the first segment of multi-focus pulse flow data; This is the second segment of multi-focus pulse flow data; Following the rule that odd numbers increase from 3 to 41, for , Divide the exposure window and extract the locally sharp exposed image from the exposure window. , ;in, for Corresponding to the i-th time window The first set of locally clear, exposed images; for Corresponding to the i-th time window The second set of locally clear exposed images; Let be the set of real numbers. The height of the exposed image; The width of the exposed image; ;21 represents the total number of time windows; Will The image is compressed into a single first fusion feature map using a pulsed-flow multiple exposure mechanism. ; Will The image is compressed into a single second fusion feature map using a pulse-flow multiple exposure mechanism. .

3. The lightweight multi-focus pulse fusion reconstruction method based on compressed sensing according to claim 2, characterized in that: Step S3 yields the dictionary matrix and the long exposure image. The specific process is as follows: Will , The exposure feature matrix is ​​extracted and fused into the multi-exposure feature matrix fusion module. ; Will , The input is fed into a lightweight pulse feature fusion module to extract and fuse fuzzy matrix features. ; Exposure feature matrix Features of fuzzy matrices Multiply the matrix features to obtain the matrix feature multiplication output. Then, apply average pooling to the matrix feature multiplication output to obtain the dictionary matrix D. Features of fuzzy matrix Long exposure images are obtained through convolution operations. ; This refers to long-exposure images within a continuous time period T.

4. The lightweight multi-focus pulse fusion reconstruction method based on compressed sensing according to claim 3, characterized in that: Obtain the exposure feature matrix The specific process is as follows: Will and Enter the first 1 respectively 1 convolution and the second 1 1. Perform convolution processing to obtain the outputs of the first convolution and the second convolution, respectively; The outputs of the first convolution and the second convolution are added together to obtain the fused feature map. The fused feature map after addition is input into the attention feature fusion module for processing, and the output of the attention feature fusion module is obtained. The output of the attention feature fusion module, the output of the first convolution, and the output of the second convolution are input into the dynamic pixel-level attention mechanism for processing to obtain the output of the dynamic pixel-level attention mechanism. The output of the dynamic pixel-level attention mechanism is input into the first activation function for processing, resulting in weights of... W The first attention map and the weights are 1 -W The second attention map; W Attention weights; With weight W The first attention map and the output of the first convolution are multiplied together to obtain the first weighted feature map; With a weight of 1 -W The second attention map is multiplied by the output of the second convolution to obtain the second weighted feature map; Add the first weighted feature map and the second weighted feature map to obtain the first weighted feature map after addition; The weighted feature map after the first summation is added to the output of the first convolution and the output of the second convolution, respectively, to obtain the weighted feature map after the second summation. Input the weighted feature map after the second addition into the third... The exposure feature matrix is ​​obtained by processing the convolution. .

5. A lightweight multi-focus pulse fusion reconstruction method based on compressed sensing according to claim 4, characterized in that: The specific process of obtaining the output of the attention feature fusion module is as follows: The fused feature map after addition is processed by local branching and global branching respectively; The fused feature maps after addition are input into the fourth [section / system / process]. 1 convolution and the fifth 1 The first convolution is processed to obtain the outputs of the fourth and fifth convolutions respectively; The process of handling local branches is as follows: The output of the fourth convolution is then channel-obfuscated to obtain the channel-obfuscated feature map. Input the feature map after channel confusion into the sixth 3rd section.

3. Processing is performed during convolution to obtain local features; The process of handling global branches is as follows: The output of the fifth convolution is input into the first 3. 3 deconvolutional layers, the second 3 3 deconvolutional layers, the third 3 The query vectors are processed in the 3rd deconvolution layer. Key vector Value vector ; Will and Multiply them to get the vector output; The output of the vector is input into the second activation function for processing to obtain the third attention map; Combine the third attention map with Multiplication Input 71 The first convolution is processed to obtain the output of the seventh convolution; The output of the seventh convolution is added to the fused feature map to obtain the global features; The global and local features are added together to obtain the output of the attention feature fusion module.

6. A lightweight multi-focus pulse fusion reconstruction method based on compressed sensing according to claim 5, characterized in that: Obtaining fuzzy matrix features The specific process is as follows: Will , The first 3×3 convolution kernel and the second 3×3 convolution kernel are input separately and processed to obtain the output of the first convolution kernel and the output of the second convolution kernel; The output of the first convolution kernel is input into the third activation function for processing, and the output of the third activation function is obtained. The output of the second convolution kernel is input into the fourth activation function for processing, and the output of the fourth activation function is obtained. The outputs of the third and fourth activation functions are concatenated by c to obtain the concatenated output of the activation functions. The outputs of the concatenated activation functions are sequentially input into the first residual block, the second residual block, the convolutional block attention module, the third 3×3 convolutional kernel, and the fifth activation function for processing to obtain the fuzzy matrix features. .

7. A lightweight multi-focus pulse fusion reconstruction method based on compressed sensing according to claim 6, characterized in that: The specific process of obtaining the optimal sparse code in step S4 is as follows: Create a clear image to be reconstructed. ; Using sparse representation and Both are represented as linear groups of sparse signals, indicating: （1）； （2）； In the formula, for The dictionary matrix; for sparse coding; for The dictionary matrix; for sparse coding; Solving for the optimal solution using formula (3) ,express: （3）； In the formula, To minimize the solution operator; Represents the regularization coefficient; Use fuzzy kernel To establish and The mapping relationship between them is represented as: （4）； In the formula, It is additive noise; This is a convolution operation; Will Decomposed into and Substituting the form into formula (4), we obtain formula (5), which means: （5）； In the formula, For fuzzy kernel matrix; Substituting formula (5) into formula (3) yields formula (6), which means: （6）； Based on formula (5), the continuous exposure time T is divided into the i-th time window. Therefore, the long exposure image of the i-th time window ,express: （7）； In the formula, Let be the fuzzy kernel matrix for the i-th time window; The additive noise is for the i-th time window; Divided into the i-th time window back, The minimization optimization formula is expressed as: （8）； exist Select Exposure Window back, The optimization formula is expressed as: （9）； In the formula, For multi-window averaging operators; For window indexing; by sparse coding initial value Starting from this point, iterative processes are performed using the sparse coding optimization module. The following operations yielded... , ,…, ; For the first Sparse coding of the sharp image to be reconstructed from the output of -1 iterations; Based on formula (9), Iterate with the dictionary matrix input sparse coding optimization module This yields the optimal sparse coding. The specific process is as follows: （10）； In the formula, For the first The sparse code after the next iteration is the optimal sparse code; It is the sixth activation function; For the objective function The gradient; It is the Lipschitz constant; This is the transpose of the fuzzy kernel matrix corresponding to the 0th time window; The image is a long-exposure image acquired within the 0th time window T0; This is the transpose of the fuzzy kernel matrix corresponding to the (N-1)th time window; For the (N-1)th time window T N-1 Long-exposure images captured internally; For a clear image to be reconstructed The transpose of the dictionary matrix.

8. A lightweight multi-focus pulse fusion reconstruction method based on compressed sensing according to claim 7, characterized in that: The specific process for designing the overall loss function is as follows: （11）； In the formula, The overall loss function; For hyperparameters; for L 1. Loss function; Let be the gradient loss function; The calculation formula is as follows: （12）； In the formula, Label the real image; To utilize lightweight multi-focus pulse fusion reconstruction networks Output clearer images with richer details; The definition is as follows: （13）； （14）； （15）； In the formula, For clearer images with richer details and pixels in real images , The horizontal gradient difference at that location; For clearer images with richer details and pixels in real images , The vertical gradient difference at that location; For clearer images with richer details, pixels , The horizontal gradient value; pixels in a real image , The horizontal gradient value; For clearer images with richer details, pixels , The gradient value in the vertical direction; pixels in a real image , The gradient value in the vertical direction; and These are the vertical and horizontal coordinates of the image pixels, respectively.

Images