Vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method and system

Abstract

The invention provides a vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method and system. The method comprises the following steps: 1, carrying out super-resolution multi-task processing; the method comprises the following steps: S1, establishing a vehicle-mounted dual-spectrum synchronous acquisition system; s2, acquiring data by using a vehicle-mounted dual-spectrum synchronous acquisition system, and establishing a high-low resolution data set of dual-spectrum pairing based on calibration parameters; s3, developing a deblurring super-resolution multi-task model with deep feature refinement, and constructing a deblurring branch based on double attention modulation and a super-resolution branch based on cascaded attention of feature enhancement; s4, carrying out style migration on the visible light image to obtain a high-resolution infrared image, and designing a dual-spectrum migration scale factor gamma and a feature attention fusion module to enhance the spatial detail features of the infrared image; according to the invention, visible light and infrared images can be synchronously obtained through the design of a double-spectrum system, a deblurring super-resolution multi-task processing method is developed, and detail features of the double-spectrum image are enhanced through information interaction.

Description

Technical Field

[0001] The present invention relates to the field of computer vision image processing, and in particular to a vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method and system. Background Art

[0002] With the continuous development of in-vehicle intelligent systems, computer vision technology is increasingly being used in the fields of autonomous driving, assisted driving, and environmental perception. When driving at night or on cloudy days, the vehicle's environmental perception becomes even more important due to the limitations of ambient lighting conditions. However, relying solely on a single visible light camera cannot provide sufficient environmental information. In order to meet this challenge, infrared cameras, as a device that can effectively obtain thermal radiation information of objects under low light conditions, can perform joint image processing with visible light cameras to provide vehicles with more comprehensive and accurate perception information in complex environments, thereby improving the safety and robustness of the system.

[0003] However, affected by the resolution of the imaging device, the acquired images are low-resolution images in most cases, and due to the change in relative speed between the vehicle and the camera sensor, the image is prone to motion blur during long exposure time, which affects the image quality. Therefore, it is very necessary to find a dual-spectrum image enhancement method that combines image deblurring and super-resolution reconstruction. In recent years, in the field of image deblurring, researchers have extracted high-level features from blurred images through convolutional neural networks or generative adversarial networks to achieve deblurring, which can significantly improve image clarity; in terms of image super-resolution reconstruction, there are methods based on convolutional neural networks, Transformer-based methods, and generative adversarial networks, which reconstruct super-resolution images by modeling the contextual information of low-resolution images.

[0004] These methods are often designed for single-spectrum images, and the existing technologies are still lacking in pertinence for the joint processing of dual-spectrum images. Therefore, the present invention proposes a vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method and system to solve the difficulties existing in the existing technologies, which is a problem that those skilled in the art need to solve urgently. Summary of the invention

[0005] The present invention proposes an on-board dual-spectrum deblurring super-resolution multi-task processing method and system, which aggregates the correlation between feature maps of different modalities through a feature correlation layer to improve the image registration accuracy, introduces a large-kernel convolution layer to expand the receptive field, uses an adaptive average pooling layer to reduce model parameters, and solves the problem of image size limitation.

[0006] The present invention adopts the following technical solutions.

[0007] A vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method comprises the following steps;

[0008] Step S1, building a vehicle-mounted dual-spectrum synchronous acquisition system;

[0009] Step S2, collecting data with a vehicle-mounted dual-spectrum synchronous acquisition system to establish a high- and low-resolution data set of dual-spectrum pairing based on calibration parameters;

[0010] Step S3, developing a deblurring super-resolution multi-task model with deep feature refinement, constructing a deblurring branch based on dual attention modulation and a super-resolution branch based on cascaded attention for feature enhancement;

[0011] Step S4: Style transfer the visible light image to obtain a high-resolution infrared image, and design a dual-spectrum transfer scale factor γ and a feature attention fusion module to enhance the spatial detail features of the infrared image.

[0012] A vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system is used to execute the vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method described above. The vehicle-mounted dual-spectrum synchronous acquisition system in step S1 includes a synchronous acquisition chip, an optical machine housing and a vehicle-mounted shock absorption adsorption system, and also includes a visible light camera arranged in pairs and an infrared camera arranged in pairs.

[0013] In step S1, the optomechanical housing is a four-aperture optomechanical housing; the vehicle-mounted dual-spectrum synchronous acquisition system includes two FLIR-BFS-U3-51S5C-C visible light cameras with an effective resolution of 2448×2048, two Guide-TWIN1212 infrared cameras with an effective resolution of 1280×1024, a synchronous acquisition chip, a four-aperture optomechanical housing and a vehicle-mounted shock-absorbing adsorption system.

[0014] The visible light cameras and infrared cameras are arranged in pairs as two shooting combinations, each shooting combination includes a visible light camera and an infrared camera; one shooting combination shoots blurred images with a long exposure time, and the other shooting combination shoots clear images with a short exposure time.

[0015] The long exposure time is greater than 13250 us; the short exposure time is less than 4950 us.

[0016] In the step S2, the preparation of high-resolution and low-resolution data sets for dual-spectrum pairing specifically includes the following steps: step A1, firstly, calibrating the intrinsic parameters and extrinsic parameters of the two cameras in the shooting combination through an external camera, so as to obtain the epipolar expression between the images taken by the two cameras; extracting features from the two collected images using a feature point extraction algorithm, and removing feature points that do not meet the limit constraints, and using the remaining feature points to achieve image registration;

[0017] Step A2, using one of the infrared images as a reference, registering it with the other images, and cutting out the overlapping area of ​​the two images after registration;

[0018] Step A3: Finally, the collected blurred images are subjected to bicubic interpolation degradation processing to establish a paired data set, including a low-resolution blurred visible light image, a high-resolution clear visible light image, a low-resolution blurred infrared image, and a high-resolution clear infrared image.

[0019] In step S3, the deblurring super-resolution multi-task model for deep feature refinement includes a deblurring branch based on dual attention modulation, a super-resolution branch based on cascaded attention for feature enhancement, and a cross-feature extraction module;

[0020] The deblurring branch based on dual attention modulation adopts UNet as the backbone network, and designs a dual attention modulation module as a module for multi-scale feature extraction, including layer normalization, 1×1 convolution, GeLU activation function, deep convolution, local attention, global attention and spatial gated feedforward modules;

[0021] The feature-enhanced cascaded attention super-resolution branch includes a 3×3 convolution, multiple cascaded high-frequency enhancement modules, a large kernel separable attention tail and an upsampling layer; the high-frequency enhancement module includes layer normalization, feature-enhanced cascaded attention, a spatial gated feedforward module, a deep convolution and a residual connection;

[0022] The cross feature extraction module includes group convolution, ReLU activation function, channel attention, spatial attention, and Sigmoid activation function.

[0023] In step S3, the feature-enhanced cascade attention includes enhanced ShuffleAttention, multi-scale large-core separable attention and residual convolution connection, and the specific implementation process is as follows:

[0024] Y=H MLSKA (H ESA ))+f DW3×3 (X) Formula 1;

[0025] Among them, X represents the input feature, Y represents the output feature, and H ESA represents the enhanced Shuffle Attention module, H MLSKA represents the multi-scale large kernel separable attention module, f DW3×3represents a 3×3 depth convolution; the enhanced Shuffle Attention includes two 1×1 convolutions, Shuffle Attention and two residual connections; the multi-scale large-core separable attention includes three gated large-core separable attention blocks of three different scales and two 1×1 convolutions.

[0026] In step S4, the feature attention fusion module includes two feature extraction branches consisting of a residual pixel attention module and a group convolution;

[0027] In step S4, the scale factor γ is used to adjust the input features of the two branches, and the visible light feature map and the infrared feature map are extracted respectively. The feature map obtained by each branch is fused by element addition to obtain the final feature map;

[0028] The scale factor γ is a learnable parameter, which is updated according to the loss L and the learning rate a during the training process, and its value is constrained between 0 and 1 by the Sigmoid function. The specific operation is expressed in the following formula:

[0029]

[0030]

[0031] Among them, I out Represents the final result image, I GT Represents a clear image of the original high resolution, represents wavelet transform, γ (t) The parameter value at the current t-th iteration, is the gradient of the loss function with respect to γ.

[0032] The vehicle-mounted dual-spectrum synchronous acquisition system is mounted on the top of the vehicle. When collecting data in step S2, the vehicle travels at a speed of 10-60 km / h, and the vehicle-mounted dual-spectrum synchronous acquisition system synchronously collects images with different blur levels.

[0033] Compared with the prior art, the present invention has the following beneficial effects: the present invention realizes an on-board dual-spectrum deblurring super-resolution multi-task processing method, which combines a deblurring network based on dual attention modulation and a super-resolution reconstruction network based on cascade attention for feature enhancement to achieve a higher-precision image enhancement effect. At the same time, the style of the visible light image is transferred, and the dual-spectrum migration scale factor γ and the feature attention fusion module are designed to fuse it with the infrared image, further enhancing the spatial detail characteristics of the infrared image. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0035] AttachedFigure 1 It is a flow chart of a vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method in an embodiment of the present invention;

[0036] Attached Figure 2 Schematic diagram of a vehicle-mounted dual-spectrum synchronous acquisition system in an embodiment of the present invention;

[0037] Attached Figure 3 A schematic diagram of a deblurring super-resolution multi-task network structure for deep feature refinement in an embodiment of the present invention;

[0038] Attached Figure 4 Schematic diagram of a deblurring branch network structure based on dual attention modulation in an embodiment of the present invention;

[0039] Attached Figure 5 Schematic diagram of the structure of a dual attention modulation module in an embodiment of the present invention;

[0040] Attached Figure 6 Schematic diagram of a super-resolution branch network structure based on cascaded attention for feature enhancement in an embodiment of the present invention;

[0041] Attached Figure 7 Schematic diagram of the structure of the cascaded attention module for feature enhancement in an embodiment of the present invention;

[0042] Attached Figure 8 Schematic diagram of the network structure of the cross-feature extraction module in an embodiment of the present invention;

[0043] Attached Fig. 9 Schematic diagram of a feature attention fusion module in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0045] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] As shown in the figure, a vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method includes the following steps:

[0048] Step S1, building a vehicle-mounted dual-spectrum synchronous acquisition system;

[0049] Step S2, collecting data with a vehicle-mounted dual-spectrum synchronous acquisition system to establish a high- and low-resolution data set of dual-spectrum pairing based on calibration parameters;

[0050] Step S3, developing a deblurring super-resolution multi-task model with deep feature refinement, constructing a deblurring branch based on dual attention modulation and a super-resolution branch based on cascaded attention for feature enhancement;

[0051] Step S4: Style transfer the visible light image to obtain a high-resolution infrared image, and design a dual-spectrum transfer scale factor γ and a feature attention fusion module to enhance the spatial detail features of the infrared image.

[0052] A vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system is used to execute the vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method described above. The vehicle-mounted dual-spectrum synchronous acquisition system in step S1 includes a synchronous acquisition chip, an optical machine housing and a vehicle-mounted shock absorption adsorption system, and also includes a visible light camera arranged in pairs and an infrared camera arranged in pairs.

[0053] In step S1, the optomechanical housing is a four-aperture optomechanical housing; the vehicle-mounted dual-spectrum synchronous acquisition system includes two FLIR-BFS-U3-51S5C-C visible light cameras with an effective resolution of 2448×2048, two Guide-TWIN1212 infrared cameras with an effective resolution of 1280×1024, a synchronous acquisition chip, a four-aperture optomechanical housing and a vehicle-mounted shock-absorbing adsorption system.

[0054] The visible light cameras and infrared cameras are arranged in pairs as two shooting combinations, each shooting combination includes a visible light camera and an infrared camera; one shooting combination shoots blurred images with a long exposure time, and the other shooting combination shoots clear images with a short exposure time.

[0055] The long exposure time is greater than 13250 us; the short exposure time is less than 4950 us.

[0056] In the step S2, the preparation of high-resolution and low-resolution data sets for dual-spectrum pairing specifically includes the following steps: step A1, firstly, calibrating the intrinsic parameters and extrinsic parameters of the two cameras in the shooting combination through an external camera, so as to obtain the epipolar expression between the images taken by the two cameras; extracting features from the two collected images using a feature point extraction algorithm, and removing feature points that do not meet the limit constraints, and using the remaining feature points to achieve image registration;

[0057] Step A2, using one of the infrared images as a reference, registering it with the other images, and cutting out the overlapping area of ​​the two images after registration;

[0058] Step A3: Finally, the collected blurred images are subjected to bicubic interpolation degradation processing to establish a paired data set, including a low-resolution blurred visible light image, a high-resolution clear visible light image, a low-resolution blurred infrared image, and a high-resolution clear infrared image.

[0059] In step S3, the deblurring super-resolution multi-task model for deep feature refinement includes a deblurring branch based on dual attention modulation, a super-resolution branch based on cascaded attention for feature enhancement, and a cross-feature extraction module;

[0060] The deblurring branch based on dual attention modulation adopts UNet as the backbone network, and designs a dual attention modulation module as a module for multi-scale feature extraction, including layer normalization, 1×1 convolution, GeLU activation function, deep convolution, local attention, global attention and spatial gated feedforward modules;

[0061] The feature-enhanced cascaded attention super-resolution branch includes a 3×3 convolution, multiple cascaded high-frequency enhancement modules, a large kernel separable attention tail and an upsampling layer; the high-frequency enhancement module includes layer normalization, feature-enhanced cascaded attention, a spatial gated feedforward module, a deep convolution and a residual connection;

[0062] The cross feature extraction module includes group convolution, ReLU activation function, channel attention, spatial attention, and Sigmoid activation function.

[0063] In step S3, the feature-enhanced cascade attention includes enhanced ShuffleAttention, multi-scale large-core separable attention and residual convolution connection, and the specific implementation process is as follows:

[0064] Y=H MLSKA (H ESA (X))+f DW3×3(X) Formula 1;

[0065] Among them, X represents the input feature, Y represents the output feature, and H ESA represents the enhanced Shuffle Attention module, H MLSKA represents the multi-scale large kernel separable attention module, f DW3×3Represents a 3×3 depth convolution; the enhanced ShuffleAttention includes two 1×1 convolutions, ShuffleAttention and two residual connections; the multi-scale large-core separable attention includes three gated large-core separable attention blocks of three different scales and two 1×1 convolutions.

[0066] In step S4, the feature attention fusion module includes two feature extraction branches consisting of a residual pixel attention module and a group convolution;

[0067] In step S4, the scale factor γ is used to adjust the input features of the two branches, and the visible light feature map and the infrared feature map are extracted respectively. The feature map obtained by each branch is fused by element addition to obtain the final feature map;

[0068] The scale factor γ is a learnable parameter, which is updated according to the loss L and the learning rate α during the training process, and its value is constrained between 0 and 1 using the Sigmoid function. The specific operation is expressed in the following formula:

[0069]

[0070]

[0071] Among them, I out Represents the final result image, I GT Represents a clear image of the original high resolution, represents wavelet transform, γ (t) The parameter value at the current t-th iteration, is the gradient of the loss function with respect to γ.

[0072] The vehicle-mounted dual-spectrum synchronous acquisition system is mounted on the top of the vehicle. When collecting data in step S2, the vehicle travels at a speed of 10-60 km / h, and the vehicle-mounted dual-spectrum synchronous acquisition system synchronously collects images with different blur levels.

[0073] Example:

[0074] This example proposes a vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method and system, referring to the accompanying drawings, including the following steps:

[0075] S1. Build a vehicle-mounted dual-spectrum synchronous acquisition system, including visible light and infrared cameras, synchronous acquisition chips, four-aperture optical machine housing and vehicle-mounted shock absorption adsorption system;

[0076] S2, establishing high and low resolution data sets based on dual spectrum pairing with calibration parameters;

[0077] S3. Develop a multi-task model for deblurring super-resolution with deep feature refinement, construct a deblurring branch based on dual attention modulation and a super-resolution branch based on cascaded attention for feature enhancement;

[0078] S4. Style transfer visible light images to obtain high-resolution infrared images, design dual-spectral transfer scale factor γ and feature attention fusion module to enhance the spatial detail characteristics of infrared images.

[0079] like Figure 2 As shown, the vehicle-mounted dual-spectrum synchronous acquisition system of step S1 includes two FLIR-BFS-U3-51S5C-C visible light cameras (effective resolution 2448×2048), two Guide-TWIN1212 infrared cameras (effective resolution 1280×1024), a synchronous acquisition chip, a four-aperture optical machine housing and a vehicle-mounted shock absorption adsorption system. The visible light and infrared cameras are paired, one pair takes blurry images with a long exposure time (greater than 13250us), and the other pair takes clear images with a short exposure time (less than 4950us). The system is mounted on the top of the vehicle, driving at a speed of 10-60km / h, and collecting images with different blur levels.

[0080] The high- and low-resolution data set production process of the dual-spectrum pairing in step S2 is as follows: first, the intrinsic and extrinsic parameters of the two cameras are obtained through camera calibration, so as to obtain the epipolar expression between the images taken by the two cameras; the features of the two collected images are extracted using the feature point extraction algorithm, and the feature points that do not meet the limit constraints are eliminated, and the remaining feature points are used to achieve image registration; one of the infrared images is used as a reference, and the other images are used to register it, and the overlapping area of ​​the two images is cropped after registration; finally, the collected blurred image is subjected to bicubic interpolation degradation processing to establish a paired data set, including a low-resolution blurred visible light image, a high-resolution clear visible light image, a low-resolution blurred infrared image, and a high-resolution clear infrared image. After processing all the data, the data set is divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0081] like Figure 3 As shown, the deblurring super-resolution multi-task model of deep feature refinement in step S3 includes a deblurring branch based on dual attention modulation, a super-resolution branch based on cascaded attention for feature enhancement, and a cross-feature extraction module. The image is input into the two branches in parallel for feature reconstruction, and the reconstructed features are fused and further extracted and refined through the cross-feature extraction module. The specific structure of each part is as follows:

[0082] like Figure 4 As shown in FIG, the deblurring branch based on dual attention modulation adopts UNet as the backbone network, and designs a dual attention modulation module as a module for multi-scale feature extraction, as shown in FIG. Figure 5 As shown in the figure, it includes layer normalization, 1×1 convolution, GeLU activation function, deep convolution, local attention, global attention and spatial gated feedforward modules. After the input features are normalized by the layer, the obtained features are subjected to feature pre-processing consisting of 1×1 convolution and deep convolution to reduce local bias in the feature extraction process. Subsequently, local attention and global attention are calculated respectively and summed. The features are further refined and extracted through 1×1 convolution, so that the model can learn the full-scale perception ability of the features, and finally input into the feedforward network to further learn more complex feature mapping relationships.

[0083] like Figure 6 As shown, the super-resolution branch based on feature-enhanced cascaded attention includes a 3×3 convolution, multiple cascaded high-frequency enhancement modules, a large kernel separable attention tail and an upsampling layer; the high-frequency enhancement module includes layer normalization, feature-enhanced cascaded attention, a spatial gated feedforward module, a deep convolution and a residual connection; as shown Figure 7 As shown in the figure, the feature-enhanced cascade attention first enhances the high-frequency detail contours of the image through enhanced Shuffle Attention, and then aggregates multi-level perceptual features through multi-scale large-kernel separable attention.

[0084] like Figure 8 As shown, the cross feature extraction module includes group convolution, ReLU activation function, channel attention, spatial attention, and Sigmoid activation function, and uses the original image and deblurred features as image priors to guide the features generated by the super-resolution network to reconstruct a high-quality image.

[0085] like Fig. 9 As shown, the feature attention fusion module of step S4 includes two residual pixel attention modules and two group convolutions. The scale factor γ is used to adjust the input features of the two branches, and the visible light feature map and the infrared feature map are extracted respectively. The feature map obtained by each branch is fused by element addition to obtain the final feature map; the scale factor γ is designed as a learnable parameter. During the training process, the parameter is updated according to the loss L and the learning rate α, and the Sigmoid function is used to constrain its value between 0 and 1. The specific operation is as follows:

[0086]

[0087] Among them, I out Represents the final result image, I GT Represents a clear image of the original high resolution, represents wavelet transform, γ (t) The parameter value at the current t-th iteration, is the gradient of the loss function with respect to γ. This step is equivalent to finding an optimal scaling factor through iterative calculation during the training process to compensate for the missing details of the infrared image through the visible light image.

[0088] During the training process of the model, the paired blurry low-resolution visible light and infrared images were used as input, and the paired clear high-resolution visible light and infrared images were used as GroundTruth. The Adam optimizer training was set, the initial learning rate was 0.0005, the batch size was set to 32, and 1500 epochs were trained. The model was saved every 10 epochs. After the training was completed, the best model was selected for testing according to the validation set indicators.

[0089] The present invention is compared with cascaded or parallel dual-task methods (GFN, FMA-Net, AFA-Net, IPT, CNLRN), and the quality of image enhancement is measured using PSNR and SSIM indicators. As shown in Table 1, as the vehicle speed increases, the greater the blur of the image, the greater the difficulty of image enhancement. The present invention achieves higher accuracy under different vehicle speeds.

[0090] Table 1: Quantitative comparison of different methods on different vehicle speed datasets under 4x super-resolution

[0091]

[0092] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method, characterized in that: The steps include: Step S1, building a vehicle-mounted dual-spectrum synchronous acquisition system; Step S2, collecting data with a vehicle-mounted dual-spectrum synchronous acquisition system to establish a high- and low-resolution data set of dual-spectrum pairing based on calibration parameters; Step S3, developing a deblurring super-resolution multi-task model with deep feature refinement, constructing a deblurring branch based on dual attention modulation and a super-resolution branch based on cascaded attention for feature enhancement; Step S4: Style transfer the visible light image to obtain a high-resolution infrared image, and design a dual-spectrum transfer scale factor γ and a feature attention fusion module to enhance the spatial detail features of the infrared image.

2. A vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system, used to execute the vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing method described in claim 1, characterized in that: The vehicle-mounted dual-spectrum synchronous acquisition system in step S1 includes a synchronous acquisition chip, an optical machine housing and a vehicle-mounted shock absorption adsorption system, and also includes a pair of visible light cameras and a pair of infrared cameras.

3. The vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system according to claim 2, characterized in that: In step S1, the optomechanical housing is a four-aperture optomechanical housing; the vehicle-mounted dual-spectrum synchronous acquisition system includes two FLIR-BFS-U3-51S5C-C visible light cameras with an effective resolution of 2448×2048, two Guide-TWIN1212 infrared cameras with an effective resolution of 1280×1024, a synchronous acquisition chip, a four-aperture optomechanical housing and a vehicle-mounted shock-absorbing adsorption system.

4. The vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system according to claim 2, characterized in that: The visible light cameras and infrared cameras are arranged in pairs as two shooting combinations, each shooting combination includes a visible light camera and an infrared camera; one shooting combination shoots blurred images with a long exposure time, and the other shooting combination shoots clear images with a short exposure time.

5. The vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system according to claim 4, characterized in that: The long exposure time is greater than 13250 us; the short exposure time is less than 4950 us.

6. The vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system according to claim 4, characterized in that: In step S2, the preparation of high- and low-resolution data sets for dual-spectrum pairing specifically includes the following steps: Step A1: First, the intrinsic parameters and extrinsic parameters of the two cameras in the shooting combination are calibrated by an external camera to obtain the epipolar expression between the images taken by the two cameras; the features of the two images are respectively extracted using a feature point extraction algorithm, and the feature points that do not meet the limit constraints are eliminated, and the remaining feature points are used to achieve image registration; Step A2, using one of the infrared images as a reference, registering it with the other images, and cutting out the overlapping area of ​​the two images after registration; Step A3: Finally, the collected blurred images are subjected to bicubic interpolation degradation processing to establish a paired data set, including a low-resolution blurred visible light image, a high-resolution clear visible light image, a low-resolution blurred infrared image, and a high-resolution clear infrared image.

7. The vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system according to claim 1, characterized in that: In step S3, the deblurring super-resolution multi-task model for deep feature refinement includes a deblurring branch based on dual attention modulation, a super-resolution branch based on cascaded attention for feature enhancement, and a cross-feature extraction module; The deblurring branch based on dual attention modulation adopts UNet as the backbone network, and designs a dual attention modulation module as a module for multi-scale feature extraction, including layer normalization, 1×1 convolution, GeLU activation function, deep convolution, local attention, global attention and spatial gated feedforward modules; The feature-enhanced cascaded attention super-resolution branch includes a 3×3 convolution, multiple cascaded high-frequency enhancement modules, a large kernel separable attention tail and an upsampling layer; the high-frequency enhancement module includes layer normalization, feature-enhanced cascaded attention, a spatial gated feedforward module, a deep convolution and a residual connection; The cross feature extraction module includes group convolution, ReLU activation function, channel attention, spatial attention, and Sigmoid activation function.

8. The vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system according to claim 7, characterized in that: In step S3, the feature-enhanced cascade attention includes enhanced Shuffle Attention, multi-scale large-core separable attention and residual convolution connection, and the specific implementation process is as follows: Y=H MLSKA (H ESA (X))+f DW3×3 (X) Formula 1; Among them, X represents the input feature, Y represents the output feature, and H ESA represents the enhanced ShuffleAttention module, H MLSKA represents the multi-scale large kernel separable attention module, f DW3×3 Represents a 3×3 depth convolution; the enhanced ShuffleAttention includes two 1×1 convolutions, ShuffleAttention and two residual connections; the multi-scale large-core separable attention includes three gated large-core separable attention blocks of three different scales and two 1×1 convolutions.

9. The vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system according to claim 1, characterized in that: In step S4, the feature attention fusion module includes two feature extraction branches consisting of a residual pixel attention module and a group convolution; In step S4, the scale factor γ is used to adjust the input features of the two branches, and the visible light feature map and the infrared feature map are extracted respectively. The feature map obtained by each branch is fused by element addition to obtain the final feature map; The scale factor γ is a learnable parameter, which is updated according to the loss L and the learning rate α during the training process, and its value is constrained between 0 and 1 using the Sigmoid function. The specific operation is expressed in the following formula: Among them, I out Represents the final result image, I GT Represents a clear image of the original high resolution, represents wavelet transform, γ (t) The parameter value at the current t-th iteration, is the gradient of the loss function with respect to γ.

10. The vehicle-mounted dual-spectrum deblurring super-resolution multi-task processing system according to claim 2, characterized in that: The vehicle-mounted dual-spectrum synchronous acquisition system is mounted on the top of the vehicle. When collecting data in step S2, the vehicle travels at a speed of 10-60 km / h, and the vehicle-mounted dual-spectrum synchronous acquisition system synchronously collects images with different blur levels.

Images