A method for image deblurring based on multi-scale feature weighting

Abstract

The present invention discloses an image deblurring method based on multi-scale feature weighting, comprising: collecting fuzzy ship plate image data of real scenes to obtain a preliminary data set, and using a fuzzy estimation network to estimate a basic fuzzy kernel and a mixing coefficient from the data; obtaining a set of clear ship plate images, using the basic fuzzy kernel and the mixing coefficient to convert them into fuzzy ship plate images, and collecting each clear-fuzzy ship plate image pair to obtain a clear fuzzy data set; constructing a deblurring model based on a multi-scale feature extraction network and integrating a feature weighted fusion module; using the clear fuzzy data set to train the deblurring model, and optimizing the deblurring model through a loss function based on the frequency domain; inputting the fuzzy image to be processed into the trained deblurring model to obtain a clear image. The present invention can better restore its detail features, improve the deblurring effect and the clarity and accuracy of the generated image as a whole.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to an image deblurring method based on multi-scale feature weighting. Background Art

[0002] As an important pillar industry for China's economic development, inland shipping undertakes the key tasks of cargo transportation and resource allocation. It plays an indispensable role in promoting the balanced development of regional economy and the effective use of resources. However, inland shipping faces the problem of ship license plate recognition in actual operation, especially in the complex and changeable inland water environment. The movement of the hull will cause blurring in the imaging process, resulting in the tailing phenomenon of the ship license plate image, which poses a challenge to the accurate recognition of the ship license plate. Improving the accuracy and robustness of ship license plate recognition can not only improve the modernization level of shipping management, but also ensure the safety and smoothness of water traffic.

[0003] The core difficulty of inland shipping license plate recognition is how to deal with the image blur caused by ship movement and environmental factors. The fuzziness of the image not only reduces the accuracy of the recognition system, but also affects its robustness. In order to overcome these difficulties, it is necessary to study and apply advanced image deblurring technologies. These technologies should be able to effectively improve the clarity of the license plate image, while having the ability to process in real time to adapt to the rapidly changing water environment. In addition, the deblurring technology should also have good generalization, be able to adapt to different water conditions and lighting changes, and ensure that high-accuracy license plate recognition can be maintained in various environments.

[0004] In order to solve the fuzzy problem in license plate recognition, intelligent technology is gradually being applied to the inland shipping industry. There are two main deblurring technologies at present: one is the image processing method based on traditional machine vision, and the other is the deblurring method based on deep learning. The traditional method relies on manually designed feature extraction, but has poor adaptability to environmental changes. In contrast, the deep learning method can automatically learn and identify the features of blurred license plates from a large amount of data by constructing a convolutional neural network model, showing stronger generalization ability and robustness; however, this type of method is insufficient in restoring the details of the license plate font. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes an image deblurring method based on multi-scale feature weighting, and the specific technical solution is as follows:

[0006] An image deblurring method based on multi-scale feature weighting includes the following steps:

[0007] S1: Collect fuzzy ship license plate image data of real scenes to obtain a preliminary data set;

[0008] S2: Use a fuzzy estimation network to estimate the basic blur kernel and mixing coefficient from the preliminary data set; the fuzzy estimation network includes an encoder and a decoder, the encoder includes five downsampling modules, each downsampling module has two layers, each layer includes a convolution layer and a LeakyReLU activation function; the output of the encoder is input into the blur kernel head and mixing coefficient head of the decoder after passing through a convolution block, the blur kernel head is used to output a set of basic blur kernels, and the mixing coefficient head is used to output the mixing coefficient; the blur kernel head and the mixing coefficient head each include five upsampling modules, each upsampling module includes a bilinear upsampling layer and three convolution layers; there is a jump connection between the second convolution layer in the downsampling module and the second convolution layer in the upsampling module, and a global average pooling layer is set at the jump connection of the blur kernel head;

[0009] S3: Obtain a set of clear ship license plate images, and use the basic blur kernel and the mixing coefficient to convert the clear ship license plate images into blurry ship license plate images to obtain a clear blurry data set;

[0010] S4: Construct a deblurring model based on a multi-scale feature extraction network and integrating a feature weighted fusion module; the multi-scale feature extraction network includes a feature encoder and a feature decoder, the feature encoder has multiple layers, each layer includes a feature extraction module and a downsampling module; the feature decoder has multiple layers, each layer includes an upsampling module and a feature extraction module; the feature encoder and the feature decoder are connected by a feature extraction module; the first input of the feature weighted fusion module at each scale is the output of the feature extraction module at the same scale in the feature encoder, that is, the low-level feature, and the second input is the output of the upsampling module at the same scale, that is, the high-level feature; after the feature weighted fusion module performs weighted fusion on the input, the output weighted fusion feature is used as the input of the feature extraction module at the same scale in the feature decoder;

[0011] S5: training the deblurring model using the clear fuzzy data set, and optimizing the deblurring model by a loss function based on the frequency domain;

[0012] S6: Input the blurred image to be processed into the trained deblurring model to obtain a clear image.

[0013] Furthermore, in S4, the feature extraction module includes two residual modules, each residual module includes a normalization layer, two convolution layers, and a channel attention module; the input feature map passes through the normalization layer and two convolution layers in sequence, and the convolution result is divided into two parts along the channel dimension and element-by-element multiplication is performed to generate a feature map, which is then input into the channel attention module, and the output of the channel attention module is multiplied with the feature map, and the multiplication result is added to the input of the residual module as the output of the residual module and as the input of the next residual module.

[0014] Furthermore, in S4, the feature weighted fusion module includes a multi-head attention module and a 1x1 convolution layer; the multi-head attention module is used to perform weighted fusion on the input low-level features and high-level features, and the Sigmoid activation function is used in the fusion process. The fused feature map passes through the 1x1 convolution layer and is input into the feature extraction module of the same scale.

[0015] Furthermore, the multi-head attention module includes: a spatial attention module, a channel attention module, and a pixel attention module;

[0016] The spatial attention feature map S obtained by the spatial attention module att The expression is as follows:

[0017]

[0018] Where Conv2d( ) represents a two-dimensional convolution operation with reflection padding; Concat( ) represents concatenation in the channel dimension; X γ Numerically equal to the sum of the low-level features and high-level features of the input X add Equal, the shape is (N, C, H, W), N represents the batch of input feature maps, C represents the number of channels of the input feature map, H represents the height of the input feature map, and W represents the width of the input feature map;

[0019] Channel attention feature map C obtained by the channel attention module att The expression is as follows:

[0020]

[0021] In the formula, Conv2d(∙,∙,1) represents a two-dimensional convolution operation using a 1x1 convolution kernel, where the first parameter is the input feature map, the second parameter is the number of output channels, and the third parameter is the convolution kernel size; r is the reduction rate of the number of channels; X αβ Numerically the same as X add Equal, shape is (N, C, H, W);

[0022] The pixel attention feature map ε obtained by the pixel attention module is expressed as follows:

[0023]

[0024] Taking the pixel attention feature map as the weight, the low-level features and high-level features are weighted and summed to obtain the weighted fusion feature F fuse The expression is as follows:

[0025]

[0026] In the formula, C 1×1 represents 1×1 convolution, F low represents the low-level features generated by the feature encoder; F high Represents the high-level features generated by the feature decoder.

[0027] Furthermore, S5 is specifically implemented through the following sub-steps:

[0028] S5.1: The blurred image X with a size of 256×256 and a number of channels of 3 is input into the deblurring model. It first passes through a 3×3 convolutional layer to generate a feature map T with a number of channels of 64.

[0029] S5.2: Extract features from the feature map T. The number of channels and size of the processed feature map remain unchanged. Then, through the downsampling module, a feature map T1 with 128 channels and a size of 128×128 is obtained.

[0030] S5.3: Repeat the S5.2 process three times to obtain: a feature map T2 with 256 channels and a size of 64×64, a feature map T3 with 512 channels and a size of 32×32, and a feature map T4 with 1024 channels and a size of 16×16;

[0031] S5.4: Extract features from feature map T4, keep the number of channels and size unchanged, and then input it into the feature decoder;

[0032] S5.5: Pass the feature map output by the previous sub-step through the upsampling module to obtain a feature map with 512 channels and a size of 32×32 ;

[0033] S5.6: Advanced Features It is input into the feature weighted fusion module with the low-level feature T3 for weighted fusion, and then input into the feature extraction module of the same scale for processing to obtain a new feature map;

[0034] S5.7: Repeat S5.5-S5.6 three times to obtain a feature map with 64 channels and a size of 256×256;

[0035] S5.8: After the feature map obtained in S5.7 is passed through a 3×3 convolutional layer to restore the number of channels to 3, it is added to the input blurred image X to obtain the output Y;

[0036] S5.9: Back-propagate the deblurring model through the focused frequency loss, that is, introduce a loss function in the frequency domain to improve the quality of image generation. The expression of the loss function FFL is as follows:

[0037]

[0038]

[0039]

[0040]

[0041] In the formula, (u, v) represents the coordinate of the spatial frequency on the spectrum, ω(u, v) represents the weight of the spatial frequency at (u, v), and F r (u,v) is the spatial frequency value of the real image at the spectrum coordinate (u,v), F f (u,v) is the spatial frequency value of the model-generated image at the spectral coordinate (u,v); (x,y) represents the coordinate of the image pixel in the spatial domain, and f(x,y) represents the pixel value of the coordinate; the value of θ is [0, 1], and 1 represents the frequency with the largest current loss; e represents the natural logarithm, and i represents the imaginary number;

[0042] S5.10: Repeat the training process of S5.1-S5.9 until the set number of training times is reached to obtain a trained deblurring model.

[0043] Furthermore, in S1, the real scene is various water environments, including different lighting conditions and weather changes; the collected blurred ship plate image data includes images of passing ships taken at different angles and heights.

[0044] The beneficial effects of the present invention are:

[0045] (1) The ship plate deblurring technology based on multi-scale feature weighting in this invention first generates a clear blur dataset of the corresponding scene by estimating the blur kernel, so that the deblurring model can maintain good robustness in practical applications. This process ensures that the deblurring model can adapt to the degree of blur in various practical scenes, thereby improving its performance in complex environments.

[0046] (2) The present invention can effectively capture and integrate the correlation information between low-level features and high-level features through a multi-scale feature weighting mechanism. Through this mechanism, the deblurring model can more comprehensively understand the image information hierarchy and improve the quality and accuracy of image restoration. The application of the multi-scale feature weighting mechanism enables the deblurring model to not only identify the basic outline of the ship plate, but also better restore its detailed features, further improving the deblurring effect.

[0047] (3) The present invention combines a loss function based on the frequency domain to further improve the image reconstruction quality by narrowing the gap in the frequency domain. The loss function in the frequency domain enables the deblurring model to focus on the frequency components that are difficult to synthesize. The deblurring model can adaptively focus on the details that are easily overlooked during the reconstruction process, thereby restoring the original image more accurately when restoring the details of the ship license plate. Through this mechanism, the deblurring model not only improves the overall clarity and accuracy of the image, but also performs well in detail restoration. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flowchart of a ship plate deblurring method based on multi-scale feature weighting proposed in an embodiment of the present invention.

[0049] Figure 2 It is a structural block diagram of the fuzzy estimation network proposed in an embodiment of the present invention.

[0050] Figure 3 It is a structural block diagram of the defuzzification model proposed in an embodiment of the present invention.

[0051] Figure 4 It is a structural block diagram of the feature weighted fusion module proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be described in detail below according to the accompanying drawings and preferred embodiments, and the purpose and effect of the present invention will become more clear. The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0053] like Figure 1 As shown, a method for image deblurring based on multi-scale feature weighting specifically includes the following steps:

[0054] S1: Collect fuzzy ship license plate image data of real scenes. The collection of these fuzzy ship license plate image data is used as a preliminary data set for subsequent fuzzy estimation network. The real scene is a variety of water environments, which include different lighting conditions and meteorological changes. In order to capture diverse image data, the camera equipment should be arranged at different angles and heights to cover as many viewing angles as possible, and regularly capture and record images of ships passing by.

[0055] S2: Use the blur estimation network to estimate the base blur kernel and mixing coefficients from the preliminary dataset.

[0056] First, we introduce the blurring process of the image, which can be described as the convolution between the clear image and the blur kernel:

[0057]

[0058] In the formula, B represents the observed blurred image, S represents the clear image, k represents the blur kernel, and n represents the noise. Represents a convolution operation.

[0059] Using the fuzzy estimation network to estimate the blur kernel in the real scene, a blurred image can get a set of basic blur kernels and the mixing coefficients corresponding to each basic blur kernel. This is achieved through the following sub-steps:

[0060] S2.1: Build a fuzzy estimation network.

[0061] like Figure 2 As shown in the figure, the blur estimation network includes an encoder and a decoder. The encoder is used to extract the features of the input blurred image, including five downsampling modules, each of which includes two layers, each of which includes a 3×3 convolution layer and a LeakyReLU activation function. After each downsampling, the number of channels of the image is doubled and the size is halved, thereby achieving multi-scale feature extraction. There are two decoders, including a blur kernel head and a mixing coefficient head; the output of the final downsampling module passes through a 3×3 convolution block to refine the features, and then is input into the blur kernel head and mixing coefficient head of the decoder respectively. The blur kernel head is used to output a set of basic blur kernels, and the mixing coefficient head is used to output mixing coefficients.

[0062] Both the blur kernel head and the mixing coefficient head include five upsampling modules, each of which includes a bilinear upsampling layer and three 3×3 convolutional layers, and each convolutional layer is followed by a LeakyReLU activation function. There is a jump connection between the second convolutional layer in the downsampling module and the second convolutional layer in the upsampling module, and a global average pooling layer is set at the jump connection of the blur kernel head.

[0063] In the blur kernel head, the input of the jump connection is first subjected to global average pooling and then input into the upsampling module. The upsampling module performs bilinear upsampling on the input of the previous module and concatenates it with the result of global average pooling in the channel dimension, and then generates the output through three convolutional layers. After five upsampling modules, 3×3 convolution and softmax function are performed to obtain B blur kernels with a size of K×K; in this embodiment, B is set to 25 and K×K is set to 33×33.

[0064] The number and size of the basic blur kernels are selected according to the requirements: the size of the basic blur kernel (K×K) determines the size of the area considered around each pixel, while the number of basic blur kernels (B) determines the number of parameters required to represent the motion blur field. A smaller K value can reduce the computational complexity, but may not capture complex motion blur patterns; similarly, a smaller B value can reduce the number of parameters of the model, but may not accurately represent the diversity of the motion blur field.

[0065] The upsampling module of the mixing coefficient head does not perform global average pooling on the input of the skip connection, and the other operations are the same, that is, in the mixing coefficient head, the upsampling module performs bilinear upsampling on the input of the previous module, concatenates it with the input of the skip connection in the channel dimension, and then generates the output through three convolutional layers. After the last convolution, the softmax function is used to ensure that the sum of the mixing coefficients adds up to 1.

[0066] In specific applications, these mixing coefficients are used to linearly combine the basic blur kernels into the actual blur kernel, which can be specifically expressed as:

[0067]

[0068]

[0069]

[0070] In the formula, K actual (x 0 ,y 0 ) represents the position (x 0 ,y 0 ) at the actual blur kernel, α b (x 0 ,y 0 ) represents the mixing coefficient of the bth basic blur kernel at the corresponding position; K b represents the bth basic blur kernel, F kernel represents the feature vector input to the fuzzy kernel head, F mix Represents the feature vector input to the mixing coefficient head.

[0071] S3: Get a set of clear license plate images, use the basic blur kernel and mixing coefficient generated by S2 to process them respectively, get the corresponding blurred license plate images, and collect the clear-blurred license plate image pairs to form a clear-blurred data set; set the training set: validation set: test set = 9:1:1.

[0072] S4: Construct a deblurring model based on a multi-scale feature extraction network and a feature weighted fusion module, as follows:

[0073] S4.1: If Figure 3 As shown in the figure, the multi-scale feature extraction network consists of a feature encoder and a feature decoder.

[0074] The feature encoder has multiple layers (four layers are set in this embodiment), each layer includes a feature extraction module and a downsampling module. The feature extraction module is used to process the input features and pass the output to the next layer; the downsampling module is used to generate low-level features of different scales. The low-level features include basic image elements such as edges and textures.

[0075] The feature decoder has multiple layers, the same number of layers as the feature encoder. Each layer includes an upsampling module and a feature extraction module. The upsampling module is used to generate high-level features of different scales. High-level features cover more complex shapes and object structures than low-level features.

[0076] The feature encoder and feature decoder are connected by a feature extraction module.

[0077] The feature extraction module includes two residual modules, each of which includes a normalization layer (using layer normalization), two convolution layers, and a channel attention module. The input feature map passes through the normalization layer and two convolution layers in sequence. The convolution result is divided into two parts along the channel dimension and they are multiplied element by element to generate a feature map, which is then input into the channel attention module. The output of the channel attention module is multiplied with the feature map generated by element-by-element multiplication. The multiplication result is added to the input of the residual module as the output of the residual module and as the input of the next residual module.

[0078] S4.2: A feature weighted fusion module Fusion is set up based on the multi-scale feature extraction network. The first input of the feature weighted fusion module at each scale is the output of the feature extraction module at the same scale in the encoder (i.e., low-level features), and the second input is the output of the upsampling module at the same scale (i.e., high-level features); the weighted fusion features output by the feature weighted fusion module are used as the input of the feature extraction module at the same scale in the feature decoder (i.e., after the upsampling module).

[0079] like Figure 4 As shown in the figure, the feature weighted fusion module includes: a multi-attention module and a 1x1 convolutional layer.

[0080] The Multi-Attention module is used to perform weighted fusion of the input low-level features and high-level features. The Sigmoid activation function is used in the fusion process. The fused feature map passes through a 1x1 convolution layer and is input into the feature extraction module of the same scale.

[0081] The Multi-Attention module includes: spatial attention module, channel attention module, pixel attention module, and Sigmoid activation function. The spatial attention feature map S obtained by the spatial attention module att The expression is as follows:

[0082]

[0083] Where Conv2d( ) represents a two-dimensional convolution operation with reflection padding; Concat( ) represents concatenation in the channel dimension; X γNumerically equal to the sum of the low-level features and high-level features of the input X add The shape is (N, C, H, W), where N represents the batch of the input feature map, C represents the number of channels of the input feature map, H represents the height of the input feature map, and W represents the width of the input feature map.

[0084] Channel attention feature map C obtained by the channel attention module att The expression is as follows:

[0085]

[0086] In the formula, Conv2d(∙,∙,1) represents a two-dimensional convolution operation using a 1x1 convolution kernel, where the first parameter is the input feature map, the second parameter is the number of output channels, and the third parameter is the convolution kernel size; r is the reduction rate of the number of channels; X αβ Numerically the same as X add Equal, the shape is (N, C, H, W).

[0087] The pixel attention feature map ε obtained by the pixel attention module is expressed as follows:

[0088]

[0089] Using the pixel attention feature map output by the Multi-Attention module as the weight, the low-level features of the feature encoder part and the high-level features of the feature decoder part are weighted and summed. The expression is as follows:

[0090]

[0091] In the formula, C 1×1 represents 1×1 convolution, F low represents the low-level features generated by the feature encoder; F high represents the high-level features generated by the feature decoder. fuse Represents the weighted fusion features output by the feature weighted fusion module.

[0092] S5: Use the clear blur dataset generated by S3 to input the deblurring model constructed by S4 for training, which is specifically achieved through the following sub-steps:

[0093] S5.1: The input blurred image X with a size of 256×256 and 3 channels is passed through a 3×3 convolutional layer to generate a 64-channel feature map T.

[0094] S5.2: Perform feature extraction on the feature map T. The number of channels and size of the feature map after processing remain unchanged. Then, through the downsampling module, a feature map T1 with 128 channels and a size of 128×128 is obtained.

[0095] S5.3: Repeat the S5.2 process three times to obtain: feature map T2 with 256 channels and a size of 64×64, feature map T3 with 512 channels and a size of 32×32, and feature map T4 with 1024 channels and a size of 16×16.

[0096] S5.4: T4 is then further processed through the feature extraction module, keeping the number of channels and size unchanged, and then input into the decoder.

[0097] S5.5: The feature map output from the previous sub-step is passed through the upsampling module to obtain a feature map with 512 channels and a size of 32×32. .

[0098] S5.6: Advanced Features It is input into the feature weighted fusion module with the low-level feature T3 for weighted fusion, and then input into the feature extraction module of the same scale for processing to obtain a new feature map.

[0099] S5.7: Repeat S5.5-S5.6 three times to obtain a feature map with 64 channels and a size of 256×256.

[0100] S5.8: The feature map obtained in S5.7 is passed through a 3×3 convolutional layer to restore the number of channels to 3, and finally added to the original blurred image X to obtain the output Y.

[0101] S5.9: Back-propagate the deblurring model through the focal frequency loss (FFL), that is, introduce a loss function in the frequency domain to help improve the quality of image generation. The expression of the loss function FFL is as follows:

[0102]

[0103]

[0104]

[0105]

[0106] In the formula, (u, v) represents the coordinate of the spatial frequency on the spectrum, ω(u, v) represents the weight of the spatial frequency at (u, v), and F r (u,v) is the spatial frequency value of the real image at the spectrum coordinate (u,v), F f(u,v) is the spatial frequency value of the model-generated image at the spectral coordinate (u,v). (x,y) represents the coordinate of the image pixel in the spatial domain, and f(x,y) represents the pixel value of the coordinate; the value of θ is [0, 1], and 1 represents the frequency with the largest current loss; e represents the natural logarithm, and i represents the imaginary number, following the Euler formula.

[0107] S5.10: Repeat the training process of S5.1-S5.9 until the set number of training times is reached to obtain a trained deblurring model; in this embodiment, the training is set to 400,000 rounds.

[0108] S6: Input the blurred image to be processed into the trained deblurring model, and finally output a clear image. The method of the present invention can improve the reliability of ship plate recognition and provide strong support for the intelligent development and management of inland shipping.

[0109] The method of the present invention combines multi-scale feature extraction and weighting mechanism, and can perform layered processing on different frequency features in the license plate image, thereby enhancing the retention of high-frequency information and improving the detail restoration effect. Compared with traditional deblurring technology and existing deep learning models, this method can not only retain more details in various blurry scenes, but also more effectively cope with complex environmental changes, thereby significantly improving the accuracy and robustness of license plate recognition in the field of inland shipping. The method of the present invention helps to improve the transportation efficiency of inland waters, reduce the risk of accidents, provide important support for the technological development of smart shipping, and has broad application prospects.

[0110] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention shall be included in the protection scope of the invention.

Claims

1. An image deblurring method based on multi-scale feature weighting, characterized in that: The following steps are involved: S1: Collect fuzzy ship license plate image data of real scenes to obtain a preliminary data set; S2: Use a fuzzy estimation network to estimate the basic blur kernel and mixing coefficient from the preliminary data set; the fuzzy estimation network includes an encoder and a decoder, the encoder includes five downsampling modules, each downsampling module has two layers, each layer includes a convolution layer and a LeakyReLU activation function; the output of the encoder is input into the blur kernel head and mixing coefficient head of the decoder after passing through a convolution block, the blur kernel head is used to output a set of basic blur kernels, and the mixing coefficient head is used to output the mixing coefficient; the blur kernel head and the mixing coefficient head each include five upsampling modules, each upsampling module includes a bilinear upsampling layer and three convolution layers; there is a jump connection between the second convolution layer in the downsampling module and the second convolution layer in the upsampling module, and a global average pooling layer is set at the jump connection of the blur kernel head; S3: Obtain a set of clear ship license plate images, and use the basic blur kernel and the mixing coefficient to convert the clear ship license plate images into blurry ship license plate images to obtain a clear blurry data set; S4: constructing a deblurring model based on a multi-scale feature extraction network and integrating a feature weighted fusion module; the multi-scale feature extraction network includes a feature encoder and a feature decoder, the feature encoder has multiple layers, and each layer includes a feature extraction module and a downsampling module; The feature decoder has multiple layers, each of which includes an upsampling module and a feature extraction module; The feature encoder and feature decoder are connected by a feature extraction module; the first input of the feature weighted fusion module at each scale is the output of the feature extraction module at the same scale in the feature encoder, that is, the low-level feature, and the second input is the output of the upsampling module at the same scale, that is, the high-level feature; after the feature weighted fusion module performs weighted fusion on the input, the output weighted fusion feature is used as the input of the feature extraction module at the same scale in the feature decoder; In S4, the feature weighted fusion module includes a multi-head attention module and a 1x1 convolution layer; the multi-head attention module is used to perform weighted fusion on the input low-level features and high-level features, and a Sigmoid activation function is used in the fusion process. The fused feature map passes through a 1x1 convolution layer and is input into a feature extraction module of the same scale; The multi-head attention module includes: a spatial attention module, a channel attention module, and a pixel attention module; The spatial attention feature map S obtained by the spatial attention module att The expression is as follows: Where Conv2d() represents a two-dimensional convolution operation with reflection padding; Concat() represents concatenation in the channel dimension; X γ Numerically equal to the sum of the low-level features and high-level features of the input X add Equal, the shape is (N, C, H, W), N represents the batch of input feature maps, C represents the number of channels of the input feature map, H represents the height of the input feature map, and W represents the width of the input feature map; Channel attention feature map C obtained by the channel attention module att The expression is as follows: In the formula, Conv2d(·,·,1) represents a two-dimensional convolution operation using a 1x1 convolution kernel, where the first parameter is the input feature map, the second parameter is the number of output channels, and the third parameter is the convolution kernel size; r is the reduction rate of the number of channels; X αβ Numerically the same as X add Equal, shape is (N, C, H, W); The pixel attention feature map ε obtained by the pixel attention module is expressed as follows: ε=Sigmoid(Conv2d(concat(X add ,S att +C att ))) Taking the pixel attention feature map as the weight, the low-level features and high-level features are weighted and summed to obtain the weighted fusion feature F fuse The expression is as follows: F fuse =C 1×1 (F low ·ε+F high ·(1-ε)+F low +F high ) In the formula, C 1×1 represents 1×1 convolution, F low represents the low-level features generated by the feature encoder; F high Represents the high-level features generated by the feature decoder; S5: training the deblurring model using the clear fuzzy data set, and optimizing the deblurring model by a loss function based on the frequency domain; S6: Input the blurred image to be processed into the trained deblurring model to obtain a clear image.

2. The image deblurring method based on multi-scale feature weighting according to claim 1, characterized in that: In S4, the feature extraction module includes two residual modules, each residual module includes a normalization layer, two convolution layers, and a channel attention module; the input feature map passes through the normalization layer and the two convolution layers in sequence, and the convolution result is divided into two parts along the channel dimension and multiplied element by element to generate a feature map, which is then input into the channel attention module, and the output of the channel attention module is multiplied with the feature map, and the multiplication result is added to the input of the residual module as the output of the residual module and as the input of the next residual module.

3. The image deblurring method based on multi-scale feature weighting according to claim 1, characterized in that: S5 is specifically implemented through the following sub-steps: S5.1: The blurred image X with a size of 256×256 and a number of channels of 3 is input into the deblurring model. It first passes through a 3×3 convolutional layer to generate a feature map T with a number of channels of 64. S5.2: Extract features from the feature map T. The number of channels and size of the processed feature map remain unchanged. Then, through the downsampling module, a feature map T1 with 128 channels and a size of 128×128 is obtained. S5.3: Repeat the S5.2 process three times to obtain: a feature map T2 with 256 channels and a size of 64×64, a feature map T3 with 512 channels and a size of 32×32, and a feature map T4 with 1024 channels and a size of 16×16; S5.4: Extract features from feature map T4, keep the number of channels and size unchanged, and then input it into the feature decoder; S5.5: Pass the feature map output by the previous sub-step through the upsampling module to obtain a feature map with 512 channels and a size of 32×32 S5.6: Advanced Features It is input into the feature weighted fusion module with the low-level feature T3 for weighted fusion, and then input into the feature extraction module of the same scale for processing to obtain a new feature map; S5.7: Repeat S5.5-S5.6 three times to obtain a feature map with 64 channels and a size of 256×256; S5.8: After the feature map obtained in S5.7 is passed through a 3×3 convolutional layer to restore the number of channels to 3, it is added to the input blurred image X to obtain the output Y; S5.9: Back-propagate the deblurring model through the focused frequency loss, that is, introduce a loss function in the frequency domain to improve the quality of image generation. The expression of the loss function FFL is as follows: ω(u,v)=|F r (u,v)-F f (u,v)| θ In the formula, H represents the height of the input feature map, W represents the width of the input feature map; (u, v) represents the coordinates of the spatial frequency on the spectrum, ω(u, v) represents the weight of the spatial frequency at (u, v), and F r (u,v) is the spatial frequency value of the real image at the spectrum coordinate (u,v), F f (u,v) is the spatial frequency value of the model-generated image at the spectral coordinate (u,v); (x,y) represents the coordinate of the image pixel in the spatial domain, and f(x,y) represents the pixel value of the coordinate; the value of θ is [0, 1], and 1 represents the frequency with the largest current loss; e represents the natural logarithm, and i represents the imaginary number; S5.10: Repeat the training process of S5.1-S5.9 until the set number of training times is reached to obtain a trained deblurring model.

4. The image deblurring method based on multi-scale feature weighting according to claim 1, characterized in that: In S1, the real scene is various water environments, including different lighting conditions and weather changes; the collected blurred ship plate image data includes images of passing ships taken at different angles and heights.

Images