Method, device, equipment and storage medium for restoring uniform motion blur of detected images

Abstract

The present application relates to a method, device, equipment and storage medium for restoring uniform motion blur of a detection image. The method inputs a single-channel motion blur detection image into a network, performs feature extraction through an encoder, performs feature enhancement through a multi-scale dilated convolution module and performs feature restoration through a decoder to obtain a clear detection image. An end-to-end motion blur restoration algorithm based on multi-scale dilated convolution and cross-scale feature fusion directly learns the mapping relationship between the blurred image and the clear image, thereby avoiding the blur kernel estimation process and the ringing effect; a multi-scale dilated convolution module is used to extract blur features of different scales, thereby improving the network's adaptability to input images of different blur scales; a cross-scale feature fusion module is used to weight and optimize the decoded features from both the channel domain and the spatial domain, thereby helping the network to extract important features, learn the mapping relationship more accurately, and improve the accuracy of uniform motion blur restoration of the detection image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for restoring uniform motion blur of a detected image. Background Art

[0002] When using an optoelectronic telescope to detect and track space targets, in some modes, the background stars and the telescope will move relative to each other within a limited exposure time, causing the stars to be imaged as strip-shaped tails on the image plane. In addition, the imaging process will be affected by various noises, such as readout noise, thermal noise, and dark current noise, and the final output image is a uniform motion-blurred image containing noise. In the motion-blurred image, the star energy is dispersed, the signal-to-noise ratio decreases, the boundary between the star and the background also becomes blurred, and the center of mass positioning result is offset, affecting the subsequent astronomical positioning accuracy. Therefore, motion-blur restoration of star detection images is the basis for achieving astronomical positioning of space targets.

[0003] The blur scale of star detection images will change with the speed of relative motion and exposure time, and noise interference is common in images. Therefore, the restoration algorithm needs to be able to adapt to input images with different blur scales and effectively suppress the interference of noise on the restoration process. However, most traditional star image motion blur restoration algorithms have poor adaptability to input images with different blur scales and are greatly affected by noise interference, resulting in inaccurate blur kernel estimation and ringing effects in restored images.

[0004] Most traditional star image motion blur restoration algorithms have poor adaptability to input images of different blur scales and are greatly affected by noise interference, resulting in inaccurate blur kernel estimation. The interference of noise causes ringing effects when some algorithms restore images. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, equipment and storage medium for restoring uniform motion blur of detected images in response to the above technical problems. This method does not require prior information of the image, does not need to estimate the blur kernel based on image adjustment parameters, and achieves the restoration of blurred images to clear images in an end-to-end manner, and has achieved good results in both simulated image and real image tests.

[0006] A method for restoring uniform motion blur of a detected image, the method comprising:

[0007] The acquired star detection image is input into the encoder to obtain shallow features, second-layer features and deep features; the encoder is used to extract features of different levels of the star detection image using multiple convolution operations and downsampling operations.

[0008] The deep features are input into the multi-scale dilated convolution module to obtain multi-scale contextual features. The multi-scale dilated convolution module is used to extract the multi-scale features of the deep features using convolution kernels with different dilation rates, and a hybrid attention mechanism is used to perform weighted tuning in the channel domain and spatial domain on the multi-scale features to extract multi-scale contextual features.

[0009] The star detection image, shallow features, second-layer features and multi-scale context features are input into the decoder to obtain a final clear detection image; wherein the decoder is used to perform convolution and upsampling on the multi-scale context features and then fuse them with the second-layer features through a cross-scale feature fusion module, perform convolution and upsampling on the obtained fused features and then fuse them with the shallow features through a cross-scale feature fusion module, and perform convolution on the obtained fused features and then perform element-wise addition on the star detection image to obtain a final clear detection image.

[0010] A detection image uniform motion blur restoration device, the device comprising:

[0011] The feature encoding module is used to input the acquired star detection image into the encoder to obtain shallow features, second-layer features and deep features; the encoder is used to extract different levels of features of the star detection image using multiple convolution operations and downsampling operations.

[0012] The feature enhancement module is used to input the deep features into the multi-scale dilated convolution module to obtain multi-scale contextual features. The multi-scale dilated convolution module is used to extract the multi-scale features of the deep features using convolution kernels with different dilation rates, and to perform weighted tuning of the multi-scale features in the channel domain and the spatial domain using a hybrid attention mechanism to extract multi-scale contextual features.

[0013] The image restoration module is used to input the star detection image, shallow features, second-layer features and multi-scale context features into the decoder to obtain the final clear detection image; wherein the decoder is used to perform convolution and upsampling on the multi-scale context features and then fuse them with the second-layer features through a cross-scale feature fusion module, perform convolution and upsampling on the obtained fused features and then fuse them with the shallow features through a cross-scale feature fusion module, and perform element-wise addition of the obtained fused features with the star detection image after convolution processing to obtain the final clear detection image.

[0014] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned detection image uniform motion blur restoration method when executing the computer program.

[0015] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for restoring uniform motion blur of a detected image.

[0016] The above-mentioned detection image uniform motion blur restoration method, device, equipment and storage medium, the method inputs a single-channel motion blur detection image into the network, performs feature extraction through the encoder, performs feature enhancement through the multi-scale dilated convolution module and performs feature restoration through the decoder, and obtains the final clear detection image. The end-to-end motion blur restoration algorithm based on multi-scale dilated convolution and cross-scale feature fusion directly learns the mapping relationship between the blurred image and the clear image, thereby avoiding the blur kernel estimation process and avoiding the ringing effect; extracts blur features of different scales through the multi-scale dilated convolution module, thereby improving the network's adaptability to input images of different blur scales; and performs weighted tuning of the decoded features from both the channel domain and the spatial domain through the cross-scale feature fusion module, helping the network to extract important features, learn the mapping relationship more accurately, and improve the accuracy of uniform motion blur restoration of the detection image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a process of restoring uniform motion blur of a detected image in one embodiment;

[0018] Figure 2 The overall structure of a detection image uniform motion blur restoration model in another embodiment;

[0019] Figure 3 Schematic diagram of dilated convolution in another embodiment, wherein (a) is a schematic diagram of dilated convolution with a dilation rate of 1, (b) is a schematic diagram of dilated convolution with a dilation rate of 2, and (c) is a schematic diagram of dilated convolution with a dilation rate of 4;

[0020] Figure 4 It is a structural diagram of a multi-scale dilated convolution module in another embodiment;

[0021] Figure 5 Schematic diagram of a cross-scale feature fusion module based on a hybrid attention mechanism in another embodiment;

[0022] Figure 6 1. A clear star map and different types of motion blurred star maps in another embodiment, wherein (a) is a point-shaped clear star map, (b) is a horizontal motion blurred star map, (c) is a vertical motion blurred star map, and (d) is a motion blurred star map in an inclined direction;

[0023] Figure 71 is a comparison of the deblurring effect of a short-tailed simulated star map in another embodiment, wherein (a) is a clear star map, (b) is a blurred star map, (c) is a deblurring effect diagram of the comparison algorithm 1, (d) is a deblurring effect diagram of the comparison algorithm 2, (e) is a deblurring effect diagram of MIMO-UNet, and (f) is a deblurring effect diagram of REDNet;

[0024] Figure 8 3D grayscale images of local star restoration results of a short-tailed simulated star map using different algorithms in another embodiment, wherein (a) is a clear star map, (b) is a blurred star map, (c) is a 3D grayscale image of the restoration result of comparison algorithm 1, (d) is a 3D grayscale image of the restoration result of comparison algorithm 2, (e) is a 3D grayscale image of the restoration result of MIMO-UNet, and (f) is a 3D grayscale image of the restoration result of REDNet;

[0025] Fig. 9 1 is a comparison of the deblurring effects of a simulated star image with a medium and long tail in another embodiment, wherein (a) is a clear star image, (b) is a blurred star image, (c) is a blurred effect image of the comparison algorithm 1, (d) is a blurred effect image of the comparison algorithm 2, (e) is a blurred effect image of MIMO-UNet, and (f) is a blurred effect image of REDNet;

[0026] Fig.10 3D grayscale images of local star restoration results of a long-tailed simulated star map using different algorithms in another embodiment, wherein (a) is a clear star map, (b) is a blurred star map, (c) is a 3D grayscale image of the restoration result of comparison algorithm 1, (d) is a 3D grayscale image of the restoration result of comparison algorithm 2, (e) is a 3D grayscale image of the restoration result of MIMO-UNet, and (f) is a 3D grayscale image of the restoration result of REDNet;

[0027] Fig.11 1 is a comparison of the deblurring effect of a long-tailed simulated star map in another embodiment, wherein (a) is a clear star map, (b) is a blurred star map, (c) is a blurred effect diagram of comparison algorithm 1, (d) is a blurred effect diagram of comparison algorithm 2, (e) is a blurred effect diagram of MIMO-UNet, and (f) is a blurred effect diagram of REDNet;

[0028] Fig.12 3D grayscale images of local star restoration results of a long-tailed simulated star map using different algorithms in another embodiment, wherein (a) is a clear star map, (b) is a blurred star map, (c) is a 3D grayscale image of the restoration result of comparison algorithm 1, (d) is a 3D grayscale image of the restoration result of comparison algorithm 2, (e) is a 3D grayscale image of the restoration result of MIMO-UNet, and (f) is a 3D grayscale image of the restoration result of REDNet;

[0029] Fig.13It is a structural block diagram of a device for restoring uniform motion blur of a detected image in one embodiment;

[0030] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] In one embodiment, Figure 1 As shown, a method for restoring uniform motion blur of a detected image is provided, and the method comprises the following steps:

[0033] Step 100: Input the acquired star detection image into the encoder to obtain shallow features, second-layer features and deep features; the encoder is used to extract features of different levels of the star detection image using multiple convolution operations and downsampling operations.

[0034] Specifically, the overall structure of the detection image uniform motion blur restoration model (abbreviated as REDNet) proposed in this application is as follows: Figure 2 The detection image uniform motion blur restoration model includes an encoder, a multi-scale hole convolution module and a decoder.

[0035] Figure 2 The encoder is on the far left, with three layers from top to bottom. Each layer consists of a 3×3 convolution layer and 8 stacked residual blocks. The difference is that the stride of the 3×3 convolution layer in the first layer is 1, mainly to extract shallow features from the image. The stride of the 3×3 convolution layer in the last two layers is 2, mainly to downsample the features. Downsampling can expand the receptive field range, allowing the network to perceive a wider range of motion blur.

[0036] The star detection images input to the network are single-channel motion blurred detection images.

[0037] Step 102: Input the deep features into the multi-scale dilated convolution module to obtain multi-scale context features; the multi-scale dilated convolution module is used to extract multi-scale features of the deep features using convolution kernels with different dilation rates, and use a hybrid attention mechanism to perform weighted tuning in the channel domain and the spatial domain on the multi-scale features to extract multi-scale context features.

[0038] Specifically, the deep features extracted by the third layer of the encoder (i.e., deep features) are followed by a multi-scale dilated convolution module, which uses convolution kernels with different dilation rates to extract features, further expand the receptive field of the network, and provide multi-scale contextual information for subsequent decoder feature restoration.

[0039] Step 104: input the star detection image, shallow features, second-layer features, and multi-scale context features into the decoder to obtain a final clear detection image; wherein the decoder is used to perform convolution and upsampling processing on the multi-scale context features and then fuse them with the second-layer features through a cross-scale feature fusion module, perform convolution and upsampling processing on the obtained fused features and then fuse them with the shallow features through a cross-scale feature fusion module, perform convolution processing on the obtained fused features and then perform element-wise addition on the star detection image to obtain a final clear detection image.

[0040] Specifically, Figure 2 The rightmost part of the decoder is symmetrical with the encoder. There are three layers from bottom to top, each of which consists of 8 stacked residual blocks and 1 convolution layer. The convolution layers of the two layers below the decoder are transposed convolutions of size 4 and stride 2, which are mainly used to upsample the feature map so that the resolution of the feature map is restored to the original dimension and the number of feature channels is reduced. The top convolution layer is a normal convolution, which is mainly used to restore the feature map to a deblurred single-channel star map. The entire decoder gradually restores the low-resolution feature map to a high-resolution image of the same size as the original image, reconstructs the image details and eliminates blur.

[0041] The encoder and decoder of the network both use residual blocks as basic modules. The residual block consists of two convolutional layers, a rectified linear unit (ReLU) and a jump connection. The residual structure can build a deeper network architecture through jump connections, effectively solving the problem of network degradation. At the same time, given the similarity in pixel values ​​between blurry images and clear images, it is reasonable for the network to learn the difference between the two. In addition, the network establishes a jump connection between the upper two layers of the encoder and decoder. Because the deep features extracted by the third layer of the encoder have undergone downsampling and multiple convolution operations, the receptive field is expanded and contains more semantic information, but lacks the spatial detail information of the shallow layer of the image. If only the deep features are input into the decoder for feature restoration, the image restored by the network will lose a lot of image details. The jump connection between the encoder and the decoder can directly provide the shallow features of the image to the decoder, so that the decoder can retain the image details when restoring the image.

[0042] In the above-mentioned detection image uniform motion blur restoration method, the method inputs a single-channel motion blur detection image into the network, performs feature extraction through the encoder, performs feature enhancement through the multi-scale dilated convolution module, and performs feature restoration through the decoder to obtain the final clear detection image. The end-to-end motion blur restoration algorithm based on multi-scale dilated convolution and cross-scale feature fusion directly learns the mapping relationship between the blurred image and the clear image, thereby avoiding the blur kernel estimation process and avoiding the ringing effect; extracts blur features of different scales through the multi-scale dilated convolution module, thereby improving the network's adaptability to input images of different blur scales; and weights and optimizes the decoded features from both the channel domain and the spatial domain through the cross-scale feature fusion module, helping the network to extract important features, learn the mapping relationship more accurately, and improve the accuracy of uniform motion blur restoration of the detection image.

[0043] In one embodiment, the encoder includes: three encoding modules, wherein the first encoding module includes a convolution layer and a first residual group; the second and third encoding modules both include: a downsampling convolution layer and a second residual group; step 100 includes: inputting the acquired star detection image into the first encoding module of the encoder, and obtaining shallow features after being processed by the convolution layer and the first residual group; inputting the shallow features into the second encoding module of the encoder, and obtaining second features after being processed by the downsampling convolution layer and the second residual group; and inputting the second features into the third encoding module of the encoder to obtain deep features.

[0044] In one embodiment, the first residual group and the second residual group each include 8 stacked residual blocks; the residual block includes two convolutional layers, a ReLU and a skip connection.

[0045] In one embodiment, the multi-scale hole convolution module includes: three hole convolution layers with different hole rates, a point convolution layer and a hybrid attention mechanism; wherein the hybrid attention mechanism includes a channel attention module and a spatial attention module; the channel attention module includes a global pooling layer, a first point convolution layer and a second point convolution layer; the spatial attention module includes: a maximum pooling layer, an average pooling layer and a convolution layer; step 102 includes: inputting the deep features into the three hole convolution layers respectively, splicing the three hole convolution results obtained and inputting them into the point convolution layer to obtain convolution features; inputting the convolution features into the channel attention module of the hybrid attention mechanism, and obtaining the channel weighted optimized features as follows:

[0046] (1)

[0047] (2)

[0048] (3)

[0049] (4)

[0050] in, represents the features after channel weighted optimization, represents the channel weight coefficient, represents the point convolution layer, represents the ReLU function, represents the Sigmoid function, Representative sub-feature map exist The pixel value at represents the global average pooling operation, Representative c Sub-feature map The proportion of all feature maps; z represents the weight vector, , subscript C Representative feature map The number of channels, Represents the convolution feature, H and W represent the height and width of the feature map respectively.

[0051] The channel weighted optimized features are input into the spatial attention module of the hybrid attention mechanism, and the spatial weighted features are obtained as follows:

[0052] (5)

[0053] (6)

[0054] in, represents the spatially weighted feature, represents maximum pooling, represents average pooling, represents the spatial attention map, represents a convolution operation with a convolution kernel size of 7×7, Represents the Sigmoid activation function.

[0055] The spatially weighted features are added to the deep features to obtain multi-scale contextual features.

[0056] Specifically, when the photoelectric telescope is in different working conditions, the relative motion speed and exposure time between the stars and the image sensor may be different, resulting in different scales of motion blur in different star detection images taken by the sensor. To perceive the blurred features in the image, it is necessary to capture the contextual information of the image. The so-called contextual information refers to the relationship between a certain pixel area in the image and its surrounding pixel areas. The pixels in the image do not exist in isolation, but are interconnected with the surrounding pixels. It is the relationship between many pixels that constitutes a complete image. The size of the neuron's receptive field reflects the size of the area of ​​the original image that it can cover. Neurons with larger receptive fields can integrate more extensive contextual information, thereby containing higher-level semantic features; conversely, neurons with smaller receptive fields tend to capture more local and detailed features. Therefore, expanding the receptive field can expand the perception of image context information, thereby capturing blurred features in the image and enabling the network to learn blur parameters more accurately.

[0057] Existing methods usually perform pooling or downsampling operations to reduce the size of feature maps, thereby increasing the receptive field, and then restore the reduced feature map to the original image size through upsampling operations. The process of reducing and then enlarging the feature map will cause loss of spatial detail information. However, dilated convolution can expand the receptive field without pooling and reduce information loss. Figure 3 The dilated convolutions with different dilation rates are shown, where (a) is a dilated convolution with a dilation rate of 1, (b) is a dilated convolution with a dilation rate of 2, and (c) is a dilated convolution with a dilation rate of 4. Figure 3 The network input size is 28×28, and is used based on the input Figure 3 In (a), the dilated convolution has a dilated rate of 1 and a receptive field of 3×3. Figure 3 (a) Output based on the use of Figure 3 In (b), the dilated convolution with a dilated rate of 2 has a receptive field of 7×7. Figure 3 (b) Output based on the use of Figure 3 In (c), the dilated convolution with a dilated rate of 4 has a receptive field of 15×15. Figure 3 It can be seen that the dilated convolution can increase the receptive field without reducing the size of the feature map.

[0058] In order to adapt to input images of different blur scales, the method of the present application proposes a multi-scale dilated convolution module in the feature enhancement stage in the middle of the network. This module samples the given input with dilated convolutions at different sampling rates in parallel to capture feature information of different scales, splices the results together to capture the contextual information of the image, and then uses a hybrid attention mechanism to perform weighted tuning of the multi-scale features in the channel domain and the spatial domain. Star images of different blur scales may require the network to capture blur features of different scales, and the multi-scale dilated convolution module can meet this requirement. For blurred stars of different sizes, the network can capture image context information and learn the blur style of the image. The specific structure of the module is as follows: Figure 4 shown.

[0059] First, the input feature map The image is sent to three dilated convolutional layers with dilation rates of 1, 3, and 5 respectively. After convolution processing, the image features are concatenated in the channel dimension to obtain multi-scale dilated convolution features. Then through a The convolution layer further integrates and reduces the multi-scale hole convolution features to obtain .

[0060] (7)

[0061] (8)

[0062] in, represents the feature concatenation operation, represents a dilated convolutional layer with a dilation rate of 1. Represents a dilated convolutional layer with a dilation rate of 3. represents a dilated convolutional layer with a dilation rate of 5. represent Convolutional layer.

[0063] Assumptions Is has The feature map of the dimensional channel, that is , the size of each sub-feature map is First, Perform global average pooling to compress the spatial dimensions of the sub-feature maps and obtain the weight coefficients of each sub-feature map. The calculation process is shown in formula (4).

[0064] Next, two 1×1 convolutional layers are used to transform the weight vector Calculate and get the adjusted weight vector , and its calculation process is shown in formula (2).

[0065] Finally, the calculated channel weight coefficient With multi-scale features Multiply and weight the features of different channels to obtain the channel weighted optimized features .

[0066] (9)

[0067] in It is a multi-scale feature No. c The feature map of the channel, It is c The weight coefficient of the channel, It is the optimization feature No. c Feature map of the channel.

[0068] After channel attention processing, the features Average pooling and maximum pooling operations are performed in the channel dimension to obtain two compressed feature maps: average pooling feature With the maximum pooling feature The two compressed features are concatenated along the channel dimension, and the concatenated features are convolved with a 7×7 convolutional layer to generate a spatial attention map. Finally, the spatial attention map is multiplied element-wise with the channel-weighted features to obtain the spatially weighted features , and its calculation process is shown in formula (5) to formula (6).

[0069] Finally, the optimized multi-scale features are connected through skip connections. With the original input features Add together to get the multi-scale context features of the final output .

[0070] (10)

[0071] in, represents multi-scale context features, represents the spatially weighted feature, Feature maps representing the input to a multi-scale atrous convolution module.

[0072] In one embodiment, the decoder includes three decoding modules; wherein the third decoding module includes a third residual group and a first upsampling layer, the second decoding module includes a first cross-scale feature fusion module, a fourth residual group and a second upsampling layer, and the first decoding module includes a second cross-scale feature fusion module, a fifth residual group and a first convolutional layer; step 104 includes: inputting the multi-scale context feature into the third decoding module, and obtaining a first intermediate decoding feature after passing through the third residual group and the first upsampling layer; inputting the first intermediate decoding feature and the second layer feature into the first cross-scale feature fusion module of the second decoding module to obtain a first fused feature; inputting the fused feature into the fourth residual group of the second decoding module, and inputting the obtained feature into the second upsampling layer to obtain a second intermediate decoding feature; inputting the second intermediate decoding feature and the shallow layer feature into the first decoding module, and obtaining a third intermediate decoding feature after passing through the second cross-scale feature fusion module, the fifth residual group and the first convolutional layer; element-wise adding the third intermediate decoding feature to the star detection image to obtain a final clear detection image.

[0073] In one embodiment, the first intermediate decoding feature and the second layer feature are input into the first cross-scale feature fusion module of the second decoding module to obtain the first fused feature, including: concatenating the first intermediate decoding feature and the second layer feature and performing a point convolution operation to obtain a feature with half the number of feature channels; calculating the channel weight coefficient of the feature with half the number of feature channels through a channel attention mechanism, multiplying the obtained channel weight coefficient and the feature with half the number of feature channels to obtain a first channel weighted feature; adjusting the weights of different pixels of the first channel weighted feature through a spatial attention mechanism, multiplying the obtained pixel weight and the first channel weighted feature to obtain an optimized feature; and adding the optimized feature and the first intermediate decoding feature element by element to obtain the first fused feature.

[0074] Specifically, in actual situations, the star detection images taken by observation equipment contain a lot of noise. Although the shallow feature map extracted by the encoder contains more spatial position information, its receptive field is small, the semantic information is weak, and there is a lot of background noise. The existing encoder-decoder structure uses jump connections to transfer the shallow features extracted by the encoder to the decoder part, and the shallow features are fused with the upsampled decoded features through simple feature splicing. This operation easily causes the background noise to continue to propagate, resulting in large fluctuations in the background noise in the restored image.

[0075] Therefore, this application proposes a cross-scale feature fusion module, which enhances the decoded features based on the hybrid attention mechanism and reduces the impact of background noise on star map restoration. The structure of the cross-scale feature fusion module is as follows: Figure 5As shown in Figure 2. The hybrid attention mechanism includes the channel attention mechanism and the spatial attention mechanism. The channel attention mechanism can improve the adaptability of the network to blurred images of different scenes and scales. When processing blurred images of different scenes or blurred images of different scales, the channel attention mechanism is used to calculate the channel weight coefficient, and the channels of the image features are re-weighted to adjust the importance of the corresponding channels, which can improve the network's response to key features, especially in the face of noisy data or in the early layers of the network. The introduction of the channel attention mechanism enables the network to enhance the expression ability of features while keeping the number of parameters relatively small, and improves the ability to suppress background noise interference. The spatial attention mechanism learns the relationship between different pixels in the image and calculates the spatial attention map to highlight the important features in the image, thereby giving higher weights to the blurred area during network processing, reducing noise interference, and thus improving the deblurring ability. Finally, the weighted optimized features are fused with the upsampled decoded features by element addition to further enhance the key position information in the spatial dimension.

[0076] The output features of the encoder after skip connection Features after upsampling with the decoder The concatenated features are obtained by splicing, and then a convolution operation with a convolution kernel size of 1×1 is performed to achieve feature dimensionality reduction, obtaining features with half the number of feature channels. .

[0077] (11)

[0078] in, represents the feature concatenation operation, The convolution kernel represents a 1×1 dimensionality reduction operation.

[0079] The reduced features are then fed into the hybrid attention mechanism. The channel attention mechanism is used to adjust the weights of different channels, enhance important channel features, and suppress irrelevant channel features. The spatial attention mechanism is then used to adjust the weights of different pixels, so that the network focuses on the fuzzy features in the image and obtains the optimized features. . Features optimized by two attention mechanisms Features after upsampling with the decoder Add element by element to make up for the loss of spatial detail information of the decoded feature after upsampling and achieve feature enhancement. Then input the enhanced features into the subsequent residual group to achieve feature decoding.

[0080] In one of the embodiments, the third residual group, the fourth residual group and the fifth residual group in the decoder are all composed of 8 stacked residual blocks, and the convolution layer in the fifth residual group is a normal convolution; the first upsampling layer and the second upsampling layer both use upsampling transposed convolution.

[0081] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0082] In a verification example, this example is based on the Tycho2 star catalog and uses star map simulation software to simulate 200 star maps, of which 160 are used to make a training set, 20 are used to make a validation set, and 20 are used to make a test set.

[0083] The motion blur generation method proposed by Gong et al. is used to simulate the motion blur effect by applying a point spread function of uniform linear motion to each pixel in the image. For each star map used in the training set and the validation set, this embodiment simulates five different motion blur situations, which have different motion speeds, directions, and tail lengths. The tail length is between 0 and 90 pixels. A total of 800 training sets and 100 validation sets are generated. In order to test the restoration performance of the algorithm under different blur lengths, this embodiment divides the tail length into three categories, namely, short tail of 0 to 20 pixels, medium tail of 20 to 40 pixels, and long tail of 40 to 80 pixels. For each type of tail length, 100 test sets are made, and a total of 300 test sets are generated.

[0084] In addition, in order to simulate the noise interference of the star map in the actual acquisition process, this embodiment adds Gaussian white noise with a mean of 0 and a standard deviation of 10 to the generated blurred star map. In order to evaluate the performance of the detection image uniform motion blur restoration network model proposed in this application, a simulation data set is used for testing in the test phase. Figure 6 It shows clear star images and different types of motion blurred star images, where (a) is a point-like clear star image, (b) is a horizontal motion blurred star image, (c) is a vertical motion blurred star image, and (d) is a motion blurred star image in an inclined direction.

[0085] (1) Experimental environment and parameter settings

[0086] This embodiment is based on the Ubuntu 18.04 operating system and uses the Python-based PyTorch deep learning framework for experiments. The experimental hardware configuration includes an AMD Ryzen 9 3900X processor (core frequency 3.8 GHz) and an NVIDIA GeForce RTX 3090 GPU. In the model training phase, this subject uses the Adam optimization algorithm and sets the batch size to 4. Each training sample is cropped to a size of 256×256 pixels. The loss function uses the L1 loss function. The initial learning rate is set to 0.0001, and the cosine annealing strategy is used to adjust the learning rate. The minimum learning rate is set to 0.000001, and the training rounds are 200. Unlike the image resolution of 256×256 in the training phase, in the testing phase, the simulation test set maintains the original image resolution for testing.

[0087] (2) Comparative experimental results and evaluation

[0088] This application compares the proposed REDNet with existing algorithms to evaluate the effectiveness of the proposed algorithm. The comparison algorithms include two traditional star image motion blur restoration algorithms and an image deblurring algorithm based on deep learning. The method proposed by Jiang et al. in the literature 1 (Jiang J, Huang JN, Zhang G J. An Accelerated Motion Blurred Star Restoration Based on Single Image [J]. IEEE Sensors Journal, 2017, 17(5):1306~1315) (comparison algorithm 1) first performs grayscale stretching on the blurred star image, and then performs Fourier transform on the grayscale stretched star image to obtain a spectrum map. Then, the motion blur angle and motion blur length are estimated by analyzing the spectrum map through Laden transform. Hou et al. proposed a method (comparative algorithm 2) in reference 2 (Hou YX, Zhao RJ, Ma YB, et al. AReal-Time Star Tailing Removal Method Based on Fast Blur Kernel Estimations[J]. Mathematical Problems in Engineering, 2021, 2021(1): 1~13). This method first performs two Fourier transforms on the blurred star map to obtain a bispectrum, and then uses principal component analysis to analyze the bispectrum to estimate the angle of motion blur. Next, the spectrum is Laden transformed according to the estimated angle to obtain a curve similar to a Sinc function. By improving the curve, the length of motion blur is obtained. Finally, the super Laplace algorithm is used to restore the image. Reference 3 (Cho SJ, Ji SW, Hong JP, et al. Rethinking Coarse-to-Fine Approach in Single Image Deblurring [C] / / InProceedings of the IEEE Conference on Computer Vision. 2021: 4621~4630.) proposed MIMO-UNet (Multi Input Multi Output UNet, MIMO-UNet), which uses a multi-input multi-output codec network to achieve image deblurring.

[0089] 1) Evaluation indicators

[0090] (a) Peak Signal to Noise Ratio (PSNR): It is the ratio of the peak signal energy to the average noise energy. It is a commonly used evaluation indicator for measuring image quality.

[0091] (b) Structural Similarity Index measure (SSIM): It evaluates the objective similarity between two images based on three aspects: image brightness, contrast, and structure.

[0092] (c) Centroid Error (CE): In order to evaluate the accuracy of the star positions in the restored star map, this embodiment uses the centroid positioning error as an evaluation indicator, which represents the deviation error between the star centroid in the restored star map and the standard star centroid in the clear star map.

[0093] 2) Qualitative analysis

[0094] (a) Simulation image results analysis

[0095] In order to test the performance of the REDNet network proposed in this application, this embodiment selects three simulated motion blurred star images with blur lengths of approximately 20, 40, and 80 pixels for testing. Different blur lengths represent different tailing conditions, 20 pixels for short tailing, 40 pixels for medium-long tailing, and 80 pixels for long tailing. Under different tailing conditions, the signal-to-noise ratio of the stars in the image is also different. The longer the tail, the lower the signal-to-noise ratio. Since the simulation is a 16-bit unsigned image, the star image shown below is an image after grayscale stretching.

[0096] Figure 7 Comparison of the deblurring effects of different algorithms on the short-tailed simulated star map, where (a) is a clear star map, (b) is a blurred star map, (c) is the deblurring effect of the comparison algorithm 1, (d) is the deblurring effect of the comparison algorithm 2, (e) is the deblurring effect of MIMO-UNet, and (f) is the deblurring effect of REDNet; Figure 8 The three-dimensional grayscale images of the local star restoration results of the short-tailed simulated star map by different algorithms, where (a) is a clear star map, (b) is a fuzzy star map, (c) is a three-dimensional grayscale image of the restoration result of the comparison algorithm 1, (d) is a three-dimensional grayscale image of the restoration result of the comparison algorithm 2, (e) is a three-dimensional grayscale image of the restoration result of MIMO-UNet, and (f) is a three-dimensional grayscale image of the restoration result of REDNet. Figure 7 and Figure 8The analysis shows that in the case of short tails, the star energy in the star map restored by the four methods tends to be concentrated in the center. However, the background noise in the star map restored by Algorithm 1 and Algorithm 2 fluctuates greatly, and is accompanied by an obvious ringing effect. From the perspective of local stars, Algorithm 1 and Algorithm 2 have poor restoration effects on low signal-to-noise ratio tailing stars, with loss of star energy, insignificant improvement in signal-to-noise ratio, and shifts in the position of restored stars. MIMO-UNet and the method of this application can effectively restore stars and concentrate star energy. The stars restored by REDNet have similar energy to those restored by MIMO-UNet, but the background fluctuations of the REDNet restored image are smaller, which means that the background noise in the image is smaller.

[0097] Fig. 9 The comparison of the deblurring effects of different algorithms on the simulated star image with medium and long tails, where (a) is a clear star image, (b) is a blurred star image, (c) is the blurred effect of the comparison algorithm 1, (d) is the blurred effect of the comparison algorithm 2, (e) is the blurred effect of MIMO-UNet, and (f) is the blurred effect of REDNet. Fig.10 The three-dimensional grayscale images of the local star restoration results of the long-tailed simulated star map by different algorithms, where (a) is a clear star map, (b) is a blurred star map, (c) is a three-dimensional grayscale image of the restoration result of the comparison algorithm 1, (d) is a three-dimensional grayscale image of the restoration result of the comparison algorithm 2, (e) is a three-dimensional grayscale image of the restoration result of MIMO-UNet, and (f) is a three-dimensional grayscale image of the restoration result of REDNet. Fig. 9 and Fig.10 It can be seen that in the case of medium and long tails, the four methods can effectively restore stars as a whole. The star map restored by REDNet is similar to the star map restored by MIMO-UNet as a whole, but the number of stars in the star map restored by REDNet is slightly more. The star morphology in the star map restored by Algorithm 1 and Algorithm 2 is short strips with high middle and low sides, and more background noise is retained, and the ringing effect of the star map restored by Algorithm 2 is more obvious. From the perspective of local stars, the star energy restored by the two traditional methods is lost, and the background noise fluctuates greatly. The local star energy restored by the method of the present application is similar to that of MIMO-UNet, but the background noise fluctuates less.

[0098] Fig.11 Comparison of the deblurring effects of different algorithms on long-tailed simulated star images, where (a) is a clear star image, (b) is a blurred star image, (c) is the blurred effect of algorithm 1, (d) is the blurred effect of algorithm 2, (e) is the blurred effect of MIMO-UNet, and (f) is the blurred effect of REDNet. Fig.12The three-dimensional grayscale images of the local star restoration results of the long-tailed simulated star map by different algorithms, where (a) is a clear star map, (b) is a fuzzy star map, (c) is a three-dimensional grayscale image of the restoration result of the comparison algorithm 1, (d) is a three-dimensional grayscale image of the restoration result of the comparison algorithm 2, (e) is a three-dimensional grayscale image of the restoration result of MIMO-UNet, and (f) is a three-dimensional grayscale image of the restoration result of REDNet. Fig.11 and Fig.12 It can be seen that in the case of long tails, the energy of the tailing stars is dispersed, and the tailing stars with low signal-to-noise ratio are almost submerged in the background noise. The number of stars in the star map restored by the four methods has decreased. Compared with algorithm 1, the stars restored are short strips, and the blur kernel estimation of algorithm 2 is inaccurate, restoring one star into two short strips, and the ringing effect is more obvious. From the perspective of local stars, both traditional methods cannot restore low signal-to-noise ratio long-tailed stars. The large fluctuations in background noise lead to a small improvement in the signal-to-noise ratio of local stars. MIMO-UNet and REDNet can restore the star morphology and concentrate the star energy, but the positions of the restored stars are offset.

[0099] Based on the above comparative tests, the REDNet proposed in this application can achieve good results on the three types of blurred simulated images, reconstruct the star morphology, preserve the star energy as much as possible, and suppress background noise. At the same time, REDNet can avoid large deviations of restored stars in the case of short tails and medium and long tails, and reduce the center of mass positioning error to a certain extent.

[0100] 3) Quantitative analysis

[0101] The evaluation indicators PSNR and SSIM are used to evaluate the deblurring performance of the proposed method from the perspective of the entire image, and the centroid error CE is used to evaluate the impact of the proposed method on the centroid positioning of stars. Since there is no corresponding true value image for the real image, the quantitative analysis in this section is performed on the simulation test set.

[0102] It can be seen from Tables 1 to 3 that the two traditional methods, Algorithm 1 and Algorithm 2, are greatly affected by noise, and the background of the restored star map is volatile and has a high noise level, accompanied by a ringing effect. Therefore, the values ​​of various indicators are lower than those of the two deep learning methods. The star energy in the restored star map by Algorithm 1 is not concentrated and the noise level is high, resulting in its peak signal-to-noise ratio and structural similarity being lower than those of Algorithm 2. The ringing effect of the literature becomes more obvious as the length of the star tail increases, resulting in many false stars being generated along the ringing direction after threshold segmentation. In addition, in the case of long tails, Algorithm 2 restores a star into two short strips, so its centroid positioning error increases with the increase of the tail length, and is higher than Algorithm 1 in the case of long tails. The quantitative analysis results of MIMO-UNet and the method of this application are relatively close in the case of short tails, but the background variance of the image restored by MIMO-UNet is large. The REDNet proposed in this application can effectively suppress noise and adapt to input images of different types and different blur scales. Therefore, REDNet is superior to MIMO-UNet in all indicators in the three tailing conditions. However, as the length of the star tail increases, its signal-to-noise ratio decreases, resulting in a decrease in the number of stars restored by REDNet, and the restored stars are offset, so the center of mass positioning error gradually increases.

[0103] Table 1 Quantitative analysis results of each algorithm on the short tail test set

[0104]

[0105] Table 2 Quantitative analysis results of each algorithm on the medium and long tail test set

[0106]

[0107] Table 3 Quantitative analysis results of each algorithm on the long tail test set

[0108]

[0109] In one embodiment, Fig.13 As shown, a detection image uniform motion blur restoration device is provided, comprising: a feature encoding module, a feature enhancement module and an image restoration module, wherein:

[0110] The feature encoding module is used to input the acquired star detection image into the encoder to obtain shallow features, second-layer features and deep features; the encoder is used to extract different levels of features of the star detection image using multiple convolution operations and downsampling operations.

[0111] The feature enhancement module is used to input the deep features into the multi-scale dilated convolution module to obtain multi-scale contextual features. The multi-scale dilated convolution module is used to extract the multi-scale features of the deep features using convolution kernels with different dilation rates, and to perform weighted tuning of the multi-scale features in the channel domain and the spatial domain using a hybrid attention mechanism to extract multi-scale contextual features.

[0112] The image restoration module is used to input the star detection image, shallow features, second-layer features and multi-scale context features into the decoder to obtain the final clear detection image; wherein the decoder is used to perform convolution and upsampling on the multi-scale context features and then fuse them with the second-layer features through a cross-scale feature fusion module, perform convolution and upsampling on the obtained fused features and then fuse them with the shallow features through a cross-scale feature fusion module, and perform element-wise addition of the obtained fused features with the star detection image after convolution processing to obtain the final clear detection image.

[0113] In one embodiment, the encoder includes: three encoding modules, wherein the first encoding module includes a convolution layer and a first residual group; the second and third encoding modules both include: a downsampling convolution layer and a second residual group; a feature encoding module, further used to input the acquired star detection image into the first encoding module of the encoder, and obtain shallow features after processing by the convolution layer and the first residual group; input the shallow features into the second encoding module of the encoder, and obtain second features after processing by the downsampling convolution layer and the second residual group; the second features are input into the third encoding module of the encoder to obtain deep features.

[0114] In one embodiment, the first residual group and the second residual group each include 8 stacked residual blocks; the residual block includes two convolutional layers, a ReLU and a skip connection.

[0115] In one embodiment, the multi-scale dilated convolution module includes: three dilated convolution layers with different dilated rates, a point convolution layer, and a hybrid attention mechanism; wherein the hybrid attention mechanism includes a channel attention module and a spatial attention module; the channel attention module includes a global pooling layer, a first point convolution layer, and a second point convolution layer; the spatial attention module includes: a maximum pooling layer, an average pooling layer, and a convolution layer; the feature enhancement module is also used to input the deep features into the three dilated convolution layers respectively, splice the three dilated convolution results obtained, and input them into the point convolution layer to obtain convolution features; input the convolution features into the channel attention module of the hybrid attention mechanism to obtain channel weighted optimized features, as shown in formulas (1) to (4). Input the channel weighted optimized features into the spatial attention module of the hybrid attention mechanism to obtain spatially weighted features as shown in formulas (5) and 6; add the spatially weighted features and the deep features to obtain multi-scale context features.

[0116] In one embodiment, the decoder includes three decoding modules; wherein the third decoding module includes a third residual group and a first upsampling layer, the second decoding module includes a first cross-scale feature fusion module, a fourth residual group and a second upsampling layer, and the first decoding module includes a second cross-scale feature fusion module, a fifth residual group and a first convolutional layer; the image restoration module is also used to input the multi-scale context feature into the third decoding module, and obtain the first intermediate decoding feature after passing through the third residual group and the first upsampling layer; the first intermediate decoding feature and the second layer feature are input into the first cross-scale feature fusion module of the second decoding module to obtain the first fused feature; the fused feature is input into the fourth residual group of the second decoding module, and the obtained feature is input into the second upsampling layer to obtain the second intermediate decoding feature; the second intermediate decoding feature and the shallow layer feature are input into the first decoding module, and after passing through the second cross-scale feature fusion module, the fifth residual group and the first convolutional layer, the third intermediate decoding feature is obtained; the third intermediate decoding feature is element-wise added with the star detection image to obtain the final clear detection image.

[0117] In one embodiment, the image restoration module is further used to concatenate the first intermediate decoding feature and the second layer feature and perform a point convolution operation to obtain a feature with half the number of feature channels; calculate the channel weight coefficient of the feature with half the number of feature channels through a channel attention mechanism, multiply the obtained channel weight coefficient and the feature with half the number of feature channels to obtain a first channel weighted feature; adjust the weights of different pixels of the first channel weighted feature through a spatial attention mechanism, multiply the obtained pixel weight and the first channel weighted feature to obtain an optimized feature; and add the optimized feature and the first intermediate decoding feature element by element to obtain a first fusion feature.

[0118] In one of the embodiments, the third residual group, the fourth residual group and the fifth residual group in the decoder are all composed of 8 stacked residual blocks, and the convolution layer in the fifth residual group is a normal convolution; the first upsampling layer and the second upsampling layer both use upsampling transposed convolution.

[0119] The specific definition of the detection image uniform motion blur restoration device can refer to the definition of the detection image uniform motion blur restoration method mentioned above, which will not be repeated here. Each module in the above detection image uniform motion blur restoration device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0120] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.14 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for restoring uniform motion blur of a detected image is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0121] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0122] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above method embodiment when executing the computer program.

[0123] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0124] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0125] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for restoring uniform motion blur of a detected image, characterized in that: The method comprises: The acquired star detection image is input into an encoder to obtain shallow features, second-layer features and deep features; the encoder is used to extract features of different levels of the star detection image by using multiple convolution operations and downsampling operations; The deep features are input into a multi-scale dilated convolution module to obtain multi-scale context features; the multi-scale dilated convolution module is used to extract multi-scale features of the deep features using convolution kernels with different dilation rates, and a hybrid attention mechanism is used to perform weighted tuning in the channel domain and the spatial domain on the multi-scale features to extract multi-scale context features; The star detection image, the shallow features, the second layer features and the multi-scale context features are input into a decoder to obtain a final clear detection image; wherein the decoder is used to perform feature fusion on the multi-scale context features after convolution and upsampling, and then perform feature fusion on the second layer features through a cross-scale feature fusion module, perform feature fusion on the obtained fused features after convolution and upsampling, and then perform feature fusion on the shallow features through a cross-scale feature fusion module, and perform element-wise addition on the obtained fused features after convolution with the star detection image to obtain a final clear detection image; The multi-scale hole convolution module includes: three hole convolution layers with different hole rates, a point convolution layer and a hybrid attention mechanism; the hybrid attention mechanism includes a channel attention module and a spatial attention module; the channel attention module includes a global pooling layer, a first point convolution layer and a second point convolution layer; the spatial attention module includes: a maximum pooling layer, an average pooling layer and a convolution layer; The deep features are input into the multi-scale hole convolution module to obtain multi-scale context features, including: The deep features are input into three hole convolution layers respectively, and the three hole convolution results are concatenated and input into the point convolution layer to obtain convolution features; The convolutional features are input into the channel attention module of the hybrid attention mechanism, and the channel weighted optimized features are obtained as follows: in, represents the features after channel weighted optimization, represents the channel weight coefficient, is the point convolution layer, represents the ReLU function, represents the Sigmoid function, Representative sub-feature map exist The pixel value at represents the global average pooling operation, Representative c Sub-feature map The proportion of all feature maps; z represents the weight vector, , subscript C Representative feature map The number of channels, Represents the convolution feature, H and W represent the height and width of the feature map respectively; The channel weighted optimized features are input into the spatial attention module of the hybrid attention mechanism, and the spatially weighted features are obtained as follows: in, represents the spatially weighted feature, represents maximum pooling, represents average pooling, represents the spatial attention map, represents a convolution operation with a convolution kernel size of 7×7, Represents the Sigmoid activation function; Adding the spatially weighted features to the deep features to obtain multi-scale context features; The encoder comprises: three encoding modules, wherein the first encoding module comprises a convolutional layer and a first residual group; the second and third encoding modules each comprise: a downsampling convolutional layer and a second residual group; The acquired star detection image is input into the encoder to obtain shallow features, second-layer features and deep features, including: Inputting the acquired star detection image into the first encoding module of the encoder, and obtaining shallow features after being processed by the convolution layer and the first residual group; Inputting the shallow layer features into the second encoding module of the encoder, and obtaining the second layer features after being processed by the downsampling convolution layer and the second residual group; The second layer features are input into the third encoding module of the encoder to obtain deep features; The decoder includes three decoding modules; the third decoding module includes a third residual group and a first upsampling layer, the second decoding module includes a first cross-scale feature fusion module, a fourth residual group and a second upsampling layer, and the first decoding module includes a second cross-scale feature fusion module, a fifth residual group and a first convolutional layer; Inputting the star detection image, the shallow layer features, the second layer features and the multi-scale context features into the decoder to obtain a final clear detection image, including: Inputting the multi-scale context features into a third decoding module, and obtaining first intermediate decoding features after passing through the third residual group and the first upsampling layer; Inputting the first intermediate decoding feature and the second layer feature into a first cross-scale feature fusion module of a second decoding module to obtain a first fused feature; Inputting the fused features into the fourth residual group of the second decoding module, and inputting the obtained features into the second upsampling layer to obtain second intermediate decoding features; Inputting the second intermediate decoding feature and the shallow feature into the first decoding module, and obtaining the third intermediate decoding feature after passing through the second cross-scale feature fusion module, the fifth residual group and the first convolution layer; The third intermediate decoding feature is element-wise added to the star detection image to obtain a final clear detection image.

2. The method according to claim 1, characterized in that The first residual group and the second residual group each include 8 stacked residual blocks; the residual block includes two convolutional layers, a ReLU and a jump connection.

3. The method according to claim 1, characterized in that Inputting the first intermediate decoding feature and the second layer feature into a first cross-scale feature fusion module of a second decoding module, and obtaining a first fusion feature includes: The first intermediate decoding feature and the second layer feature are concatenated and subjected to a point convolution operation to obtain a feature with half the number of feature channels; The feature with half the number of feature channels is used to calculate the channel weight coefficient through the channel attention mechanism, and the obtained channel weight coefficient is multiplied by the feature with half the number of feature channels to obtain the first channel weighted feature; The weighted features of the first channel are used to adjust the weights of different pixels through a spatial attention mechanism, and the obtained pixel weights are multiplied by the weighted features of the first channel to obtain optimized features; The optimized feature and the first intermediate decoding feature are added element by element to obtain a first fused feature.

4. The method according to claim 1, characterized in that: The third residual group, the fourth residual group and the fifth residual group in the decoder are all composed of 8 stacked residual blocks, and the convolution layer in the fifth residual group is a normal convolution; the first upsampling layer and the second upsampling layer both use upsampling transposed convolution.

5. A detection image uniform motion blur restoration device, characterized in that: The device comprises: A feature encoding module is used to input the acquired star detection image into an encoder to obtain shallow features, second-layer features and deep features; the encoder is used to extract features of different levels of the star detection image using multiple convolution operations and downsampling operations; A feature enhancement module, used to input the deep features into a multi-scale dilated convolution module to obtain multi-scale context features; the multi-scale dilated convolution module is used to extract multi-scale features of the deep features using convolution kernels with different dilation rates, and to perform weighted tuning of the multi-scale features in the channel domain and the spatial domain using a hybrid attention mechanism to extract multi-scale context features; An image restoration module is used to input the star detection image, the shallow features, the second layer features and the multi-scale context features into a decoder to obtain a final clear detection image; wherein the decoder is used to perform feature fusion on the multi-scale context features after convolution and upsampling, and on the second layer features through a cross-scale feature fusion module; perform feature fusion on the obtained fused features after convolution and upsampling, and on the shallow features through a cross-scale feature fusion module; perform element-wise addition on the obtained fused features after convolution and convolution with the star detection image to obtain a final clear detection image; The multi-scale hole convolution module includes: three hole convolution layers with different hole rates, a point convolution layer and a hybrid attention mechanism; the hybrid attention mechanism includes a channel attention module and a spatial attention module; the channel attention module includes a global pooling layer, a first point convolution layer and a second point convolution layer; the spatial attention module includes: a maximum pooling layer, an average pooling layer and a convolution layer; The feature enhancement module is further used to input the deep features into three hole convolution layers respectively, splice the three hole convolution results and input them into the point convolution layer to obtain convolution features; input the convolution features into the channel attention module of the hybrid attention mechanism, and obtain the channel weighted optimized features as follows: in, represents the features after channel weighted optimization, represents the channel weight coefficient, is the point convolution layer, represents the ReLU function, represents the Sigmoid function, Representative sub-feature map exist The pixel value at represents the global average pooling operation, Representative c Sub-feature map The proportion of all feature maps; z represents the weight vector, , subscript C Representative feature map The number of channels, Represents the convolution feature, H and W represent the height and width of the feature map respectively; The channel weighted optimized features are input into the spatial attention module of the hybrid attention mechanism, and the spatially weighted features are obtained as follows: in, represents the spatially weighted feature, represents maximum pooling, represents average pooling, represents the spatial attention map, represents a convolution operation with a convolution kernel size of 7×7, Represents the Sigmoid activation function; Adding the spatially weighted features to the deep features to obtain multi-scale context features; The encoder includes: three encoding modules, wherein the first encoding module includes a convolution layer and a first residual group; the second and third encoding modules each include: a downsampling convolution layer and a second residual group; a feature encoding module, further used for inputting the acquired star detection image into the first encoding module of the encoder, and obtaining shallow features after being processed by the convolution layer and the first residual group; inputting the shallow features into the second encoding module of the encoder, and obtaining second features after being processed by the downsampling convolution layer and the second residual group; and inputting the second features into the third encoding module of the encoder to obtain deep features; The decoder includes three decoding modules; the third decoding module includes a third residual group and a first upsampling layer, the second decoding module includes a first cross-scale feature fusion module, a fourth residual group and a second upsampling layer, and the first decoding module includes a second cross-scale feature fusion module, a fifth residual group and a first convolutional layer; the image restoration module is also used to input the multi-scale context feature into the third decoding module, and obtain a first intermediate decoding feature after passing through the third residual group and the first upsampling layer; the first intermediate decoding feature and the second layer feature are input into the first cross-scale feature fusion module of the second decoding module to obtain a first fusion feature; the fusion feature is input into the fourth residual group of the second decoding module, and the obtained feature is input into the second upsampling layer to obtain a second intermediate decoding feature; the second intermediate decoding feature and the shallow layer feature are input into the first decoding module, and after passing through the second cross-scale feature fusion module, the fifth residual group and the first convolutional layer, a third intermediate decoding feature is obtained; the third intermediate decoding feature is element-wise added to the star detection image to obtain a final clear detection image.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images