Method, device, equipment and memory for restoring non-uniform motion blur of detected images

Abstract

The present application relates to a method, device, equipment and memory for restoring non-uniform motion blur of detected images. The method uses a multi-scale feature extraction module in the encoder to extract context information of different ranges at the same scale, and then performs multi-scale feature fusion of feature maps of different resolutions between the encoder and the decoder to enhance the strip blur features in the image and provide rich feature representation for non-uniform motion blur restoration; the two work together to enable the network to adapt to input images of different blur scales, learn blur features of different targets in the image, and improve the restoration effect of blurred images. The method solves the problem that the existing algorithm cannot accurately estimate multiple blur kernels and has poor restoration effect.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and memory for restoring non-uniform motion blur of a detected image. Background Art

[0002] In space debris detection images, stars are imaged as strips due to motion blur. Space debris may be imaged as dots or strips, and the image is likely to contain multiple pieces of space debris. The blur pattern of all stars remains consistent, while the motion blur patterns between stars and debris, and between different pieces of debris, are inconsistent, that is, there are different motion blur situations in the image. This type of image is defined as a non-uniform motion blurred image, which requires an algorithm to estimate multiple blur patterns to achieve accurate restoration.

[0003] Space debris detection images contain a large number of stars, and the blur scales of stars in different detection images are different. Therefore, the network needs to be able to adapt to input images with different blur scales. In addition, there are space debris with inconsistent motion in the space debris detection images, which also requires the network to have the ability to perceive different blur features in the image. However, most traditional algorithms mainly deal with uniform motion blurred star images with only stars, and cannot effectively estimate the multiple blur kernels in the space debris detection images. The restored images have obvious ringing effects at the space debris. Most deep learning-based methods mainly deal with 8-bit color natural images. The image input needs to be normalized before training the network. The space debris detection images taken are mostly 16-bit single-channel grayscale images with higher dynamic range and accuracy. Normalization will cause information loss. If only the data input method is modified, the network will not be compatible and the restoration effect of the real image will be poor. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, equipment and memory for restoring non-uniform motion blur of detected images to address the problem that existing algorithms cannot accurately estimate multiple blur kernels and have poor restoration effects.

[0005] A method for restoring non-uniform motion blur of a detected image, the method comprising:

[0006] The acquired space debris detection image is input into the encoder to obtain output features of three scales; the encoder includes three encoding modules, which include a convolutional layer, a multi-scale feature extraction module and a residual group; the multi-scale feature extraction module is used to extract multi-scale features at the same resolution using dilated convolutions and strip pooling operations with different dilation rates, and fuse the multi-scale features.

[0007] The output features of the three scales are input into the feature enhancement layer to obtain three layers of fused features; the feature enhancement layer includes three multi-scale feature fusion modules, and the multi-scale feature fusion module is used to perform multi-scale feature fusion on the output features of the three scales so that the fused features of each layer contain the information of the other two layers.

[0008] The three-layer fusion features are input into the decoder to obtain a restored single-channel clear image; the decoder includes three decoding modules, the third decoding module is used to process the third-layer fusion features through residual groups and upsampling to obtain the first decoding features, the second decoding module is used to fuse the first decoding features and the second-layer fusion features, and then process them through residual blocks and upsampling to obtain the second decoding features, and the third decoding module is used to fuse the second decoding features and the first-layer fusion features, and then use residual blocks and convolution operations to obtain the restored single-channel clear image.

[0009] A detection image non-uniform motion blur restoration device, the device comprising:

[0010] The feature encoding module is used to input the acquired space debris detection image into the encoder to obtain output features of three scales; the encoder includes three encoding modules, and the encoding module includes a convolution layer, a multi-scale feature extraction module and a residual group; the multi-scale feature extraction module is used to use dilated convolutions with different dilation rates and strip pooling operations to perform multi-scale feature extraction on features at the same resolution, and fuse the multi-scale features.

[0011] The feature fusion enhancement module is used to input the output features of the three scales into the feature enhancement layer to obtain three-layer fusion features; the feature enhancement layer includes three multi-scale feature fusion modules, and the multi-scale feature fusion module is used to perform multi-scale feature fusion on the output features of the three scales so that the fused features of each layer contain the information of the other two layers.

[0012] The detection image non-uniform motion blur restoration module is used to input the three-layer fusion features into the decoder to obtain a restored single-channel clear image; the decoder includes 3 decoding modules, the third decoding module is used to process the third-layer fusion features through residual groups and upsampling to obtain the first decoding features, the second decoding module is used to fuse the first decoding features and the second-layer fusion features, and then process them through residual blocks and upsampling to obtain the second decoding features, and the third decoding module is used to fuse the second decoding features and the first-layer fusion features, and then use residual blocks and convolution operations to obtain a restored single-channel clear image.

[0013] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned detection image non-uniform motion blur restoration method when executing the computer program.

[0014] A computer readable memory stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned detection image non-uniform motion blur restoration method.

[0015] The above-mentioned detection image non-uniform motion blur restoration method, device, equipment and memory, the method uses a multi-scale feature extraction module in the encoder to extract context information of different ranges at the same scale, and then performs multi-scale feature fusion of feature maps of different resolutions between the encoder and the decoder to enhance the strip blur features in the image and provide rich feature representation for non-uniform motion blur restoration; the two work together to enable the network to adapt to input images of different blur scales, learn blur features of different targets in the image, and improve the restoration effect of blurry images. The problem that the existing algorithm cannot accurately estimate multiple blur kernels and has poor restoration effect is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a flow chart of a method for restoring non-uniform motion blur of a detected image in one embodiment;

[0017] Figure 2 It is an overall structural diagram of a multi-scale codec network MSEDNet in another embodiment;

[0018] Figure 3 A structural diagram of a multi-scale feature extraction module in another embodiment;

[0019] Figure 4 It is a structural diagram of a multi-scale feature fusion module in another embodiment;

[0020] Figure 5 Partial non-uniform motion blurred star images of a non-uniform motion blurred simulation data set in another embodiment, wherein (a) is a blurred star image including a bar star and tracked debris, (b) is a blurred star image including a bar star and two bar fragments, (c) is a blurred star image including a bar star and four bar fragments, and (d) is a blurred star image including a bar star and six bar fragments;

[0021] Figure 6 3D grayscale images of a clear star image, a fuzzy star image and their fragments in another embodiment, wherein (a) is a clear star image point fragment, (b) is a clear star image point fragment 3D grayscale image, (c) is a fuzzy star image point fragment, and (d) is a fuzzy star image point fragment 3D grayscale image;

[0022] Figure 7 The comparison of the deblurring effect of a simulated star map containing a fragment in another embodiment, wherein (a) is the comparison algorithm 1, (b) is the comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet;

[0023] Figure 8 3D grayscale images of fragment restoration results of different algorithms in another embodiment, where (a) is comparison algorithm 1, (b) is comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet;

[0024] Fig. 9 Schematic diagram of a clear star map and a blurred star map containing space debris in another embodiment, wherein (a) is a clear star map containing three point-shaped debris, and (b) is a blurred star map containing three strip-shaped debris;

[0025] Fig.10 Different fragments and their three-dimensional grayscale images in a clear star map in another embodiment, wherein (a) is fragment 1, (b) is fragment 2, (c) is fragment 3, (d) is a three-dimensional grayscale image of fragment 1, (e) is a three-dimensional grayscale image of fragment 2, and (f) is a three-dimensional grayscale image of fragment 3;

[0026] Fig.11 Different fragments and their three-dimensional grayscale images in a blurred star map in another embodiment, wherein (a) is fragment 1, (b) is fragment 2, (c) is fragment 3, (d) is a three-dimensional grayscale image of fragment 1, (e) is a three-dimensional grayscale image of fragment 2, and (f) is a three-dimensional grayscale image of fragment 3;

[0027] Fig.12 1 is a comparison of the deblurring effect of a simulated star map containing space debris in another embodiment, wherein (a) is comparison algorithm 1, (b) is comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet;

[0028] Fig.13 1 is a comparison of the deblurring effect of fragment 1 in another embodiment, where (a) is comparison algorithm 1, (b) is comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet;

[0029] Fig.14 The comparison of the deblurring effect of fragment 2 in another embodiment, where (a) is the comparison algorithm 1, (b) is the comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet;

[0030] Fig.15 The comparison of the deblurring effect of fragment 3 in another embodiment, where (a) is the comparison algorithm 1, (b) is the comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet;

[0031] Fig.16 3D grayscale images of the restoration results of fragment 1 using different algorithms in another embodiment, where (a) is comparison algorithm 1, (b) is comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet;

[0032] Fig.17 3D grayscale images of the restoration results of fragment 2 using different algorithms in another embodiment, where (a) is comparison algorithm 1, (b) is comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet;

[0033] Fig.18 3D grayscale images of the restoration results of fragment 3 using different algorithms in another embodiment, where (a) is comparison algorithm 1, (b) is comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet;

[0034] Fig.19 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] In space debris detection images, debris may appear as dots or strips with tails, or even multiple pieces of space debris with inconsistent tails may appear in the same image. In addition, there are a large number of stars with tails in the image. These factors lead to multiple blur styles in the image, which increases the difficulty of motion blur restoration. When designing the network, it is not only necessary for the network to be able to adapt to input images with different blur scales, but also to have the ability to perceive multiple blur features in the image.

[0037] In one embodiment, Figure 1 As shown, a method for restoring non-uniform motion blur of a detected image is provided, and the method comprises the following steps:

[0038] Step 100: Input the acquired space debris detection image into the encoder to obtain output features of three scales; the encoder includes three encoding modules, and the encoding module includes a convolution layer, a multi-scale feature extraction module and a residual group; the multi-scale feature extraction module is used to use dilated convolutions with different dilation rates and strip pooling operations to perform multi-scale feature extraction on features at the same resolution, and fuse the multi-scale features.

[0039] Specifically, non-uniform motion blur means that different areas in the image may have different blur scales and blur directions. Multi-scale features can help the algorithm learn the blur features in the image more accurately by analyzing the information of the image at different scales. This application makes improvements based on the encoder-decoder structure and proposes a multi-scale codec network MSEDNet for realizing non-uniform motion blur restoration of the detected image. The network uses a multi-scale feature extraction module to extract contextual information of different ranges at the same scale, and then performs multi-scale feature fusion of feature maps of different resolutions between the encoder and decoder to provide rich feature representation for non-uniform motion blur restoration. The network structure of MSEDNet is as follows: Figure 2 shown.

[0040] Each encoding module of the encoder consists of a 3×3 convolutional layer, a multi-scale feature extraction module and 8 stacked residual blocks. The 3×3 convolutional layer is responsible for converting the input single-channel grayscale image into a multi-channel feature map or downsampling the feature map. The multi-scale feature extraction module uses dilated convolutional layers with different dilation rates and strip pooling to perform multi-scale feature extraction on feature maps at the same resolution, and fuses features with different receptive fields. The stacked residual blocks can further capture more complex image features.

[0041] Step 102: Input the output features of the three scales into the feature enhancement layer to obtain three layers of fused features; the feature enhancement layer includes three multi-scale feature fusion modules, and the multi-scale feature fusion module is used to perform multi-scale feature fusion on the output features of the three scales so that each layer of fused features contains information from the other two layers.

[0042] Specifically, in order to utilize the information contained in feature maps of different resolutions, the network proposed in this application performs multi-scale feature fusion on the output features of the encoder at three scales, so that the fused features of each layer contain the information of the other two layers, achieve feature complementarity, learn the representation of fuzzy features at different resolutions, and help the network adapt to blurs of different scales. In addition, the multi-scale feature fusion module uses a multi-strip attention mechanism to perceive the fuzzy features of different areas in the image, helping the network learn multiple fuzzy styles in the image.

[0043] Step 104: input the three-layer fusion features into the decoder to obtain a restored single-channel clear image; the decoder includes 3 decoding modules, the third decoding module is used to process the third-layer fusion features through a residual group and upsampling to obtain a first decoding feature, the second decoding module is used to fuse the first decoding feature and the second-layer fusion features, and then process them through a residual block and upsampling to obtain a second decoding feature, and the third decoding module is used to fuse the second decoding feature and the first-layer fusion features, and then use a residual block and convolution operation to obtain a restored single-channel clear image.

[0044] Specifically, the third layer of the decoder uses 8 stacked residual blocks to decode the output features of the encoder, and uses a transposed convolution of size 4×4 and stride 2 to achieve feature map upsampling. The second layer has a similar structure to the third layer, but a 1×1 convolution layer is added before the input residual group is decoded to fuse the output features of the encoder with the upsampled decoded features. The first layer has a similar structure to the second layer, except that the transposed convolution layer is replaced by a 3×3 convolution layer to achieve feature dimensionality reduction and convert the multi-channel feature map into a single-channel clear image.

[0045] In the above-mentioned detection image non-uniform motion blur restoration method, the method uses a multi-scale feature extraction module in the encoder to extract context information of different ranges at the same scale, and then performs multi-scale feature fusion of feature maps of different resolutions between the encoder and the decoder to enhance the strip blur features in the image and provide rich feature representation for non-uniform motion blur restoration; the two work together to enable the network to adapt to input images of different blur scales, learn blur features of different targets in the image, and improve the restoration effect of blurry images. This solves the problem that the existing algorithm cannot accurately estimate multiple blur kernels and has poor restoration effect.

[0046] In one embodiment, the convolution layers in the second and third encoding modules are down-sampling convolution layers; step 100 includes: inputting the acquired space debris detection image into the first encoding module of the encoder, and obtaining output features of the first scale after passing through the convolution layer, the multi-scale feature extraction module, and the residual group; inputting the output features of the first scale into the second encoding module of the encoder, and obtaining output features of the second scale after passing through the down-sampling convolution layer, the multi-scale feature extraction module, and the residual group; inputting the output features of the second scale into the third encoding module of the encoder, and obtaining output features of the third scale after passing through the down-sampling convolution layer, the multi-scale feature extraction module, and the residual group.

[0047] In one embodiment, the multi-scale feature extraction module includes: a horizontal hole convolution branch, a vertical hole convolution branch, a standard hole convolution branch, a horizontal strip pooling branch, a vertical strip pooling branch and a point convolution; in the multi-scale feature extraction module: the features input to the module are processed by the horizontal hole convolution branch, the vertical hole convolution branch, the standard hole convolution branch, the horizontal strip pooling branch and the vertical strip pooling branch, the obtained features are spliced ​​and then reduced in dimension by point convolution to obtain fused features, the fused features are added to the features input to the module, and the features extracted by the multi-scale feature extraction module are:

[0048] (1)

[0049] (2)

[0050] (3)

[0051] (4)

[0052] (5)

[0053] (6)

[0054] (7)

[0055] in, Represents the features extracted by the multi-scale feature extraction module, Represents the features input to the module, Represents the features extracted by the horizontal hole convolution branch, Represents a 1×3 dilated convolution kernel with a dilation rate of 1. Represents a 1×3 dilated convolution kernel with a dilation rate of 3. Represents a 1×3 dilated convolution kernel with a dilation rate of 5. Represents the features extracted by the vertical hole convolution branch; Represents a 3×1 dilated convolution kernel with a dilation rate of 1. Represents a 3×1 dilated convolution kernel with a dilation rate of 3. Represents a 3×1 dilated convolution kernel with a dilation rate of 5. Represents the features extracted by the standard dilated convolution branch, Represents a 3×3 dilated convolution kernel with a dilation rate of 1. Represents a 3×3 dilated convolution kernel with a dilation rate of 3. Represents a 3×3 dilated convolution kernel with a dilation rate of 5; represents the horizontal pooling operation, represents the vertical pooling operation, Represents an extended operation, Represents a 1×1 convolution kernel, which is used to achieve feature dimensionality reduction. is the feature concatenation operation, For fusion features.

[0056] Specifically, space debris detection images may contain blurs of different scales. When the relative motion between the target and the telescope is slow, a short and thick tail appears in the image; when the relative motion between the target and the telescope is fast, a thin and long tail appears in the image. In order to extract multi-scale features, some existing algorithms usually cascade multiple codec networks, each of which processes images of one resolution, to achieve coarse-to-fine restoration. However, this method is time-consuming and difficult to train.

[0057] In view of the stripe-like blur pattern formed after the target is tailed, this application proposes a multi-scale feature extraction module (MSFEM) in the encoder part. This module combines standard dilated convolution, strip-shaped dilated convolution and strip-shaped pooling to extract rich and diverse features, thereby helping the network to improve the image restoration effect.

[0058] The multi-scale feature extraction module extracts features of different scales by using convolution kernels with different dilation rates, so that the network can understand and learn blur styles of different scales. The horizontal dilated convolution branch and the vertical dilated convolution branch are specially designed to capture features in the horizontal and vertical directions, which helps the network to more accurately learn the local context information of the strip blur features in the horizontal and vertical directions. The strip pooling operation can obtain global context information in the horizontal and vertical directions, helping the network to learn the relative position relationship of the target in the image. Finally, the features of different branches are fused through the feature splicing operation, which can combine feature information of different types and scales to provide a richer and more comprehensive feature representation for subsequent feature restoration. The specific module structure is as follows: Figure 3 As shown, the calculation process is as follows:

[0059] First, input features After five different branches, namely the horizontal dilated convolution branch, the vertical dilated convolution branch, the standard dilated convolution branch, the horizontal strip pooling branch and the vertical strip pooling branch, five different features are obtained as shown in formulas (1) to (5): , , , , .

[0060] Since the horizontal pooling operation and the vertical pooling operation result in the vertical tensor and the horizontal tensor respectively, is the vertical convolution kernel, is the horizontal convolution kernel, is an expansion operation. The dilated convolution kernel adds zero elements to the standard convolution kernel. Using a dilated convolution kernel alone will cause information loss. Therefore, in the same branch, the features extracted by multiple convolution kernels with different dilation rates are added and fused to make up for the features. Then the features extracted by the five branches are concatenated, and 1×1 convolution is used for feature dimensionality reduction. Finally, the features are added to the original input features for feature enhancement. The calculation process is shown in formulas (6) and (7).

[0061] The role of the multi-scale feature extraction module in the encoder part is mainly reflected in the following aspects: ① Feature diversity: By extracting features at different scales, multi-scale feature extraction can capture the details and structural information in the image, which helps to learn the blur style more accurately and restore a clear image. ② Scale adaptability: There may be blurs of different scales in non-uniform motion blurred images. Multi-scale feature extraction can help the network better adapt to scale changes and improve deblurring performance. ③ Context information utilization: Multi-scale feature extraction can capture local and global context information by design, helping the network learn the relative position relationship of objects in the image.

[0062] In one embodiment, the residual group in the encoder and the decoder is composed of 8 stacked residual blocks; the residual block includes two convolutional layers, a ReLU and a jump connection, and step 102 includes: fusing the output features of the first scale with the output features of the second scale and the third scale in a first multi-scale feature fusion module to obtain the first layer of fused features; fusing the output features of the second scale with the output features of the first scale and the third scale in a second multi-scale feature fusion module to obtain the second layer of fused features; fusing the output features of the third scale with the output features of the first scale and the second scale in a third multi-scale feature fusion module to obtain the third layer of fused features.

[0063] In one embodiment, the output features of the first scale are fused with the output features of the second scale and the third scale in a first multi-scale feature fusion module to obtain a first layer of fused features, including: downsampling and feature dimension reduction operations on the output features of the first scale to obtain a first feature to be fused having the same size and number of channels as the second layer features; upsampling and feature dimension increase operations on the output features of the third scale to obtain a second feature to be fused having the same size and number of channels as the second layer features; adding the first feature to be fused, the second feature to be fused, and the output features of the second scale by element-wise addition to obtain a combined feature; and performing a global average pooling on the combined feature. After being processed by the first convolution layer, the first point convolution layer and the second point convolution layer, the channel weight vector corresponding to the feature is obtained; the channel weight vector is activated by the Sigmoid function and then fused with the combined feature to obtain the initial fused feature; the initial fused feature is passed through three pooling branches composed of horizontal pooling kernels and vertical pooling kernels of different sizes to extract the global context and local context in the corresponding direction to obtain strip pooling features of three scales; the multi-scale strip pooling features obtained by adding the strip pooling features of three scales are passed through the point convolution layer and the sigmoid function to obtain the attention mask; the attention mask is multiplied by the initial fused feature to obtain the weighted optimized feature.

[0064] In one embodiment, the initial fusion features are passed through three pooling branches consisting of horizontal pooling cores and vertical pooling cores of different sizes to extract the global context and local context in the corresponding direction, and the strip pooling features of three scales are obtained as follows:

[0065] (8)

[0066] (9)

[0067] (10)

[0068] in, Represents the strip pooling feature of the first scale, Represents the strip pooling feature of the second scale, Represents the strip pooling feature of the third scale, , and Represent the pooling kernels respectively. , , The horizontal pooling operation, , and Represent the pooling kernels respectively. , , The vertical pooling operation, is a combination of features, for Convolution kernel, for Convolution kernel, is a 3×3 convolution kernel, For extended operations.

[0069] Specifically, the multi-scale feature extraction module performs multi-scale feature extraction on a feature map at one resolution, and the multi-scale feature fusion module fuses feature maps of different resolutions.

[0070] In the encoder part, the network downsamples the feature map to expand the receptive field and generate feature maps of different scales. The receptive field of high-resolution feature maps is small and usually contains rich spatial detail information; while the receptive field of low-resolution feature maps is large, each pixel represents information in a larger range, and the feature map usually contains rich semantic information, which can help the network identify the overall trend of the image. Using multi-scale feature fusion technology to fuse feature maps of different resolutions can help the network more comprehensively identify and understand the blur style in the image and accurately locate the blur position. In addition, the blur degree in high-resolution feature maps is large, and the blur degree in low-resolution feature maps is small. Multi-scale feature fusion can help the network better cope with different degrees and types of blur, adapt to more blur conditions, and improve its adaptability in practical applications. The traditional encoder-decoder structure directly passes the output features of the encoding layer to the decoding layer of the same layer without using features of adjacent scales. In addition, most existing algorithms use the channel attention mechanism to weighted tune the feature channels, but for image deblurring tasks, it is more important to combine the spatial attention mechanism to weighted tune the pixels to perceive and emphasize the blurred areas in the image. For non-uniform motion blurred images, it is necessary to have the ability to perceive blur features of different scales in the image.

[0071] In order to enhance the transmission between information flows, this application designs a multi-scale feature fusion module MSFFM (Multi Scale Feature Fusion Module, MSFFM), which fuses the features of different resolutions output by the encoder. Since the strip attention mechanism can capture the global context information and local context information of the image, it adjusts the spatial pixel weights to highlight important features. Based on the characteristics of non-uniform motion blur, this application designs a multi-strip attention mechanism in the feature fusion stage to perceive the blur features of different scales in the image and perform feature enhancement. The module structure is as follows Figure 4 As shown in the figure, the features of different layers are first recalibrated and fused according to the channel weights to suppress useless features, and then the spatial dimensions are adjusted through a multi-strip attention mechanism, giving higher weights to the blurred areas in the image, so that the network pays more attention to the blurred areas. Since the operation of feature fusion at each layer is similar, the following takes the second layer feature fusion as an example to specifically introduce the module process:

[0072] The output features of different layers of the encoder have different sizes and different numbers of channels. It is necessary to first downsample and reduce the dimension of the first layer features, and upsample and increase the dimension of the third layer features so that the features to be fused have the same size and number of channels as the second layer features. Then, the features of the three scales are combined by element-wise addition to obtain the combined features. , perform global average pooling on the features in the spatial dimension to obtain the weight vector about the channel , and then compress the channel dimension of the weight vector through a 1×1 convolution layer, reducing the amount of calculation while obtaining a more compact weight vector ,in , and then the weight vector The channel weight vector corresponding to the feature is obtained by expanding the channel through a 1×1 convolution layer Finally, the Sigmoid activation function is used on the channel weight vector to generate the channel weight coefficient corresponding to the feature, and then the multi-scale features are re-weighted and fused to obtain the initial fusion feature The initially fused features are then passed through three different sizes of horizontal pooling kernels and vertical pooling kernels to extract the global context and local context in the corresponding direction (the calculation process is shown in formulas (8) to (10)). The strip pooling features of different scales are added together to obtain the multi-scale strip pooling features. , and finally the feature Input a 1×1 convolution layer and a Sigmoid activation function to get the attention mask , attention mask and initial fusion features Multiply to get the weighted optimized features ; The calculation process is as follows:

[0073] (11)

[0074] (12)

[0075] (13)

[0076] Since the horizontal pooling operation and the vertical pooling operation result in the vertical tensor and the horizontal tensor respectively, is the vertical convolution kernel, is the horizontal convolution kernel, It is a normal 3×3 convolution kernel.

[0077] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0078] In a verification example, a star image non-uniform motion blur dataset is first constructed, and the specific steps are as follows:

[0079] (1) Clear star map simulation

[0080] First, based on the Tycho2 star catalog, 200 clear star maps were generated using star map simulation software. The clear star maps only contain point-like stars.

[0081] (2) Motion blurred star map simulation

[0082] For each clear star image, five motion blurred star images are generated by simulating different motion lengths and motion angles. The motion length is between 0 and 90 pixels. These motion blurred star images simulate the motion of stars relative to the observing device.

[0083] (3) Point fragment simulation

[0084] Simulate 100 different point-shaped light spots and simulate point-shaped fragments of different positions and sizes.

[0085] (4) Strip fragment simulation

[0086] For each point-like debris image, a strip-like debris image is simulated to simulate the trailing phenomenon caused by the relative motion between space debris and the observation equipment.

[0087] (5) Construction of non-uniform motion blur dataset

[0088] First, from the 1000 sets of generated clear star images and their corresponding blurred star images, 800 sets were divided as training sets, 100 sets as validation sets, and the remaining 100 sets as test sets.

[0089] Next, the training set is further divided into 10 subsets, each containing 80 sets of star images.

[0090] Then, for the first nine subsets, we randomly sampled 80 times from the point fragment image, each time extracting the same number of point fragments and adding them to the clear star image of each subset. At the same time, we added the corresponding strip fragments to the corresponding motion blurred star image.

[0091] Finally, for the remaining subset, one point-shaped fragment is randomly selected from the point-shaped fragment image and added to the clear star image and the motion blurred star image to simulate the situation where the tracked fragment is point-shaped.

[0092] The validation set and test set are processed in the same way as the training set to construct a simulated dataset with non-uniform motion blur.

[0093] Figure 5The figure shows some non-uniform motion blurred star images of the non-uniform motion blur simulation data set, where (a) is a blurred star image containing a bar star and tracked fragments, (b) is a blurred star image containing a bar star and two bar fragments, (c) is a blurred star image containing a bar star and four bar fragments, and (d) is a blurred star image containing a bar star and six bar fragments.

[0094] (1) Experimental environment and parameter settings

[0095] In the network training phase of the experiment in this embodiment, in order to reduce the memory usage, the blurred star image in the non-uniform motion blur dataset is first randomly cropped into image blocks of 256 pixels × 256 pixels. Then, 4 blurred star image blocks are randomly selected each time, and after random rotation and random flipping data enhancement operations, they are sent to the motion blur restoration network. In addition, in order to train the network, the L1 norm between the clear star image and the restored star image is selected as the loss function of the motion blur restoration network in the experiment, and the Adam optimizer is used to iteratively optimize the motion blur restoration network, and its optimization parameters are and are 0.9 and 0.999 respectively. The initial learning rate of the motion blur restoration network is , gradually decreases to The training process lasted for 200 rounds in total. Finally, the entire network framework was implemented based on the Pytorch deep learning framework, and related experiments were carried out using the NVIDIA GeForce RTX 3090 graphics card.

[0096] (2) Comparative experimental results and evaluation

[0097] The comparison algorithms selected in this embodiment include: Comparison Algorithm 1 is the restoration algorithm based on grayscale stretching and improved Richardson-Lucy proposed by Jiang et al. in Reference 1 (Jiang J, Huang JN, Zhang G J. An Accelerated Motion Blurred Star Restoration Based on Single Image [J]. IEEE Sensors Journal, 2017, 17(5): 1306~1315), Comparison Algorithm 2 is the restoration algorithm based on dual spectrum and super Laplace model proposed by Hou et al. in Reference 2 (Hou YX, ZhaoR J, Ma YB, et al. A Real-Time Star Tailing Removal Method Based on FastBlur Kernel Estimations [J]. Mathematical Problems in Engineering, 2021, 2021(1): 1~13), and the multi-input multi-output codec network MIMO-UNet. In order to objectively evaluate the performance of the network proposed in this application, this embodiment uses peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and centroid error (CE) as evaluation indicators for the entire image.

[0098] 1) Qualitative analysis

[0099] In order to test the restoration effect of the MSEDNet proposed in this application on the non-uniform motion blurred star map, this section selects simulated star maps with different blur conditions for testing, including: ① a motion blurred star map with one point-like fragment, in which the blur in the image is the motion blur of the trailing star and the Gaussian blur of the point-like fragment; ② a motion blurred star map with three strip-like fragments, in which the blur in the image is motion blur, but the motion blur length and motion blur angle between the star and the three fragments, and between the three fragments are inconsistent.

[0100] Figure 6 The figures show a clear star image, a fuzzy star image and three-dimensional grayscale images of their fragments, wherein (a) is a clear star image point fragment, (b) is a clear star image point fragment three-dimensional grayscale image, (c) is a fuzzy star image point fragment, and (d) is a fuzzy star image point fragment three-dimensional grayscale image; Figure 7 The figure shows the comparison of the deblurring effect of a simulated star map containing one fragment, where (a) is the comparison algorithm 1, (b) is the comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet; Figure 8The three-dimensional grayscale images of the fragment restoration results of different algorithms are shown, where (a) is comparison algorithm 1, (b) is comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet.

[0101] Depend on Figures 6 to 8 It can be seen that for a blurred image containing a point-shaped fragment, both traditional methods cannot effectively restore the shape and energy of the point-shaped fragment, the gray value is seriously reduced and the background noise fluctuates greatly. The four deep learning methods can restore the shape of the point-shaped fragments and concentrate the fragment energy while restoring the blurred star map. Among them, the energy of the fragment restored by MIMO-UNet is inaccurate and higher than the energy of the fragment in the clear star map; the energy of the point-shaped space fragments in the restored star map by MSEDNet is the best.

[0102] Fig. 9 Shown are schematic diagrams of a clear star map and a blurred star map containing space debris, where (a) is a clear star map containing three point-shaped fragments, and (b) is a blurred star map containing three strip-shaped fragments. Fig.10 Shown are different fragments and their three-dimensional grayscale images in a clear star image, where (a) is fragment 1, (b) is fragment 2, (c) is fragment 3, (d) is the three-dimensional grayscale image of fragment 1, (e) is the three-dimensional grayscale image of fragment 2, and (f) is the three-dimensional grayscale image of fragment 3. Fig.11 Shown are different fragments in the blurred star map and their three-dimensional grayscale images, where (a) is fragment 1, (b) is fragment 2, (c) is fragment 3, (d) is the three-dimensional grayscale image of fragment 1, (e) is the three-dimensional grayscale image of fragment 2, and (f) is the three-dimensional grayscale image of fragment 3. Fig.12 The figure shows the comparison of the deblurring effect of the simulated star map containing space debris, where (a) is the comparison algorithm 1, (b) is the comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet; Fig.13 The figure shows the comparison of the deblurring effect of fragment 1, where (a) is the comparison algorithm 1, (b) is the comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet. Fig.14 The figure shows the comparison of the deblurring effect of fragment 2, where (a) is the comparison algorithm 1, (b) is the comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet. Fig.15 The figure shows the comparison of the deblurring effect of fragment 3, where (a) is the comparison algorithm 1, (b) is the comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet. Fig.16 The figure shows the three-dimensional grayscale images of the restoration results of fragment 1 using different algorithms, where (a) is comparison algorithm 1, (b) is comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet. Fig.17The figure shows the three-dimensional grayscale images of the restoration results of fragment 2 using different algorithms, where (a) is comparison algorithm 1, (b) is comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet. Fig.18 The figure shows the three-dimensional grayscale images of the restoration results of fragment 3 using different algorithms, where (a) is comparison algorithm 1, (b) is comparison algorithm 2, (c) is MIMO-UNet, and (d) is MSEDNet.

[0103] Depend on Figures 9 to 18 It can be seen that for the blurred image containing three strip-shaped fragments, the grayscale value of fragment 3 is higher, forming an obvious grayscale difference with the background, resulting in the appearance of a directional frequency domain component in the spectrum diagram. This frequency domain component is aliased with the frequency domain component generated by the trailing stars, resulting in the inability of both traditional algorithms to accurately estimate the blur kernel from the spectrum diagram, making it impossible to restore the entire image, resulting in an obvious ringing effect. MIMO-UNet has a poor morphological restoration effect for brighter fragments. The analysis of this situation is as follows: MIMO-UNet multiplies the downsampled feature map with the features extracted from the downsampled blurred image on the encoder side, making the feature value of the highlight area abnormally large, and the decoder side cannot accurately restore it. MSEDNet has a relatively stable restoration effect on fragments.

[0104] 2) Quantitative analysis

[0105] The peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) are used as evaluation indicators to evaluate the performance of the entire image. In addition, the centroid error (CE) is also used in this embodiment to measure the improvement effect of the centroid positioning accuracy after the image is restored using the method of the present application.

[0106] Table 1 Quantitative analysis results of each algorithm on the test set

[0107]

[0108] The peak signal-to-noise ratio and structural similarity of the MSEDNet proposed in this application are higher than those of multiple comparison algorithms. MSEDNet has greater advantages in restoring star maps containing space debris.

[0109] In one embodiment, a detection image non-uniform motion blur restoration device is provided, comprising: a feature encoding module, a feature fusion enhancement module and a detection image non-uniform motion blur restoration module, wherein:

[0110] The feature encoding module is used to input the acquired space debris detection image into the encoder to obtain output features of three scales; the encoder includes three encoding modules, and the encoding module includes a convolution layer, a multi-scale feature extraction module and a residual group; the multi-scale feature extraction module is used to use dilated convolutions with different dilation rates and strip pooling operations to perform multi-scale feature extraction on features at the same resolution, and fuse the multi-scale features.

[0111] The feature fusion enhancement module is used to input the output features of the three scales into the feature enhancement layer to obtain three-layer fusion features; the feature enhancement layer includes three multi-scale feature fusion modules, and the multi-scale feature fusion module is used to perform multi-scale feature fusion on the output features of the three scales so that the fused features of each layer contain the information of the other two layers.

[0112] The detection image non-uniform motion blur restoration module is used to input the three-layer fusion features into the decoder to obtain a restored single-channel clear image; the decoder includes 3 decoding modules, the third decoding module is used to process the third-layer fusion features through residual groups and upsampling to obtain the first decoding features, the second decoding module is used to fuse the first decoding features and the second-layer fusion features, and then process them through residual blocks and upsampling to obtain the second decoding features, and the third decoding module is used to fuse the second decoding features and the first-layer fusion features, and then use residual blocks and convolution operations to obtain a restored single-channel clear image.

[0113] In one embodiment, the convolution layers in the second and third encoding modules are downsampling convolution layers; the feature encoding module is further used to input the acquired space debris detection image into the first encoding module of the encoder, and obtain the output features of the first scale after passing through the convolution layer, the multi-scale feature extraction module, and the residual group; the output features of the first scale are input into the second encoding module of the encoder, and obtain the output features of the second scale after passing through the downsampling convolution layer, the multi-scale feature extraction module, and the residual group; the output features of the second scale are input into the third encoding module of the encoder, and obtain the output features of the third scale after passing through the downsampling convolution layer, the multi-scale feature extraction module, and the residual group.

[0114] In one embodiment, the multi-scale feature extraction module in the feature encoding module includes: a horizontal hole convolution branch, a vertical hole convolution branch, a standard hole convolution branch, a horizontal strip pooling branch, a vertical strip pooling branch and a point convolution; in the multi-scale feature extraction module: the features input to the module are processed by the horizontal hole convolution branch, the vertical hole convolution branch, the standard hole convolution branch, the horizontal strip pooling branch and the vertical strip pooling branch, the obtained features are concatenated and then dimensionally reduced by point convolution to obtain fused features, the fused features are added to the features input to the module, and the features extracted by the multi-scale feature extraction module as shown in formulas (1) to (7) are obtained.

[0115] In one embodiment, the residual group in the encoder and the decoder is composed of 8 stacked residual blocks; the residual block includes two convolutional layers, a ReLU and a jump connection, and a feature fusion enhancement module, which is also used to fuse the output features of the first scale with the output features of the second scale and the third scale in a first multi-scale feature fusion module to obtain the first layer of fused features; the output features of the second scale are fused with the output features of the first scale and the third scale in a second multi-scale feature fusion module to obtain the second layer of fused features; the output features of the third scale are fused with the output features of the first scale and the second scale in a third multi-scale feature fusion module to obtain the third layer of fused features.

[0116] In one embodiment, the feature fusion enhancement module is further used to downsample and reduce the dimension of the output features of the first scale to obtain a first feature to be fused having the same size and number of channels as the second layer features; upsample and increase the dimension of the output features of the third scale to obtain a second feature to be fused having the same size and number of channels as the second layer features; add the first feature to be fused, the second feature to be fused, and the output features of the second scale by element-wise addition to obtain a combined feature; and process the combined feature through a global average pooling layer, a first point convolution layer, and a second point convolution layer to obtain a The channel weight vector corresponding to the feature is obtained; the channel weight vector is activated by the Sigmoid function and then fused with the combined feature to obtain the initial fused feature; the initial fused feature is passed through three pooling branches composed of horizontal pooling kernels and vertical pooling kernels of different sizes to extract the global context and local context in the corresponding direction to obtain strip pooling features of three scales; the multi-scale strip pooling features obtained by adding the strip pooling features of three scales are passed through the point convolution layer and the sigmoid function to obtain the attention mask; the attention mask is multiplied by the initial fused feature to obtain the weighted optimized feature.

[0117] In one of the embodiments, the feature fusion enhancement module is also used to extract the global context and local context in the corresponding direction by passing the initial fused features through three pooling branches composed of horizontal pooling cores and vertical pooling cores of different sizes, so as to obtain strip pooling features of three scales as shown in formulas (8) to (10).

[0118] The specific definition of the detection image non-uniform motion blur restoration device can refer to the definition of the detection image non-uniform motion blur restoration method mentioned above, which will not be repeated here. Each module in the above-mentioned detection image non-uniform motion blur restoration device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0119] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.19 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for restoring non-uniform motion blur of a detected image is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0120] Those skilled in the art will understand that Fig.19 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above method embodiment when executing the computer program.

[0122] In one embodiment, a computer readable memory is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0123] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0124] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for restoring non-uniform motion blur of a detected image, characterized in that: The method comprises: The acquired space debris detection image is input into an encoder to obtain output features of three scales; the encoder includes three encoding modules, the encoding module includes a convolution layer, a multi-scale feature extraction module and a residual group; the multi-scale feature extraction module is used to extract multi-scale features of features at the same resolution by using dilated convolution and strip pooling operations with different dilated rates, and fuse the multi-scale features; The output features of the three scales are input into the feature enhancement layer to obtain three layers of fused features; the feature enhancement layer includes three multi-scale feature fusion modules, and the multi-scale feature fusion module is used to perform multi-scale feature fusion on the output features of the three scales so that the fused features of each layer contain the information of the other two layers; The three-layer fusion features are input into the decoder to obtain a restored single-channel clear image; the decoder includes three decoding modules, the third decoding module is used to process the third-layer fusion features through residual groups and upsampling to obtain a first decoding feature, the second decoding module is used to fuse the first decoding feature and the second-layer fusion features, and then process them through residual blocks and upsampling to obtain a second decoding feature, and the third decoding module is used to fuse the second decoding feature and the first-layer fusion features, and then use residual blocks and convolution operations to obtain a restored single-channel clear image.

2. The method according to claim 1, characterized in that The convolutional layers in the second and third encoding modules are downsampling convolutional layers; The acquired space debris detection image is input into the encoder to obtain output features at three scales, including: Inputting the acquired space debris detection image into the first encoding module of the encoder, and obtaining the output features of the first scale after passing through the convolution layer, the multi-scale feature extraction module, and the residual group; Inputting the output features of the first scale into the second encoding module of the encoder, and obtaining the output features of the second scale after passing through the downsampling convolution layer, the multi-scale feature extraction module, and the residual group; The output features of the second scale are input into the third encoding module of the encoder, and after passing through a downsampling convolution layer, a multi-scale feature extraction module, and a residual group, output features of the third scale are obtained.

3. The method according to claim 2, characterized in that The multi-scale feature extraction module includes: a horizontal hole convolution branch, a vertical hole convolution branch, a standard hole convolution branch, a horizontal strip pooling branch, a vertical strip pooling branch and a point convolution; in the multi-scale feature extraction module: The features input to the module are processed by the horizontal dilated convolution branch, the vertical dilated convolution branch, the standard dilated convolution branch, the horizontal strip pooling branch, and the vertical strip pooling branch. The obtained features are concatenated and then dimensionally reduced by point convolution to obtain fused features. The fused features are added to the features input to the module to obtain the features extracted by the multi-scale feature extraction module: ； ； ； ； ； ； ； in, Represents the features extracted by the multi-scale feature extraction module, Represents the features input to the module, Represents the features extracted by the horizontal hole convolution branch, Represents a 1×3 dilated convolution kernel with a dilation rate of 1. Represents a 1×3 dilated convolution kernel with a dilation rate of 3. Represents a 1×3 dilated convolution kernel with a dilation rate of 5. Represents the features extracted by the vertical hole convolution branch; Represents a 3×1 dilated convolution kernel with a dilation rate of 1. Represents a 3×1 dilated convolution kernel with a dilation rate of 3. Represents a 3×1 dilated convolution kernel with a dilation rate of 5. Represents the features extracted by the standard dilated convolution branch, Represents a 3×3 dilated convolution kernel with a dilation rate of 1. Represents a 3×3 dilated convolution kernel with a dilation rate of 3. Represents a 3×3 dilated convolution kernel with a dilation rate of 5; represents the horizontal pooling operation, represents the vertical pooling operation, Represents an extended operation, represents a 1×1 convolution kernel, is the feature concatenation operation, For fusion features.

4. The method according to claim 1, characterized in that: The residual group in the encoder and the decoder is composed of 8 stacked residual blocks; the residual block includes two convolutional layers, a ReLU and a jump connection.

5. The method according to claim 1, characterized in that The output features of the three scales are input into the feature enhancement layer to obtain three layers of fusion features, including: The output features of the first scale are fused with the output features of the second scale and the third scale in the first multi-scale feature fusion module to obtain the first layer of fused features; The output features of the second scale are fused with the output features of the first scale and the third scale in the second multi-scale feature fusion module to obtain the second layer of fused features; The output features of the third scale are fused with the output features of the first scale and the second scale in the third multi-scale feature fusion module to obtain the third layer of fused features.

6. The method according to claim 5, characterized in that The output features of the first scale are fused with the output features of the second scale and the third scale in the first multi-scale feature fusion module to obtain the first layer of fusion features, including: Downsampling and feature dimension reduction operations are performed on the output features of the first scale to obtain first features to be fused that have the same size and number of channels as the second-layer features; upsampling and feature dimension increase operations are performed on the output features of the third scale to obtain second features to be fused that have the same size and number of channels as the second-layer features; the first features to be fused, the second features to be fused, and the output features of the second scale are added element by element to obtain a combined feature; After the combined features are processed by a global average pooling layer, a first point convolution layer, and a second point convolution layer, a channel weight vector corresponding to the features is obtained; The channel weight vector is activated by a Sigmoid function and then fused with the combined feature to obtain an initial fused feature; The initial fusion features are passed through three pooling branches consisting of horizontal pooling cores and vertical pooling cores of different sizes to extract global context and local context in corresponding directions, thereby obtaining strip pooling features of three scales; The multi-scale strip pooling features obtained by adding the strip pooling features of three scales are passed through the point convolution layer and the sigmoid function to obtain the attention mask; The attention mask and the initial fused feature are multiplied to obtain a weighted optimized feature.

7. The method according to claim 6, characterized in that The initial fusion features are passed through three pooling branches composed of horizontal pooling cores and vertical pooling cores of different sizes to extract the global context and local context in the corresponding direction, and the strip pooling features of three scales are obtained as follows: ； ； ； in, Represents the strip pooling feature of the first scale, Represents the strip pooling feature of the second scale, Represents the strip pooling feature of the third scale, , and Represent the pooling kernels respectively. , , The horizontal pooling operation, , and Represent the pooling kernels respectively. , , The vertical pooling operation, is a combination of features, for Convolution kernel, for Convolution kernel, is a 3×3 convolution kernel, For extended operations.

8. A detection image non-uniform motion blur restoration device, characterized in that: The device comprises: A feature encoding module is used to input the acquired space debris detection image into an encoder to obtain output features of three scales; the encoder includes three encoding modules, and the encoding module includes a convolution layer, a multi-scale feature extraction module and a residual group; the multi-scale feature extraction module is used to extract multi-scale features of features at the same resolution using dilated convolutions and strip pooling operations with different dilated rates, and fuse the multi-scale features; A feature fusion enhancement module is used to input the output features of the three scales into the feature enhancement layer to obtain three-layer fusion features; the feature enhancement layer includes three multi-scale feature fusion modules, and the multi-scale feature fusion module is used to perform multi-scale feature fusion on the output features of the three scales so that the fused features of each layer contain the information of the other two layers; The detection image non-uniform motion blur restoration module is used to input the three-layer fusion features into the decoder to obtain a restored single-channel clear image; the decoder includes 3 decoding modules, the third decoding module is used to process the third-layer fusion features through a residual group and upsampling to obtain a first decoding feature, the second decoding module is used to fuse the first decoding feature and the second-layer fusion features, and then process them through a residual block and upsampling to obtain a second decoding feature, and the third decoding module is used to fuse the second decoding feature and the first-layer fusion features, and then use a residual block and convolution operation to obtain a restored single-channel clear image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer readable memory having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images