Image motion deblurring method of intelligent reconnaissance equipment

Abstract

The invention relates to the technical field of image processing, and provides an image motion deblurring method for intelligent reconnaissance equipment, which comprises the following steps: acquiring a target image through the intelligent reconnaissance equipment, sending the target image to edge computing equipment for motion deblurring processing, and outputting a generated image; a trained semi-supervised generative adversarial network composed of a generator and a discriminator is carried in the edge computing device; wherein the generator comprises a feature pyramid network, and is used for performing motion deblurring processing on an input image and outputting a generated image; the discriminator comprises a discriminator network based on a PatchGAN network and is used for judging whether the generated clear image is true or false; when the output of the discriminator is true, outputting the generated image output by the generator as an image motion deblurring result; and when the output of the discriminator is false, updating the parameters of the generator according to a preset loss function and re-outputting the generated image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to an image motion deblurring method for an intelligent reconnaissance equipment. Background Art

[0002] As a highly integrated, technologically sophisticated, and functionally complex high-tech optoelectronic equipment, an optoelectronic pod uses optoelectronic sensors in different bands such as visible light, infrared, laser, and millimeter wave, and through a precise stabilization platform and a high-speed airborne control system, realizes functions such as searching, tracking, identifying, and measuring targets on land, sea, and air. However, due to the influence of various factors, such as motion blur, haze, sandstorm and other bad weather and data transmission errors, the images collected by the optoelectronic pod appear degraded, and the degradation of the images will seriously affect the clarity and usability of the images.

[0003] Currently, traditional image motion blur algorithms mainly calculate the blur kernel of the degraded image and then try to reverse infer the clear image. However, due to the inability to fully restore the complete blur kernel of the degraded image, the restoration effect of traditional algorithms is often unsatisfactory. With the development of deep learning, more and more scholars have turned to using convolutional neural networks to extract the feature information of the degraded image in the hope of achieving a better image restoration effect. However, deep learning algorithms require a large amount of real data sets for training, and it is difficult to obtain real data sets based on blurred images, resulting in the existing data sets mainly being blurred images generated by algorithms, thus directly affecting the quality of the image restoration effect. Summary of the Invention

[0004] The present invention provides an image motion deblurring method for an intelligent reconnaissance equipment to overcome the defects of poor image restoration effect and low real-time performance in the above-mentioned prior art.

[0005] To solve the above technical problems, the technical solution of the present invention is as follows:

[0006] An image motion deblurring method for an intelligent reconnaissance equipment, comprising the following steps:

[0007] Collect a target image through an intelligent reconnaissance equipment, send the target image to an edge computing device for motion deblurring processing, and output a generated image;

[0008] A semi-supervised generative adversarial network composed of a generator and a discriminator that has been trained is installed in the edge computing device;

[0009] Among them, the generator includes a feature pyramid network for performing motion deblurring processing on the input image and outputting a generated image;

[0010] The discriminator includes a discriminator network based on the PatchGAN network, which is used to judge the authenticity of the generated clear image; when the discriminator outputs true, the generated image output by the generator is output as the image motion deblurring result; when the discriminator outputs false, the parameters of the generator are updated according to a preset loss function and a new generated image is output.

[0011] Furthermore, the present invention also proposes a device, including a memory and a processor, where computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor executes all or part of the steps of the image motion deblurring method proposed by the present invention.

[0012] Furthermore, the present invention also proposes a storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, all or part of the steps of the image motion deblurring method proposed by the present invention are implemented.

[0013] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0014] When the present invention performs image acquisition on an optoelectronic pod, due to too long exposure time, when the object moves, it is easy to cause local motion blur phenomena such as blurring and ghosting of the photographed object. A new model algorithm is established, and a semi-supervised generative adversarial network is adopted to generate clear images in a self-supervised manner. The feature pyramid network and the dense network are incorporated into this model, greatly improving the detailed texture part of the image; at the same time, based on the PatchGAN network, an improvement is made to increase its receptive field to improve the discrimination effect of the network, indirectly improving the image restoration ability of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic flowchart of the image motion deblurring method shown according to an embodiment of the present invention.

[0016] Figure 2 It is an architecture diagram of the generator shown according to an embodiment of the present invention.

[0017] Figure 3 It is an architecture diagram of the discriminator shown according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0019] The terms used in the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The singular forms "a", "the", and "said" used in the present invention and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0020] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0021] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Embodiment 1

[0023] This embodiment proposes an image motion deblurring method for intelligent reconnaissance equipment, as Figure 1 shown, which is a schematic flow diagram of the image motion deblurring method of this embodiment.

[0024] In the image motion deblurring method proposed in this embodiment, the following steps are included:

[0025] Collect a target image through an intelligent reconnaissance device, send the target image to an edge computing device for motion deblurring processing, and output a generated image;

[0026] A semi-supervised generative adversarial network composed of a generator and a discriminator that has been trained is installed in the edge computing device;

[0027] Among them, the generator includes a feature pyramid network for performing motion deblurring processing on the input image and outputting a generated image;

[0028] The discriminator includes a discriminator network based on the PatchGAN network, which is used to judge the authenticity of the generated clear image. When the discriminator outputs true, the generated image output by the generator is used as the image motion deblurring result and output. When the discriminator outputs false, the parameters of the generator are updated according to a preset loss function and a new generated image is output.

[0029] In this embodiment, when image acquisition is performed on an optoelectronic pod, due to too long exposure time, when an object moves, it is easy to cause local motion blur phenomena such as blurring and smear of the photographed object. A new model algorithm is established, and a semi-supervised generative adversarial network is used to generate clear images in a self-supervised manner. The feature pyramid network and the dense network are integrated into this model, greatly improving the detailed texture part of the image. At the same time, based on the PatchGAN network, an improvement is made to increase its receptive field to improve the discrimination effect of the network, indirectly improving the image restoration ability of the algorithm.

[0030] In an alternative embodiment, the edge computing device uses a Jetson AGX Orin edge computing device.

[0031] As an exemplary illustration, the Jetson AGX Orin edge computing device is a GPU-accelerated computing platform with a computing power of about 200 TOPS. This device integrates a powerful GPU and has excellent parallel computing capabilities, which enables it to process multiple image processing tasks simultaneously and speed up the processing speed. Secondly, Jetson AGX Orin supports multiple mainstream programming languages, such as C / C++ and Python, and is compatible with a wide range of open-source libraries and tools, which provides rich tools and resources for the development and implementation of image processing algorithms. Thirdly, it has rich software development tools and documentation support, providing a flexible software programming environment and architecture. Users can freely customize and optimize image processing algorithms according to their needs to obtain better performance and results. Finally, this device is suitable for the power-constrained pod environment. This enables this embodiment to reduce energy consumption while ensuring high performance.

[0032] In an alternative embodiment, the generator includes a feature extraction module and a splicing module based on the feature pyramid network structure.

[0033] Among them, the feature extraction module includes a 7×7 convolutional layer and 4 dense convolutional connection modules connected in sequence, which are used to extract and output five first feature maps of different scales.

[0034] The feature extraction module further includes a channel splicing unit, which is used to upsample each level of the first feature map to the same size and then add them through an add layer to generate four second feature maps of different scales.

[0035] The second feature maps output by the feature extraction module are respectively input into the splicing module after undergoing upsampling operations of 8 times, 4 times, 2 times, and 1 time, for upsampling the second feature maps to the same size and then performing a dimension addition operation to obtain a third feature map;

[0036] The first feature map of the first scale output by the 7×7 convolutional layer is input into the splicing module, for upsampling the third feature map to the same size and then performing a dimension addition operation with the first feature map to obtain a fourth feature map;

[0037] The input image is input into the splicing module, for upsampling the fourth feature map to the same size and then performing a feature value addition operation with the input image to obtain a generated image.

[0038] Exemplarily, as Figure 2 shown, it is the architecture diagram of the generator of this embodiment.

[0039] Exemplarily, in the feature pyramid network structure of the generator, when inputting a single three-channel image of 256×256, the 7×7 convolutional layer and 4 dense convolutional connection modules extract and output 5 first feature maps of different scales, and their scales are 128, 64, 32, 16, and 8 respectively.

[0040] When the first feature map with a scale of 8 passes through the channel splicing unit, it only passes through the convolutional layer and outputs a second feature map with a scale of 8; after the second feature map with a scale of 8 is upsampled to double its original size, it then performs an addition operation with the feature values of the secondary first feature map to generate a second feature map with a scale of 16. The second feature map with a scale of 16 is further upsampled to double its original size and then performs an addition operation with the feature values of the secondary first feature map to generate a second feature map with a scale of 32. The second feature map with a scale of 32 is further upsampled to double its original size and then performs an addition operation with the feature values of the secondary first feature map to generate a second feature map with a scale of 64. Thus, 4 second feature maps of different scales are obtained.

[0041] Further, the second feature maps with scales of 8, 16, 32, and 64 respectively undergo upsampling operations of 8 times, 4 times, 2 times, and 1 time, and in the splicing module, a concat dimension addition operation is performed on the second feature maps of the same size to obtain a third feature map with a feature map size of 64×64.

[0042] Further, the third feature map is subjected to a dimension addition operation with the first feature map with a scale of 128 through a 2-fold upsampling to obtain a fourth feature map with a size of 128×128; finally, after another 2-fold upsampling, it performs a feature value addition with the original image, and finally outputs a generated image with a size of 256×256.

[0043] Among them, the 7×7 convolutional layer in the feature extraction module is used to retain the feature information of the blurred image as much as possible. A large convolutional kernel can extract most of the information of the image as much as possible. Placing it after the input layer can not only extract most of the feature information, but also will not significantly increase the actual number of weights.

[0044] Further optionally, the dense convolutional connection module includes a dense connection unit and a deformable convolutional unit.

[0045] Among them, the dense connection unit includes a BN layer, a ReLU layer, and a Transition layer connected in sequence; the Transition layer includes a BN layer, a ReLU layer, a 1×1 convolutional kernel, and an average pooling layer connected in sequence;

[0046] The deformable convolutional unit is used to calculate the offset of each pixel of the input feature map, and perform a convolutional operation after adjusting the sampling position of the convolutional kernel according to the offset.

[0047] Compared with the residual block, the dense connection unit is placed in front of the convolutional layer, which can generate better results. The purpose of batch normalization is to change the pixel values in the image to values within the normal distribution from 0 to 1. The purpose of doing this is to make all values fall into the area that is more sensitive to the activation function, which can expand the gradient and also speed up the model convergence.

[0048] The role of the 1×1 convolutional kernel set in the Transition layer is to reduce the number of channels, which is beneficial to improving the training speed of the model. If the number of channels is not restricted, a huge amount of data will be generated, which is very likely to cause memory overflow and is not conducive to training. The role of the average pooling layer is to reduce the feature map to half of its original size to generate feature maps of different scales.

[0049] In this embodiment, a deformable convolutional unit is selected. Compared with ordinary convolution, it has the ability to learn and adapt to the deformation of irregular objects. For deformable convolution, no matter how complex the object is, its deformation amount can be learned. The receptive field after deformable convolution is not rectangular but irregular in shape, and can completely cover the object surface. The receptive field of the image after standard convolution cannot completely cover the object surface, while the receptive field of the image after deformable convolution can completely cover the object surface.

[0050] Further optionally, the splicing module includes 4 concat layers for respectively performing dimensional addition operations on the second feature maps that have undergone upsampling operations of 8 times, 4 times, 2 times, and 1 time; the last concat layer is connected to a channel splicing unit, and the output end of each add layer is connected to a channel splicing unit; wherein, the channel splicing unit includes 2 convolutional layers with the same padding value.

[0051] Further optionally, the output ends of the 7×7 convolutional layer and the dense convolutional connection module are respectively connected to 1×1 convolutional kernels for unifying the dimensions of the output feature maps and making the number of channels generated by the first feature maps of each scale unified.

[0052] In an optional embodiment, the discriminator includes 6 convolutional layers; among them, in the second-level convolutional layer to the fourth-level convolutional layer connected in sequence, each convolutional layer is connected to an instance normalization layer and an activation function layer; the receptive field of the discriminator is 142.

[0053] The discriminator in this embodiment improves the PatchGAN network. By expanding its receptive field to evaluate the evaluation value of each local area, the receptive field can cover the surface of the object as much as possible, so that all the feature information of the object can be obtained, and then the evaluation value of this area can be better evaluated.

[0054] Exemplarily, as Figure 3 shown, it is the architecture diagram of the discriminator of this embodiment.

[0055] The original PatchGAN network is composed of 5 convolutional layers, and its receptive field is 70. This embodiment expands its receptive field by adding one convolutional layer. The receptive field of the improved discriminator network is 142, which improves the discrimination ability of the discriminator without increasing a large number of parameters. Finally, a matrix with a size of 16×16×1 is generated. By averaging the matrix values, it is finally evaluated whether this single image is real. If the picture is fake, the feedback data is sent to the generator for data update.

[0056] As an exemplary illustration, the calculation formula of the receptive field is as follows:

[0057] field=(output size -1)·S + k size

[0058] wherein, output size represents the output size of the image; S is the stride; k size is the convolutional kernel size. It can be seen that compared with the original PatchGAN network in this embodiment, the receptive field range has doubled.

[0059] In an alternative embodiment, the method further comprises the following steps:

[0060] Collect training data and train the generative adversarial network; wherein, in combination with the L1 mean absolute error loss and the perceptual loss, use the VGG16 network to extract the feature maps of the generated images, and calculate the generator loss value loss through the mean square error G ; Its expression is:

[0061] loss G =λ·L1 + γL perceptual

[0062]

[0063] wherein, L1 represents the L1 error loss function, and the principle of the L1 loss function is to take the difference between the true value y i and the generated value f(x i ) and finally take the average; L perceptual represents the perceptual loss function, and the perceptual loss function refers to comparing the feature vector output after convolving the real sample with the feature vector obtained from the constructed sample. Generally, the mean square error function is used for feature map comparison to make their content in feature information similar; λ is the weight coefficient; γ is the weight coefficient; y i is the true value, f(x i ) is the pixel value of the i-th pixel in the generated image, n is the number of pixels in the generated image; φ p,q (·) represents the feature map obtained in the q-th layer of the p-th convolution in the VGG16 network, W p,q and H p,q respectively represent the width and height of the feature map; I HR and I LR respectively represent the real image and the generated image;

[0064] Use the WGAN-GP network to optimize the objective function of the generative adversarial network and calculate the discriminator loss value loss D ; Its expression is:

[0065]

[0066] wherein, E[·] represents the expected value of the distribution function; Pg represents the real data distribution, Pr represents the noise distribution, represents the interpolation data distribution; D(·) represents the output of the discriminator; represents the sample sampled from the real data distribution; x represents the sample sampled from the noise distribution; represents the gradient operator.

[0067] In this embodiment, the VGG network is selected to extract the image feature map. In the field of image reconstruction, the mean squared error loss function is generally used as the image pixel space loss function. However, this loss cannot generate good image samples and is gradually replaced by the perceptual loss function. The perceptual loss calculates the square of the difference of multi-dimensional feature maps. Therefore, using the perceptual loss can construct higher-quality images.

[0068] The real image and the generated image are put into the VGG16 network to obtain the feature maps. It is set to compare the feature maps obtained from the third layer of convolution in the third convolution of the VGG16 network, and the loss value is obtained by using the mean squared error.

[0069] In an alternative embodiment, the method further includes the following steps:

[0070] The generated image output by the edge computing device is transmitted back to the cloud server through the network protocol; at the same time, the edge computing device forms a self-organizing network and transmits the generated image to a preset mobile terminal;

[0071] The cloud server analyzes the received generated image, generates real-time and / or offline reports, visualization charts and / or alarm signals and transmits them to a preset mobile terminal.

[0072] The edge computing device and the cloud in this embodiment form a closely collaborative architecture, making the links such as image data acquisition, real-time processing, cloud analysis, and mobile terminal presentation more efficient, reliable, and effective. This provides users with faster, more flexible, and intelligent image analysis and data processing services.

[0073] Exemplarily, in the specific implementation process, the user remotely controls the movement of the pod-mounted device through the mobile terminal device and sends an image acquisition instruction to the cloud intelligent control computing center. The cloud dispatches a collaborative work task to the edge computing device according to the user instruction, and the edge computing device acquires image data according to the task content.

[0074] Furthermore, the edge computing device is used for image processing tasks. The GPU of the edge computing device is used for parallel computing to accelerate the image deblurring image processing algorithm. Then the optimized image processing algorithm and model are run to improve the processing speed and accuracy.

[0075] The edge computing device performs real-time processing, calculation, and storage on the image data collected by the camera, and then returns the processing result to the cloud. The cloud analyzes the processing result of multi-machine collaboration and presents the analysis result on the mobile terminal device according to the user's needs. In addition, there is also a self-organizing network between the edge computing devices to present the image results processed by the edge devices on the mobile terminal device, completing the image motion deblurring process.

[0076] The optoelectronic pod applied in this embodiment is a camera device mounted on a robotic arm or a jib crane, and is used for monitoring and controlling the pod operation. In the optoelectronic pod, image acquisition is performed by a camera.

[0077] Exemplarily, the image acquisition process mainly includes the following steps:

[0078] 1) Use a high-resolution camera: The camera in the optoelectronic sensor of the optoelectronic pod consists of a single-photon camera, an infrared camera, and a high-resolution camera.

[0079] 2) Install and position the camera: Install the camera on the optoelectronic pod, and determine its viewing range and field of view by adjusting the angle and position. The position and angle of the camera should be adjusted according to the specific application situation so as to be able to comprehensively capture the image information of the pod operation area.

[0080] 3) Connect and configure the camera: Integrate and install the camera into the optoelectronic pod, and perform corresponding configurations as needed. The configurations include setting parameters such as image resolution, frame rate, exposure time, and sensor gain.

[0081] 4) Perform image acquisition: Once the camera is connected and configured, the user can start remotely controlling the movement of the pod-mounted device through a mobile device and send commands. The camera will regularly generate image frames and transmit them through digital signals. The edge computing device of the optoelectronic pod will receive and process these image frames, thereby realizing real-time monitoring of the pod operation area.

[0082] Through the above steps, the optoelectronic pod can realize image acquisition of the operation area.

[0083] Furthermore, a single blurred image collected by the optoelectronic pod is subjected to feature extraction by a feature extraction module based on a feature pyramid network structure, and feature maps of different scales are output; secondly, a single three-channel image of 256×256 is input, and feature maps are extracted through a dense convolution connection module. The feature maps of each scale are subjected to upsampling and addition operations to generate a clear image. Finally, after a dimension addition operation, a feature map is output, and through upsampling and dimension addition operations, the final clear generated image is obtained.

[0084] The generated image output by the generator is input into the discriminator for judgment. The discriminator increases the convolutional layer through the PatchGAN network to expand the receptive field and improve the discrimination ability of the discriminator. The discriminator finally generates a matrix of 16×16×1 size, and the authenticity of the image is evaluated by averaging the matrix values.

[0085] Furthermore, the processed image data results can be temporarily or permanently stored within the edge computing device. Meanwhile, the edge computing device transmits the processing results back to the cloud server via a network protocol. After receiving the image data results transmitted back by the edge computing device, the cloud server uses more powerful computing capabilities, storage, and algorithms for further analysis and processing.

[0086] The results processed by the cloud can be presented through a mobile device. Through real-time or offline reports, visual charts, alerts, or other forms of information presentation.

[0087] An ad-hoc network can also be formed among the edge computing devices within the pod to share and present the processed image results of each other on the mobile device. This collaboration can improve real-time performance, accuracy, and reliability.

[0088] When performing cloud-edge-end collaborative processing on the optoelectronic pod in this embodiment, the entire system will be integrated through real-time data processing of the pod, enabling it to comprehensively handle various processing services.

[0089] Embodiment 2

[0090] This embodiment proposes a device, including a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor is caused to execute all or part of the steps of the image motion deblurring method proposed in Embodiment 1.

[0091] Embodiment 3

[0092] This embodiment proposes a storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, all or part of the steps of the image motion deblurring method proposed in Embodiment 1 are implemented.

[0093] Exemplarily, the storage medium includes but is not limited to various media that can store program codes, such as USB flash drives, external hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0094] Exemplarily, the instructions, programs, code sets, or instruction sets can be implemented using conventional programming languages.

[0095] Exemplarily, the processor includes but is not limited to smartphones, personal computers, servers, network devices, etc., and is used to execute all or part of the steps of the image motion deblurring method described in Embodiment 1.

[0096] The terms used in the drawings are for illustrative purposes only and should not be construed as limiting the present invention;

[0097] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all implementation manners here. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A method for image motion deblurring of intelligent reconnaissance equipment, characterized in that: The following steps are involved: Collect target images through intelligent reconnaissance equipment, send the target images to edge computing devices for motion deblurring, and output generated images; The edge computing device is equipped with a trained semi-supervised generative adversarial network consisting of a generator and a discriminator; Wherein, the generator includes a feature pyramid network, which is used to perform motion deblurring on the input image and output a generated image; The discriminator includes a discriminator network based on the PatchGAN network, which is used to judge the true or false of the generated clear image; when the discriminator output is true, the generated image output by the generator is output as the image motion deblurring result; when the discriminator output is false, the parameters of the generator are updated according to a preset loss function and the generated image is re-output.

2. The image motion deblurring method according to claim 1, characterized in that: The generator includes a feature extraction module and a splicing module based on a feature pyramid network structure; wherein: The feature extraction module includes a 7×7 convolution layer and four dense convolution connection modules connected in sequence, which are used to extract and output first feature maps of five different scales; The feature extraction module also includes a channel splicing unit, which is used to upsample the first feature maps of each level to the same size and then add them through an add layer to generate four second feature maps of different scales; The second feature map output by the feature extraction module is respectively input into the splicing module after being upsampled by 8 times, 4 times, 2 times and 1 times, so as to perform a dimensional addition operation after upsampling the second feature map to the same size, thereby obtaining a third feature map; The first feature map of the first scale output by the 7×7 convolutional layer is input into a concatenation module, which is used to upsample the third feature map to the same size and then perform a dimension addition operation with the first feature map to obtain a fourth feature map; The input image is input to a stitching module, which is used to upsample the fourth feature map to the same size and then perform a feature value addition operation with the input image to obtain a generated image.

3. The image motion deblurring method according to claim 2, characterized in that: The dense convolution connection module includes a dense connection unit and a deformable convolution unit; The densely connected unit includes a BN layer, a ReLU layer, and a Transition layer connected in sequence; the Transition layer includes a BN layer, a ReLU layer, a 1×1 convolution kernel, and an average pooling layer connected in sequence; The deformable convolution unit is used to calculate the offset of each pixel of the input feature map, and adjust the sampling position of the convolution kernel according to the offset and then perform the convolution operation.

4. The image motion deblurring method according to claim 2, characterized in that: The concatenation module includes 4 concat layers, which are used to perform dimension addition operations on the second feature maps that have been upsampled by 8 times, 4 times, 2 times, and 1 times, respectively; The last concat layer is connected to a channel splicing unit, and the output end of each add layer is connected to a channel splicing unit; wherein the channel splicing unit includes two convolutional layers with the same padding value.

5. The image motion deblurring method according to claim 2, characterized in that: In the generator, the output ends of the 7×7 convolution layer and the dense convolution connection module are respectively connected to 1×1 convolution kernels.

6. The image motion deblurring method according to claim 1, characterized in that: The discriminator includes 6 convolutional layers; among the sequentially connected 2nd to 4th convolutional layers, each convolutional layer is connected to an instance normalization layer and an activation function layer; the receptive field of the discriminator is 142.

7. The image motion deblurring method according to any one of claims 1 to 6, characterized in that: The method further comprises the following steps: Collect training data and train the generative adversarial network; in which, the VGG16 network is used to extract the feature map of the generated image by combining the L1 mean absolute error loss and the perceptual loss, and the generator loss value loss is calculated by the mean square error G ; Its expression is: loss G =λ·L1+γL perceptual Among them, L1 represents the L1 mean absolute error loss, L perceptual represents the perceptual loss; λ is the weight coefficient; γ is the weight coefficient; y i is the true value, f(x i ) is the i-th pixel value in the generated image, n is the number of pixels in the generated image; φ p,q (·) represents the feature map obtained in the qth layer of the pth convolution in the VGG16 network, W p,q and H p,q Respectively represent the width and height of the feature map; I HR and I LR Represent the real image and the generated image respectively; Use the WGAN-GP network to optimize the objective function of the generated adversarial network and calculate the discriminator loss value D ; Its expression is: Where E[·] represents the expected value of the distribution function; Pg represents the real data distribution, Pr represents the noise distribution, represents the interpolated data distribution; D(·) represents the output of the discriminator; represents samples sampled from the real data distribution; x represents samples sampled from the noise distribution; Represents the gradient operator.

8. The image motion deblurring method according to any one of claims 1 to 6, characterized in that: The method further comprises the following steps: The generated image output by the edge computing device is transmitted back to the cloud server through the network protocol; at the same time, the edge computing device forms a self-organizing network and transmits the generated image to a preset mobile terminal; The cloud server analyzes the received generated images, generates real-time and / or offline reports, visualization charts and / or alarm signals, and transmits them to a preset mobile terminal.

9. A device comprising a memory and a processor, wherein the memory stores computer-readable instructions, characterized in that: When the computer-readable instructions are executed by the processor, the processor executes all or part of the steps of the image motion deblurring method according to any one of claims 1 to 8.

10. A storage medium having computer-readable instructions stored thereon, characterized in that: When the computer-readable instructions are executed by a processor, all or part of the steps of the image motion deblurring method according to any one of claims 1 to 8 are implemented.

Images