Method for deblurring optical images acquired by underwater robots of hydropower dam
By combining depth estimation, bilateral mesh, and feature reconstruction modules, the problem of image blurring for underwater robots was solved, achieving efficient and accurate image deblurring and improving image quality and processing efficiency for hydropower station dam inspection.
Patent Information
- Application Number
- CN202410230193.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-29
- Publication Date
- 2026-01-20
- Estimated Expiration
- 2044-02-29
AI Technical Summary
Optical images acquired by underwater robots often suffer from blurring, distortion, and noise, affecting image clarity and accuracy and limiting their application in hydropower dam inspection.
A combined approach of depth estimation module, bilateral grid module, and feature reconstruction module is adopted. The depth estimation module estimates the depth map, the bilateral grid module processes the image edge information, and the feature reconstruction module fuses the depth information and bilateral grid information to generate a clear image.
It achieves efficient and accurate image deblurring of real-time video streams acquired by underwater robots, improving image quality and processing efficiency, providing more accurate color perception and depth information, enhancing the ability to extract edge features, and improving the speed and adaptability of image processing.
Smart Images

Figure CN118071643B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing and recognition, in particular to a method for deblurring optical images obtained by underwater robots of hydropower station dams. BACKGROUND
[0002] Underwater robots are widely used in the detection and maintenance of hydropower station dams. By carrying an optical imaging system, underwater robots can obtain image information of dam structures in underwater environments, providing important data support for dam maintenance and repair. However, due to the reflection of particles in water on light, when the reflected light reaches the camera, it will cause scattering effects on imaging, scattering makes the details of imaging blurred, affecting the quality of the image, and due to the complex influence of different depths, positions, seasons and air temperature on water, the difficulty of image deblurring is increased. The optical images obtained by underwater robots often have problems such as blurring, distortion and noise. These problems seriously affect the clarity and accuracy of the image, thereby limiting the application effect of underwater robots in the detection of hydropower station dams.
[0003] At present, some methods for deblurring underwater images have been proposed. Common methods are mainly divided into two categories: underwater single image deblurring methods based on prior knowledge and underwater single image deblurring methods based on learning. The deblurring method based on prior knowledge estimates the intermediate variable and the background light according to the physical prior assumption, and then further estimates the transmission function, but the estimation error of the intermediate variable may be amplified, which finally affects the deblurring effect. Early single image deblurring methods based on learning focus on estimating the transmission function and background light through neural networks, and then restoring underwater blurred images. However, in the case of a single image, it is often difficult to accurately estimate the transmission function and the global background light, so the deblurring effect is not very ideal, and these methods are not truly end-to-end algorithms, which cannot directly map input images to output images. By using a deep convolutional neural network, a clear image can be estimated directly from a blurred underwater image through step-by-step optimization. However, on the one hand, methods based on physical models are usually limited by prior knowledge, which leads to color distortion in the deblurring task, and poor generalization ability in different scenes. On the other hand, although learning-based methods can better adapt to different scenes, they usually do not effectively utilize the depth information in the image, which leads to the difficulty of completely removing the blur phenomenon in the distance of the image, and the edge information may be missing when generating deblurred images. SUMMARY
[0004] The technical problem to be solved by the present application is to provide a method for deblurring optical images obtained by underwater robots of hydropower station dams, which can effectively remove underwater image blur and improve the image quality and processing efficiency of underwater robots in the detection of hydropower station dams.
[0005] To solve the above technical problems, the technical scheme adopted by the present application is: a method for deblurring optical images obtained by underwater robots of a hydropower station dam, comprising the following steps:
[0006] S1, the optical imaging system of the underwater robot platform real-time acquires video stream data of the underwater environment;
[0007] S2, input the collected video stream data of the underwater environment into the deblurring network for processing, and the construction process of the deblurring network is as follows:
[0008] S201, constructing a depth estimation module to estimate a depth map
[0009] The depth estimation module scales the original underwater blurred picture, and obtains a depth map through the depth estimation module;
[0010] S202, constructing a bilateral grid module to process edge information of the image
[0011] The bilateral grid module scales the original underwater blurred picture, and then extracts a feature map through the Unet network, and obtains a bilateral grid feature map through global average pooling;
[0012] S203, constructing a feature reconstruction module to fuse depth information and bilateral grid information
[0013] The feature reconstruction module guides the bilateral network feature map extracted by the bilateral network module with the depth map obtained in step S201 to obtain a sampling feature map, then performs element-wise dot product on the sampling feature map obtained by the feature reconstruction module and the feature map extracted by the Unet network to generate a reconstruction feature map, and the generated reconstruction feature map is subjected to residual error with the original image through two convolution layers to estimate a deblurred image.
[0014] S204, network model training.
[0015] In the preferred scheme, in step S201, the depth estimation module is composed of an encoder and a decoder, the encoder is composed of a series of consecutive dilated convolution modules CDC and a local-global feature interaction module LGFI, and the decoder adopts bilinear upsampling and a reflective nonlinear convolution module RCE to restore the features of four stages of the encoder into a depth map with the same size as the input.
[0016] In the preferred scheme, the CDC module extracts multi-scale features using dilated convolution, and uses dilated convolution with different expansion rates at each stage, and the operation of the dilated convolution is as follows:
[0017]
[0018] Wherein, w represents the value of the convolution kernel, γ indicates the expansion rate for convolution input x[i], i indicates the value of the i-th position feature map, k is the index of the convolution kernel weight, and K is the number of convolution kernel weights;
[0019] When the CDC module inputs the feature map X of HxWxC, the output expression of the CDC module is:
[0020]
[0021] Wherein, X is the input feature map of the CDC module, is the output feature map of the CDC module, L P represents point convolution, L PG indicates point-by-point convolution operation and GELU activation, L BN represents a batch normalization layer, Conv r is a 3x3 dilated convolution with an expansion rate of γ;
[0022] When the LGFI module inputs the feature map X of HxWxC, the feature map will be mapped to Q, K and V first, and the specific expression is as follows:
[0023] Q=XW q ,K=XW k ,V=XW v (3)
[0024]
[0025]
[0026] Wherein, X is the input feature map of the LGFI module, is the output feature map of the LGFI module, W q , W k , W v is a parameter to be learned, L LN represents a normalization layer, Q, K and V represent query, key and value respectively, and Attention(Q, K, V) is a cross-covariance attention mechanism;
[0027] Attention(Q,K,V ) = V · Softmax(Q T · K) (6)
[0028] Wherein, Softmax(Q T ·K) represents calculating the similarity between query and key by using Softmax.
[0029] In the preferred scheme, the decoder adopts a reflective nonlinear convolution module RCE, and the specific operation is as follows:
[0030]
[0031] wherein X is an input feature map, is an output feature map, Conv is a 3x3 convolution, L ELU is an exponential linear unit ELU activation function, L RP is a reflective padding layer, the main operation of which is to symmetrically fill a pixel around the edge.
[0032] In a preferred solution, in the step S202, the picture is first fixed to a size of (256, 256) using bicubic interpolation, the multiscale features of the image are extracted by Unet to generate a feature map of (3, 256, 256), and then the feature map is converted into a low-resolution 12x16x16x16 bilateral grid feature through global average pooling.
[0033] In a preferred solution, in the step S203, the depth map is first upsampled to the original image size using bilinear interpolation, the upsampled depth map is then spliced with the two grid maps to form a three-channel sampling map, the three-channel sampling map is then used to sample the bilateral grid feature using cubic linear interpolation to form a sampling feature map with an original image size of (640, 480) and a channel number of 12, and finally the element-wise dot product of the (12, 640, 480) feature map obtained by the feature reconstruction module and the (3, 640, 480) feature map extracted by Unet is performed to generate a reconstruction feature map with a channel number of three and an original image size, the expression of which is as follows: Figure One
[0034]
[0035] wherein C i is the i-th channel feature extracted by Unet, f(k) is the feature map after sampling, is the output of the feature reconstruction module.
[0036] In a preferred solution, in the step S204, the training of the network model uses L1 loss and perceptual loss, wherein the perceptual loss uses the mean square error of the output feature map with a receptive field of 68x68 of VGG19, and the loss function L is expressed as follows:
[0037] L = λ1L1+λ2L p (9)
[0038] wherein λ1 is the coefficient of the L1 loss function, and λ2 is the coefficient of the perceptual loss function. p
[0039] The expression of the L1 loss function is as follows:
[0040]
[0041] wherein I(i) represents the value of the pixel at the i-th position of the clear image, I dehaze represents the pixel value at the i-th position of the underwater blurred image;
[0042] L p The loss function is specifically as follows:
[0043]
[0044] wherein VGG N represents the feature map with a VGG19 receptive field of N×N, I represents the clear image, and I dehaze represents the underwater blurred image.
[0045] The optical image deblurring method provided by the application has the following
[0046] Beneficial effects:
[0047] 1. The method realizes efficient and accurate image deblurring processing of real-time video streams collected by underwater robots, and provides important technical support for detection and maintenance of hydropower station dams.
[0048] 2. Step S201 introduces a lightweight depth estimation module, which can effectively estimate the distance of underwater objects on the basis of accelerating calculation processing. This method not only optimizes the utilization of computing resources, but also helps to improve the quality of underwater image color restoration, providing more accurate color perception and depth information for underwater vision systems.
[0049] 3. Step S202 adopts an image scaling combined with bilateral grid feature extraction method, which not only optimizes the calculation speed, but also enhances the accurate extraction of edge features, effectively improving the overall processing performance.
[0050] 4. The feature reconstruction module of step S203 extracts features through a bilateral network guided by a depth map, realizes the generation of a sampling feature map, significantly improves the reconstruction quality of image details, and accelerates the image processing flow. This strategy effectively enhances the feature extraction capability, improves the image processing speed and adaptability, and ensures the high resolution and clarity of the reconstructed image. BRIEF DESCRIPTION OF DRAWINGS
[0051] The application will be further described below in conjunction with the drawings and examples:
[0052] Figure 1 The network architecture diagram of the application;
[0053] Figure 2 The depth estimation module diagram of the application;
[0054] Figure 3 A processing flowchart for the CDC module of the application;
[0055] Figure 4 A processing flowchart for the LGFI module of the application;
[0056] Figure 5 A processing flowchart for the RCE module of the application;
[0057] Figure 6 A Unet network diagram;
[0058] Figure 7 A feature reconstruction module diagram;
[0059] Figure 8 A before-and-after comparison diagram for underwater image deblurring using the method; DETAILED DESCRIPTION
[0060] In combination Figures 1-7 The specific embodiments of the application are further described in detail.
[0061] A method for deblurring optical images obtained by underwater robots of a hydropower station dam, comprising the following steps:
[0062] S1, the optical imaging system of the underwater robot platform real-time takes the video stream data of the underwater environment.
[0063] S2, input the video stream data collected in the underwater environment into the deblurring network for processing, as shown in Figure 1 The construction process of the deblurring network is as follows:
[0064] S201, construct a depth estimation module to estimate a depth map
[0065] The depth estimation module scales the original (640, 480) underwater blurred picture to (320, 320), and obtains a 1x320x320 depth map through the depth estimation module, and upsamples the depth map to 1x640x480.
[0066] As shown in Figure 2 The depth estimation module is composed of an encoder and a decoder, the encoder is composed of a series of consecutive expansion convolution modules CDC and a local-global feature interaction module LGFI, and the decoder adopts bilinear upsampling and a reflection nonlinear convolution module RCE to restore the features of the four stages of the encoder into a depth map with the same size as the input.
[0067] As shown in Figure 3As shown, the CDC module extracts multi-scale features using dilated convolution. In each stage, a dilated convolution with different dilation rates is used to achieve a larger receptive field while keeping the size of the output feature map unchanged. The operation of dilated convolution is as follows:
[0068]
[0069] where w represents the value of the convolution kernel, γ represents the dilation rate for the convolution input x[i], i is the value of the i-th position feature map, k is the index of the convolution kernel weight, and K is the number of convolution kernel weights.
[0070] When the CDC module inputs a feature map X of HxWxC, the output expression of the CDC module is:
[0071]
[0072] where X is the input feature map of the CDC module, is the output feature map of the CDC module, L P represents point convolution, L PG represents point-wise convolution operation and GELU activation, L BN represents a batch normalization layer, Conv r is a 3x3 dilated convolution with a dilation rate of γ.
[0073] As shown in Figure 4 , when the LGFI module inputs a feature map X of HxWxC, the feature map will first be mapped to Q, K, and V. The specific expression is as follows:
[0074] Q=XW q ,K=XW k ,V=XW v (3)
[0075]
[0076]
[0077] where X is the input feature map of the LGFI module, is the output feature map of the LGFI module, W q , W k , W v are parameters to be learned, Q, K, and V represent Query, Key, and Value, respectively, and Attention(Q, K, V) is a cross-covariance attention mechanism.
[0078] Attention(Q, K, V)=V·Softmax(Q T ·K) (6)
[0079] where Softmax(Q T • K) represents the similarity between the query and the key is calculated using Softmax, and these similarities are converted into probabilities (weights) to determine the importance of each value in the final output.
[0080] The encoder first scales the original image of (640, 480) to the size of 320x320, and performs 3 global average pooling operations on the 3x320x320 image to obtain 2, 4 and 8 times down-sampling pictures respectively. In the first stage, the original picture is subjected to a down-sampling convolution layer with a step of 2 and a convolution kernel of 3 to obtain a feature map of 48x160x160; in the second stage, the 48x160x160 feature map obtained in the first stage is spliced with the 2 times down-sampled 3x160x160 picture to obtain a feature map of 51x160x160, and then subjected to a down-sampling layer to obtain a feature map of 48x80x80, and then subjected to 3 CDC modules, the expansion convolution expansion rates in the CDC modules are 1, 2 and 3 respectively, to obtain a feature map of 48x80x80, and finally subjected to an LGFI module to obtain an attention feature map of 48x80x80; in the third stage, the down-sampled 48x80x80 feature map obtained in the second stage, the 4 times down-sampled 3x80x80 picture and the 48x80x80 attention feature map output in the second stage are spliced to obtain a feature map of 99x80x80, and then subjected to a down-sampling layer to obtain a feature map of 80x40x40, and then subjected to 3 CDC modules, the expansion convolution expansion rates in the CDC modules are 1, 2 and 3 respectively, to obtain a feature map of 80x40x40, and finally subjected to an LGFI module to obtain an attention feature map of 80x40x40; in the fourth stage, the down-sampled 80x40x40 feature map obtained in the third stage, the 8 times down-sampled 3x80x60 picture and the 80x40x40 attention feature map output in the third stage are spliced to obtain a feature map of 163x40x40, and then subjected to a down-sampling layer to obtain a feature map of 128x20x20, and then subjected to 9 CDC modules, the expansion convolution expansion rates in the CDC modules are 1, 2, 3, 1, 2, 3, 2, 4 and 6 respectively, to obtain a feature map of 128x20x20, and finally subjected to an LGFI module to obtain an attention feature map of 128x20x20.
[0081] As shown in Figure 5 , the decoder adopts a reflective nonlinear convolution module RCE, adopts a reflection padding (ReflectionPad2d) to prevent loss of spatial information, and adopts an exponential linear unit (ELU) activation function to increase the nonlinearity of the model. The specific operation is as follows:
[0082]
[0083] wherein X is an input feature map, is an output feature map, Conv is a 3x3 convolution, L ELU is an exponential linear unit ELU activation function, L RP is a reflective padding layer, the main operation is to fill a pixel around the edge symmetrically.
[0084] In the embodiment, the fourth stage of the decoder first passes the attention feature map of 128x20x20 obtained by the fourth stage of the encoder through an RCE module to obtain a feature map of 64x20x20, and adopts bilinear upsampling to obtain a feature map of 64x40x40; the third stage of the decoder splices the feature map of 64x40x40 obtained by upsampling of the fourth stage of the decoder and the attention feature map of 80x40x40 obtained by the third stage of the encoder to obtain a feature map of 144x40x40, and passes the feature map through an RCE module to obtain a feature map of 64x40x40; the second stage of the decoder passes the feature map of 64x40x40 obtained by upsampling of the third stage of the decoder through an RCE module to obtain a feature map of 40x40x40, adopts bilinear upsampling to obtain a feature map of 40x80x80, splices the obtained feature map of 40x80x80 and the attention feature map of 48x80x80 obtained by the second stage of the encoder to obtain a feature map of 88x80x80, and passes the feature map through an RCE module to obtain a feature map of 40x80x80; the feature map of 40x80x80 output by the second stage of the decoder is passed through an RCE module to obtain a feature map of 24x80x80, and bilinear upsampling is adopted to obtain a feature map of 24x160x160; finally, the feature map of 24x160x160 is passed through an RCE module to obtain a feature map of 24x160x160, bilinear upsampling is adopted to obtain a feature map of 24x320x320, the feature map is passed through an RCE module to obtain a feature map of 1x320x320, and an activation function is adopted to output a depth map.
[0085] S202, construct a bilateral grid module to process edge information of the image
[0086] Due to the low contrast and edge blur of the blurred image and other factors, the intensity values of two pixel points on the boundary are very close. By adopting bilateral filtering, the edge information of the image can be better processed. The bilateral grid module scales the original (640, 480) underwater blurred picture to (256, 256), and then extracts a feature map of 3x256x256 through the Unet network, and obtains a bilateral grid feature map of 12x16x16x16 through global average pooling.
[0087] As shown in Figure 7 , first, fix the original (640, 480) image to (256, 256) size by using bicubic interpolation, and then pass the image through the Unet network to obtain a feature map of 3x256x256, and finally obtain a bilateral grid feature map of 12x16x16x16 through global average pooling. Figure 6The Unet shown extracts multi-scale features from the image, generating a (3,256,256) feature map, which is then converted into a low-resolution 12×16×16×16 bilateral grid feature map through global average pooling.
[0088] S203, Construct a feature reconstruction module that integrates depth information and bilateral mesh information.
[0089] To better utilize the low-resolution features extracted by bilateral meshes, this invention proposes a bilateral mesh slicing method guided by depth information.
[0090] The feature reconstruction module uses the 1×640×480 depth map obtained in step S201 to guide the sampling of the 12×16×16×16 bilateral mesh feature map extracted by the bilateral network module, and then performs the principal element dot product between the sampled feature map obtained by the feature reconstruction module and the network-extracted feature map of Unet to generate the reconstructed feature map.
[0091] First, the depth map is upsampled to the original image size using bilinear interpolation. Then, the sampled depth map is stitched together with two mesh maps to form a three-channel sampled map. Next, the two-sided mesh features are sampled using cubic linear interpolation on the three-channel sampled map to form a sampled feature map with dimensions (640, 480) equal to the original image size and 12 channels. Finally, the (12, 640, 480) feature map obtained by the feature reconstruction module is multiplied element-wise with the (3, 640, 480) feature map extracted by Unet to generate a three-channel feature map with dimensions equal to the original image size and 12 channels. Figure One The reconstructed feature map of sample size is expressed as follows:
[0092]
[0093] Among them, C i Let f(k) be the feature of the i-th channel extracted by Unet, and f(k) be the sampled feature map. This is the output of the feature reconstruction module.
[0094] The feature reconstruction module first upsamples the 1×320×320 depth map obtained from the depth estimation module to the original image dimensions of 1×640×480 using bilinear interpolation. Then, it concatenates the sampled 1×640×480 depth map with two 1×640×480 grid maps to form a 3×640×480 three-channel sampled image. Next, it uses this three-channel sampled image to sample a 12×16×16×16 bilateral grid feature map using cubic linear interpolation, resulting in a 12×640×480 feature map with dimensions equal to the original image size. Finally, it compares this feature map with the original image dimensions. Figure FiveThe feature reconstruction module performs element-wise dot products with the features extracted by Unet to generate a 3×640×480 reconstructed feature map. The generated reconstructed feature map is then processed by two convolutional layers and residuals are calculated with the original image to estimate the deblurred image.
[0095] Through the above steps, this invention achieves efficient and accurate image deblurring processing of real-time video streams collected by underwater robots, providing important technical support for the inspection and maintenance of hydropower station dams.
[0096] The network model is trained using L1 loss and perceptual loss. The perceptual loss uses the mean squared error of the output feature map of VGG19 with a receptive field of 68x68 to enhance the model's learning of image details, style and color.
[0097] The loss function L is expressed as follows:
[0098] L = λ1L1 + λ2L p (9)
[0099] Where λ1 is the coefficient of the L1 loss function, and λ2 is the coefficient of the L1 loss function. p The coefficients of the perceptual loss function;
[0100] The L1 loss function expression is as follows:
[0101]
[0102] Where I(i) represents the value of the pixel at the i-th position in the sharp image, I dehaze This represents the pixel value at the i-th position of the blurred underwater image;
[0103] L p The loss function is as follows:
[0104]
[0105] Among them, VGG N This represents a feature map with an N×N receptive field for VGG19, where I represents a clear image. dehaze This represents a blurred underwater image.
[0106] The deblurred image is evaluated using a loss function L. Specifically, L1 evaluates the pixel difference between the deblurred image I and the original sharp image; while L... p The evaluation assesses the difference between the deblurred image and the original sharp image on the VGG19 network output feature map.
[0107] Taking underwater images collected at Gezhouba Dam as an example, a comparison of images before and after deblurring is shown below. Figure 8As shown in the pictures, the reconstruction quality of image details is significantly improved after processing, ensuring high resolution and clarity of the reconstructed image.
Claims
1. A method for deblurring optical images acquired by an underwater robot at a hydroelectric dam, characterized in that, Includes the following steps: S1, The optical imaging system of the underwater robot platform acquires video stream data of the underwater environment in real time; S2. The video stream data of the underwater environment is collected and input into the deblurring network for processing. The construction process of the deblurring network is as follows: S201, Construct the depth estimation module to estimate the depth map. The depth estimation module scales up the original blurred underwater image and then obtains a depth map. The depth estimation module consists of an encoder and a decoder. The encoder consists of a series of consecutive dilated convolutional modules (CDC) and a local global feature interaction module (LGFI). The decoder uses a bilinear upsampling and reflection nonlinear convolutional module (RCE) to restore the features of the encoder's four stages into a depth map of the same size as the input. The CDC module utilizes dilated convolution to extract multi-scale features, employing dilated convolution with different dilation rates at each stage. The dilated convolution operation is as follows: (1); Where w represents the value of the convolution kernel, Indicates the input used for convolution. The inflation rate is denoted by i, where i represents the value of the feature map at the i-th location, k is the index of the convolution kernel weights, and K is the number of convolution kernel weights. When the CDC module input Feature map At that time, the output expression of the CDC module is: (2); in, This is the input feature map for the CDC module. This is the output feature map of the CDC module. Represents point convolution. This represents pointwise convolution and GELU activation. Represents the batch normalization layer. It is The expansion rate is dilated convolution; When the LGFI module inputs Feature map At that time, the feature map will first be mapped to The specific expression is as follows: (3); (4); (5); in, This is the input feature map for the LGFI module. This is the output feature map of the LGFI module. , , These are the parameters to be learned. Represents the normalization layer. These represent query, key, and value, respectively. For cross-covariance attention mechanism; (6); in, Representatives utilize Calculate the similarity between the query and the key; S202, Construct a bilateral mesh module to process image edge information. The bilateral grid module scales up the original blurred underwater image, then extracts the feature map through the Unet network, and finally obtains the bilateral grid feature map through global average pooling. S203, Construct a feature reconstruction module that integrates depth information and bilateral mesh information. The feature reconstruction module uses the depth map obtained in step S201 to guide the bilateral network feature map extracted by the bilateral network module to obtain the sampled feature map. Then, the sampled feature map obtained by the feature reconstruction module is multiplied element-wise with the network extracted feature map of Unet to generate the reconstructed feature map. The generated reconstructed feature map is passed through two convolutional layers and residuals are calculated with the original image to estimate the deblurred image. S204, Network model training.
2. The method for deblurring optical images acquired by an underwater robot at a hydroelectric dam according to claim 1, characterized in that, The decoder employs a reflective nonlinear convolutional module (RCE), and its specific operation is as follows: (7); in, Given the input feature map, To output the feature map, for convolution, For the exponential linear unit ELU activation function, As a reflective fill layer, the main operation is to symmetrically fill a circle of pixels around the edge.
3. The method for deblurring optical images acquired by an underwater robot at a hydroelectric dam according to claim 1, characterized in that, In step S202, the image is first fixed to a position using bicubic interpolation. Size, multi-scale features of the image are extracted using Unet, and generated The feature maps are then transformed into low-resolution images through global average pooling. Bilateral grid feature.
4. The method for deblurring optical images acquired by an underwater robot at a hydroelectric dam according to claim 1, characterized in that, In step S203, the depth map is first upsampled to the original image size using bilinear interpolation. Then, the sampled depth map is stitched together with the two mesh maps to form a three-channel sampled image. Finally, the three-channel sampled image is used to sample the bilateral mesh features using cubic linear interpolation to form a sampled image with dimensions of [missing information]. Original image size and number of channels: The sampled feature map is then used by the feature reconstruction module. Feature maps and Unet extracted The feature map is used to perform element-wise dot products to generate a reconstructed feature map with three channels and the same dimensions as the original image. The expression is as follows: (8); in, The first one extracted for Unet Characteristics of each channel This is the sampled feature map. This is the output of the feature reconstruction module.
5. The method for deblurring optical images acquired by an underwater robot at a hydroelectric dam according to claim 1, characterized in that, In step S204, the network model is trained using L1 loss and perceptual loss, where the perceptual loss uses the receptive field of VGG19. The mean squared error of the output feature map, and the expression for the loss function L, are as follows: (9); in, for The coefficients of the loss function, for The coefficients of the perceptual loss function; The loss function expression is as follows: (10); in, (i) represents a clear image. The value of each pixel at a given location. The first image representing the underwater blurred image The pixel value at each position; The loss function is as follows: (11); in, VGG19's sensing field is Feature map, Represents a clear image. This represents a blurred underwater image.
Citation Information
Patent Citations
Edge-based deep learning image motion blur removing method
CN111028177A