An image depth completion method in a foggy scene

Abstract

The application discloses a kind of image depth completion methods under fog scene, comprising:1. using camera to collect image data, while using laser radar to collect point cloud information and generate sparse depth map;2. establish fog atmospheric scattering model, utilize the extinction coefficient generated randomly to generate fog simulation image set;3. build convolutional neural network, utilize fog simulation image set to train, obtain extinction coefficient prediction network model, output the extinction coefficient of fog image;4. build the depth completion network of double-branch stem, input fog image and sparse depth map into depth completion network to process and fuse, generate dense depth map.The application can generate dense depth map by image depth completion, to enhance the applicability and effectiveness of depth information, so that depth map can be utilized in more applications.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image data processing and image enhancement, in particular to an image depth completion method in a foggy scene. BACKGROUND

[0002] Depth acquisition is a key basic problem in computer vision, which can accurately perceive the geometric structure in the image, and is the basis and key for many downstream applications such as scene reconstruction, three-dimensional target identification, automatic driving and robot navigation. Especially in the application of automatic driving and robot navigation, most of the scenes are outdoor environments. For outdoor scenes, the current mainstream and reliable depth acquisition method is to use a laser radar. However, high-precision laser radar equipment is expensive, and a 64-line scanning laser radar can only provide a depth map with a density of about 5%. Moreover, in a foggy scene, the scattering and attenuation of laser signals by fog, as well as the increase in signal noise caused by scattering, will exacerbate the sparsity of the depth map generated by the laser radar, thereby directly affecting the safety and reliability of the application system. SUMMARY

[0003] The present application provides an image depth completion method in a foggy scene to generate a dense depth map through image depth completion, thereby making up for the sparsity of the depth map collected by the laser radar in a foggy scene, enhancing the applicability and effectiveness of the depth information, and making the depth map available in more applications.

[0004] To achieve the above-mentioned application purposes, the present application adopts the following technical solutions:

[0005] The image depth completion method in a foggy scene according to the present application has the following steps:

[0006] A. Using a camera to collect the gth image of the target scene , and The dimension of , wherein represents the height of the image, represents the width of the image, represents the number of channels of the image;

[0007] Using a laser radar to synchronously collect the gth point cloud information of the target scene , and generate the gth sparse depth map , and The dimension of , wherein 1 represents the number of channels of the sparse depth map;

[0008] B. Randomly generating the gth extinction coefficient with a value range of [0.1, 1] A foggy atmospheric scattering model was established, and the foggy atmospheric scattering model was then used to analyze... A simulation was conducted to obtain the following results: Foggy images tagged This leads to the generation of a set of simulated foggy images;

[0009] C. Construct an image extinction coefficient prediction network, input a set of simulated foggy images, and train the network using the Adam optimization algorithm to obtain the optimal image extinction coefficient prediction model.

[0010] D. Construct a deep completion network with a dual-branch backbone for use in... and Feature extraction and fusion are performed to obtain the g-th dense depth map. ;

[0011] E. Based on and Constructing the mean absolute error loss Therefore, the Adam optimization algorithm is used to train the deep completion network of the dual-branch backbone, and the calculation is performed. To update network parameters until The process continues until convergence, thus obtaining the optimal dual-branch backbone depth completion network model, which is used to output the dense depth map of the corresponding image.

[0012] The image depth completion method for foggy scenes described in this invention is also characterized in that C includes:

[0013] C1. Construct an image extinction coefficient prediction network, which includes: Q downsampling blocks and T fully connected layers; wherein, the q-th level downsampling block is composed of a q-th level multi-layer two-dimensional convolutional layer and a q-th level downsampling layer;

[0014] The q-th level multi-layer two-dimensional convolutional layer includes: Y two-dimensional convolutional layers, Y batch normalization layers, and Y ReLU activation function layers, wherein the kernel size of the y-th two-dimensional convolutional layer is... ,y=1,2,…,Y;

[0015] When q=1 The input is the q-th level downsampled block, which is processed to output the q-th level downsampled feature map. ;

[0016] When q=2,3,…,Q, the downsampled feature map of level q-1 will be... The input q-th level downsampled block is processed to obtain the q-th level downsampled feature map. Thus, the final output Q-level downsampled feature map is obtained from the Q-level downsampled block. ;

[0017] After processing through T full connection layers, output the predicted value of extinction coefficient ;

[0018] C2, based on and the true value of the extinction coefficient Establish the mean square error loss , so as to train the image extinction coefficient prediction network by using Adam optimization algorithm, and calculate to update the network parameters until converge, thereby obtaining the optimal image extinction coefficient prediction model.

[0019] Further, the predicted value of the extinction coefficient in C1 is obtained as follows:

[0020] When t=1, input the t-th full connection layer for processing, and output the t-th feature map ;

[0021] When t=2, 3,…, T-1, the (t-1)-th feature map is input into the t-th full connection layer for processing, and the t-th feature map is obtained ;

[0022] When t=T, the (T-1)-th feature map is input into the t-th full connection layer for processing, thereby outputting the predicted value of the extinction coefficient ;

[0023] Further, the deep completion network in D is composed of a first main body and a second main body with the same structure; wherein any main body comprises in turn: a first convolutional layer, a Z-level residual block, Z deconvolutional layers, a second convolutional layer; wherein the z-th residual block comprises: a z-th fog convolutional layer, a z-th batch normalization layer, a z-th ReLU activation function;

[0024] D1, the first convolutional layer in the first main body performs feature extraction processing on and to obtain the g-th initial convolutional feature map , and the dimension of , wherein represents the number of channels of ;

[0025] D2, the Z-level residual block in the first main body processes to obtain the Z-th residual feature map :

[0026] D3, will The data is sequentially input into Z deconvolution layers for deconvolution processing to obtain the g-th deconvolution feature map. ;

[0027] D4, the second convolutional layer in the first main branch Feature extraction is performed to obtain the g-th dense depth map of the first backbone. ,and The dimension is ,in, express The number of channels;

[0028] D5. The second main branch follows the process from D1 to D4. and Feature extraction and fusion are performed to obtain the g-th dense depth map of the second backbone. ,and The dimension is ,in, express The number of channels;

[0029] D6, will and After stitching along the channel dimension, a new g-th dense depth map is obtained. .

[0030] Furthermore, D2 includes:

[0031] When z=1, The residual feature map is used as the input and output of the (z-1)th module. The data is then input into the z-th fog convolutional layer of the first backbone, thereby obtaining the g-th z-th global contrast feature using equation (1). :

[0032] (1)

[0033] In equation (1), express The upper position is pixel values, express The upper position is Pixel values;

[0034] The z-level fog convolution layer will Divided into The g-th image block, the z-th level, and the d-th image block are obtained using equation (2). Local contrast features :

[0035] (2)

[0036] In formula (2), represents the number of rows of image blocks, represents the number of columns of image blocks, and M=N; represents the height of each image block, and , represents the width of each image block, and ; represents the pixel value at the position ; represents the pixel value at the position ;

[0037] The z-th level fog convolution layer obtains the g-th z-th level global entropy feature by formula (3) :

[0038] (3)

[0039] In formula (3), represents the brightness value of a pixel in represents the probability that the brightness value at the position is , and has:

[0040] (4)

[0041] In formula (4), is the Dirac function;

[0042] The z-th level fog convolution layer obtains the g-th z-th level original image feature by formula (5) :

[0043] (5)

[0044] In formula (5), represents the image background of the inherent atmospheric light A mapping of the surrounding environment; represents the position of any pixel point in is the transmittance at the position , and has:

[0045] (6)

[0046] In formula (6), Representing the position depth of the scene at the position, The extinction coefficient of the foggy image output by the extinction coefficient prediction model;

[0047] When z=2,3,…,Z, the as the input of the z-th residual block, and the g-th residual feature map output by the z-th residual block is obtained; thus the g-th residual feature map output by the Z-th residual block is obtained, and The dimension of , wherein, indicates the number of channels of .

[0048] Further, the D3 comprises:

[0049] When z=1, the z-th deconvolution layer processes , and outputs the g-th z-th deconvolution feature map :

[0050] When z=2,3,…,Z, the z-th deconvolution layer performs residual connection processing on the g-th (z-1)-th deconvolution feature map and the g-th (Z-z)-th residual feature map using formula (7), and obtains the g-th z-th deconvolution feature map ; thus the g-th Z-th deconvolution feature map output by the Z-th deconvolution layer is obtained, and The dimension of , wherein, indicates the height of , indicates the width of , indicates the number of channels of .

[0051] (7).

[0052] The electronic device comprises a memory and a processor, and the characteristic lies in that the memory is used to store a program supporting the processor to execute the image depth completion method in a foggy scene, and the processor is configured to execute the program stored in the memory.

[0053] The computer readable storage medium stores a computer program, and the characteristic lies in that the computer program is run by the processor to execute the steps of the image depth completion method in a foggy scene.

[0054] Compared with the prior art, the present application has the beneficial effects that:

[0055] 1. The present application is aimed at the problem that the depth map generated by laser radar collection is relatively sparse, and in foggy weather scenes, the scattering and attenuation of fog on laser signals, as well as the increase in signal noise caused by scattering, will exacerbate the sparsity of the depth map generated by laser radar collection. A method for image depth completion in foggy weather scenes is proposed: by establishing a depth completion network with a double-branch backbone, using an image-guided depth completion method, the original sparse depth map is completed to generate a dense depth map. This method can use the rich texture information of the RGB image to guide the completion of the sparse depth map, thereby greatly improving the practicality and applicability of the sparse depth map, and providing more accurate and complete three-dimensional information for computer vision, autonomous driving, robot navigation and other fields.

[0056] 2. The present application generates a set of foggy weather simulation images by simulating sunny weather images into foggy weather images using a foggy weather atmospheric scattering model. This simulation method has the advantages of low cost, high efficiency, controllable parameters, good scene consistency, and high safety compared to directly collecting images in foggy weather scenes. This method can be used for large-scale data set generation and model robustness testing, and can provide strong support for research and application in actual collection difficulties or unsafe conditions. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The present application is a method for image depth completion in foggy weather scenes, and the overall flowchart is shown in the figure;

[0058] Figure 2 The present application is a schematic diagram of the extinction coefficient prediction network;

[0059] Figure 3 The present application is a schematic diagram of the double-branch backbone depth completion network;

[0060] Figure 4 The present application is a schematic diagram of the fog convolution layer. DETAILED DESCRIPTION

[0061] In this embodiment, a method for image depth completion in foggy weather scenes is used to synchronize the image data and point cloud information collected by the camera and laser radar, and to simulate the collected sunny weather images into foggy weather scenes using a foggy weather atmospheric scattering model to generate a set of foggy weather simulation images. An extinction coefficient prediction model is trained to output the extinction coefficient of the foggy weather images. Finally, a depth completion network with a double-branch backbone is built to input the foggy weather images and sparse depth maps into the depth completion network for processing and fusion to generate a dense depth map. The present application can generate a dense depth map through image depth completion, thereby enhancing the applicability and effectiveness of the depth information, and making the depth map available in more applications. Specifically, as Figure 1As shown, it includes the following steps:

[0062] A. Use a camera to capture the g-th image of the target scene. ,and The dimension is ,in, Indicates the height of the image. Indicates the width of the image. This represents the number of channels in the image; based on the image captured by the camera. The camera intrinsic parameter matrix J was obtained using Zhang Zhengyou's calibration method and the MABTLAB calibration tool.

[0063] The g-th point cloud information of the target scene is acquired simultaneously using a lidar system. And generate the g-th sparse depth map. ;and The dimension is Where 1 represents the number of channels in the sparse depth map; the PnP algorithm is used to obtain the camera's extrinsic parameters. The coordinate system of the lidar point cloud is transformed to the camera coordinate system using equation (1):

[0064] (1)

[0065] In equation (1), These are the coordinates of the lidar point cloud. O represents the coordinates of the LiDAR point cloud in the camera coordinate system, O is the rotation matrix, and I is the translation vector.

[0066] Equation (2) is used to project the transformed 3D points onto a 2D image plane to generate a sparse depth map:

[0067] (2)

[0068] In equation (2), These are pixel coordinates on the image plane.

[0069] B. Generate a set of simulated foggy images:

[0070] B1. Randomly generate the g-th extinction coefficient with a value range of [0.1, 1]. A foggy atmospheric scattering model is established, which is represented by equation (3):

[0071] (3)

[0072] In equation (3), express The position of any pixel in the array; It is a simulated foggy image at the location Pixel value at; is the pixel value of the haze-free image at position ; is the transmittance at position ; is the ambient light inherent to the surrounding environment; represents the attenuated light of the object reflection light after scattering by the haze; represents the ambient light after scattering by the haze, which compensates for the scattered object reflection light, and has:

[0073] (4)

[0074] In formula (4), represents the scene depth at position .

[0075] B2, simulating the haze atmosphere scattering model on to obtain a haze image tagged with , and further generating a set of haze simulation images.

[0076] C, building an image extinction coefficient prediction network model:

[0077] C1, building an image extinction coefficient prediction network as shown in Figure 2 , which includes Q down-sampling blocks, T fully connected layers in sequence; wherein the qth down-sampling block is composed of a qth multi-layer two-dimensional convolution layer and a qth down-sampling layer;

[0078] The qth multi-layer two-dimensional convolution layer includes Y two-dimensional convolution layers, Y batch normalization layers and Y ReLU activation function layers, wherein the convolution kernel size of the yth two-dimensional convolution layer is ,y=1,2,…,Y;

[0079] Standardizing the image , the pixel value of the image is scaled to the range of [0, 1], thereby improving the training efficiency and the stability of the model.

[0080] When q=1, the input is processed in the qth down-sampling block, and the qth down-sampling feature map is outputted.

[0081] When q=2,3,…,Q, the q-1th down-sampling feature map is inputted into the qth down-sampling block for processing, and the qth down-sampling feature map is obtained; thereby the final Qth down-sampling feature map is outputted from the Qth down-sampling block.

[0082] After processing through T full connection layers, output feature map .

[0083] When t = 1, Input into the t-th full connection layer for processing, output the t-th feature map ;

[0084] When t = 2, 3, …, T-1, the (t-1)-th feature map Input into the t-th full connection layer for processing, and obtain the t-th feature map ;

[0085] When t = T, the (T-1)-th feature map Input into the t-th full connection layer for processing, thereby output the T-th feature map That is, the predicted value of the extinction coefficient ;

[0086] C2, based on And the true value of the extinction coefficient Establish the mean square error loss , so as to train the image extinction coefficient prediction network by using the Adam optimization algorithm, and calculate To update the network parameters until Converge, thereby obtaining the optimal image extinction coefficient prediction model; the extinction coefficient output by the image extinction coefficient prediction network model will be involved in the subsequent construction of the fog convolutional layer, so as to improve the adaptability of the deep completion network to the foggy environment.

[0087] D, build a deep completion network with a double-branch backbone, which is used for feature extraction and fusion processing of And Obtain the g-th dense depth map ;

[0088] The deep completion network is composed of a first backbone and a second backbone with the same structure; any backbone comprises, in sequence, a first convolutional layer, a Z-level residual block, Z deconvolutional layers, and a second convolutional layer; the z-th level residual block comprises a z-th level fog convolutional layer, a z-th level batch normalization layer, and a z-th level ReLU activation function. Figure 3 It is a schematic diagram of a double-branch backbone deep completion network.

[0089] D1, the first convolutional layer in the first backbone performs feature extraction processing on And Obtain the g-th initial convolutional feature map , and the dimension of is , wherein Indicates the number of channels of the input image;

[0090] D2, the Z-th residual block pair in the first trunk is processed to obtain the Z-th residual feature map :

[0091] When z = 1, the Z-th residual feature map is taken as the input of the first module and the residual feature map is output , and is input into the Z-th fog convolutional layer of the first trunk, so that the g-th Z-th global contrast feature is obtained by using formula (5) :

[0092] (5)

[0093] In formula (5), represents the pixel value at the position of ; represents the pixel value at the position of ;

[0094] The Z-th fog convolutional layer divides the Z-th residual feature map into N image blocks, and obtains the g-th Z-th local contrast feature of the d-th image block by using formula (6) :

[0095] (6)

[0096] In formula (6), represents the number of rows of the image blocks divided, represents the number of columns of the image blocks divided, and M = N; represents the height of each image block, and , represents the width of each image block, and ; represents the pixel value at the position of ; represents the pixel value at the position of ; represents the pixel value at the position of ;

[0097] The Z-th fog convolutional layer obtains the g-th Z-th global entropy feature by using formula (7) :

[0098] (7)

[0099] In formula (7),​​​​​​​​ represents the luminance value of the middle pixel; represents at the position , the probability of the luminance value being , and has:

[0100] (8)

[0101] In formula (8), is the Dirac function.

[0102] The z-th fog convolution layer obtains the g-th original image feature of the z-th level by formula (9) :

[0103] (9)

[0104] In formula (9), represents the image background of which is mapped by the ambient light A inherent in the surrounding environment; represents the position of any pixel point in , and is the transmittance at the position , and has:

[0105] (10)

[0106] In formula (10), represents the scene depth at the position , and is the extinction coefficient of the foggy image output by the extinction coefficient prediction model;

[0107] When z = 2, 3, …, Z, take as the input of the z-th residual block, and obtain the g-th residual feature map of the corresponding output; thus the g-th residual feature map of the final output is obtained from the Z-th residual block, and the dimension of , wherein represents the number of channels of ;

[0108] Figure 4 is a schematic diagram of a fog convolution layer. By designing four kinds of image features, different aspects of information of the image can be captured, and these information can be combined to make the convolution layer learn more complex and rich feature expressions, thereby improving the robustness of the model and more robustly processing various noises and interference.

[0109] D3, taking The Z deconvolution layers are sequentially inputted to perform deconvolution processing, and the gth deconvolution feature map is obtained ;

[0110] When z=1, the zth deconvolution layer processes the gth deconvolution feature map , and outputs the gth zth deconvolution feature map :

[0111] When z=2, 3, …, Z, the zth deconvolution layer performs residual connection processing on the gth (z-1)th deconvolution feature map and the gth (Z-z)th residual feature map using formula (11), and obtains the gth zth deconvolution feature map ; thus, the gth Zth deconvolution feature map as the final output is obtained from the Zth deconvolution layer , and the dimension of is , wherein represents the height of , represents the width of , represents the number of channels of .

[0112] (11)

[0113] D4, the second convolution layer in the first backbone performs feature extraction processing on , and obtains the gth first backbone dense depth map , and the dimension of is , wherein represents the number of channels of .

[0114] D5, the second backbone performs feature extraction and fusion processing on and according to the processes of D1-D4, and thus obtains the gth second backbone dense depth map , and the dimension of is , wherein represents the number of channels of .

[0115] D6, after and are spliced in the channel dimension, a new gth dense depth map is obtained.

[0116] E, an average absolute error loss is constructed based on and . , and the Adam optimization algorithm is used to train the deep completion network with the double-branch backbone, and is calculated to update the network parameters until converges, thereby obtaining an optimal deep completion network model with the double-branch backbone for outputting a dense depth map corresponding to an image.

[0117] In this embodiment, an electronic device is designed, which includes a memory and a processor. The memory is configured to store a program supporting the processor to execute an image depth completion method in a foggy scene. The processor is configured to execute the program stored in the memory.

[0118] In this embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the image depth completion method in the foggy scene are executed.

Claims

1. A method for image depth completion in foggy scenes, characterized in that, Includes the following steps: A. Use a camera to capture the g-th image of the target scene. ,and The dimension is ,in, Indicates the height of the image. Indicates the width of the image. Indicates the number of channels in an image; The g-th point cloud information of the target scene is acquired simultaneously using a lidar system. And generate the g-th sparse depth map. ,and The dimension is , where 1 represents the number of channels in the sparse depth map; B. Randomly generate the g-th extinction coefficient with a value range of [0.1, 1]. A foggy atmospheric scattering model was established, and the foggy atmospheric scattering model was then used to analyze... A simulation was conducted to obtain the following results: Foggy images tagged This leads to the generation of a set of simulated foggy images; C. Construct an image extinction coefficient prediction network, input a set of simulated foggy images, and train the network using the Adam optimization algorithm to obtain the optimal image extinction coefficient prediction model. D. Construct a deep completion network with a dual-branch backbone for use in... and Feature extraction and fusion are performed to obtain the g-th dense depth map. ; E. Based on and Constructing the mean absolute error loss Therefore, the Adam optimization algorithm is used to train the deep completion network of the dual-branch backbone, and the calculation is performed. To update network parameters until The process continues until convergence, thus obtaining the optimal dual-branch backbone depth completion network model, which is used to output the dense depth map of the corresponding image.

2. The image depth completion method for foggy scenes according to claim 1, characterized in that, The C includes: C1. Construct an image extinction coefficient prediction network, which includes: Q downsampling blocks and T fully connected layers; wherein, the q-th level downsampling block is composed of a q-th level multi-layer two-dimensional convolutional layer and a q-th level downsampling layer; The q-th level multi-layer two-dimensional convolutional layer includes: Y two-dimensional convolutional layers, Y batch normalization layers, and Y ReLU activation function layers, wherein the kernel size of the y-th two-dimensional convolutional layer is... ,y=1,2,…,Y; When q=1 The input is the q-th level downsampled block, which is processed to output the q-th level downsampled feature map. ; When q=2,3,…,Q, the downsampled feature map of level q-1 will be... The input q-th level downsampled block is processed to obtain the q-th level downsampled feature map. Thus, the final output Q-level downsampled feature map is obtained from the Q-level downsampled block. ; After processing by T fully connected layers, the predicted value of the output extinction coefficient is obtained. ; C2, based on and the true value of the extinction coefficient Establish mean square error loss Therefore, the Adam optimization algorithm is used to train the image extinction coefficient prediction network and calculate... To update network parameters until The optimal image extinction coefficient prediction model is obtained by converging until convergence.

3. The image depth completion method for foggy scenes according to claim 2, characterized in that, The predicted value of the extinction coefficient in C1 It is obtained through the following process: When t=1, The input is processed in the t-th level fully connected layer, and the output is the t-th level feature map. ; When t=2,3,…,T-1, the (t-1)th level feature map The input is processed in the t-th level fully connected layer to obtain the t-th level feature map. ; When t=T, the (T-1)th level feature map The input is processed in the t-th level fully connected layer to output the predicted value of the extinction coefficient. .

4. The image depth completion method for foggy scenes according to claim 3, characterized in that, The depth completion network in D consists of a first backbone and a second backbone with the same structure; wherein each backbone includes, in sequence: a first convolutional layer, a Z-level residual block, Z deconvolutional layers, and a second convolutional layer; wherein the z-level residual block includes: a z-level fog convolutional layer, a z-level batch normalization layer, and a z-level ReLU activation function. D1, the first convolutional layer in the first main branch and Feature extraction is performed to obtain the g-th initial convolutional feature map. ,and The dimension is ,in, express The number of channels; D2, Z-level residual block pairs in the first main branch The process is performed to obtain the Z-level residual feature map. : D3, will The data is sequentially input into Z deconvolution layers for deconvolution processing to obtain the g-th deconvolution feature map. ; D4, the second convolutional layer in the first main branch Feature extraction is performed to obtain the g-th dense depth map of the first backbone. ,and The dimension is ,in, express The number of channels; D5. The second main branch follows the process from D1 to D4. and Feature extraction and fusion are performed to obtain the g-th dense depth map of the second backbone. ,and The dimension is ,in, express The number of channels; D6, will and After stitching along the channel dimension, a new g-th dense depth map is obtained. .

5. The image depth completion method for foggy scenes according to claim 4, characterized in that, The D2 includes: When z=1, The residual feature map is used as the input and output of the (z-1)th module. The data is then input into the z-th fog convolutional layer of the first backbone, thereby obtaining the g-th z-th global contrast feature using equation (1). : (1) In equation (1), express The upper position is pixel values, express The upper position is Pixel values; The z-level fog convolution layer will Divided into The g-th image block, the z-th level, and the d-th image block are obtained using equation (2). Local contrast features : (2) In equation (2), express The number of rows into which the image is divided. express The number of columns into which the image is divided, where M=N; This represents the height of each image block, and , This represents the width of each image patch, and ; express The upper position is Pixel values; express The upper position is Pixel values; The z-th level fog convolutional layer uses equation (3) to obtain the g-th z-th level global entropy feature. : (3) In equation (3), express The brightness value of the middle pixel; express In position At this location, the brightness value is The probability of , and we have: (4) In equation (4), It is the Dirac function; The z-th fog convolutional layer uses equation (5) to obtain the g-th z-th original image feature. : (5) In equation (5), This represents the mapping of the inherent atmospheric light A in the surrounding environment. Image background; express The position of any pixel in the image. For position The transmittance, and we have: (6) In equation (6), Indicates position Scene depth at that location The extinction coefficient is the output of the extinction coefficient prediction model for foggy images. When z = 2, 3, ..., Z, As input to the z-th level residual block, the g-th residual feature map of the corresponding output is obtained. Thus, the g-th residual feature map of the final output is obtained from the Z-th residual block. ,and The dimension is ,in, express The number of channels.

6. The image depth completion method for foggy scenes according to claim 5, characterized in that, The D3 includes: When z=1, the z-th level deconvolution layer pairs Process the data and output the g-th and z-th deconvolutional feature map. : When z=2,3,…,Z, the z-th level deconvolution layer uses equation (7) to process the g-th (z-1)-th level deconvolution feature map. and the g-th Zz-th residual feature map Perform residual connection processing to obtain the g-th z-th deconvolution feature map. Thus, the final output g-th Z-th deconvolutional feature map is obtained from the Z-th deconvolutional layer. ,and The dimension is ,in, express height, express width, express The number of channels; (7)。 7. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing the image depth completion method in a foggy scene as described in any one of claims 1-6, and the processor is configured to execute the program stored in the memory.

8. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when run by the processor, executes the steps of the image depth completion method for foggy scenes as described in any one of claims 1-6.

Images