Image depth completion method in foggy day scene

Abstract

The invention discloses an image depth completion method in a foggy day scene, and the method comprises the steps: 1, collecting image data through employing a camera, and collecting point cloud information through employing a laser radar, and generating a sparse depth map; 2, establishing a foggy weather atmospheric scattering model, and generating a foggy weather simulation image set by using a randomly generated extinction coefficient; 3, building a convolutional neural network, training by using the foggy day simulation image set to obtain an extinction coefficient prediction network model, and outputting the extinction coefficient of the foggy day image; and 4, constructing a double-branch trunk depth completion network, and inputting the foggy day image and the sparse depth map into the depth completion network for processing and fusion to generate a dense depth map. According to the method, the dense depth map can be generated through image depth completion, so that the applicability and effectiveness of depth information are enhanced, and the depth map can be utilized in more applications.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing and image enhancement, and in particular to an image depth completion method in a foggy scene. Background Art

[0002] Depth acquisition is a key basic problem in computer vision. It can accurately perceive the geometric structure in the image and is the basis and key to many downstream applications (such as scene reconstruction, 3D target recognition, autonomous driving and robot navigation, etc.). Especially in the applications of autonomous driving and robot navigation, most scenes are outdoor environments. For outdoor scenes, the current mainstream and reliable way to acquire depth is to use LiDAR. However, high-precision LiDAR equipment is expensive, and a 64-line scanning LiDAR can only provide a depth map with a density of about 5%. In addition, in foggy scenes, the scattering and attenuation of the laser signal by the fog, as well as the increase in signal noise caused by the scattering, will increase the sparsity of the depth map generated by the LiDAR acquisition, which will directly affect the safety and reliability of the application system. Summary of the invention

[0003] In response to the problems existing in the above-mentioned scheme, the invention provides an image depth completion method in foggy scenes, hoping to generate a dense depth map through image depth completion to make up for the sparse depth map collected by the lidar in foggy scenes, thereby enhancing the applicability and effectiveness of the depth information and enabling the depth map to be used in more applications.

[0004] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:

[0005] The image depth completion method in a foggy scene of the present invention is characterized in that it comprises the following steps:

[0006] A. Use the camera to capture the gth image of the target scene ,and The dimension is ,in, Indicates the height of the image. Indicates the width of the image. Indicates the number of channels of the image;

[0007] Use LiDAR to synchronously collect the g-th point cloud information of the target scene , and generate the g-th sparse depth map ,and The dimension is , where 1 represents the number of channels of the sparse depth map;

[0008] B. Randomly generate the gth extinction coefficient in the range [0.1,1] , and establish a foggy atmospheric scattering model, so as to use the foggy atmospheric scattering model to Simulate and obtain Foggy images for labels , and then generate a set of foggy simulated images;

[0009] C. Build an image extinction coefficient prediction network, input a set of foggy simulated images, and use the Adam optimization algorithm to train the network to obtain the optimal image extinction coefficient prediction model;

[0010] D. Build a dual-branch backbone deep completion network for and Perform feature extraction and fusion processing to obtain the g-th dense depth map ;

[0011] E. Based on and Constructing mean absolute error loss , so as to use the Adam optimization algorithm to train the dual-branch trunk depth completion network and calculate To update the network parameters until Until convergence, the optimal dual-branch backbone depth completion network model is obtained, which is used to output the dense depth map of the corresponding image.

[0012] The image depth completion method in a foggy scene according to the present invention is also characterized in that C includes:

[0013] C1. Build an image extinction coefficient prediction network, which includes: Q downsampling blocks and T fully connected layers in sequence; wherein the qth downsampling block is composed of the qth multi-layer two-dimensional convolutional layer and the qth downsampling layer;

[0014] The q-th multi-layer two-dimensional convolutional layer includes: Y two-dimensional convolutional layers, Y batch normalization layers and Y ReLU activation function layers, wherein the convolution kernel size of the y-th two-dimensional convolutional layer is ,y=1,2,…,Y;

[0015] When q=1, Input the qth level downsampling block for processing and output the qth level downsampling feature map ;

[0016] When q=2,3,…,Q, the q-1th level downsampled feature map Input to the qth level downsampling block for processing, and obtain the qth level downsampling feature map ; Thus, the Q-th level downsampling block finally outputs the Q-th level downsampling feature map ;

[0017] After processing by T fully connected layers, the predicted value of the extinction coefficient is output ;

[0018] C2, based on and the true value of the extinction coefficient Establishing mean squared error loss , so as to use the Adam optimization algorithm to train the image extinction coefficient prediction network and calculate To update the network parameters until Until convergence, the optimal image extinction coefficient prediction model is obtained.

[0019] Furthermore, the predicted value of the extinction coefficient in C1 is It is obtained by the following process:

[0020] When t=1, Input the t-th level fully connected layer for processing and output the t-th level feature map ;

[0021] When t=2,3,…,T-1, the t-1th level feature map Input into the t-th level fully connected layer for processing and obtain the t-th level feature map ;

[0022] When t=T, the T-1th level feature map The input is processed in the t-th level fully connected layer to output the predicted value of the extinction coefficient ;

[0023] Furthermore, the depth completion network in D is composed of a first backbone and a second backbone with the same structure; wherein any backbone includes in sequence: a first convolutional layer, a Z-level residual block, Z deconvolutional layers, and a second convolutional layer; wherein the z-th level residual block includes: a z-th level fog convolutional layer, a z-th level batch normalization layer, and a z-th level ReLU activation function;

[0024] D1, the first convolutional layer pair in the first backbone and Perform feature extraction to obtain the g-th initial convolution feature map ,and The dimension is ,in, express The number of channels;

[0025] D2, Z-level residual block pairs in the first backbone Processing is performed to obtain the Z-th level residual feature map :

[0026] D3. Input the Z deconvolution layers in sequence for deconvolution processing to obtain the g-th deconvolution feature map ;

[0027] D4, the second convolutional layer pair in the first backbone Perform feature extraction to obtain the g-th first trunk dense depth map ,and The dimension is ,in, express The number of channels;

[0028] D5, the second trunk follows the process of D1-D4 and Perform feature extraction and fusion processing to obtain the g-th second trunk dense depth map ,and The dimension is ,in, express The number of channels;

[0029] D6. and After splicing in the channel dimension, the new g-th dense depth map is obtained .

[0030] Further, the D2 comprises:

[0031] When z=1, As the input of the z-1th module and output the residual feature map , and input into the z-th fog convolution layer of the first backbone, so as to obtain the g-th z-th global contrast feature using formula (1): :

[0032] (1)

[0033] In formula (1), express The upper position is The pixel value of express The upper position is The pixel value of

[0034] The z-th fog convolutional layer will Divide into image blocks, and use formula (2) to get the g-th z-th level d-th image block The local contrast feature :

[0035] (2)

[0036] In formula (2), express The number of rows that the image is divided into blocks, express The number of columns divided into image blocks, and M=N; represents the height of each image block, and , represents the width of each image block, and ; express The upper position is The pixel value of express The upper position is The pixel value of

[0037] The z-th fog convolution layer uses formula (3) to obtain the g-th z-th global entropy feature :

[0038] (3)

[0039] In formula (3), express The brightness value of the pixel in ; express In Location At The probability of , and:

[0040] (4)

[0041] In formula (4), is the Dirac function;

[0042] The z-th fog convolution layer uses formula (5) to obtain the g-th z-th original image feature :

[0043] (5)

[0044] In formula (5), Represents the mapping of the inherent atmospheric light A in the surrounding environment Image background; express The position of any pixel in For location The transmittance is:

[0045] (6)

[0046] In formula (6), Indicates location The scene depth at is the extinction coefficient of the foggy image output by the extinction coefficient prediction model;

[0047] When z=2,3,…,Z, As the input of the z-th level residual block, and get the corresponding output g-th residual feature map ; Thus, the g-th residual feature map of the final output is obtained by the Z-th level residual block ,and The dimension is ,in, express The number of channels.

[0048] Furthermore, the D3 includes:

[0049] When z=1, the z-th deconvolution layer is Process and output the g-th z-th level deconvolution feature map :

[0050] When z=2,3,…,Z, the z-th level deconvolution layer uses formula (7) to deconvolute the g-th z-1-th level feature map And the g-th Zz-th level residual feature map Perform residual connection processing and obtain the g-th z-th level deconvolution feature map ; Thus, the final output of the g-th Z-th deconvolution feature map is obtained by the Z-th deconvolution layer ,and The dimension is ,in, express Height, express The width of express The number of channels;

[0051] (7).

[0052] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the image depth completion method in a foggy scene, and the processor is configured to execute the program stored in the memory.

[0053] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program is characterized in that when the computer program is executed by a processor, the steps of the image depth completion method in a foggy scene are executed.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. The present invention aims at the fact that the depth map generated by laser radar is relatively sparse, and in foggy scenes, the scattering and attenuation of the laser signal by fog, as well as the increase in signal noise caused by scattering, will aggravate the sparsity of the depth map generated by laser radar. A method for image depth completion in foggy scenes is proposed: by establishing a dual-branch trunk depth completion network, the image-guided depth completion method is used to complete the original sparse depth map, thereby generating a dense depth map. This method can use the rich texture information of RGB images to guide the completion of sparse depth maps, thereby greatly improving the practicality and scope of application of sparse depth maps, and providing more accurate and complete three-dimensional information for computer vision, autonomous driving, robot navigation and other fields.

[0056] 2. The present invention simulates sunny images into foggy images by using a foggy atmospheric scattering model, thereby generating a set of simulated foggy images. Compared with directly collecting images in foggy scenes, this simulation method has the advantages of low cost, high efficiency, controllable parameters, good scene consistency, and high safety. This method can be applied to the generation of large-scale data sets and the robustness testing of models, and can provide strong support for research and application in situations where actual collection is difficult or unsafe. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is an overall flow chart of an image depth completion method in a foggy scene according to the present invention;

[0058] Figure 2 is a schematic diagram of the extinction coefficient prediction network of the present invention;

[0059] Figure 3 This is a schematic diagram of a dual-branch trunk depth completion network of the present invention;

[0060] Figure 4 Schematic diagram of the fog convolution layer of the present invention. DETAILED DESCRIPTION

[0061] In this embodiment, a method for image depth completion in a foggy scene uses a camera and a lidar to synchronously collect image data and point cloud information, and simulates the collected sunny images into images in a foggy scene through a foggy atmospheric scattering model to generate a foggy simulated image set, and trains an extinction coefficient prediction model to output the extinction coefficient of the foggy image, and finally builds a double-branch trunk depth completion network, inputs the foggy image and the sparse depth map into the depth completion network for processing and fusion, and generates a dense depth map; the present invention can generate a dense depth map through image depth completion, thereby enhancing the applicability and effectiveness of depth information, so that the depth map can be used in more applications; specifically, Figure 1As shown, the following steps are included:

[0062] A. Use the camera to capture the gth image of the target scene ,and The dimension is ,in, Indicates the height of the image. Indicates the width of the image. Indicates the number of channels of the image; according to the image collected by the camera , use Zhang Zhengyou calibration method and MABTLAB calibration tool to obtain the camera intrinsic parameter matrix J;

[0063] Use LiDAR to synchronously collect the g-th point cloud information of the target scene , and generate the g-th sparse depth map ;and The dimension is , where 1 represents the number of channels of the sparse depth map; the PnP algorithm is used to obtain the external parameters of the camera ; Use formula (1) to transform the coordinate system of the lidar point cloud to the camera coordinate system:

[0064] (1)

[0065] In formula (1), are the coordinates of the LiDAR point cloud, are the coordinates of the corresponding lidar point cloud in the camera coordinate system, O is the rotation matrix, and I is the translation vector.

[0066] The transformed 3D points are projected onto the 2D image plane using equation (2) to generate a sparse depth map:

[0067] (2)

[0068] In formula (2), are the pixel coordinates on the image plane.

[0069] B. Generate foggy simulated image set:

[0070] B1, randomly generate the gth extinction coefficient in the range [0.1,1] , establish the foggy atmospheric scattering model, the model is expressed by formula (3):

[0071] (3)

[0072] In formula (3), express The position of any pixel in ; is a simulated foggy image at position The pixel value at ; is a fog-free image at position The pixel value at ; For location The transmittance at ; It is the atmospheric light inherent in the surrounding environment; Indicates the attenuated light after the reflected light of the target object is scattered by fog; It represents the ambient light after the fog scatters the atmospheric light, which is used to make up for the scattered reflected light of the object, and has:

[0073] (4)

[0074] In formula (4), Indicates location The depth of the scene at.

[0075] B2. Using the foggy atmospheric scattering model Simulate and obtain Foggy images for labels , and then generate a set of foggy simulated images.

[0076] C. Build an image extinction coefficient prediction network model:

[0077] C1. Build an image extinction coefficient prediction network such as Figure 2 As shown, it includes: Q downsampling blocks and T fully connected layers in sequence; wherein the qth level downsampling block is composed of the qth level multi-layer two-dimensional convolutional layer and the qth level downsampling layer;

[0078] The qth multi-layer 2D convolutional layer includes: Y 2D convolutional layers, Y batch normalization layers and Y ReLU activation function layers, where the convolution kernel size of the yth 2D convolutional layer is ,y=1,2,…,Y;

[0079] For images Normalization is performed to scale the pixel values ​​of the image to the range of [0, 1], thereby improving training efficiency and model stability.

[0080] When q=1, Input the qth level downsampling block for processing and output the qth level downsampling feature map ;

[0081] When q=2,3,…,Q, the q-1th level downsampled feature map Input to the qth level downsampling block for processing, and obtain the qth level downsampling feature map ; Thus, the Q-th level downsampling block finally outputs the Q-th level downsampling feature map ;

[0082] After processing by T fully connected layers, the output feature map .

[0083] When t=1, Input the t-th level fully connected layer for processing and output the t-th level feature map ;

[0084] When t=2,3,…,T-1, the t-1th level feature map Input into the t-th level fully connected layer for processing and obtain the t-th level feature map ;

[0085] When t=T, the T-1th level feature map The input is processed in the t-th level fully connected layer to output the T-th level feature map That is, the predicted value of the extinction coefficient ;

[0086] C2, based on and the true value of the extinction coefficient Establishing mean squared error loss , so as to use the Adam optimization algorithm to train the image extinction coefficient prediction network and calculate To update the network parameters until The algorithm converges until the optimal image extinction coefficient prediction model is obtained. The extinction coefficient output by the image extinction coefficient prediction network model will be used to build the subsequent fog convolution layer to improve the adaptability of the depth completion network to foggy environments.

[0087] D. Build a dual-branch backbone deep completion network for and Perform feature extraction and fusion processing to obtain the g-th dense depth map ;

[0088] The deep completion network is composed of a first backbone and a second backbone with the same structure; wherein any backbone includes in sequence: a first convolutional layer, a Z-level residual block, Z deconvolutional layers, and a second convolutional layer; wherein the z-th level residual block includes: a z-th level fog convolutional layer, a z-th level batch normalization layer, and a z-th level ReLU activation function; Figure 3 This is a schematic diagram of a dual-branch trunk depth completion network;

[0089] D1, the first convolutional layer pair in the first backbone and Perform feature extraction to obtain the g-th initial convolution feature map ,and The dimension is ,in, express The number of channels;

[0090] D2, Z-level residual block pairs in the first backbone Processing is performed to obtain the Z-th level residual feature map :

[0091] When z=1, As the input of the z-1th module and output the residual feature map , and input into the z-th fog convolution layer of the first backbone, so as to obtain the g-th z-th global contrast feature using formula (5): :

[0092] (5)

[0093] In formula (5), express The upper position is The pixel value of express The upper position is The pixel value of

[0094] The z-th fog convolutional layer will Divide into image blocks, and use formula (6) to get the g-th z-th level d-th image block The local contrast feature :

[0095] (6)

[0096] In formula (6), express The number of rows that the image is divided into blocks, express The number of columns divided into image blocks, and M=N; represents the height of each image block, and , represents the width of each image block, and ; express The upper position is The pixel value of express The upper position is The pixel value of

[0097] The z-th fog convolution layer uses formula (7) to obtain the g-th z-th global entropy feature :

[0098] (7)

[0099] In formula (7), express The brightness value of the pixel in ; express In Location At The probability of , and:

[0100] (8)

[0101] In formula (8), is the Dirac function.

[0102] The z-th fog convolution layer uses formula (9) to obtain the g-th z-th original image feature :

[0103] (9)

[0104] In formula (9), Represents the mapping of the inherent atmospheric light A in the surrounding environment Image background; express The position of any pixel in For location The transmittance is:

[0105] (10)

[0106] In formula (10), Indicates location The scene depth at is the extinction coefficient of the foggy image output by the extinction coefficient prediction model;

[0107] When z=2,3,…,Z, As the input of the z-th level residual block, and get the corresponding output g-th residual feature map ; Thus, the g-th residual feature map of the final output is obtained by the Z-th level residual block ,and The dimension is ,in, express The number of channels;

[0108] Figure 4 This is a schematic diagram of the fog convolution layer. By designing four image features, it is possible to capture information from different aspects of the image. By combining this information, the convolution layer can learn more complex and rich feature expressions, thereby improving the robustness of the model and more robustly handling the effects of various noises and interferences.

[0109] D3. Input Z deconvolution layers in sequence for deconvolution processing, and obtain the g-th deconvolution feature map ;

[0110] When z=1, the z-th deconvolution layer is Process and output the g-th z-th level deconvolution feature map :

[0111] When z=2,3,…,Z, the z-th level deconvolution layer uses formula (11) to deconvolute the g-th z-1-th level feature map And the g-th Zz-th level residual feature map Perform residual connection processing and obtain the g-th z-th level deconvolution feature map ; Thus, the final output of the g-th Z-th deconvolution feature map is obtained by the Z-th deconvolution layer ,and The dimension is ,in, express Height, express The width of express The number of channels;

[0112] (11)

[0113] D4, the second convolutional layer pair in the first backbone Perform feature extraction to obtain the g-th first trunk dense depth map ,and The dimension is ,in, express The number of channels.

[0114] D5, the second trunk follows the process of D1-D4 and Perform feature extraction and fusion processing to obtain the g-th second trunk dense depth map ,and The dimension is ,in, express The number of channels;

[0115] D6. and After splicing in the channel dimension, the new g-th dense depth map is obtained .

[0116] E. Based on and Constructing mean absolute error loss , so as to use the Adam optimization algorithm to train the deep completion network of the dual-branch backbone and calculate To update the network parameters until Until convergence, the optimal dual-branch backbone depth completion network model is obtained, which is used to output the dense depth map of the corresponding image.

[0117] In this embodiment, an electronic device is designed, including a memory and a processor. The memory is used to store a program that supports the processor to execute the image depth completion method in the foggy scene, and the processor is configured to execute the program stored in the memory.

[0118] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image depth completion method in a foggy scene are executed.

Claims

1. A method for image depth completion in foggy scenes, characterized in that: The steps include: A. Use the camera to capture the gth image of the target scene ,and The dimension is ,in, Indicates the height of the image. Indicates the width of the image. Indicates the number of channels of the image; Use LiDAR to synchronously collect the g-th point cloud information of the target scene , and generate the g-th sparse depth map ,and The dimension is , where 1 represents the number of channels of the sparse depth map; B. Randomly generate the gth extinction coefficient in the range [0.1,1] , and establish a foggy atmospheric scattering model, so as to use the foggy atmospheric scattering model to Simulate and obtain Foggy images for labels , and then generate a set of foggy simulated images; C. Build an image extinction coefficient prediction network, input a set of foggy simulated images, and use the Adam optimization algorithm to train the network to obtain the optimal image extinction coefficient prediction model; D. Build a dual-branch backbone deep completion network for and Perform feature extraction and fusion processing to obtain the g-th dense depth map ; E. Based on and Constructing mean absolute error loss , so as to use the Adam optimization algorithm to train the dual-branch trunk depth completion network and calculate To update the network parameters until Until convergence, the optimal dual-branch backbone depth completion network model is obtained, which is used to output the dense depth map of the corresponding image.

2. The image depth completion method in a foggy scene according to claim 1, characterized in that: The C includes: C1. Build an image extinction coefficient prediction network, which includes: Q downsampling blocks and T fully connected layers in sequence; wherein the qth downsampling block is composed of the qth multi-layer two-dimensional convolutional layer and the qth downsampling layer; The q-th multi-layer two-dimensional convolutional layer includes: Y two-dimensional convolutional layers, Y batch normalization layers and Y ReLU activation function layers, wherein the convolution kernel size of the y-th two-dimensional convolutional layer is ,y=1,2,…,Y; When q=1, Input the qth level downsampling block for processing and output the qth level downsampling feature map ; When q=2,3,…,Q, the q-1th level downsampled feature map Input to the qth level downsampling block for processing, and obtain the qth level downsampling feature map ; Thus, the Q-th level downsampling block finally outputs the Q-th level downsampling feature map ; After processing by T fully connected layers, the predicted value of the extinction coefficient is output ; C2, based on and the true value of the extinction coefficient Establishing mean squared error loss , so as to use the Adam optimization algorithm to train the image extinction coefficient prediction network and calculate To update the network parameters until Until convergence, the optimal image extinction coefficient prediction model is obtained.

3. The image depth completion method in a foggy scene according to claim 2, characterized in that: The predicted value of the extinction coefficient in C1 It is obtained by the following process: When t=1, Input the t-th level fully connected layer for processing and output the t-th level feature map ; When t=2,3,…,T-1, the t-1th level feature map Input into the t-th level fully connected layer for processing and obtain the t-th level feature map ; When t=T, the T-1th level feature map The input is processed in the t-th level fully connected layer to output the predicted value of the extinction coefficient .

4. The image depth completion method in a foggy scene according to claim 3, characterized in that: The depth completion network in D is composed of a first backbone and a second backbone with the same structure; wherein any backbone includes in sequence: a first convolutional layer, a Z-level residual block, Z deconvolutional layers, and a second convolutional layer; wherein the z-th level residual block includes: a z-th level fog convolutional layer, a z-th level batch normalization layer, and a z-th level ReLU activation function; D1, the first convolutional layer pair in the first backbone and Perform feature extraction to obtain the g-th initial convolution feature map ,and The dimension is ,in, express The number of channels; D2, Z-level residual block pairs in the first backbone Processing is performed to obtain the Z-th level residual feature map : D3. Input the Z deconvolution layers in sequence for deconvolution processing to obtain the g-th deconvolution feature map ; D4, the second convolutional layer pair in the first backbone Perform feature extraction to obtain the g-th first trunk dense depth map ,and The dimension is ,in, express The number of channels; D5, the second trunk follows the process of D1-D4 and Perform feature extraction and fusion processing to obtain the g-th second trunk dense depth map ,and The dimension is ,in, express The number of channels; D6. and After splicing in the channel dimension, the new g-th dense depth map is obtained .

5. The method for image depth completion in foggy scenes according to claim 4, characterized in that: The D2 includes: When z=1, As the input of the z-1th module and output the residual feature map , and input into the z-th fog convolution layer of the first backbone, so as to obtain the g-th z-th global contrast feature using formula (1): : (1) In formula (1), express The upper position is The pixel value of express The upper position is The pixel value of The z-th fog convolutional layer will Divide into image blocks, and use formula (2) to get the g-th z-th level d-th image block The local contrast feature : (2) In formula (2), express The number of rows that the image is divided into blocks, express The number of columns divided into image blocks, and M=N; represents the height of each image block, and , represents the width of each image block, and ; express The upper position is The pixel value of express The upper position is The pixel value of The z-th fog convolution layer uses formula (3) to obtain the g-th z-th global entropy feature : (3) In formula (3), express The brightness value of the pixel in ; express In Location At The probability of , and: (4) In formula (4), is the Dirac function; The z-th fog convolution layer uses formula (5) to obtain the g-th z-th original image feature : (5) In formula (5), Represents the mapping of the inherent atmospheric light A in the surrounding environment Image background; express The position of any pixel in For location The transmittance is: (6) In formula (6), Indicates location The scene depth at is the extinction coefficient of the foggy image output by the extinction coefficient prediction model; When z=2,3,…,Z, As the input of the z-th level residual block, and get the corresponding output g-th residual feature map ; Thus, the g-th residual feature map of the final output is obtained by the Z-th level residual block ,and The dimension is ,in, express The number of channels.

6. The method for image depth completion in foggy scenes according to claim 5, characterized in that: The D3 includes: When z=1, the z-th deconvolution layer is Process and output the g-th z-th level deconvolution feature map : When z=2,3,…,Z, the z-th level deconvolution layer uses formula (7) to deconvolute the g-th z-1-th level feature map And the g-th Zz-th level residual feature map Perform residual connection processing and obtain the g-th z-th level deconvolution feature map ; Thus, the final output of the g-th Z-th deconvolution feature map is obtained by the Z-th deconvolution layer ,and The dimension is ,in, express Height, express The width of express The number of channels; (7)。 7. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the image depth completion method in a foggy scene as described in any one of claims 1-6, and the processor is configured to execute the program stored in the memory.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for image depth completion in a foggy scene described in any one of claims 1 to 6 are executed.

Images