Depth map super-resolution reconstruction method based on diffusion model

Abstract

The invention belongs to the field of depth image processing, and particularly relates to a depth image super-resolution reconstruction method based on a diffusion model, and the method comprises the steps: obtaining a high-resolution depth image and a high-resolution color image, inputting the high-resolution depth image and the high-resolution color image into a trained depth image super-resolution reconstruction model, a reconstructed super-resolution depth map is obtained; the depth map super-resolution reconstruction model comprises a first guide generation network, a guide estimation network and a depth map super-resolution network; according to the method, the guide information is generated by utilizing the guide generation network according to the multi-modal image, and the guide information is estimated by utilizing the guide estimation network based on the diffusion model according to the low-resolution depth image, so that priori knowledge required by the depth image can be better restored, and the guide estimation information is obtained; and the guidance information and the guidance estimation information are combined to guide the reconstruction of the depth map, so that the reconstruction precision of the depth map can be effectively improved, and particularly, the method has excellent performance in the recovery of complex textures and edge details.

Description

Technical Field

[0001] The present invention belongs to the field of depth image processing, and in particular relates to a depth map super-resolution reconstruction method based on a diffusion model. Background Art

[0002] Depth images play an important role in 3D scene understanding and are widely used in 3D reconstruction, virtual reality (VR), augmented reality (AR) and other fields. Due to the limitations of modern depth sensors, depth images are usually low resolution (LR) and noisy. However, when acquiring a depth map, it is relatively easy to obtain a high-resolution color image in the same scene. Therefore, the information in the color image can be used to guide the reconstruction of a high-resolution depth map.

[0003] Existing guided deep image reconstruction methods are mainly divided into the following four categories: 1) Methods based on pixel-to-pixel mapping: This method regards guided super-resolution as a pixel-to-pixel mapping of the guide image (high-resolution color image) to the source image domain (high-resolution depth image). It is implemented through a parameterized multi-layer perceptron, and the weights are updated by minimizing the difference between the source image and the downsampled target image. 2) Methods based on deep reconstruction: Using a deep learning framework, a convolutional neural network for deep image super-resolution reconstruction guided by color images is constructed to achieve super-resolution reconstruction of depth maps. 3) Hybrid methods based on interpolation and reconstruction: Based on deep learning, traditional image processing techniques, such as interpolation algorithms, are combined to improve the effect of super-resolution reconstruction. 4) Methods based on feature fusion: Through feature extraction and fusion strategies, the features of depth images and color images are combined to improve the quality of super-resolution reconstruction.

[0004] However, existing depth map super-resolution techniques have problems such as high computational complexity, limited feature extraction, and poor edge detail recovery. Therefore, a new depth map super-resolution method is urgently needed to improve reconstruction accuracy and efficiency. Summary of the invention

[0005] In order to solve the above-mentioned problems in the prior art, the present invention adopts a depth map super-resolution reconstruction method based on a diffusion model, including: obtaining a high-resolution depth map and a high-resolution color map, inputting the high-resolution depth map and the high-resolution color map into a trained depth map super-resolution reconstruction model, and obtaining a reconstructed super-resolution depth map; the depth map super-resolution reconstruction model includes: a first guided generation network GGN, a guided estimation network GRN and a depth map super-resolution network DSRN; the training process of the depth map super-resolution reconstruction model includes:

[0006] S1, obtain a high-resolution depth map and a high-resolution color map, perform average pooling on the high-resolution depth map to obtain a low-resolution depth map;

[0007] S2, inputting the high-resolution depth map, the low-resolution depth map and the high-resolution color map into a first guide generation network GGN to extract guide information;

[0008] S3, inputting the guidance information and the low-resolution depth map into the guidance estimation network GRN to generate guidance estimation information;

[0009] S4, inputting the low-resolution depth map, the guidance information, the guidance estimation information and the high-resolution color map into a depth map super-resolution network DSRN to obtain a reconstructed super-resolution depth map;

[0010] S5. Calculate the loss function value according to the reconstructed super-resolution depth map, update the model parameters according to the loss function value, and obtain the trained depth map super-resolution reconstruction model when the loss function value is minimum.

[0011] The first guidance generation network GGN extracts guidance information including:

[0012] S21, down-sampling the high-resolution depth map and the high-resolution color map to obtain a down-sampled map;

[0013] S22, splicing the downsampled image and the low-resolution depth image, and processing the spliced ​​image using a residual block to obtain guidance information.

[0014] The guided estimation network GRN includes: a diffusion model and a second guided generation network GGN; wherein the diffusion model includes a noise adding network and a noise removing network, and the second guided generation network GGN has the same structure as the first guided generation network GGN; the guided estimation network GRN processes the guided information and the low-resolution depth map including:

[0015] S31, input the guidance information into the noise adding network for forward noise adding, and obtain the noise data G T :

[0016] S32: Input the low-resolution depth map into the second guide generation network GGN to obtain condition information C G ;

[0017] S33, condition information C G and the noise data G T Input the denoising network to get the guided estimation information

[0018] The depth map super-resolution network DSRN includes: a depth map feature extraction module group, a color map feature extraction module group and a feature fusion module group; the depth map super-resolution network DSRN processes the low-resolution depth map, the guidance information, the guidance estimation information and the high-resolution color map including:

[0019] S41, convolve the low-resolution depth map to obtain a first feature map; input the guidance information, the guidance estimation information and the first feature map into the depth map feature extraction module group to obtain a depth feature;

[0020] S42, convolve the high-resolution color image to obtain a second feature map; input the second feature map into the color image feature extraction module group to obtain color features;

[0021] S43, inputting the depth features and the color features into the feature fusion module group for feature fusion, combining the fusion result with the low-resolution depth map, and obtaining a reconstructed super-resolution depth map.

[0022] The depth map feature extraction module group includes: multiple DFE Blocks, convolution layers and ESA Blocks connected in series in sequence; the depth map feature extraction module group processes the guidance information, guidance estimation information and the first feature map including: inputting the guidance information, guidance estimation information and the first feature map into multiple DFE Blocks connected in series in sequence, inputting the output of the last DFE Block into the convolution layer, combining the output of the convolution layer and the first feature map, and inputting the combined features into the ESABlock; wherein the DFE Block is a depth map feature extraction block, and the ESABlock is a spatial attention enhancement module.

[0023] DFE Block includes: a preliminary processing module, a self-attention feature extraction module and a channel attention feature extraction module; DFE Block processes the guidance information, the guidance estimation information and the first feature map including: inputting the guidance information, the guidance estimation information and the first feature map into the preliminary processing module, inputting the output of the preliminary processing module into the self-attention feature extraction module, combining the output of the self-attention feature extraction module with the first feature map to obtain a combined feature; inputting the combined feature into the channel attention feature extraction module; combining the output of the channel attention feature extraction module with the combined feature to obtain the output of DFE Block.

[0024] The preliminary processing module processes the guidance information, the guidance estimation information and the first feature map, including: combining the guidance information and the guidance estimation information, and successively shaping, upsampling and convolving the combined information to obtain the guidance feature; normalizing the first feature map, combining the normalized result with the guidance feature, and obtaining the output of the preliminary processing module.

[0025] The color image feature extraction module group includes: multiple CFE Blocks, convolutional layers and ESA Blocks connected in series in sequence; the color image feature extraction module group processes the second feature map by: inputting the second feature map into multiple DFE Blocks connected in series in sequence, inputting the output of the last DFE Block into the convolutional layer, combining the output of the convolutional layer and the second feature map, and inputting the combined features into the ESABlock; wherein the CFE Block is a color image feature extraction block.

[0026] CFE Block includes: a first convolution branch, a second convolution branch and a channel attention feature extraction module; DFEBlock processes the second feature map including: dividing the second feature map into two sub-feature maps according to the channel dimension, inputting the two sub-feature maps into the first convolution branch and the second convolution branch respectively, splicing the outputs of the first convolution branch and the second convolution branch, combining the spliced ​​result with the second feature map, inputting the combined result into the channel attention feature extraction module, combining the output of the channel attention feature extraction module with the combined result, and obtaining the output of CFE Block.

[0027] The feature fusion module group includes two feature fusion modules. The depth feature and color feature are input into the first feature fusion module, and the output of the first feature fusion module and the color feature are input into the second feature fusion module; the output of the second feature fusion module is combined with the low-resolution depth map to obtain a reconstructed super-resolution depth map.

[0028] Beneficial effects:

[0029] 1. The present invention generates guidance information using a guided generation network GGN according to a multimodal high-resolution depth map, a low-resolution depth map and a high-resolution color map; estimates the guidance information using a guided estimation network GRN based on a diffusion model according to the low-resolution depth map, which can better restore the prior knowledge required for the depth map, obtain guided estimation information, and combine the guided information and the guided estimation information to guide the reconstruction of the depth map, which can effectively improve the reconstruction accuracy of the depth map, especially in the recovery of complex textures and edge details; 2. The depth map super-resolution network (DSRN) of the present invention extracts color features of the high-resolution color map, extracts depth features of the low-resolution depth map using the guided information and the guided estimation information, and effectively improves the spatial resolution of the low-resolution depth map by fusing the color features with the depth features, thereby effectively improving the reconstruction accuracy of the depth map; 3. The guided estimation network GRN based on the diffusion model of the present invention has a simple structure, can significantly reduce the demand for computing resources, and improve processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1A framework diagram of a depth map super-resolution reconstruction method based on a diffusion model provided by an embodiment of the present invention;

[0031] Figure 2 A framework diagram of a first guidance generation network GGN provided in an embodiment of the present invention;

[0032] Figure 3 A framework diagram of a guided estimation network GRN provided by an embodiment of the present invention;

[0033] Figure 4 A framework diagram of a depth map super-resolution network DSRN provided in an embodiment of the present invention;

[0034] Figure 5 A framework diagram of a feature fusion module provided in an embodiment of the present invention;

[0035] Figure 6 A framework diagram of a depth map feature extraction module provided in an embodiment of the present invention;

[0036] Figure 7 A framework diagram of a color image feature extraction block provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] like Figure 1 As shown, the present invention adopts a depth map super-resolution reconstruction method based on a diffusion model, including: obtaining a high-resolution depth map and a high-resolution color map, inputting the high-resolution depth map and the high-resolution color map into a trained depth map super-resolution reconstruction model, and obtaining a reconstructed super-resolution depth map; the depth map super-resolution reconstruction model includes: a first guided generation network GGN, a guided estimation network GRN and a depth map super-resolution network DSRN; the training process of the depth map super-resolution reconstruction model includes:

[0039] S1, obtain a high-resolution depth map and a high-resolution color map, perform average pooling on the high-resolution depth map to obtain a low-resolution depth map;

[0040] S2, inputting the high-resolution depth map, the low-resolution depth map and the high-resolution color image into a first guide generation network GGN to extract guide information;

[0041] S3, inputting the guidance information and the low-resolution depth map into the guidance estimation network GRN to generate guidance estimation information;

[0042] S4, inputting the low-resolution depth map, the guidance information, the guidance estimation information and the high-resolution color map into a depth map super-resolution network DSRN to obtain a reconstructed super-resolution depth map;

[0043] S5. Calculate the loss function value according to the reconstructed super-resolution depth map, update the model parameters according to the loss function value, and obtain the trained depth map super-resolution reconstruction model when the loss function value is minimum.

[0044] Average pooling of the high-resolution depth map includes: dividing the high-resolution depth map into non-overlapping 2×2 pixel blocks, and then calculating the average value of the pixels in each block as a pixel in the corresponding low-resolution depth map to obtain the low-resolution depth map.

[0045] like Figure 2 As shown, the first guidance generation network GGN extracts guidance information including:

[0046] S21, down-sampling the high-resolution depth map and the high-resolution color map to obtain a down-sampled map;

[0047] The high-resolution depth map and color image are downsampled using pixel-unshuffle according to the scale factor, including: dividing the image into 2×2 area blocks, each block is rearranged to a different output channel, and the resolution of the output downsampled image is reduced to half of the original image.

[0048] S22, concatenate the downsampled image and the low-resolution depth image, process the concatenated image using the residual block, and convert the result of the residual block processing into Compress (Compress) to obtain the boot information Where H and W are the height and width respectively, and C is the number of channels.

[0049] like Figure 3 As shown, the guided estimation network GRN includes: a diffusion model and a second guided generation network GGN, the diffusion model includes a noise adding network and a noise removing network; the guided estimation network GRN processes the guided information and the low-resolution depth map including:

[0050] S31, guide information Input the noise adding network for forward noise adding to obtain the noise data G T :

[0051]

[0052] Among them, q(GT |G) is the case where the guidance information G is given, the noise data G T distribution of , T is the total number of time steps, represents Gaussian distribution, G T represents the noisy data at time step T, α T represents the degree of noise addition at time step T, represents the identity matrix, Indicates the degree of cumulative noise addition in the first T time steps.

[0053] S32: Input the low-resolution depth map into the second guide generation network GGN to obtain condition information C G ; The architecture of the second guided generation network GGN is exactly the same as the first guided generation network GGN except for the input;

[0054] S33, condition information C G and the noise data G T Input the denoising network, and after T iterations, the guided estimation information is obtained

[0055] Since the guidance information required by DSRN is very compact, a simple denoising network is selected to estimate the noise, and the denoising iteration is started at the Tth time step to obtain the guidance estimation information.

[0056]

[0057] in, The denoising network is based on the input information and condition information C G The predicted noise, is the denoising result at time step t, α t is the degree of noise addition at time step t, Degree to add to the accumulated noise of the previous t time steps.

[0058] like Figure 4 As shown, the depth map super-resolution network DSRN includes: a depth map feature extraction module group, a color map feature extraction module group and a feature fusion module group; the depth map super-resolution network DSRN processes the low-resolution depth map, the guidance information, the guidance estimation information and the high-resolution color map including:

[0059] S41, convolve the low-resolution depth map to obtain a first feature map; input the guidance information, the guidance estimation information and the first feature map into the depth map feature extraction module group to obtain a depth feature;

[0060] The depth map feature extraction module group includes: multiple DFE Blocks, convolution layers and ESA Blocks connected in series in sequence; the depth map feature extraction module group processes the guidance information, guidance estimation information and the first feature map including: inputting the guidance information, guidance estimation information and the feature map into multiple DFE Blocks connected in series in sequence, inputting the output of the last DFE Block into the convolution layer, adding the output of the convolution layer and the first feature map, and inputting the added features into the ESABlock; wherein, DFEBlock is a depth map feature extraction module, and ESA Block is a spatial attention enhancement module.

[0061] ESABlock processes the added features by inputting the added features into the convolution layer, inputting the output of the convolution layer into the spatial attention layer, and fusing the output of the spatial attention layer with the added features to obtain more discriminative features.

[0062] like Figure 6 As shown, the DFE Block includes a preliminary processing module, a self-attention feature extraction module and a channel attention feature extraction module; the DFE Block processes the guidance information, the guidance estimation information and the first feature map, including: inputting the guidance information, the guidance estimation information and the first feature map into the preliminary processing module, inputting the output of the preliminary processing module into the self-attention feature extraction module, adding the output of the self-attention feature extraction module and the first feature map to obtain a combined feature; inputting the combined feature into the channel attention feature extraction module; adding the output of the channel attention feature extraction module and the combined feature to obtain the output of the DFE Block.

[0063] The preliminary processing module processes the guidance information, the guidance estimation information and the first feature map, including: splicing the guidance information and the guidance estimation information, reshaping, nearest up sampling and 1×1 convolution of the spliced ​​information G′ in turn to obtain the guidance feature; normalizing the first feature map, adding the normalized result and the guidance feature to obtain the output of the preliminary processing module.

[0064] Specifically, the process can be described as:

[0065]

[0066] The depth map feature extraction module group includes: multiple DFE Blocks, convolution layers and ESA Blocks connected in series in sequence; the depth map feature extraction module group processes the guidance information, guidance estimation information and the first feature map including: inputting the guidance information, guidance estimation information and the feature map into multiple DFE Blocks connected in series in sequence, inputting the output of the last DFE Block into the convolution layer, adding the output of the convolution layer and the first feature map, and inputting the added features into the ESA Block; wherein the DFE Block is a depth map feature extraction module, and the ESA Block is a spatial attention enhancement module.

[0067] The self-attention feature extraction module includes a 1×1 convolution layer, a 3×3 convolution layer, a self-attention module, and a 1×1 convolution layer connected in series; the channel attention feature extraction module includes a normalization layer (LN), a 1×1 convolution layer, a Gelu activation layer, a 1×1 convolution layer, and a channel attention module (Channel Attention) connected in series.

[0068] S42, convolve the high-resolution color image to obtain a second feature map; input the second feature map into the color image feature extraction module group to obtain color features;

[0069] Since the extraction of color image features does not require parsing the information from the guide, the high-resolution color image is directly input into the color image feature extraction module group after convolution to obtain the color feature x0.

[0070] The color image feature extraction module group includes: multiple CFE Blocks, convolutional layers and ESA Blocks connected in series in sequence; the color image feature extraction module group processes the second feature map by: inputting the second feature map into multiple DFE Blocks connected in series in sequence, inputting the output of the last DFE Block into the convolutional layer, adding the output of the convolutional layer and the second feature map, and inputting the added features into the ESABlock.

[0071] like Figure 7 As shown, the CFE Block includes: a first convolution branch, a second convolution branch and a channel attention feature extraction module; the DFE Block processes the second feature map including: dividing the second feature map into two sub-feature maps according to the channel dimension, inputting the two sub-feature maps into the first convolution branch and the second convolution branch respectively, concatenating the outputs of the first convolution branch and the second convolution branch, adding the concatenated result to the second feature map, inputting the added result into the channel attention feature extraction module, adding the output of the channel attention feature extraction module to the added result, and obtaining the output of the CFE Block.

[0072] The first convolution branch includes: a 1×1 convolution layer and a 3×3 convolution layer; the first convolution branch processes the sub-feature map by: performing 1×1 convolution on the sub-feature map, performing 1×1 convolution and 3×3 convolution on the features after the 1×1 convolution, multiplying the features after the 1×1 convolution and the 3×3 convolution, and performing 3×3 convolution on the multiplied features to obtain the output of the first convolution branch.

[0073] The second convolution branch includes a 1×1 convolutional layer and a 3×3 convolutional layer connected in series.

[0074] The channel attention feature extraction module includes a normalization layer (LN), a 1×1 convolution layer, a Gelu activation layer, a 1×1 convolution layer, and a channel attention module (Channel Attention) connected in series.

[0075] Adding the concatenated result to the second feature map includes: performing convolution on the concatenated result, and adding the convolution result to the second feature map.

[0076] S43, inputting the depth features and the color features into the feature fusion module group for feature fusion, combining the fusion result with the low-resolution depth map, and obtaining a reconstructed super-resolution depth map.

[0077] The feature fusion module group includes two feature fusion modules FFM. The depth feature and color feature are input into the first feature fusion module, and the output of the first feature fusion module and the color feature are input into the second feature fusion module. The low-resolution depth map is bicubic upsampling, and the output of the second feature fusion module is added to the low-resolution depth map after bicubic upsampling to obtain a reconstructed super-resolution depth map.

[0078] like Figure 5 As shown in the figure, FFM processes the depth features and color features by: aligning the color features and the depth features through grid sampling (GridSample), processing the aligned color features through convolutional layers and Sigmoid activation functions, and multiplying the processed color features with the aligned depth features to obtain fused features, which can retain the effective components of the depth and color features for reconstruction while weakening other non-essential features; further refining the fused features through the residual block to obtain the final fused feature map.

[0079] Optionally, the loss function is a mean square error loss function, and the model is updated by calculating the average value of the square difference of pixel values ​​between the reconstructed image and the target image.

[0080] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A depth map super-resolution reconstruction method based on a diffusion model, characterized in that: include: Obtain a high-resolution depth map and a high-resolution color map, and input the high-resolution depth map and the high-resolution color map into a trained depth map super-resolution reconstruction model to obtain a reconstructed super-resolution depth map; The depth map super-resolution reconstruction model includes: a first guide generation network GGN, a guide estimation network GRN and a depth map super-resolution network DSRN; the training process of the depth map super-resolution reconstruction model includes: S1, obtain a high-resolution depth map and a high-resolution color map, perform average pooling on the high-resolution depth map to obtain a low-resolution depth map; S2, inputting the high-resolution depth map, the low-resolution depth map and the high-resolution color map into a first guide generation network GGN to extract guide information; S3, inputting the guidance information and the low-resolution depth map into the guidance estimation network GRN to generate guidance estimation information; S4, inputting the low-resolution depth map, the guidance information, the guidance estimation information and the high-resolution color map into a depth map super-resolution network DSRN to obtain a reconstructed super-resolution depth map; S5. Calculate the loss function value according to the reconstructed super-resolution depth map, update the model parameters according to the loss function value, and obtain the trained depth map super-resolution reconstruction model when the loss function value is minimum.

2. The method for super-resolution reconstruction of a depth map based on a diffusion model according to claim 1, characterized in that: The first guidance generation network GGN extracts guidance information including: S21, down-sampling the high-resolution depth map and the high-resolution color map to obtain a down-sampled map; S22, splicing the downsampled image and the low-resolution depth image, and processing the spliced ​​image using a residual block to obtain guidance information.

3. The method for super-resolution reconstruction of a depth map based on a diffusion model according to claim 1, characterized in that: The guided estimation network GRN includes: a diffusion model and a second guided generation network GGN; wherein the diffusion model includes a noise adding network and a noise removing network, and the second guided generation network GGN has the same structure as the first guided generation network GGN; the guided estimation network GRN processes the guided information and the low-resolution depth map including: S31, input the guidance information into the noise adding network for forward noise adding, and obtain the noise data G T : S32: Input the low-resolution depth map into the second guide generation network GGN to obtain condition information C G ; S33, condition information C G and the noise data G T Input the denoising network to get the guided estimation information 4. The method for super-resolution reconstruction of a depth map based on a diffusion model according to claim 1, characterized in that: The depth map super-resolution network DSRN includes: a depth map feature extraction module group, a color map feature extraction module group and a feature fusion module group; the depth map super-resolution network DSRN processes the low-resolution depth map, the guidance information, the guidance estimation information and the high-resolution color map including: S41, convolve the low-resolution depth map to obtain a first feature map; input the guidance information, the guidance estimation information and the first feature map into the depth map feature extraction module group to obtain a depth feature; S42, convolve the high-resolution color image to obtain a second feature map; input the second feature map into the color image feature extraction module group to obtain color features; S43, inputting the depth features and the color features into the feature fusion module group for feature fusion, combining the fusion result with the low-resolution depth map, and obtaining a reconstructed super-resolution depth map.

5. The method for super-resolution reconstruction of a depth map based on a diffusion model according to claim 4, characterized in that: The depth map feature extraction module group includes: multiple DFE Blocks, convolution layers and ESA Blocks connected in series in sequence; the depth map feature extraction module group processes the guidance information, guidance estimation information and the first feature map including: inputting the guidance information, guidance estimation information and the first feature map into multiple DFE Blocks connected in series in sequence, inputting the output of the last DFE Block into the convolution layer, combining the output of the convolution layer and the first feature map, and inputting the combined features into the ESA Block; wherein, DFEBlock is a depth map feature extraction block, and ESA Block is a spatial attention enhancement module.

6. The method for super-resolution reconstruction of a depth map based on a diffusion model according to claim 5, characterized in that: DFE Block includes: a preliminary processing module, a self-attention feature extraction module and a channel attention feature extraction module; DFEBlock processes the guidance information, the guidance estimation information and the first feature map, including: inputting the guidance information, the guidance estimation information and the first feature map into the preliminary processing module, inputting the output of the preliminary processing module into the self-attention feature extraction module, combining the output of the self-attention feature extraction module with the first feature map to obtain a combined feature; inputting the combined feature into the channel attention feature extraction module; combining the output of the channel attention feature extraction module with the combined feature to obtain the output of DFEBlock.

7. The method for super-resolution reconstruction of a depth map based on a diffusion model according to claim 6, characterized in that: The preliminary processing module processes the guidance information, the guidance estimation information and the first feature map, including: combining the guidance information and the guidance estimation information, and successively shaping, upsampling and convolving the combined information to obtain the guidance feature; normalizing the first feature map, combining the normalized result with the guidance feature, and obtaining the output of the preliminary processing module.

8. The method for super-resolution reconstruction of a depth map based on a diffusion model according to claim 4, characterized in that: The color image feature extraction module group includes: multiple CFE Blocks, convolutional layers and ESA Blocks connected in series in sequence; the color image feature extraction module group processes the second feature map by: inputting the second feature map into multiple DFE Blocks connected in series in sequence, inputting the output of the last DFE Block into the convolutional layer, combining the output of the convolutional layer and the second feature map, and inputting the combined features into the ESA Block; wherein the CFE Block is a color image feature extraction block, and the ESABlock is a spatial attention enhancement module.

9. The method for super-resolution reconstruction of a depth map based on a diffusion model according to claim 8, characterized in that: The CFE Block includes: a first convolution branch, a second convolution branch and a channel attention feature extraction module; the DFE Block processes the second feature map including: dividing the second feature map into two sub-feature maps according to the channel dimension, inputting the two sub-feature maps into the first convolution branch and the second convolution branch respectively, splicing the outputs of the first convolution branch and the second convolution branch, combining the spliced ​​result with the second feature map, inputting the combined result into the channel attention feature extraction module, combining the output of the channel attention feature extraction module with the combined result, and obtaining the output of the CFE Block.

10. The method for super-resolution reconstruction of a depth map based on a diffusion model according to claim 1, characterized in that: The feature fusion module group includes two feature fusion modules. The depth feature and color feature are input into the first feature fusion module, and the output of the first feature fusion module and the color feature are input into the second feature fusion module; the output of the second feature fusion module is combined with the low-resolution depth map to obtain a reconstructed super-resolution depth map.

Images