A 3D Reconstruction Method and System Based on Frequency-Spatial Dual-Domain Features and Cascaded Optimization

Abstract

This invention discloses a 3D reconstruction method and system based on frequency-space dual-domain features and cascaded optimization. The method includes: acquiring a sequence of multi-view images of the target scene; using a frequency-space dual-path feature extraction network to extract local texture in the spatial domain and encode global structure in the frequency domain for each image, and fusing them to generate a dual-domain enhanced feature map; constructing grouped cost volumes sequentially from coarse to fine based on a cascaded pyramid strategy, and using a 3D attention mechanism to regularize the cost volumes and suppress matching noise; adaptively adjusting the depth search range between each cascaded stage based on the uncertainty of depth estimation in the previous stage; and finally generating a dense 3D point cloud by backprojecting the fine depth map. This invention significantly improves the reconstruction integrity and geometric accuracy of weak texture and high reflectivity areas through the synergy of frequency domain prior and attention mechanism, and is suitable for high-fidelity 3D reconstruction of complex objects such as spatial targets and industrial parts.

Description

Technical Field

[0001] This invention relates to the fields of image processing and 3D vision technology, and more specifically, to a 3D reconstruction method and system based on frequency-space dual-domain features and cascade optimization. Background Technology

[0002] Multi-view stereo 3D reconstruction is a fundamental task in computer vision and is widely used in scenarios such as autonomous driving, augmented reality, industrial precision measurement, and on-orbit maintenance of non-cooperative spacecraft. Currently, the mainstream deep learning 3D reconstruction methods are based on a coarse-to-fine cascaded cost volume architecture, which has achieved good reconstruction results in general scenarios.

[0003] However, current cascaded reconstruction algorithms still suffer from significant performance degradation when dealing with complex targets with large areas of weak texture and highly reflective materials. Existing technologies include: Scheme 1, standard spatial domain feature extraction; Scheme 2, traditional variance cost volume aggregation; and Scheme 3, conventional cost volume regularization. These existing technologies have the following drawbacks when applied:

[0004] 1. Unable to effectively recover the complete depth of weak textures and highly reflective target surfaces; point cloud is severely incomplete.

[0005] 2. The cost body construction process contains too much redundant information, making it impossible to accurately determine the matching similarity;

[0006] 3. Unable to preserve high-frequency details and sharp edges of geometric shapes while performing regularized noise reduction;

[0007] 4. The depth range in the cascade stage is fixed, lacking the ability to dynamically adjust based on observation results, resulting in wasted computation. Summary of the Invention

[0008] The purpose of this invention is to provide a three-dimensional reconstruction method and system based on frequency-space dual-domain features and cascade optimization, which solves the problems of poor reconstruction integrity, large matching noise, and loss of geometric details in the prior art for weak texture and highly reflective areas.

[0009] The first aspect of this invention provides a three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization, comprising the following steps:

[0010] Acquire a sequence of two-dimensional images of the target scene from N different perspectives, wherein one of the two-dimensional images in the sequence is used as a reference view and the remaining images are used as source views;

[0011] A dual-path feature extraction module is used to extract images from each viewpoint to generate corresponding dual-domain enhanced feature maps, wherein the dual-domain enhanced feature maps fuse local texture edge and global frequency structure information;

[0012] The dual-domain enhanced feature maps from N perspectives are subjected to differentiable homography transformation according to the preset number of depth assumption planes to generate an initial cost volume, and the initial cost volume is calibrated and aggregated to obtain the enhanced cost volume.

[0013] The enhanced cost volume is fed into a preset network model to obtain a probability volume, and then the expected depth map and pixel-level uncertainty map are calculated along the depth dimension.

[0014] Between cascaded stages, the depth search range for the next stage is adaptively expanded and sampled based on the desired depth map and the pixel-level uncertainty map, and a fine depth map is output in the final stage, based on which a 3D point cloud of the target scene is generated.

[0015] In this scheme, the step of performing dual-path feature extraction on the image from each viewpoint to generate a corresponding dual-domain enhanced feature map specifically includes:

[0016] Perform standard convolution on the spatial path of the input viewpoint image to extract local detail features and generate spatial features;

[0017] In the frequency domain path, a fast Fourier transform is performed on the input viewpoint image to map it to the frequency domain. After adjusting the weights of each frequency component through a learnable frequency modulation function, an inverse Fourier transform is performed to reconstruct it back to the spatial domain, generating frequency features.

[0018] The spatial features and the frequency features are fused element by element to obtain a dual-domain enhanced feature map that simultaneously possesses local texture discrimination and global structural integrity.

[0019] In this scheme, the method further includes: performing feature recalibration on the dual-domain enhanced feature map and adaptively aggregating the multi-receptive-field convolutional response, decomposing the multi-receptive-field convolutional response into low-frequency approximation components, horizontal detail components and vertical detail components and then fusing them to generate an enhanced multi-scale discriminative feature map, wherein the multi-scale discriminative feature map is used to construct the cost body.

[0020] In this scheme, the step of performing a differentiable homography transformation on the dual-domain enhanced feature maps from N viewpoints according to a preset number of depth assumption planes to generate an initial cost volume, and then calibrating and aggregating the initial cost volume to obtain the enhanced cost volume, specifically includes:

[0021] Receive the multi-scale discriminative feature maps generated by each view at multiple scales, calculate the variance of all the dual-domain enhanced feature maps under each depth assumption, and stack all the variances to form an initial cost body;

[0022] The initial cost volume is grouped along the channel dimension, and a lightweight 3D convolution and sigmoid activation are applied to each cost volume to obtain the grouped attention weights.

[0023] The weights are multiplied element-wise with the corresponding grouped cost bodies to generate the enhanced cost body.

[0024] In this scheme, the step of feeding the enhanced cost volume into a preset network model to obtain a probability volume, and then calculating the expected depth map and pixel-level uncertainty map along the depth dimension, specifically includes:

[0025] The preset network model includes an encoder, a decoder, and a 3DU-Net architecture network model with skip connections;

[0026] The enhanced cost body is input into the 3DU-Net architecture network model, wherein at each skip connection, a set of three-dimensional attention operators that simultaneously consider channel, spatial and depth dimensions are inserted to filter out mismatch noise during the regularization process, and the regularized cost body is output.

[0027] A Softmax operation is performed on the regularized cost volume along the depth dimension to obtain a depth probability volume reflecting the probability of each pixel at different depths. The expected depth map for guiding the next level of sampling at the current stage is calculated by probability weighted averaging, as well as an uncertainty map that measures the reliability of the estimate.

[0028] In this scheme, between the cascaded stages, the depth search range for the next stage is adaptively expanded and sampled based on the desired depth map and the pixel-level uncertainty map, and a fine depth map is output in the final stage. A 3D point cloud of the target scene is generated based on the fine depth map. Specifically, this includes:

[0029] During the transition between adjacent cascaded stages, obtain the expected depth map and uncertainty map of the current stage;

[0030] Construct an initial depth interval based on the standard interval of the current depth and the number of assumptions;

[0031] Based on the uncertainty of each pixel, a scaling factor is applied proportionally to pixels with high uncertainty to dynamically stretch the depth search range;

[0032] The depth hypothesis plane of uniformly resampled sub-pixels within the adaptively determined upper and lower boundaries is used as the input for the refinement search in the next stage.

[0033] The process iterates until the final stage outputs the final detailed depth map, and combines the camera intrinsics to back-project the pixels of the reference view into the three-dimensional space to generate the three-dimensional point cloud of the target scene.

[0034] A second aspect of the present invention also provides a three-dimensional reconstruction system based on frequency-space dual-domain features and cascaded optimization, comprising a memory and a processor. The memory includes a three-dimensional reconstruction method program based on frequency-space dual-domain features and cascaded optimization. When the processor executes the three-dimensional reconstruction method program based on frequency-space dual-domain features and cascaded optimization, it performs the following steps:

[0035] Acquire a sequence of two-dimensional images of the target scene from N different perspectives, wherein one of the two-dimensional images in the sequence is used as a reference view and the remaining images are used as source views;

[0036] A dual-path feature extraction module is used to extract images from each viewpoint to generate corresponding dual-domain enhanced feature maps, wherein the dual-domain enhanced feature maps fuse local texture edge and global frequency structure information;

[0037] The dual-domain enhanced feature maps from N perspectives are subjected to differentiable homography transformation according to the preset number of depth assumption planes to generate an initial cost volume, and the initial cost volume is calibrated and aggregated to obtain the enhanced cost volume.

[0038] The enhanced cost volume is fed into a preset network model to obtain a probability volume, and then the expected depth map and pixel-level uncertainty map are calculated along the depth dimension.

[0039] Between cascaded stages, the depth search range for the next stage is adaptively expanded and sampled based on the desired depth map and the pixel-level uncertainty map, and a fine depth map is output in the final stage, based on which a 3D point cloud of the target scene is generated.

[0040] A third aspect of the present invention provides a computer-readable storage medium comprising a machine program for a three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization, wherein when the program is executed by a processor, it implements the steps of the three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization as described in any of the preceding claims.

[0041] A fourth aspect of the present invention provides a computer program product comprising computer program code, wherein when the computer program code is run on a computer, the computer implements the steps of a three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization as described in any of the preceding claims.

[0042] The fifth aspect of the present invention provides an electronic device, the electronic device comprising: a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the steps of a three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization as described in any of the preceding claims.

[0043] The present invention discloses a three-dimensional reconstruction method and system based on frequency-space dual-domain features and cascade optimization, which has the following beneficial effects:

[0044] 1. By extracting features through both frequency and space, the Fourier spectrum is used to enable the network to perceive global low-frequency contours, thereby greatly improving the depth continuity of weak textures and highly reflective surfaces, and significantly improving the integrity of the reconstructed point cloud compared with conventional methods.

[0045] 2. By combining grouped cost volume aggregation with 3D attention regularization, redundant channel noise is removed from both the feature source and cost matching levels, and the cross-view geometrically consistent response is amplified, resulting in a significant reduction in voids and anomalous flying points on the metal plane.

[0046] 3. By using an uncertainty-based adaptive depth sampling mechanism, the search range is automatically expanded in high uncertainty areas to cover more possible values, while low uncertainty areas maintain high-resolution positioning, achieving high-fidelity reproduction of boundaries and micro-part structures without significantly increasing video memory. Attached Figure Description

[0047] Figure 1 The diagram illustrates the steps of a three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization according to the present invention.

[0048] Figure 2 The diagram illustrates the frequency-space dual-path enhancement architecture of a three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization according to the present invention.

[0049] Figure 3 A schematic diagram of a wavelet-enhanced feature pyramid architecture for a three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization according to the present invention is shown.

[0050] Figure 4 A schematic diagram of an efficient grouping cost calibration architecture for a three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization according to the present invention is shown.

[0051] Figure 5 The diagram illustrates an adaptive depth-sensing refinement architecture for a 3D reconstruction method based on frequency-space dual-domain features and cascaded optimization according to the present invention.

[0052] Figure 6 A block diagram of a three-dimensional reconstruction method and system based on frequency-space dual-domain features and cascade optimization according to the present invention is shown. Detailed Implementation

[0053] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0054] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0055] Currently, the implementation schemes most similar to this invention are mainly the following: Scheme 1, Scheme 2, or Scheme 3.

[0056] Among them, the first scheme is standard spatial domain feature extraction: it only uses local spatial convolution stacking to extract features, which limits the receptive field and makes it difficult to extract discriminative matching primitives when encountering smooth surfaces, resulting in large-area holes and discontinuities in the depth map; the second scheme is traditional variance cost volume aggregation: the multi-view matching aggregation method is simple and lacks effective screening of redundant information in the channel dimension, resulting in a large amount of noise mixed into the cost volume input and regularization process, which is very easy to produce matching ambiguity when facing high reflectivity; the third scheme is conventional cost volume regularization: existing regularization networks mostly use simple 3D convolution concatenation, without a screening mechanism for the continuity of the depth direction, making it difficult to retain small sharp corners and edges while suppressing abnormal fluctuations, resulting in deformed or overly smooth geometric structures.

[0057] In view of the shortcomings of the prior art mentioned in the background section, the purpose of this invention is to provide a three-dimensional reconstruction method and system based on frequency-space dual-domain features and cascade optimization, wherein this invention aims to achieve the following objectives:

[0058] (1) High-frequency details and global structure are taken into account, and complete reconstruction is achieved under harsh lighting and material conditions;

[0059] (2) Improve the expression quality of the cost volume to achieve high-fidelity deep reconstruction with minimal parameter overhead;

[0060] (3) It endows the network with the ability to dynamically adjust the receptive field and the upper limit of the depth range, and adapts to the current geometric topology. Therefore, the core innovation and key protection point are:

[0061] 1. A dual-path feature extraction structure that operates in both the spatial and frequency domains adaptively captures global low-frequency structures through a learnable frequency modulation function, complementing local spatial textures;

[0062] 2. A cost volume aggregation and regularization technique combining grouped channel attention and channel-space-depth 3D attention;

[0063] 3. An adaptive depth range sampling strategy driven by pixel-level uncertainty connects multiple cascaded stages of depth inference.

[0064] Specifically, Figure 1 The diagram illustrates the steps of a three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization according to the present invention.

[0065] like Figure 1 As shown, this invention discloses a three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization, comprising the following steps:

[0066] S102, acquire a sequence of two-dimensional images of the target scene from N different perspectives, wherein one of the two-dimensional images in the sequence is used as a reference view and the remaining images are used as source views;

[0067] S104, perform a dual-path feature extraction module on the image from each viewpoint to generate a corresponding dual-domain enhanced feature map, wherein the dual-domain enhanced feature map integrates local texture edge and global frequency structure information;

[0068] S106, Perform differentiable homography transformation on the dual-domain enhanced feature maps of N viewpoints according to the preset number of depth assumption planes to generate an initial cost volume, and calibrate and aggregate the initial cost volume to obtain an enhanced cost volume;

[0069] S108, The enhanced cost body is fed into a preset network model to obtain a probability body, and then the expected depth map and pixel-level uncertainty map are calculated along the depth dimension.

[0070] S110, between the cascaded stages, the depth search range of the next stage is adaptively expanded and sampled based on the expected depth map and the pixel-level uncertainty map, and a fine depth map is output in the final stage, and a 3D point cloud of the target scene is generated based on the fine depth map.

[0071] It should be noted that, in this embodiment, the technical points of the present invention are frequency-space dual-domain enhancement, cost volume multi-dimensional attention denoising, and coarse-to-fine adaptive depth calculation. Specifically, it is applied to the visual measurement of targets with a large number of smooth curved surfaces and highly reflective shells, such as non-cooperative spacecraft and debris. In these scenarios, the present invention can overcome the recurring problems of depth holes and edge collapse, and provide high-precision and surface-consistent point cloud data for robotic arm capture and reverse engineering.

[0072] Specifically, in this embodiment, when applied, the present invention first obtains a dual-domain enhanced feature map that combines local texture details and global structural information through dual-path feature extraction using parallel spatial domain convolutional paths and frequency domain Fourier transform paths; then, through channel-spatial attention recalibration, dynamic convolutional kernel adaptive aggregation, and wavelet decomposition fusion, a multi-scale discriminative feature map is generated; subsequently, according to the depth hypothesis plane preset in the cascade stage, an initial cost volume is constructed through differentiable homography transformation and cross-view variance calculation, and the enhanced cost volume is obtained by calibrating and aggregating it using grouped channel attention; the enhanced cost volume is fed into a 3DU-Net regularized network with embedded 3D attention skip connections, outputting a depth probability volume and solving for the desired depth map and pixel-level uncertainty map; during the progression of the cascade stage, the depth search range of the next stage is adaptively stretched or shrunk based on the uncertainty map, and the depth hypothesis is resampled; after iterating to the final stage, a dense 3D point cloud of the target scene is generated by back-projection of the fine depth map combined with camera intrinsic parameters.

[0073] According to an embodiment of the present invention, the step of performing dual-path feature extraction on the image from each viewpoint to generate a corresponding dual-domain enhanced feature map specifically includes:

[0074] Perform standard convolution on the spatial path of the input viewpoint image to extract local detail features and generate spatial features;

[0075] In the frequency domain path, a fast Fourier transform is performed on the input viewpoint image to map it to the frequency domain. After adjusting the weights of each frequency component through a learnable frequency modulation function, an inverse Fourier transform is performed to reconstruct it back to the spatial domain, generating frequency features.

[0076] The spatial features and the frequency features are fused element by element to obtain a dual-domain enhanced feature map that simultaneously possesses local texture discrimination and global structural integrity.

[0077] It should be noted that, in this embodiment, for the input viewpoint image, a Fast Fourier Transform is performed on the feature representation of the same input image along the frequency domain path, mapping it from the spatial domain to the frequency domain to obtain the frequency components reflecting the global brightness distribution and structural contours of the image, such as... Figure 2 As shown, with explicit encoding and complementary global structural information to spatial domain features, for the input feature map C is the number of channels, H is the height, and W is the width. The frequency path first projects the input into a frequency-aware representation using a frequency projection operator: ,in, This represents a frequency domain projection, specifically capable of capturing the global frequency components of the feature representation. Simultaneously, local detail features are extracted along the spatial path using stacked standard convolutional layers to generate a spatial feature map. This path inherits the ability of traditional convolutional neural networks to capture local information such as texture gradients and edge directions, providing stable local matching primitives for subsequent cross-view matching and ensuring the object surface possesses the ability to discriminate texture regions. Specifically, a learnable frequency modulation function... It is applied to adaptively adjust the global structural response in the projection domain, and the modulated representation is then passed through the reconstruction operator. Reconstructing back into the spatial domain, thus generating spatial features with frequency enhancement: .

[0078] Specifically, in this embodiment, the low-frequency components in the frequency domain correspond to the smooth changes in the overall shape of the object, while the high-frequency components correspond to edge transitions and detailed textures. The present invention introduces a learnable frequency modulation function to adaptively weight and adjust the response intensity of each frequency component, enabling the network to autonomously decide whether to rely on global contour information or local high-frequency details based on the material characteristics of the current scene. After modulation, the weighted frequency domain representation is reconstructed back to the spatial domain through inverse Fourier transform to generate a frequency feature map with global structure perception capability.

[0079] Furthermore, in this embodiment, the spatial feature map output by the spatial path and the frequency feature map output by the frequency path are fused element-wise to generate the final dual-domain enhanced feature map. Specifically, the fusion process enables the two complementary feature information to be mutually calibrated in a unified representation space: in texture-rich regions, spatial features dominate, providing accurate pixel-level matching basis; in areas with weak texture or smooth surfaces with high reflectivity, frequency features compensate for the limitation of the local convolutional receptive field, and provide reliable contextual clues for matching by carrying global low-frequency structural information. This dual-domain collaborative working mechanism described in this invention fundamentally solves the problem of matching ambiguity and depth inference failure caused by the lack of local texture when traditional single spatial domain feature extraction faces typical non-cooperative space target surfaces such as satellite heat shields and metal shells, providing a more robust and discriminative feature foundation for subsequent cost volume construction and depth estimation.

[0080] According to an embodiment of the present invention, the method further includes: performing feature recalibration on the dual-domain enhanced feature map and adaptively aggregating the multi-receptive-field convolutional response, decomposing the multi-receptive-field convolutional response into low-frequency approximation components, horizontal detail components and vertical detail components and then fusing them to generate an enhanced multi-scale discriminative feature map, wherein the multi-scale discriminative feature map is used to construct the cost body.

[0081] It should be noted that, in this embodiment, in the second stage of the feature extraction network, in order to explicitly capture both coarse structural information and fine edge details, specifically, as shown in... Figure 3 The diagram shows a schematic of the wavelet-enhanced feature pyramid architecture, which uses a two-dimensional discrete wavelet transform to decompose the input into three parts:

[0082] (1) Low-frequency approximation is obtained by 2×2 average pooling. ;

[0083] (2) Horizontal details are obtained by pooling along the absolute difference in height. ;

[0084] (3) Pooling is performed along the absolute difference in width to obtain vertical details. Each part is processed by a separate convolutional layer:

[0085]

[0086] in, This represents a learnable 1×1 convolutional transformation. Subsequently, each feature is scaled up to its original spatial size and concatenated. Finally, the concatenated features are fused through 1×1 convolution, batch normalization, and ReLU to generate the output features. By explicitly modeling and aggregating multi-frequency components, this invention enhances both the global structure and fine details in high-level features, which is beneficial for subsequent cost volume construction and depth estimation.

[0087] According to an embodiment of the present invention, the step of performing a differentiable homography transformation on the dual-domain enhanced feature maps of N viewpoints according to a preset number of depth assumption planes to generate an initial cost volume, and calibrating and aggregating the initial cost volume to obtain an enhanced cost volume, specifically includes:

[0088] Receive the multi-scale discriminative feature maps generated by each view at multiple scales, calculate the variance of all the dual-domain enhanced feature maps under each depth assumption, and stack all the variances to form an initial cost body;

[0089] The initial cost volume is grouped along the channel dimension, and a lightweight 3D convolution and sigmoid activation are applied to each cost volume to obtain the grouped attention weights.

[0090] The weights are multiplied element-wise with the corresponding grouped cost bodies to generate the enhanced cost body.

[0091] It should be noted that, in this embodiment, after the multi-view feature extraction is completed, each view has generated a corresponding multi-scale discriminative feature map. In this invention, the camera coordinate system of the reference view is used as a reference, and under the preset number of depth assumption planes and depth interval in the current cascading stage, the multi-scale discriminative feature maps of all source views are mapped one by one to the depth assumption planes of the reference view through differentiable homography transformation.

[0092] Specifically, in this embodiment, for each depth hypothesis plane, the pixel-wise variance of the feature maps after mapping all views on that plane is calculated, and the variances of each plane are stacked along the depth dimension to form the initial cost volume. Variance serves as a concise and efficient cross-view value. Figure 1 Consistency measures can reflect the degree of matching and agreement of multi-view features at the corresponding depth hypothesis with minimal computational overhead—the smaller the variance, the more consistent the features of each view at that depth are, and the higher the probability that the depth hypothesis is the true depth.

[0093] Furthermore, in this embodiment, based on this, the present invention further introduces efficient grouped cost volume aggregation to calibrate the initial cost volume. The initial cost volume is uniformly divided into several groups along the channel dimension. A lightweight 3D convolution operation is independently applied to each group. Local depth-spatial context features within the group are extracted using learnable convolution kernels. Subsequently, a sigmoid activation function is used to generate the group's attention weights. These weights reflect the importance of each channel group in the cost volume to the depth inference task. Then, the attention weights of each group are multiplied element-wise with the original cost volume features within the corresponding group to complete the adaptive recalibration of the cost volume. This grouping strategy achieves fine calibration of the channel dimension while maintaining extremely low parameter counts: high weights are applied to channel groups containing effective matching signals to enhance their expressive power, while low weights are applied to channel groups containing noise or redundant information to suppress their interference. The resulting enhanced cost volume retains key cross-view geometric consistency information while effectively filtering out matching noise introduced by factors such as weak textures and high reflectivity, providing a higher-quality input foundation for subsequent 3D attention regularization and depth inference.

[0094] Specifically, in this embodiment, as Figure 4 As shown, given the feature map for each view, the variance between different views is first calculated to measure the consistency of the features: ,in, Indicates the first The feature volume of each view For the number of views, To calculate the aggregated variance volume, this invention employs a lightweight grouped channel attention mechanism. ,in, 3D convolution attention operators representing grouping. It uses the Sigmoid function, and the final aggregation cost volume is calculated as follows: ,in, Representing element-wise multiplication, it improves the discriminative power of cost volume by highlighting information transmission channels and maintaining low parameter costs, while preserving the high-frequency details necessary for accurate depth estimation.

[0095] According to an embodiment of the present invention, the step of feeding the enhanced cost volume into a preset network model to obtain a probability volume, and then calculating the desired depth map and pixel-level uncertainty map along the depth dimension, specifically includes:

[0096] The preset network model includes an encoder, a decoder, and a 3DU-Net architecture network model with skip connections;

[0097] The enhanced cost body is input into the 3DU-Net architecture network model, wherein at each skip connection, a set of three-dimensional attention operators that simultaneously consider channel, spatial and depth dimensions are inserted to filter out mismatch noise during the regularization process, and the regularized cost body is output.

[0098] A Softmax operation is performed on the regularized cost volume along the depth dimension to obtain a depth probability volume reflecting the probability of each pixel at different depths. The expected depth map for guiding the next level of sampling at the current stage is calculated by probability weighted averaging, as well as an uncertainty map that measures the reliability of the estimate.

[0099] It should be noted that, in this embodiment, the process of feeding the enhanced cost volume into a preset network model to obtain the probability volume, and then calculating the expected depth map and pixel-level uncertainty map along the depth dimension is as follows: The preset network model adopts a 3D network including an encoder, decoder, and skip connections. The U-Net architecture enhances the cost volume by first extracting multi-scale depth-spatial geometric features through layer-by-layer 3D convolution and downsampling in the encoding path. Then, in the decoding path, spatial resolution is gradually restored through upsampling. At each skip connection position, an efficient 3D attention operator is inserted. This operator performs joint attention modeling on the channel, spatial, and depth dimensions of the cost volume, enabling the network to adaptively enhance the response weights of regions with high geometric consistency while suppressing mismatch noise and abnormal fluctuations in the depth direction introduced by weak textures or highly reflective regions. The final output is a regularized cost volume with clear geometric structure and sufficient noise filtering. A Softmax operation is performed on the regularized cost volume along the depth dimension, transforming the matching cost value on each depth hypothesis plane into a depth probability volume. The expected depth map for the current stage is then calculated by probability weighted averaging. At the same time, a pixel-level uncertainty map is obtained by quantizing the variance or entropy of the probability distribution. These two are used as the depth estimation output for the current stage and the prior driving signal for adaptive depth sampling in the next stage, respectively.

[0100] Furthermore, in this embodiment, the present invention introduces a 3D channel attention mechanism at the skip connections of the 3D regularized 3DU-Net, assigning higher weights to feature layers with strong depth continuity, thereby enhancing the ability of 3D convolution to smoothly fit complex geometric structures with minimal computational overhead.

[0101] According to an embodiment of the present invention, the step of adaptively expanding and sampling the depth search range of the next stage based on the expected depth map and the pixel-level uncertainty map between cascaded stages, and outputting a fine depth map in the final stage, and generating a 3D point cloud of the target scene based on the fine depth map, specifically includes:

[0102] During the transition between adjacent cascaded stages, obtain the expected depth map and uncertainty map of the current stage;

[0103] Construct an initial depth interval based on the standard interval of the current depth and the number of assumptions;

[0104] Based on the uncertainty of each pixel, a scaling factor is applied proportionally to pixels with high uncertainty to dynamically stretch the depth search range;

[0105] The depth hypothesis plane of uniformly resampled sub-pixels within the adaptively determined upper and lower boundaries is used as the input for the refinement search in the next stage.

[0106] The process iterates until the final stage outputs the final detailed depth map, and combines the camera intrinsics to back-project the pixels of the reference view into the three-dimensional space to generate the three-dimensional point cloud of the target scene.

[0107] It should be noted that, in this embodiment, the preset network model adopts a 3DU-Net architecture network model with built-in three-dimensional attention skip connections. The enhanced cost volume generated in the previous stage is fed into this network. In the encoder part, the spatial resolution of the cost volume is gradually compressed and the receptive field is expanded through layer-by-layer 3D convolution and downsampling operations, extracting multi-scale depth-spatial geometric features layer by layer from local to global. In the decoder part, the spatial resolution is gradually restored through upsampling and skip connections, fusing shallow detail features and deep semantic features in the encoding path. Unlike conventional 3DU-Net, which only performs simple feature splicing at skip connections, this invention inserts a set of efficient three-dimensional attention operators at each skip connection. These operators simultaneously perform joint attention modeling on the channel dimension, spatial dimension, and depth dimension of the cost volume features: in the channel dimension, the contribution weight of different feature channels to the geometric consistency judgment is learned; in the spatial dimension, significant geometric regions such as object boundaries and corners are located; and in the depth dimension, feature layers with continuous consistency in the depth direction are identified and given higher responses.

[0108] Furthermore, in this embodiment, through the collaborative attention mechanism of these three dimensions, the network can adaptively focus on feature regions with high geometric confidence during the regularization process, while effectively suppressing mismatch noise and abnormal fluctuations introduced by weak texture surfaces, highly reflective materials or cross-view occlusion. After the entire encoding-decoding process, the network outputs a regularization cost body with fully filtered noise and clear geometric structure.

[0109] Specifically, in this embodiment, after obtaining the regularized cost volume, the present invention performs a Softmax normalization operation along its depth dimension, transforming the matching cost value on each depth hypothesis plane into a depth probability volume representing the probability distribution. For each pixel in the reference view, the higher its probability value on a certain depth hypothesis, the greater the likelihood that the depth is the true depth. Based on this probability volume, the expected depth map for the current stage is calculated using a probability-weighted average method—that is, a weighted summation is performed with each depth hypothesis value as a variable and the corresponding probability value as a weight, to obtain the expected depth estimate for each pixel. Simultaneously, a pixel-level uncertainty map is calculated using the depth probability volume. Specifically, uncertainty can be measured by the variance or information entropy of the depth probability distribution: if the probability distribution of a pixel is highly concentrated near a certain depth hypothesis, its uncertainty is low and the estimation result is highly reliable; if the probability distribution is relatively dispersed, it indicates that multiple depth hypotheses have a certain probability, resulting in high uncertainty and low reliability of the estimation result. The expected depth map serves as the depth estimate output for the current stage, guiding subsequent point cloud generation; while the uncertainty map acts as prior input for the next stage, driving the adaptive depth range sampling module to dynamically adjust the depth search interval at the pixel level. This integrated output method of depth estimation and confidence assessment allows subsequent processing to allocate computational resources strategically. It maintains a compact search range in geometrically simple and easily estimated regions to save computation, while expanding the search range in geometrically complex and difficult-to-estimate regions to ensure that correct depths are not overlooked.

[0110] It is worth mentioning that, in order to improve the efficiency and robustness of generating cascaded multi-view depth assumptions, this invention also proposes an adaptive depth range sampling strategy, such as... Figure 5 Given the depth estimate from the previous stage and the corresponding pixel-level uncertainty The adaptive depth range sampling strategy constructs an initial depth range around each pixel based on the standard depth interval Δd and the number of assumptions N. Then, the interval is adaptively expanded according to the uncertainty: the higher the uncertainty, the larger the expansion factor. This results in a wider search scope, with the upper and lower limits being updated sequentially:

[0111] ;

[0112] Finally, the depth assumption value for each pixel will be... Uniform sampling is performed within: The adaptive depth range sampling strategy of the present invention enables high-uncertainty pixels to explore a wider depth range while keeping low-uncertainty pixels compact, thereby improving overall accuracy and efficiency without introducing additional learnable parameters.

[0113] In one embodiment of the invention, the network model of the present invention is trained specifically within the PyTorch framework, using the Adam optimizer with parameters set as follows: , Learning rate strategy: The initial learning rate is set to 0.001, and a multi-step decay strategy is adopted, halving the learning rate at the 10th, 12th and 14th epochs. The total number of training epochs is 16 epochs. The number of depth hypothesis planes in the three cascaded stages are set to 48, 32 and 8 respectively, with corresponding depth intervals of 4.0, 1.0 and 0.25 times the baseline depth interval. This setting allows the depth search range to gradually transition from broad coverage to finer detail. The batch size is set to 2 to ensure the stability of the training gradient.

[0114] In one embodiment of the present invention, two of the most representative public datasets are selected for a two-layer verification of "basic performance - generalization ability". In this embodiment, the DTU dataset is used for specific application. The DTU dataset is one of the most widely used standard benchmark datasets in the field of multi-view stereo reconstruction. It was constructed by the Computer Vision Laboratory of the Technical University of Denmark and is specifically used to evaluate multi-view stereo reconstruction. Figure 3 The geometric recovery capability of the dimensional reconstruction algorithm in a controlled environment is therefore widely used as a benchmark dataset for model training and performance evaluation in academic research. Therefore, the specific dataset collection will not be described in detail in this embodiment. It should be noted that the dataset collection process was completed in a highly controlled laboratory environment.

[0115] Furthermore, in this embodiment, the DTU dataset contains 124 different indoor scenes, which include various types of objects, such as model buildings, sculptures, industrial parts, and objects with complex textures or reflective properties. These scenes vary considerably in terms of geometry, surface texture, and material reflective properties, thus providing a challenging testing environment for multi-view stereoscopic algorithms.

[0116] In the experimental setup, this invention follows the data partitioning method widely used in current MVS research, selecting 79 scenarios from 124 scenarios for model training, 18 scenarios for validation, and 22 scenarios for testing and evaluation. Through experiments on the DTU dataset, the basic geometric reconstruction capability, depth estimation accuracy, and multi-view information fusion capability of the model in a controlled environment can be effectively evaluated.

[0117] The evaluation metrics were calculated using the official MATLAB evaluation script, which includes the following three core metrics:

[0118] (1) Accuracy: The average distance from the reconstructed point cloud to the true point cloud reflects the accuracy of the point cloud.

[0119] (2) Completeness: The average distance from the ground truth point cloud to the reconstructed point cloud, reflecting the coverage of the point cloud.

[0120] (3) Overall error: the average of accuracy and completeness. The lower the value, the better the reconstruction quality.

[0121] To comprehensively verify the effectiveness of each component, the ablation experiment on the DTU dataset was divided into two parts:

[0122] (1) Enhancement in feature extraction networks.

[0123] (2) Cost volume aggregation and depth estimation strategy.

[0124] This invention gradually introduces the proposed frequency-space dual-domain features, channel spatial attention, dynamic kernel aggregation, and wavelet pyramid modules to examine their individual contributions and their combined contributions. The quantitative results are summarized in Tables 1 and 2, where lower values ​​are better.

[0125] Table 1. Ablation Study Results of Feature Extraction Module .

[0126] In Table 1, when only the frequency-space dual-domain feature module is used, the model's accuracy is 0.365, completeness is 0.359, and overall error is 0.352. This result indicates that introducing frequency domain information during feature extraction can effectively improve the model's basic performance. Further adding channel-space attention reduces the completeness from 0.359 to 0.352, demonstrating that the attention mechanism can effectively recalibrate the feature response, thereby enhancing the model's feature representation ability in areas with weak texture. Subsequently, introducing dynamic kernel aggregation further optimizes the model's accuracy to 0.362, while maintaining the completeness at 0.353, resulting in more balanced overall performance. This indicates that multi-scale dynamic kernel aggregation enhances the adaptability of feature representation through adaptive receptive field while maintaining the stability of geometric reconstruction. Finally, introducing a wavelet pyramid module further reduces the completeness to 0.349, and the overall error reaches 0.357, indicating that the multi-scale feature enhancement strategy can further improve the completeness of the reconstruction results.

[0127] Furthermore, the contributions of grouped cost volume, 3D attention, and depth sampling were evaluated in the experiments. The results are summarized in Table 2. When only the grouped cost volume was used, the model's accuracy was 0.366, completeness was 0.354, and the overall error was 0.360, indicating that it could capture consistency information between multiple views to a certain extent, thereby improving the reconstruction effect. After introducing 3D attention, the model's accuracy and completeness were 0.373 and 0.361, respectively, with an overall error of 0.367. This result shows that by jointly modeling and aggregating features in the channel, spatial, and depth dimensions, 3D attention can effectively alleviate the feature mismatch problem and improve feature representation ability. Further adding depth sampling further optimized the model performance, with accuracy at 0.366, completeness reduced to 0.352, and overall error at 0.359. This indicates that the adaptive depth-aware refinement mechanism can further improve the completeness of the reconstruction results, thereby improving the overall reconstruction performance.

[0128] Table 2. Ablation Study Results of Aggregation and Depth Estimation Module .

[0129] Figure 6 A block diagram of a three-dimensional reconstruction system based on frequency-space dual-domain features and cascade optimization according to the present invention is shown.

[0130] like Figure 6 As shown, this invention discloses a three-dimensional reconstruction system based on frequency-space dual-domain features and cascaded optimization, including a memory and a processor. The memory includes a three-dimensional reconstruction method program based on frequency-space dual-domain features and cascaded optimization. When the processor executes the three-dimensional reconstruction method program based on frequency-space dual-domain features and cascaded optimization, it implements the following steps:

[0131] Acquire a sequence of two-dimensional images of the target scene from N different perspectives, wherein one of the two-dimensional images in the sequence is used as a reference view and the remaining images are used as source views;

[0132] A dual-path feature extraction module is used to extract images from each viewpoint to generate corresponding dual-domain enhanced feature maps, wherein the dual-domain enhanced feature maps fuse local texture edge and global frequency structure information;

[0133] The dual-domain enhanced feature maps from N perspectives are subjected to differentiable homography transformation according to the preset number of depth assumption planes to generate an initial cost volume, and the initial cost volume is calibrated and aggregated to obtain the enhanced cost volume.

[0134] The enhanced cost volume is fed into a preset network model to obtain a probability volume, and then the expected depth map and pixel-level uncertainty map are calculated along the depth dimension.

[0135] Between cascaded stages, the depth search range for the next stage is adaptively expanded and sampled based on the desired depth map and the pixel-level uncertainty map, and a fine depth map is output in the final stage, based on which a 3D point cloud of the target scene is generated.

[0136] It should be noted that when the three-dimensional reconstruction system based on frequency-space dual-domain features and cascade optimization disclosed in this invention is applied, the specific process corresponds to the three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization described in the above embodiments. Since the specific implementation details of the system application are consistent with the content of the three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization described above, no further details will be provided in this embodiment.

[0137] A third aspect of the present invention provides a computer-readable storage medium comprising a three-dimensional reconstruction method program based on frequency-space dual-domain features and cascaded optimization. When the three-dimensional reconstruction method program based on frequency-space dual-domain features and cascaded optimization is executed by a processor, it implements the steps of the three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization as described in any of the preceding claims.

[0138] The fourth aspect of the present invention provides a computer program product comprising: computer program code, which, when run on a computer, causes the computer to execute any of the methods described in the embodiments of the three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization.

[0139] The fifth aspect of the present invention provides an electronic device, the electronic device comprising: a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the steps of a three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization as described in any of the preceding claims.

[0140] The terms “component,” “module,” “system,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0141] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0142] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0143] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0146] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0147] This invention discloses a three-dimensional reconstruction method and system based on frequency-space dual-domain features and cascaded optimization. By using frequency-space joint features, it effectively resists the degradation of weak textures in the scene and eliminates matching ambiguity through grouped cost volumes and three-dimensional attention. This enables the perception of the geometry of complex targets with high integrity, providing unprecedented data processing capabilities for automated spatial operations and filling the gaps in the existing technology for the complete geometric recovery of non-cooperative targets.

[0148] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization, characterized in that, Includes the following steps: Acquire a sequence of two-dimensional images of the target scene from N different perspectives, wherein one of the two-dimensional images in the sequence is used as a reference view and the remaining images are used as source views; A dual-path feature extraction module is used for each viewpoint image to generate a corresponding dual-domain enhanced feature map. Specifically, this includes: performing standard convolution on the input viewpoint image in the spatial path to extract local detail features and generate spatial features; performing a fast Fourier transform on the input viewpoint image in the frequency domain to map it to the frequency domain, adjusting the weights of each frequency component through a learnable frequency modulation function, and then performing an inverse Fourier transform to reconstruct it back to the spatial domain to generate frequency features; fusing the spatial features and the frequency features element-wise to obtain a dual-domain enhanced feature map that simultaneously possesses local texture discrimination and global structural integrity; wherein, the dual-domain enhanced feature map integrates local texture edge and global frequency structure information. The dual-domain enhanced feature map is recalibrated and adaptively aggregated with multi-receptive-field convolutional responses. The multi-receptive-field convolutional responses are decomposed into low-frequency approximation components, horizontal detail components, and vertical detail components and then fused to generate an enhanced multi-scale discriminative feature map, wherein the multi-scale discriminative feature map is used to construct the cost volume. The dual-domain enhanced feature maps from N perspectives are subjected to differentiable homography transformation according to the preset number of depth assumption planes to generate an initial cost volume, and the initial cost volume is calibrated and aggregated to obtain the enhanced cost volume. The enhanced cost volume is fed into a preset network model to obtain a probability volume, and then the expected depth map and pixel-level uncertainty map are calculated along the depth dimension. Specifically, the preset network model includes an encoder, a decoder, and a 3DU-Net architecture network model with skip connections; the enhanced cost volume is input into the 3DU-Net architecture network model, wherein at each skip connection, a set of three-dimensional attention operators that simultaneously consider the channel, spatial, and depth dimensions are inserted to filter out mismatch noise during the regularization process, and a regularized cost volume is output; a Softmax operation is performed on the regularized cost volume along the depth dimension to obtain a depth probability volume reflecting the probability of each pixel at different depths, and the expected depth map used to guide the next level of sampling and the uncertainty map measuring the reliability of the estimation are calculated by probability weighted averaging. Between cascaded stages, the depth search range for the next stage is adaptively expanded and sampled based on the desired depth map and the pixel-level uncertainty map, and a fine depth map is output in the final stage, based on which a 3D point cloud of the target scene is generated.

2. The three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization according to claim 1, characterized in that, The process of generating an initial cost volume by performing a differentiable homography transformation on dual-domain enhanced feature maps from N viewpoints according to a preset number of depth assumption planes, and then calibrating and aggregating the initial cost volume to obtain the enhanced cost volume, specifically includes: Receive the multi-scale discriminative feature maps generated by each view at multiple scales, calculate the variance of all the dual-domain enhanced feature maps under each depth assumption, and stack all the variances to form an initial cost body; The initial cost volume is grouped along the channel dimension, and a lightweight 3D convolution and sigmoid activation are applied to each cost volume to obtain the grouped attention weights. The weights are multiplied element-wise with the corresponding grouped cost bodies to generate the enhanced cost body.

3. The three-dimensional reconstruction method based on frequency-space dual-domain features and cascaded optimization according to claim 2, characterized in that, Between the cascaded stages, the depth search range for the next stage is adaptively expanded and sampled based on the desired depth map and the pixel-level uncertainty map, and a fine depth map is output in the final stage. A 3D point cloud of the target scene is generated based on the fine depth map, specifically including: During the transition between adjacent cascaded stages, obtain the expected depth map and uncertainty map of the current stage; Construct an initial depth interval based on the standard interval of the current depth and the number of assumptions; Based on the uncertainty of each pixel, a scaling factor is applied proportionally to pixels with high uncertainty to dynamically stretch the depth search range; The depth hypothesis plane of uniformly resampled sub-pixels within the adaptively determined upper and lower boundaries is used as the input for the refinement search in the next stage. The process iterates until the final stage outputs the final detailed depth map, and combines the camera intrinsics to back-project the pixels of the reference view into the three-dimensional space to generate the three-dimensional point cloud of the target scene.

4. A three-dimensional reconstruction system based on frequency-space dual-domain features and cascaded optimization, characterized in that, The system includes a memory and a processor. The memory contains a program for a 3D reconstruction method based on frequency-space dual-domain features and cascaded optimization. When the processor executes the program for the 3D reconstruction method based on frequency-space dual-domain features and cascaded optimization, it performs the following steps: Acquire a sequence of two-dimensional images of the target scene from N different perspectives, wherein one of the two-dimensional images in the sequence is used as a reference view and the remaining images are used as source views; A dual-path feature extraction module is used for each viewpoint image to generate a corresponding dual-domain enhanced feature map. Specifically, this includes: performing standard convolution on the input viewpoint image in the spatial path to extract local detail features and generate spatial features; performing a fast Fourier transform on the input viewpoint image in the frequency domain to map it to the frequency domain, adjusting the weights of each frequency component through a learnable frequency modulation function, and then performing an inverse Fourier transform to reconstruct it back to the spatial domain to generate frequency features; fusing the spatial features and the frequency features element-wise to obtain a dual-domain enhanced feature map that simultaneously possesses local texture discrimination and global structural integrity; wherein, the dual-domain enhanced feature map integrates local texture edge and global frequency structure information. The dual-domain enhanced feature map is recalibrated and adaptively aggregated with multi-receptive-field convolutional responses. The multi-receptive-field convolutional responses are decomposed into low-frequency approximation components, horizontal detail components, and vertical detail components and then fused to generate an enhanced multi-scale discriminative feature map, wherein the multi-scale discriminative feature map is used to construct the cost volume. The dual-domain enhanced feature maps from N perspectives are subjected to differentiable homography transformation according to the preset number of depth assumption planes to generate an initial cost volume, and the initial cost volume is calibrated and aggregated to obtain the enhanced cost volume. The enhanced cost volume is fed into a preset network model to obtain a probability volume, and then the expected depth map and pixel-level uncertainty map are calculated along the depth dimension. Specifically, the preset network model includes an encoder, a decoder, and a 3DU-Net architecture network model with skip connections; the enhanced cost volume is input into the 3DU-Net architecture network model, wherein at each skip connection, a set of three-dimensional attention operators that simultaneously consider the channel, spatial, and depth dimensions are inserted to filter out mismatch noise during the regularization process, and a regularized cost volume is output; a Softmax operation is performed on the regularized cost volume along the depth dimension to obtain a depth probability volume reflecting the probability of each pixel at different depths, and the expected depth map used to guide the next level of sampling and the uncertainty map measuring the reliability of the estimation are calculated by probability weighted averaging. Between cascaded stages, the depth search range for the next stage is adaptively expanded and sampled based on the desired depth map and the pixel-level uncertainty map, and a fine depth map is output in the final stage, based on which a 3D point cloud of the target scene is generated.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a three-dimensional reconstruction method program based on frequency-space dual-domain features and cascade optimization. When the three-dimensional reconstruction method program based on frequency-space dual-domain features and cascade optimization is executed by a processor, it implements the steps of the three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization as described in any one of claims 1 to 3.

6. A computer program product, characterized in that, The computer program product includes computer program code, which, when run on a computer, enables the computer to implement the steps of a three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization as described in any one of claims 1 to 3.

7. An electronic device, characterized in that, The electronic device includes: A processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the steps of a three-dimensional reconstruction method based on frequency-space dual-domain features and cascade optimization as described in any one of claims 1 to 3.

Images