Multi-scale disparity computation module, method, device and medium based on haar wavelet transform

Abstract

The application discloses a multi-scale disparity calculation module and method based on Haar wavelet transform, equipment and medium, the application first constructs a feature pyramid by using Haar wavelet transform, extracts high-frequency and low-frequency details, and then adopts a coarse-to-fine strategy to perform disparity calculation. The application can effectively solve the problem that random initialization converges slowly or falls into local optimization in a weak texture area, and meanwhile, wavelet detail components can provide more abundant texture and edge information for cost calculation. The application has the advantages of high precision, strong robustness and high speed in dealing with the disparity calculation task of a weak texture, non-texture or repeated texture scene.

Description

Technical Field

[0001] This invention relates to the field of computer vision and 3D reconstruction technology, and in particular to a multi-scale disparity calculation module, method, device and medium based on Haar wavelet transform. Background Technology

[0002] Multi-view stereo matching (MVS) is a core technology for recovering high-precision 3D geometric models from a set of images with known camera parameters. Among them, PatchMatch-based algorithms, such as Gipuma, have attracted much attention due to their efficiency and accuracy. These algorithms estimate the disparity plane for each pixel by randomly initializing disparity and normal vector maps and iteratively propagating optimization.

[0003] However, the existing technology has the following technical problems:

[0004] 1. Blindness of initialization: Random initialization strategy is inefficient, and the convergence speed depends on the law of large numbers. In scenarios with a large depth range, a large number of iterations are required, resulting in a large amount of computation.

[0005] 2. Limited information utilization: Traditional cost functions are usually based on the color (RGB) and gradient (L2 norm) information of the original image, which are sensitive to changes in illumination and have difficulty providing sufficient matching constraints in areas lacking texture information, resulting in slow convergence and low computational efficiency.

[0006] 3. Computation time issue: Although highly parallel computing has been achieved on GPUs, performing a large number of iterative calculations directly at the original scale is still relatively time-consuming when processing high-resolution images.

[0007] Therefore, there is an urgent need in this field for a disparity calculation method that can provide better initialization, utilize richer image features, and converge efficiently. Summary of the Invention

[0008] To overcome the technical problems of high computational cost, poor computational efficiency, and long computation time in existing MVS PatchMatch-type algorithms, this invention provides a multi-scale disparity calculation module, method, device, and medium based on Haar wavelet transform. By introducing multi-scale analysis and wavelet transform features, the quality of disparity optimization is effectively improved, and the ability to discriminate matching costs is enhanced, thereby significantly improving the accuracy, completeness, and efficiency of 3D reconstruction.

[0009] To achieve the above-mentioned technical objectives, the technical solution of the present invention is as follows:

[0010] A multi-scale disparity calculation module based on Haar wavelet transform includes:

[0011] The wavelet pyramid construction unit is used to perform multi-level Haar wavelet decomposition on the input multi-view image, thereby generating multiple high- and low-frequency wavelet component images of each image in the multi-view image at different resolutions, i.e., different scales; the multi-view image includes two images of the same object taken from two different angles.

[0012] The coarse-to-fine optimization unit is used to build and optimize the disparity map and surface normal vector map scale-wise based on the high- and low-frequency wavelet component images generated by the wavelet pyramid construction unit, in ascending order of resolution from lowest to highest. During optimization, firstly, two initial blank images with the same resolution are built based on the lowest resolution (coarsest scale) of the high- and low-frequency wavelet component images. Each pixel in these images corresponds to a pixel in the high- and low-frequency wavelet component images. Then, disparity and normal vector values ​​are randomly initialized for each pixel in the two initial blank images within a preset range to construct the initial disparity map and initial surface normal vector map, thus establishing the initial disparity plane for each pixel. Iterative propagation is then performed to optimize the current scale, and aggregation is calculated during the iterative propagation process. When calculating the matching cost, the average absolute difference of each component within a preset-size window of the multi-view image at this scale is calculated using all high- and low-frequency wavelet component images. Weights are assigned to each component, and the summation is used as the aggregation matching cost for that pixel. After optimization at this scale, the optimized disparity map and surface normal map are upsampled, and disparity and normal values ​​are randomly added or subtracted for all pixels within a preset range to add random perturbation. This perturbation is then used as the initial disparity map and initial surface normal map at a higher scale. The iteration propagation is repeated at higher scales, and during this process, the aggregation matching cost of pixels is calculated in the same way based on the high- and low-frequency wavelet component images at the corresponding scale to achieve optimization, until optimization is completed at the highest resolution.

[0013] Furthermore, the high and low frequency wavelet component images generated by the wavelet pyramid construction unit include a set of images of approximate components, horizontal detail components, vertical detail components, and diagonal detail components generated by each image at different resolutions.

[0014] Furthermore, in the coarse-to-fine optimization unit, the expression for calculating the aggregation matching cost is:

[0015] .

[0016] Where α, β, γ, δ are the approximate component weight coefficient, the horizontal detail component weight coefficient, the vertical detail component weight coefficient, and the diagonal detail component weight coefficient, respectively, and α + β + γ + δ = 1; , , , π represents the sum of the average absolute differences of the wavelet approximation components, horizontal detail components, vertical detail components, and diagonal detail components between the corresponding high- and low-frequency wavelet component images of two multi-view images within a neighborhood window N centered at pixel p; π represents the disparity plane at point p.

[0017] Furthermore, the expression for calculating the sum of the mean absolute differences of the wavelet approximation components, horizontal detail components, vertical detail components, and diagonal detail components is as follows:

[0018] .

[0019] in , representing different components, q and p represent corresponding points in the two images of the multi-view image, namely the reference image and the target image; This represents the wavelet component value at point q in the reference image at scale l. V represents the wavelet component value of the corresponding point on the target image i after projection onto the parallax plane π; V is the set of target images pre-selected by the 3D reconstruction algorithm.

[0020] This invention also provides a multi-scale disparity calculation method based on Haar wavelet transform, comprising the following steps:

[0021] Step 1: Perform multi-level Haar wavelet decomposition on the input multi-view image to generate multiple high- and low-frequency wavelet component images of each image in the multi-view image at different resolutions, i.e., different scales; the multi-view image includes two images of the same object taken from two different angles.

[0022] Step 2: Based on the high- and low-frequency wavelet component images generated in Step 1, establish and optimize the disparity map and surface normal vector map scale by scale, from the lowest to the highest resolution. During optimization, firstly, based on the lowest resolution (coarsest scale) in the high- and low-frequency wavelet component images, establish two initial blank images with the same resolution, where each pixel corresponds to a pixel in the high- and low-frequency wavelet component images. Then, randomly initialize the disparity value and normal vector value for each pixel in the two initial blank images within a preset range to construct the initial disparity map and initial surface normal vector map, thereby establishing the initial disparity plane for each pixel. Then, perform iterative propagation to optimize the current scale, and calculate the aggregation matching cost during the iterative propagation process. Using all high- and low-frequency wavelet component images of the multi-view image at this scale, the mean absolute difference of each component within a preset window size is calculated, and weights are assigned to each component before summing them to serve as the aggregation matching cost for that pixel. After optimization at this scale, the optimized disparity map and surface normal map are upsampled, and disparity and normal values ​​are randomly added or subtracted for all pixels within a preset range to add random perturbation. This perturbation is then used as the initial disparity map and initial surface normal map at a higher scale. The iteration propagation is then repeated at higher scales, and during the iterative propagation process, the aggregation matching cost of the pixel is calculated in the same way based on the high- and low-frequency wavelet component images at the corresponding scale to achieve optimization, until optimization is completed at the highest resolution.

[0023] Furthermore, in step 1, the generated high and low frequency wavelet component images include a set of images of approximate components, horizontal detail components, vertical detail components, and diagonal detail components generated for each image at different resolutions.

[0024] Furthermore, in step 2, the expression for calculating the aggregation matching cost is:

[0025] .

[0026] Where α, β, γ, δ are the approximate component weight coefficient, the horizontal detail component weight coefficient, the vertical detail component weight coefficient, and the diagonal detail component weight coefficient, respectively, and α + β + γ + δ = 1; , , , π represents the sum of the average absolute differences of the wavelet approximation components, horizontal detail components, vertical detail components, and diagonal detail components between the corresponding high- and low-frequency wavelet component images of two multi-view images within a neighborhood window N centered at pixel p; π represents the disparity plane at point p.

[0027] Furthermore, in step 2, the expression for calculating the sum of the mean absolute differences of the wavelet approximation components, horizontal detail components, vertical detail components, and diagonal detail components is as follows:

[0028] .

[0029] in , representing different components, q and p represent corresponding points in the two images of the multi-view image, namely the reference image and the target image; This represents the wavelet component value at point q in the reference image at scale l. V represents the wavelet component value of the corresponding point on the target image i after projection onto the parallax plane π; V is the set of target images pre-selected by the 3D reconstruction algorithm.

[0030] The present invention also provides an electronic device, comprising:

[0031] One or more processors;

[0032] Storage device for storing one or more programs;

[0033] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0034] The present invention also provides a computer-readable medium storing a computer program that, when executed by a processor, implements the aforementioned method.

[0035] The technical advantage of this invention lies in that, compared with the prior art, this invention has the following advantages:

[0036] 1. Higher reconstruction accuracy and completeness: The coarse-to-fine strategy provides high-quality initial values ​​for fine-scale optimization, especially in weak texture areas, effectively avoiding the optimization process from getting trapped in local optima and reducing reconstruction holes and noise.

[0037] 2. Enhanced robustness: By utilizing the multi-component features (approximate, horizontal, vertical, and diagonal) of wavelet decomposition to jointly calculate the matching cost, compared to using only color information, it is less sensitive to factors such as illumination changes and noise, and has stronger matching and discrimination capabilities.

[0038] 3. Superior edge preservation properties: The horizontal and vertical detail components explicitly enhance the matching constraints at the image edges, making the reconstructed 3D geometric boundaries clearer and more accurate.

[0039] 4. Potential efficiency improvements: At coarse scales, the optimization speed of iterative propagation is extremely fast due to the low image resolution. Although the computational overhead of wavelet decomposition is increased, the overall computational efficiency is still effectively improved by reducing the number of iterations required at fine scales.

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the workflow of the multi-scale disparity calculation module based on Haar wavelet transform in an embodiment of the present invention.

[0042] Figure 2 The image shows a comparison of the results of the original gipuma algorithm and the gipuma algorithm integrated with the present invention in performing disparity calculations on dinosaurs in the Middlebury dataset. (a) is a color disparity map of four views obtained by the gipuma algorithm for dinosaurs, and (b) is a color disparity map of four views obtained by the gipuma algorithm integrated with the present invention for dinosaurs.

[0043] Figure 3 The image shows a comparison of the results of the original gipuma algorithm and the gipuma algorithm integrated with the present invention in performing disparity calculations on the temple in the Middlebury dataset. (a) is a color disparity map of the four views obtained by the gipuma algorithm for the temple, and (b) is a color disparity map of the four views obtained by the gipuma algorithm integrated with the present invention for the temple. Detailed Implementation

[0044] See Figure 1 The multi-scale disparity calculation module based on Haar wavelet transform provided in this embodiment includes a wavelet pyramid construction unit. This unit performs multi-level Haar wavelet decomposition on the input multi-view image, decomposing the image at a series of progressively smaller resolutions (from fine to coarse) to separate information from different components. In this embodiment, multiple high- and low-frequency wavelet component images at different resolutions (i.e., different scales) are generated for each image in the generated multi-view image. These include approximate component, horizontal detail component, vertical detail component, and diagonal detail component images. The image sizes decrease sequentially at different scales, thus constructing a pyramid-shaped structure.

[0045] In this embodiment, the multi-view image includes two images of the same object taken from two different angles. These two images form the reference image and the target image in the existing PatchMatch algorithm for 3D reconstruction.

[0046] The coarse-to-fine optimization unit in this embodiment is used to build and optimize the disparity map and surface normal vector map scale-by-scale based on the high- and low-frequency wavelet component images generated by the wavelet pyramid construction unit, in order from the lowest resolution to the highest resolution. For example, the multi-level Haar wavelet decomposition used is divided into 3 levels, corresponding to a resolution reduction from the original resolution to 1 / 8 of the original resolution. The specific number of levels can also be adjusted during implementation according to specific needs.

[0047] Then, based on the PatchMatch algorithm, optimization is performed through iterative propagation. The specific iterative propagation process is similar to the existing PatchMatch algorithm. During the iterative propagation process, specific optimization methods such as spatial propagation, view propagation, temporal propagation, and planar optimization can be used as needed. These methods all involve the use of aggregation matching costs. This embodiment makes corresponding improvements to the specific calculation of aggregation matching costs, and the specific explanation of this part is given below.

[0048] First, based on the lowest resolution (i.e., the coarsest scale) of the high- and low-frequency wavelet component images, two initial blank images with the same resolution are created. Then, disparity and normal vector values ​​are randomly initialized for each pixel in the two initial blank images within a preset range to construct an initial disparity map and an initial surface normal vector map, thereby establishing an initial disparity plane for each pixel. The blank images created at this time correspond to the step in the PatchMatch algorithm where random disparity and normal vector values ​​are added to each pixel of the reference image during initialization. The preset range in the random initialization is also based on the range set in the existing PatchMatch algorithm. Since multi-level Haar wavelet decomposition in this embodiment will generate multiple different scales, and these scales are not the same as the resolution of the original input multi-view image, and since this embodiment starts optimization from the coarsest scale to improve the computational speed of iterative propagation at low resolution, blank images with the same resolution as the coarsest scale are directly created here, and each pixel in the blank image corresponds to a pixel in the high- and low-frequency wavelet component images, and iterative propagation is started for optimization.

[0049] When calculating the specific aggregation matching cost, based on the coarse-to-fine optimization unit provided in this embodiment, the average absolute difference of each component within a preset-size window of the multi-view image at this scale is calculated using all high- and low-frequency wavelet component images of the multi-view image at this scale. Weights are then assigned to each component, and the summation is used as the aggregation matching cost for that pixel. After completing the optimization at this scale, the optimized disparity map and surface normal map are upsampled, and disparity and normal values ​​are randomly added or subtracted for all pixels within a preset range to add random perturbation. This is then used as the initial disparity map and initial surface normal map at a higher scale. The iteration propagation is then repeated at higher scales, and during the iterative propagation process, the aggregation matching cost of the pixels is calculated in the same way based on the high- and low-frequency wavelet component images at the corresponding scale to achieve optimization, until optimization is completed at the highest resolution. The use of aggregation matching cost is based on the existing PatchMatch algorithm. That is, the aggregation matching cost is used to evaluate the quality of the current disparity plane of a pixel. Specifically, the aggregation matching cost of a pixel using the current disparity plane is calculated and compared with the aggregation matching cost of using the disparity plane of other pixels in the neighborhood during the propagation process. If the aggregation cost of using the disparity plane of other pixels in the neighborhood is smaller, then the disparity plane is updated, that is, the disparity plane of other pixels is adopted; otherwise, it remains unchanged.

[0050] Specifically, the expression for calculating the aggregation matching cost is:

[0051] .

[0052] Where α, β, γ, δ are the approximate component weight coefficient, the horizontal detail component weight coefficient, the vertical detail component weight coefficient, and the diagonal detail component weight coefficient, respectively, and α + β + γ + δ = 1; , , , π represents the sum of the average absolute differences of the wavelet approximation components, horizontal detail components, vertical detail components, and diagonal detail components between the corresponding high- and low-frequency wavelet component images of two multi-view images within a neighborhood window N centered at pixel p; π represents the disparity plane at point p.

[0053] The expression for calculating the sum of the mean absolute differences of the wavelet approximation components, horizontal detail components, vertical detail components, and diagonal detail components is as follows:

[0054] .

[0055] in , representing different components, q and p represent corresponding points in the two images of the multi-view image, namely the reference image and the target image; This represents the wavelet component value at point q in the reference image at scale l. V represents the wavelet component value of the corresponding point on the target image i after projection onto the parallax plane π; V is the set of target images pre-selected by the 3D reconstruction algorithm. The target view set here is also determined based on the existing PatchMatch algorithm. For example, the PatchMatch algorithm inputs 6 images from different angles for multi-view stereo matching. One of these images is used as a reference image. Then, one of the other five images can be arbitrarily selected as the target image to form a combination. The PatchMatch algorithm will match and score these combinations and select the top few combinations with higher scores to perform multi-view stereo matching. The corresponding images in these combinations that serve as target images are the pre-selected target image combinations.

[0056] Next, this embodiment will integrate the provided multi-scale disparity calculation method based on Haar wavelet transform into the open-source Gipuma algorithm, replacing its original disparity initialization and cost calculation steps, and provide a further overall explanation of the Gipuma algorithm. It should be noted that, depending on specific needs, this embodiment can also be integrated into other PatchMatch algorithms to replace the original disparity initialization and cost calculation content, such as PatchMatch Stereo, ACMH, etc., which can also improve the performance of the corresponding algorithms.

[0057] In this embodiment, the parameter settings are as follows: the wavelet decomposition level is set to 3 (L=3), resulting in 3 different resolutions, or 3 scales, during decomposition. Let l represent all scales, l = 0, 1, 2, 3, where 1, 2, and 3 are the 3 scales generated in the wavelet decomposition, 3 represents the coarsest scale (lowest resolution), and 0 represents the original scale (original resolution of the multi-view image). The weighting coefficients are set to α=0.5, β=0.2, γ=0.2, δ=0.1, representing the approximate component weighting coefficient, horizontal detail component weighting coefficient, vertical detail component weighting coefficient, and diagonal detail component weighting coefficient, respectively, which are the weights corresponding to the specific components given by the corresponding component images. The neighborhood window N is 9×9 pixels in size. It should be noted that all the above parameters can be adjusted according to specific circumstances to adapt to the multi-view stereo matching needs of different images.

[0058] The Gipuma algorithm, after integrating the multi-scale disparity calculation method of this embodiment, mainly includes the following steps:

[0059] S1: The cv2.waveletDecompose function of the OpenCV library is used to perform 3-level Haar wavelet decomposition on the input reference image and target image to obtain the approximate (LL) and detail (LH horizontal, HL vertical, HH diagonal) components of each level, thereby constructing a feature pyramid.

[0060] S2: Scale the original camera intrinsics to the current scale to ensure that the homography / projection corresponding to the same plane assumption is correct at this scale. Camera intrinsics include focal length, principal point, image size, and parallax range, etc.

[0061] S3: At the coarsest scale (l=3), randomly initialize the disparity plane for each pixel. The disparity plane is determined by the disparity value and the normal vector. The initialization range covers the disparity range in S2, following the original Gipuma "law of large numbers" covering strategy. Since the actual physical plane in 3D space can be represented by a single equation... To describe, by derivation, in an epipolar-corrected image pair, the disparity d corresponding to this physical plane can be expressed as: Therefore, the equation This refers to the disparity plane corresponding to each pixel. The disparity d can be converted to and from the depth z using d = b * f / z. In this embodiment, the disparity is randomly initialized, and b in the conversion expression represents the baselines of the two views, while f represents the camera focal length.

[0062] S4: Execute the PatchMatch loop of Gipuma at the coarsest scale (l = 3), but replace the cost calculation function in its kernel with the C_l(p, π) calculation method proposed in this invention, that is:

[0063] .

[0064] S5: Perform multiple iterations of optimization at the coarsest scale (l = 3), using a red-black checkerboard pattern, which is divided into black pixel optimization and red pixel optimization. This checkerboard pattern avoids race conditions in parallel computing.

[0065] S5: Check the planar parameters of multiple neighboring pixels, including the four directly adjacent pixels (top, bottom, left, and right), the distant neighboring pixels (5 pixels away), and the neighboring pixels in the diagonal direction, and propagate the optimal planar parameters of the neighboring pixels to the current pixel.

[0066] S6: In view propagation, find the optimal planar parameters of the corresponding point pixel of the current pixel on the right view and propagate them to the current pixel.

[0067] S7: In plane refinement, a better solution is found by randomly perturbing the current plane parameters. The disparity perturbation range is set to start from max_disparity / 2.0f and divided by 10 each time. The normal vector perturbation range is set to start from 1.0 and divided by 4 each time. Iteration continues until deltaZ < 0.01f.

[0068] S8: After optimization at scale l=3, disparity map D3 and normal map N3 are upsampled to scale l=2 using bilinear interpolation. The disparity map values ​​are multiplied by 2, and the new disparity values ​​for each pixel are randomly perturbed within ±5% of their range to generate initial values ​​for scale l=2.

[0069] S9: Repeat the optimization process of S2-S7 at the l=2 scale.

[0070] S10: Continue processing the scale of l = 1, and finally perform final optimization using the original Gipuma's RGB+ gradient cost function at the original scale (l = 0), and output the result.

[0071] Here, when performing step S10, the final optimization can also be performed based on the cost calculation of this embodiment at the original scale (l = 0).

[0072] Based on the original gipuma algorithm and the gipuma algorithm integrating the method provided in this invention, disparity calculation was performed on two weakly textured scenes, dinosaur and temple, on the publicly available Middlebury dataset. The detection results are as follows: Figure 2 and Figure 3 As shown, where Figure 2 (a) shows the color disparity maps of the four views obtained by the original gipuma algorithm for dinosaur with blockseze=9 and iter=12 settings. Figure 3 Image (a) shows the color disparity maps of the template obtained by the original GIPuma algorithm under the same settings, from four different perspectives. The image clearly shows blurred or even broken edges, with jagged and blurred object outlines; smooth areas contain many random noise spots or outliers (blocky disparity holes, disparity jumps), etc. Figure 2 and Figure 3The four images shown in (b) are four color disparity maps obtained from four different perspectives after the method provided by this invention iterated 5, 3, 2, and 2 times at blocksize=9 and from the lowest resolution scale to the highest resolution scale, respectively. It can be seen that the edges are clearer and sharper, the object outlines are more distinct, the jaggedness is greatly improved, and the disparity transition in smooth areas is uniform and delicate, without obvious mottled noise or abnormal bright spots. Furthermore, using this embodiment on a computing platform with CPU: AMD EPYC 7713 64-Core Processor and graphics card: Super 4080s, selecting 9 target images, the average calculation time for gipuma is approximately 2 seconds. Compared to the original algorithm, this embodiment takes approximately 1.8 seconds on the same platform, demonstrating faster calculation speed while ensuring better disparity calculation results, and a significant improvement in algorithm performance.

[0073] According to embodiments of the present invention, the present invention also provides an electronic device and a computer-readable medium.

[0074] Electronic devices include:

[0075] One or more processors;

[0076] Storage device for storing one or more programs;

[0077] When the one or more programs are executed by the one or more processors, the one or more processors implement a multi-scale disparity calculation method based on Haar wavelet transform.

[0078] In practical use, users can interact with servers, which are also electronic devices, via a network to receive or send messages. Terminal devices are generally various electronic devices equipped with a display and used through a human-computer interface, including but not limited to smartphones, tablets, laptops, and desktop computers. Various specific application software can be installed on these terminal devices as needed, including but not limited to web browsers, instant messaging software, social media platforms, and shopping apps.

[0079] A server is a network service provider that offers various services. The multi-scale disparity calculation method based on Haar wavelet transform provided in this embodiment is generally executed by the server. In practical applications, under necessary conditions, terminal devices can also directly execute the multi-scale disparity calculation method based on Haar wavelet transform.

[0080] Similarly, the computer-readable medium of the present invention stores a computer program thereon, which, when executed by a processor, implements the multi-scale disparity calculation method based on Haar wavelet transform according to embodiments of the present invention.

Claims

1. A multi-scale disparity computation apparatus based on Haar wavelet transform, characterized by, The application relates to a method for constructing a multi-view image, comprising the following steps: A wavelet pyramid construction unit is used to perform multi-level Haar wavelet decomposition on an input multi-view image, so as to generate multiple high-frequency and low-frequency wavelet component images of each image in the multi-view image at different resolutions, i.e. different scales; the multi-view image comprises two images taken at two different angles of the same object; A coarse-to-fine optimization unit is used to establish and optimize a disparity map and a surface normal vector map according to the high-frequency and low-frequency wavelet component images generated by the wavelet pyramid construction unit, in a scale order from the lowest resolution to the highest resolution; when optimization is performed, firstly, two initial blank images with the same resolution are established based on the lowest resolution, i.e. the coarsest scale, in the high-frequency and low-frequency wavelet component images, wherein each pixel point corresponds to a pixel point in the high-frequency and low-frequency wavelet component images, then a disparity value and a normal vector value are randomly initialized in a preset range for each pixel point in the two initial blank images, so as to construct an initial disparity map and an initial surface normal vector map, and thus an initial disparity plane of each pixel is established; then iteration propagation is performed to optimize the current scale, and when the aggregate matching cost is calculated in the iteration propagation process, the average absolute difference of each component in a preset size window corresponding to a pixel point is calculated based on all the high-frequency and low-frequency wavelet component images of the multi-view image at the current scale, and the aggregate matching cost of the pixel point is calculated by summing the average absolute difference of each component after weighting; after optimization at the current scale is completed, the optimized disparity map and surface normal vector map are up-sampled, and the disparity value and the normal vector value of all pixel points are randomly increased or decreased in a preset change range, so that random disturbance is added, and the initial disparity map and the initial surface normal vector map at a higher scale are obtained, then iteration propagation is repeated at the higher scale, and the aggregate matching cost of the pixel point is calculated in the same way based on the high-frequency and low-frequency wavelet component images at the corresponding scale to realize optimization, until optimization at the highest resolution is completed.

2. The apparatus for multi-scale disparity computation based on Haar wavelet transform according to claim 1, wherein, The high-frequency and low-frequency wavelet component images generated by the wavelet pyramid construction unit comprise an image set of an approximation component, a horizontal detail component, a vertical detail component and a diagonal detail component generated at different resolutions.

3. The apparatus for computing multi-scale disparity based on Haar wavelet transform according to claim 2, wherein, In the coarse-to-fine optimization unit, the expression for calculating the aggregate matching cost is: ； wherein a, b, g, d are the approximate component weight coefficient, horizontal detail component weight coefficient, vertical detail component weight coefficient and diagonal detail component weight coefficient respectively, and a + b + g + d = 1; 、 、 、 respectively represent the sum of average absolute difference of wavelet approximate component, horizontal detail component, vertical detail component and diagonal detail component between two image corresponding high and low frequency wavelet component images of multi-view images in a neighborhood window N centered at pixel p; p represents the disparity plane of p point.

4. The apparatus for computing multi-scale disparity based on Haar wavelet transform according to claim 3, wherein, The calculation expression of the sum of the average absolute differences of the wavelet approximation component, the horizontal detail component, the vertical detail component and the diagonal detail component is: ； wherein , represent the corresponding points of two images of the multi-view image, i.e. the reference image and the target image; represents the wavelet component value of the reference image at scale l, at point q; represents the wavelet component value of the corresponding point on the target image i after projection by the disparity plane p; V is a set of target images pre-selected by the three-dimensional reconstruction algorithm.

5. A method for multi-scale disparity computation based on Haar wavelet transform, characterized in that, The method comprises the following steps: Step 1, multi-level Haar wavelet decomposition is performed on an input multi-view image, so as to generate multiple high-frequency and low-frequency wavelet component images of each image in the multi-view image at different resolutions, i.e. different scales; the multi-view image comprises two images taken at two different angles of the same object; Step 2, a coarse-to-fine optimization unit is used to establish and optimize a disparity map and a surface normal vector map according to the high-frequency and low-frequency wavelet component images generated by the wavelet pyramid construction unit, in a scale order from the lowest resolution to the highest resolution; when optimization is performed, firstly, two initial blank images with the same resolution are established based on the lowest resolution, i.e. the coarsest scale, in the high-frequency and low-frequency wavelet component images, wherein each pixel point corresponds to a pixel point in the high-frequency and low-frequency wavelet component images, then a disparity value and a normal vector value are randomly initialized in a preset range for each pixel point in the two initial blank images, so as to construct an initial disparity map and an initial surface normal vector map, and thus an initial disparity plane of each pixel is established; then iteration propagation is performed to optimize the current scale, and when the aggregate matching cost is calculated in the iteration propagation process, the average absolute difference of each component in a preset size window corresponding to a pixel point is calculated based on all the high-frequency and low-frequency wavelet component images of the multi-view image at the current scale, and the aggregate matching cost of the pixel point is calculated by summing the average absolute difference of each component after weighting; after optimization at the current scale is completed, the optimized disparity map and surface normal vector map are up-sampled, and the disparity value and the normal vector value of all pixel points are randomly increased or decreased in a preset change range, so that random disturbance is added, and the initial disparity map and the initial surface normal vector map at a higher scale are obtained, then iteration propagation is repeated at the higher scale, and the aggregate matching cost of the pixel point is calculated in the same way based on the high-frequency and low-frequency wavelet component images at the corresponding scale to realize optimization, until optimization at the highest resolution is completed. Step 2, according to the high-low frequency wavelet component images generated in step 1, establish and optimize the disparity map and surface normal vector map in scale order from the lowest resolution to the highest resolution; when optimizing, first, based on the lowest resolution, i.e. the coarsest scale, in the high-low frequency wavelet component images, two initial blank images with the same resolution are established, wherein each pixel point corresponds to a pixel point in the high-low frequency wavelet component images, then the disparity value and the normal vector value of each pixel point in the two initial blank images are randomly initialized in a predetermined range to construct an initial disparity map and an initial surface normal vector map, thereby establishing an initial disparity plane for each pixel; then iterative propagation is performed to optimize the current scale, and when calculating the aggregated matching cost in the iterative propagation process, the average absolute difference of each component in the preset size window corresponding to the pixel point is calculated based on all high-low frequency wavelet component images of the multi-view image at the current scale, and the weighted sum is taken as the aggregated matching cost of the pixel point; after the optimization at the current scale is completed, the optimized disparity map and surface normal vector map are up-sampled, and the disparity value and the normal vector value of all pixel points are randomly increased or decreased in a predetermined change range, thereby realizing the addition of random disturbance, and then serving as the initial disparity map and the initial surface normal vector map at a higher scale, then repeating the iterative propagation at the higher scale, and calculating the aggregated matching cost of the pixel point in the same way based on the high-low frequency wavelet component images at the corresponding scale to realize optimization in the iterative propagation process, until the optimization at the highest resolution is completed.

6. The method according to claim 5, wherein, In step 1, the generated high-low frequency wavelet component images include a set of images of the approximation component, the horizontal detail component, the vertical detail component and the diagonal detail component generated at different resolutions.

7. The method according to claim 6, wherein, In step 2, the expression for calculating the aggregated matching cost is: ； wherein a, b, g, d are the approximate component weight coefficient, horizontal detail component weight coefficient, vertical detail component weight coefficient and diagonal detail component weight coefficient respectively, and a + b + g + d = 1; 、 、 、 respectively represent the sum of average absolute difference of wavelet approximate component, horizontal detail component, vertical detail component and diagonal detail component between two image corresponding high and low frequency wavelet component images of multi-view images in a neighborhood window N centered at pixel p; p represents the disparity plane of point p.

8. The method of claim 6, wherein the method is based on Haar wavelet transform. In step 2, the calculation expression of the sum of the average absolute differences of the wavelet approximation component, the horizontal detail component, the vertical detail component and the diagonal detail component is: ； wherein represents the corresponding points of two images of the multi-view image, i.e. the reference image and the target image; represents the wavelet component value of the reference image at the scale / and at the point q; represents the wavelet component value of the corresponding point on the target image i after projection by the disparity plane π; V is a set of target images pre-selected by the three-dimensional reconstruction algorithm.

9. An electronic device, comprising: comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in any one of claims 5-8.

10. A computer readable medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the method as claimed in any one of claims 5-8. The computer program is executed by the processor to implement the method as claimed in any one of claims 5-8.

Images