A parallel method for patch matching based on two-level segmentation units

Abstract

The present invention discloses a parallel method for patch matching based on two-level segmentation units, including: Step 1: Obtain the left and right images of a stereo matching pair; Step 2: Regard the input images as graph structures, and construct computational layers S1 and S2 composed of segmentation units; Step 3: Randomly initialize a label value for each pixel; Step 4: Parallelly implement label space propagation in computational layer S1; Step 5: Parallelly implement label plane refinement in computational layer S2; Step 6: Repeat Step 4) and Step 5) until the disparity map converges to obtain the sub-pixel disparity maps of the left and right images; Step 7: Detect the disparity values of pixel points that do not satisfy the consistency constraint by using the left and right disparity maps, and perform post-processing operations on them; Step 8: Convert the disparity map into a depth map by using the parameters of the binocular camera to obtain the three-dimensional topography of the scene. The present invention achieves the purposes of reducing the computational time complexity of the patch matching algorithm, enabling efficient parallel computing, and quickly converging the optimization results.

Description

Technical Field

[0001] The invention belongs to the technical field of visual non-contact measurement, and in particular relates to a parallel method for facet matching based on two-level segmentation units. Background Art

[0002] Stereo vision technology has always been a hot topic in the field of machine vision and computer vision, and is widely used in unmanned driving, robot navigation, earth observation, virtual reality, cultural heritage protection, etc. As a depth perception technology, it has the characteristics of simple system, high degree of automation, non-contact and dense point cloud. Stereo matching, as the core technology of stereo vision, aims to calculate the same-name pixels in images of different perspectives, and then use the difference in imaging coordinates of different perspectives to restore the three-dimensional shape of the object according to the triangular geometric relationship. Early stereo matching algorithms usually use the integer disparity space model of the front view plane. This simplified model reduces the size of the label space and reduces the calculation time of the disparity optimization algorithm. However, the front view plane model and the integer disparity space can only obtain discrete disparity maps, and a later interpolation algorithm is required to further obtain sub-pixel disparity maps. This method is difficult to ensure the accuracy of sub-pixel disparity maps.

[0003] In order to obtain sub-pixel disparity maps, plane label models are widely used. The goal of the stereo matching algorithm is to calculate an optimal plane label for each pixel. The plane label model can not only obtain sub-pixel disparity values, but also obtain the normal vector of the pixel plane, which is beneficial for subsequent point cloud stitching and 3D reconstruction. The patch matching algorithm can efficiently calculate the nearest neighbor field of each pixel. The algorithm only retains one or several labels and cost values ​​of the pixel, which not only reduces the memory usage of the algorithm, but also has high computational efficiency, making it widely used in binocular stereo matching, multi-view stereo matching and optical flow tasks.

[0004] Bleyer et al. first applied the patch matching algorithm to the stereo matching task (Bleyer M, Rhemann C, Rother C. PatchMatch Stereo-Stereo Matching with Slanted Support Windows[C]. 2011 Proceedings of the British Machine Vision Conference, 2011). This algorithm uses a single pixel as the calculation unit and can only use adaptive weights for cost aggregation, resulting in high computational time complexity; at the same time, the cost aggregation values ​​of adjacent pixels with the same label cannot be reused, resulting in a large amount of repeated calculations.

[0005] Considering that pixels in local areas of an image have the same label value, Lu et al. performed superpixel segmentation on the image in the patch filtering algorithm (Lu J, Li Y, Yang H, et al. Patch Match Filter: Edge-Aware Filtering Meets Randomized Search for Visual Correspondence [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (9): 1866-1879.), and used the segmentation unit as the calculation unit to complete the label space propagation and label refinement. Using superpixels as calculation units enables cost aggregation to use a linear time complexity cost filtering algorithm, thereby reducing the computational time complexity of the cost filter, while reducing the number of calculation units and avoiding a large number of repeated calculations. However, the patch filtering algorithm has overlapping label update areas constructed with superpixel units, which makes the algorithm unable to implement parallel computing; in addition, the strategy of using representative pixels to refine the labels of segmentation unit pixels reduces the utilization rate of label calculation, resulting in slow label convergence of pixels in texture-rich areas. Summary of the invention

[0006] In view of the shortcomings of the current patch matching algorithm in stereo matching tasks, such as high computational complexity and inability to perform parallel calculations, the purpose of the present invention is to propose a patch matching algorithm with low computational complexity and the ability to perform efficient parallel calculations on CPUs and GPUs. Similar to the traditional algorithm, this method still uses the ideas of spatial propagation and plane refinement of the patch matching algorithm; the difference is that the proposed algorithm uses regular-shaped segmentation units as calculation units, and constructs optimization layers of two scale segmentation units to achieve the purpose of spatial propagation and random search, respectively, to achieve efficient parallel calculations. At the same time, a plane refinement strategy based on the segmentation unit is proposed to accelerate the convergence speed of the label during the optimization process.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A parallel method for patch matching based on a two-stage segmentation unit comprises the following steps:

[0009] Step 1: Obtain the left and right images of the stereo vision matching pair and calculate the pixel disparity search range;

[0010] Step 2: Consider the left and right images as graph structures, construct the computational layers S1 and S2 consisting of two-level segmentation units, and construct the corresponding subgraphs with the segmentation units;

[0011] Step 3: Use random initialization method to randomly generate a plane label for each pixel;

[0012] Step 4: In the calculation layer S1, the segmentation unit is used as the calculation unit to perform spatial propagation of the pixel label;

[0013] Step 5: In the calculation layer S2, the plane refinement of the pixel labels is completed using the segmentation unit as the calculation unit;

[0014] Step 6: Repeat steps 4) and 5) until the disparity map converges, and use the same process to obtain the label of each pixel in the right image.

[0015] Step 7: Use the left and right disparity maps to determine the erroneous disparity value in the disparity map, and post-process the erroneous disparity value;

[0016] Step 8: Use the parameters of the binocular camera to convert the disparity map into a depth map to obtain the three-dimensional shape of the scene.

[0017] A further improvement of the present invention is that the specific implementation method of step 1) is as follows:

[0018] Step 1.1: Arrange the left and right cameras to form a binocular system according to the scene characteristics, and use Zhang Zhengyou chessboard to calibrate the camera's internal and external parameters;

[0019] Step 1.2: Use the calibrated binocular system to capture the left and right perspective images of the scene, and perform epipolar correction on the images according to the system parameters so that the image coordinates of the same-named pixels in the left and right images only change in the X direction.

[0020] A further improvement of the present invention is that the specific implementation method of step 2 is as follows:

[0021] Step 2.1: Consider the input image as a graph, where pixels represent nodes in the graph; segment the graph into K1 rectangular regions with a side length of k1 pixels; each segmentation unit is used as a computational unit to form a computational layer S1; extend the segmentation unit outward by r pixels to construct a subgraph of any segmentation unit S1(k), with a subgraph size of a rectangular region of (k1+2r)×(k1+2r), where r represents the radius of the filter window;

[0022] Step 2.1: The image is segmented into rectangles with a side length of l2 pixels. The entire image is divided into K2 rectangular areas. Each segmentation unit serves as a calculation unit to form a calculation layer S2. All segmentation units are extended outward by r pixels to construct an arbitrary segmentation unit S2(k) sub-image. The sub-image size is a rectangular area of ​​(k2+2r)×(k2+2r), where r represents the filter window radius.

[0023] A further improvement of the present invention is that the specific implementation method of step 3 is as follows:

[0024] For each pixel, a disparity value d0 is randomly generated within the disparity range, and a unit vector is randomly initialized. As the normal vector of the plane, the label l of pixel p is calculated by the pixel disparity value and the normal vector p =(a p ,b p ,c p ), where a p =-n x / n y ; b p =-n y / n z ;c p =(n x x0+n y y0+n z d0) / n z .

[0025] A further improvement of the present invention is that the specific implementation method of step 4 is as follows:

[0026] Step 4.1: Randomly sample N in any segmentation unit S1(k) in the calculation layer S1 s pixels, and the label values ​​of the pixels form the recommended label sequence

[0027] Step 4.2: Use the segmentation unit S1(k) as the calculation unit to sequentially extract the recommended labels l from the recommended label sequence s ; Calculate the matching cost of all pixels under the recommended label in the sub-image of segmentation unit S1(k);

[0028] Step 4.3: Complete cost aggregation in the subgraph of the segmentation unit S1(k); this is achieved by using a guide filter or a cross filter linear cost filter algorithm;

[0029] Step 4.4: Complete the pixel label update, that is, for any pixel p in the segmentation unit S1(k), when C(p,l p )>C(p,l s ) when the current pixel recommends the label and its cost value, otherwise the label value and cost value are not updated; where C(·) represents the matching cost after cost aggregation, l s Indicates the recommended tag, l p Indicates the current label of the pixel.

[0030] A further improvement of the present invention is that the specific implementation method of step 5 is as follows:

[0031] Step 5.1: On the calculation layer S2, randomly sample a pixel in any segmentation unit S2(k) to obtain the pixel's disparity value d0 and normal vector At the same time, random sampling is used to obtain a disparity value and an offset of the normal vector. and Δ n , the disparity value and normal vector of the generated recommended label are expressed as and in and is the offset sampling space, and Represents the maximum sampling range of disparity value and normal vector respectively; the initial spatial range is set to The spatial range decreases exponentially with the number of calculations, that is, and The termination condition is set to

[0032] Step 5.2: Disparity value based on recommended labels and the normal vector Calculate plane parameters

[0033] Step 5.3: Using steps 4.2 to 4.3, calculate the matching cost, cost aggregation and update the pixel labels in the sub-graph of the segmentation unit S2(k) according to the recommended labels;

[0034] Step 5.4: Repeat steps 5.1 to 5.3 until the calculation meets the termination condition.

[0035] A further improvement of the present invention is that the specific implementation method of step 6 is as follows:

[0036] Repeat steps 4 and 5. When the mismatch rate of the disparity map after adjacent optimization times changes less than the set threshold, stop the calculation and obtain the disparity map of the current image. When the right image calculates the same-name pixel points in the left image, the disparity value is negative, and the calculation process of the disparity map of the left image is used to obtain the disparity map of the right image.

[0037] A further improvement of the present invention is that the specific implementation method of step 7 is as follows:

[0038] Step 7.1: Use left-right consistency detection to find out the pixels in the disparity map that do not meet the left-right consistency constraint, and set them as the wrong disparity value;

[0039] Step 7.2: For pixels with incorrect disparity values, traverse forward and backward along the X direction to obtain the first pixel with a correct disparity value, and select the minimum value of the two disparity values ​​to fill the disparity value of the current pixel;

[0040] Step 7.3: Use median filtering to filter the disparity map.

[0041] A further improvement of the present invention is that the specific implementation method of step 8 is as follows:

[0042] The binocular system parameters are used to convert the disparity map into a depth map in the camera coordinate system.

[0043]

[0044] Where f is the focal length of the camera, B is the binocular baseline distance, and the image coordinates of the pixel point p are (x p ,y p ), whose disparity value is d p , the corresponding three-dimensional coordinates in the camera coordinate system are (X p ,Y p ,Z p ).

[0045] The present invention has at least the following beneficial technical effects:

[0046] The present invention proposes a parallel method for patch matching based on two-level segmentation units, which can efficiently and parallelly search for pixel label values ​​in CPU and GPU hardware. The algorithm first uses two scales to segment the image to construct two optimization layers, wherein the segmentation unit adopts a regular shape so that the label update can be completed independently in the segmentation unit, ensuring the parallel computing capability of the algorithm; in the two-level segmentation unit calculation layer, the large-size segmentation unit calculation layer mainly completes spatial propagation, and the large size of the segmentation unit reduces the number of segmentation units and also expands the distance of pixel label spatial propagation; the calculation layer with smaller segmentation units completes label refinement, and the smaller-size segmentation unit is used to satisfy the pixels in the segmentation unit to have the same label value as much as possible, thereby improving the effectiveness of label refinement. The design of the two-level segmentation unit calculation layer meets the parallel computing requirements of spatial propagation and plane refinement, and can also accelerate the convergence speed of labels. Based on the plane refinement of the segmentation unit, each sampling generates a recommended label, and not only samples the label offset, but also samples the label to be optimized. This strategy can improve the utilization rate of optimization and accelerate the convergence speed of labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The figure is a calculation flow chart of the present invention.

[0048] Figure 2 It is a schematic diagram of the regular shape segmentation and calculation unit of the graph. Figure 2 (a) Schematic diagram of the segmentation map. Figure 2 (b) is an arbitrary segmentation unit and sub-graph.

[0049] Figure 3 It is a schematic diagram of the calculation layer of two-level segmentation units of different scales, where Figure 3 (a) is the large-scale segmentation unit calculation layer, Figure 3(b) is the small-scale segmentation unit calculation layer.

[0050] Figure 4 The time comparison of the proposed algorithm with parallel computing and non-parallel computing on CPU.

[0051] Figure 5 is the disparity map effect of the proposed algorithm, where Figure 5 (a) is the left image in the matching pair. Figure 5 (b) Figure 5 (a) Standard disparity map, Figure 5 (c) is the disparity map obtained by the algorithm.

[0052] Figure 6 3D point cloud reconstruction results. DETAILED DESCRIPTION

[0053] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to be able to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with implementation examples.

[0054] The present invention provides a parallel method for patch matching based on two-level segmentation units. It uses regular shapes to segment images and uses segmentation units as calculation units and label update areas to ensure the parallel computing capability of the algorithm. The calculation layer of the two-level segmentation units is designed. The layer with smaller segmentation unit size is used for plane refinement calculation, and the layer with larger segmentation unit size is used for space propagation calculation. It ensures that the two processes can be efficiently and alternately calculated in parallel. The space propagation within the large-sized segmentation unit is equivalent to the space propagation between the small-sized segmentation units. During the plane refinement process in the segmentation unit, the labels to be optimized and the label offsets are randomly sampled at the same time to improve the convergence speed of the labels. Its calculation process is as follows: Figure 1 As shown, the specific steps include:

[0055] Step 1: Obtain the left and right images of the stereo vision matching pair and calculate the pixel disparity search range;

[0056] In this example, we select the stereo matching pair images published by the Middlebury dataset for illustration. The images in the dataset include left and right matching pairs of images, and the maximum disparity search range d max , the focal length f and baseline distance B of the binocular system.

[0057] Step 2: Construct regular segmentation units based on the input image. The two segmentation units form two computing layers S1 and S2. At the same time, construct a corresponding sub-graph for each segmentation unit, such as Figure 2 (b)

[0058] Step 2.1: Consider the input image as a graph, where pixels represent nodes in the graph; use a rectangle with a side length of k1 pixels to segment the graph, and the entire graph is divided into K1 segmentation units. Figure 2 (a) shows the segmentation of the image. The dotted rectangle is a segmentation unit S1(k). At the same time, the segmentation unit is extended outward by r pixels to construct a subgraph of the segmentation unit S1(k) for cost calculation and cost aggregation, as shown in Figure 2 As shown in (b), r represents the filter window radius; the above operations construct the S1 calculation layer.

[0059] Step 2.2: Use step 2.1 to construct the S2 calculation layer, the difference is that the rectangular size of the segmentation unit is k2×k2. In the algorithm design, the S1 layer is used for spatial propagation and the S2 layer is used for plane refinement. Therefore, k1 ≥ 1.5k2 is usually set.

[0060] Step 3: Start matching point calculation. First, randomly initialize a plane label for each pixel;

[0061] Each pixel has 3 variables: disparity value d, plane normal vector Plane Parameters p =(a p ,b p ,c p ). The conversion relationship between parallax and plane parameters is d p =a p p x +b p p y +c p The disparity values ​​and plane normals are used in a random search for plane refinement, and the plane parameters are used as labels for neighborhood propagation.

[0062] During the random initialization of the algorithm, the algorithm disparity space [0,d max ] randomly samples a disparity value d and generates a unit vector As the normal vector of the plane, the label parameter l of pixel p is calculated p =(a p ,b p ,c p ), where a p =-n x / n y ; b p =-n y / nz ;c p =(n x x0+n y y0+n z d p ) / n z .

[0063] Step 4: In the computing layer S1, the spatial propagation of labels is realized using the segmentation unit as the computing unit;

[0064] Step 4.1: Randomly sample N in any partition unit S1(k) in the computation layer S1 s pixels, the label values ​​of the pixels form the recommended label sequence L = {l s |n=1,...,N s};

[0065] Step 4.2: In the segmentation unit S1(k), cost calculation, cost aggregation and label update are performed on all labels in the label sequence. s Calculate the cost of all pixels in unit R1(k). s The calculated disparity value of the left image pixel p is d p , then it corresponds to the matching point in the right image The coordinates are x R =x L -d p The color value of the sub-pixel position is obtained by bilinear interpolation of the image. The "color + gradient" cost description operator is used here:

[0066]

[0067] Where: α represents the weight ratio of color value and gradient value; τ col and τ grad are the cutoff thresholds for color value and gradient value respectively. The parameters are set as {α,τ col ,τ grad}:={0.9,10,2}

[0068] Step 4.3: Complete cost aggregation in subgraph R1(k); the cost aggregation operation can be completed quickly by using classic linear cost filtering algorithms such as guide filtering and cross filtering;

[0069] Step 4.4: Complete the pixel label update, that is, for any pixel p in the segmentation unit S1(k), when C(p,l p )>C(p,l s ) when the current pixel takes the recommended label l s and its cost value, otherwise the label value and cost value are not updated;

[0070] This process obtains recommended labels through random sampling, and performs unified cost calculation, cost aggregation and label update on the recommended labels within the segmentation unit, thereby realizing the propagation of the pixel labels of the sampling points within the entire segmentation unit; thereby expanding the distance of label propagation and making it less likely for the algorithm to fall into local minima.

[0071] Step 5: In the calculation layer S2, the plane refinement of the pixel labels is completed using the segmentation unit as the calculation unit;

[0072] Step 5.1: On the calculation layer S2, randomly sample a pixel in any segmentation unit S2 to obtain the pixel's disparity value d0 and normal vector At the same time, random sampling is used to obtain a disparity value and an offset of the normal vector. and Δ n , the disparity value and normal vector of the generated recommended label are expressed as and Then calculate the plane parameters based on the disparity value and the normal vector;

[0073] in and is the offset range, and Represents the maximum sampling space of disparity value and normal vector respectively; the initial space range is set to where d max is the maximum disparity search value;

[0074] Step 5.2: Disparity value based on recommended labels Normal vector Calculate plane parameters

[0075] Step 5.3: Use steps 4.2 to 4.3 to calculate the matching cost according to the recommended label in the segmentation unit S2(k) and its sub-graph R2(k), aggregate the cost and update the label of the pixel;

[0076] Step 5.4: Repeat steps 5.1 to 5.3, and use the spatial range to decrease exponentially with the number of calculations, that is, and The termination condition is set to

[0077] In this calculation process, during a round of random search, each time a random sample is used to obtain a label offset, a pixel label is randomly sampled in the segmentation unit. After each recommended label is generated, the label update is completed before the next random search is performed to generate a recommended label. This is done to increase the probability of sampling a representative label in the segmentation unit, and on the other hand, it can also increase the amount of label refinement.

[0078] Step 6: Repeat steps 4) and 5) until the disparity map converges, and use the same process to obtain the label of each pixel in the right image. Figure 4 The time comparison of the proposed algorithm using CPU parallel accelerated computing and without parallel computing on three sets of data sets (Tsukuba, Venus and Cones) is shown. Figure 5 The initial left disparity map obtained by the algorithm and the distribution map of mismatched points calculated based on the standard disparity map are shown (the mismatched points in the non-occluded area are marked in gray, the mismatched points in the occluded area are marked in black, and the threshold of mismatched points is 0.5 pixels).

[0079] Step 7: Using the obtained disparity maps corresponding to the left and right images, the error disparity value is detected by using consistency constraints, and the error disparity value is processed by post-processing methods such as disparity value filling and median filtering;

[0080] Step 8: Use the parameters of the binocular camera to convert the disparity map into a depth map to obtain the three-dimensional shape of the scene.

[0081] The binocular system parameters are used to convert the disparity map into a depth map in the camera coordinate system.

[0082]

[0083] Where f is the focal length of the camera, B is the binocular baseline distance, and the image coordinates of the pixel point p are (x p ,y p ), whose disparity value is d p , the corresponding three-dimensional coordinates in the camera coordinate system are (X p ,Y p ,Z p ). Figure 6 The post-processed disparity map in Figure (5) is converted into a three-dimensional point cloud result under camera parameters.

[0084] Although the present invention has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements may be made thereto based on the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of protection claimed by the present invention.

Claims

1. A parallel method for patch matching based on two-level segmentation units, characterized in that: The following steps are involved: Step 1: Obtain the left and right images of the stereo vision matching pair and calculate the pixel disparity search range; Step 2: Consider the left and right images as graph structures, construct the computational layers S1 and S2 consisting of two-level segmentation units, and construct the corresponding subgraphs with the segmentation units; the specific implementation method is as follows: Step 2.1: Consider the input image as a graph, where pixels represent nodes in the graph; segment the graph into rectangles with a side length of k1 pixels, and the entire graph is segmented into K1 rectangular regions; each segmentation unit is used as a computational unit to form a computational layer S1; extend the segmentation unit outward by r pixels to construct a subgraph R1(k) of any segmentation unit S1(k), with a subgraph size of (k1+2r)×(k1+2r) rectangular region, where r represents the radius of the filter window; Step 2.2: Segment the image into rectangles with a side length of l2 pixels. The entire image is divided into K2 rectangular areas. Each segmentation unit is used as a calculation unit to form a calculation layer S2. Extend all segmentation units outward by r pixels to construct a sub-image of any segmentation unit S2(k). The sub-image size is a rectangular area of ​​(k2+2r)×(k2+2r), where r represents the radius of the filter window. Step 3: Use random initialization method to randomly generate a plane label for each pixel; Step 4: In the calculation layer S1, the segmentation unit is used as the calculation unit to perform spatial propagation of the pixel label; Step 5: In the calculation layer S2, the plane refinement of the pixel label is completed using the segmentation unit as the calculation unit; the specific implementation method is as follows: Step 5.1: On the calculation layer S2, randomly sample a pixel in any segmentation unit S2(k) to obtain the pixel's disparity value d0 and normal vector At the same time, random sampling is used to obtain a disparity value and an offset of the normal vector. and Δ n , regenerate the disparity value and normal vector of the new recommended label as and in and is the offset sampling space, and Represents the maximum sampling value of the disparity value and the normal vector respectively; the initial spatial range is set to The spatial range decreases exponentially with the number of calculations, that is, and The termination condition is set to Step 5.2: Disparity value based on recommended labels and the normal vector Calculate plane parameters Step 5.3: Use steps 4.2 to 4.3 to calculate the matching cost according to the recommended label in the segmentation unit S2(k) and its sub-graph R2(k), aggregate the cost and update the label of the pixel; Step 5.4: Repeat steps 5.1 to 5.3 until the calculation meets the termination condition. Step 6: Repeat steps 4) and 5) until the disparity map converges, and use the same process to obtain the label of each pixel in the right image. Step 7: Use the left and right disparity images to determine the erroneous disparity value in the disparity image, and fill the erroneous disparity value; Step 8: Use the parameters of the binocular camera to convert the disparity map into a depth map to obtain the three-dimensional shape of the scene.

2. A parallel patch matching method based on two-stage segmentation units according to claim 1, characterized in that: The specific implementation method of step 1) is as follows: Step 1.1: Arrange the left and right cameras to form a binocular system according to the scene characteristics, and use Zhang Zhengyou chessboard to calibrate the camera's internal and external parameters; Step 1.2: Use the calibrated binocular system to capture the left and right perspective images of the scene, and perform epipolar correction on the images according to the system parameters so that the image coordinates of the same-named pixels in the left and right images only change in the X direction.

3. A parallel face matching method based on two-stage segmentation units according to claim 2, characterized in that: The specific implementation method of step 3 is as follows: For each pixel, a disparity value d0 is randomly generated within the disparity range, and a unit vector is randomly initialized. As the normal vector of the plane, the label l of pixel p is calculated by the pixel disparity value and the normal vector p =(a p ,b p ,c p ), where a p =-n x / n y ; b p =-n y / n z ;c p =(n x x0+n y y0+n z d0) / n z .

4. The parallel method for patch matching based on two-stage segmentation units according to claim 3, characterized in that: The specific implementation method of step 4 is as follows: Step 4.1: Randomly sample N in any segmentation unit S1(k) in the calculation layer S1 s pixels, and the label values ​​of the pixels form the recommended label sequence Step 4.2: Use the segmentation unit S1(k) as the calculation unit to sequentially extract the recommended labels l from the recommended label sequence s ; Calculate the matching cost of all pixels under the recommended label in the sub-image of segmentation unit S1(k); Step 4.3: Complete cost aggregation in the subgraph of the segmentation unit S1(k); this is achieved by using a guide filter or a cross filter linear cost filter algorithm; Step 4.4: Complete the pixel label update, that is, for any pixel p in the segmentation unit S1(k), when C(p,l p )>C(p,l s ) when the current pixel takes the recommended label and its cost value, otherwise the label value and cost value are not updated; where C(·) represents the matching cost after cost aggregation, l s Indicates the recommended tag, l p Indicates the current label of the pixel.

5. The parallel method for patch matching based on two-stage segmentation units according to claim 4, characterized in that: The specific implementation method of step 6 is as follows: Repeat steps 4 and 5. When the change in the disparity map mismatch rate after adjacent optimization times is less than the set threshold, stop the calculation and obtain the disparity map of the current image. When the right image calculates the same-name pixel points in the left image, the disparity value is negative, and the calculation process of the left image disparity map is used to obtain the disparity map of the right image.

6. A parallel face matching method based on two-stage segmentation units according to claim 5, characterized in that: The specific implementation method of step 7 is as follows: Step 7.1: Use the consistency detection of the left and right disparity maps to find out the pixels in the disparity map that do not meet the consistency constraint, and determine that the disparity value of the pixel is an incorrect disparity value; Step 7.2: For pixels with incorrect disparity values, traverse forward and backward along the X direction to obtain the first pixel with a correct disparity value, and select the minimum value of the two disparity values ​​to fill the disparity value of the current pixel; Step 7.3: Use median filtering to filter the disparity map.

7. A parallel face matching method based on two-stage segmentation units according to claim 6, characterized in that: The specific implementation method of step 8 is as follows: The binocular system parameters are used to convert the disparity map into a depth map in the camera coordinate system. Where f is the focal length of the camera, B is the binocular baseline distance, and the image coordinates of the pixel point p are (x p ,y p ), whose disparity value is d p , the corresponding three-dimensional coordinates in the camera coordinate system are (X p ,Y p ,Z p ).

Images