A stereo matching method based on smoothing filtering and neighborhood pixel auxiliary

Abstract

This invention relates to a stereo matching method based on smoothing filtering and neighborhood pixel assistance. It incorporates neighboring points of the left and right points to be matched into the matching process, using the distance of the neighboring points from the point to be matched and whether they lie on epipolar lines for differential judgment. The judgment conditions differ for points in different neighborhood locations. On the left and right absolute phase maps, if the absolute phase values ​​of the actual pair of points to be matched differ, the phase matching of neighboring points will perform certain judgment compensation, ensuring that the two points with the highest similarity in absolute phase in a local region are successfully matched, thereby improving matching accuracy.

Description

Technical Field

[0001] This invention relates to the field of stereo matching technology, specifically to a stereo matching method based on smoothing filtering and neighborhood pixel assistance, which is applicable to phase-based structured light binocular cameras. Background Technology

[0002] When a binocular structured light 3D camera projects a set of structured light patterns onto a target object, the left and right cameras respectively acquire a set of corresponding images of the target object with structured light applied to its surface. Stereo matching is a crucial step in 3D reconstruction using a binocular structured light 3D camera. Within the shared field of view of the left and right cameras, for a point in the left image, if that point was not occluded when the right camera captured it, then finding that point in the right image is the process of stereo matching.

[0003] The 3D reconstruction algorithm within a binocular structured light 3D camera first performs epipolar correction on the two sets of images acquired by the left and right cameras, ensuring that corresponding points in the left and right images lie on the same epipolar line. Then, it performs structured light decoding on the two sets of images to obtain the corresponding absolute phase maps of the left and right cameras, such as... Figure 1 This represents the state of the absolute phase map obtained after stereo correction and decoding.

[0004] Stereo matching using left and right absolute phase maps traditionally employs point-to-point matching based on phase values ​​and epipolar constraints. Theoretically, the absolute phase value increases along each epipolar direction of the absolute phase map, and the phase value at each point is unique, making this traditional method quite reasonable. However, due to the presence of noise and phase transition points, such as... Figure 2 The figure shows the phase value state on a certain epipolar line. The absolute phase value is not strictly increasing on the epipolar line, which can easily lead to errors in the matching point pairs. Summary of the Invention

[0005] To address the problems existing in the prior art, the present invention aims to provide a stereo matching method based on smoothing filtering and neighborhood pixel assistance. It introduces a strategy of smoothing filtering and neighborhood pixel assistance to judge the phase values ​​within the neighborhood range of the left and right points after smoothing noise, making stereo matching more accurate.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A stereo matching method based on smoothing filtering and neighborhood pixel assistance, the method comprising the following steps:

[0008] Step 1: Input the left and right absolute phase images;

[0009] Step 2: Use a 3×3 filter core to perform smoothing filtering on the absolute phase map to smooth and denoise the absolute phase values ​​in the epipolar direction.

[0010] Step 3: Examine the smoothed left and right phase maps. Use the points on the left absolute phase map as the base points for stereo matching, and the points on the right absolute phase map that are on the same polar line as the left map as the target points to be matched; or, use the points on the right absolute phase map as the base points for stereo matching, and the points on the left absolute phase map that are on the same polar line as the left map as the target points to be matched.

[0011] Step 4: Take a point on the left or right absolute phase map and obtain the difference ΔP0 between two adjacent pixels on the smoothed horizontal epipolar line in the direction of the horizontal epipolar line; use a 3×3 neighborhood matching kernel for the selected point, where (1,1) is the base point for stereo matching on the left or right absolute phase map, and the other points are the neighborhood points of the base point. Record the absolute phase values ​​of the base point and its 8 neighborhood points.

[0012] Step 5: On the same polar line where the base point is located on the right absolute phase map and the left absolute phase map, take a point as the target point to be matched; or on the same polar line where the base point is located on the left absolute phase map and the right absolute phase map, take a point as the target point to be matched; use a 3×3 neighborhood matching kernel for the target point to be matched, where (1,1) is the point, and record the absolute phase values ​​of the point and its 8 neighboring points;

[0013] Step 6: Perform difference processing on the absolute phase values ​​of the 9 points recorded in the left absolute phase map and the absolute phase values ​​of the 9 points recorded in the right absolute phase map;

[0014] Step 7: Use formulas (1) to (5) to judge the data after the difference processing; where Abs() represents taking the absolute value within the parentheses, ΔP(0,0) represents the difference between the phase values ​​at position (0,0) of the left and right absolute phase diagrams, and so on;

[0015] Abs(ΔP(0,0)+ΔP(0,1)+ΔP(0,2)) < 3ΔP0 Formula (1)

[0016] Abs(ΔP(1,0)) <ΔP0 Formula (2)

[0017] Abs(ΔP(1,1)) <ΔP0 / 2 Formula (3)

[0018] Abs(ΔP(1,2)) <ΔP0 Formula (4)

[0019] Abs(ΔP(2,0)+ΔP(2,1)+ΔP(2,2)) < 3ΔP0 Formula (5)

[0020] Step 8: If a point on the selected right absolute phase map or left absolute phase map satisfies the above 5 judgment formulas, then the point is considered a matching point. Record the coordinates of the points on the left and right phase maps, and repeat steps 4-7.

[0021] Step 9: If a point on the selected right absolute phase map or left absolute phase map does not satisfy the above 5 judgment formulas, repeat steps 5-7. When all points on the corresponding epipolar line of the right absolute phase map or left absolute phase map are taken and none of them satisfy the above judgment formulas, discard the point on the left absolute phase map or right absolute phase map, consider it an invalid point, and start executing step 4 again.

[0022] Step 10: Once all points on the left or right absolute phase map have undergone the above steps, terminate the stereo matching and output the matching result.

[0023] After adopting the above scheme, the present invention uses a smoothing filter method to remove noise and phase jump points on the absolute phase map, thereby reducing the impact of phase jump points on stereo matching. Figure 3 The filtering kernel uses the phase values ​​of neighboring pixels to control the phase value of the center pixel. After smoothing filtering, no jump points appear in the epipolar direction, and the phase value increase becomes smoother. Simultaneously, this invention incorporates neighboring points of the left and right points to be matched into the matching process, using the distance of neighboring points from the point to be matched and whether they are on the epipolar line for differential judgment processing. The judgment conditions differ for points in different neighboring locations. On the left and right absolute phase maps, if the absolute phase values ​​of the actual pair of points to be matched differ, the phase matching of neighboring points will perform certain judgment compensation, ensuring that the two points with the highest similarity in absolute phase in the local region are successfully matched, thereby improving matching accuracy. Attached Figure Description

[0024] Figure 1 This represents the state of the absolute phase map obtained after stereo correction and decoding.

[0025] Figure 2 This refers to the phase state on a certain polar line;

[0026] Figure 3 This is a flowchart of the present invention;

[0027] Figure 4 This is a schematic diagram of the filter kernel of the present invention;

[0028] Figure 5 This is a schematic diagram of the matching kernel of the present invention. Detailed Implementation

[0029] like Figure 3 As shown, this invention discloses a stereo matching method based on smoothing filtering and neighborhood pixel assistance, which includes the following steps:

[0030] Step 1: Input the left and right absolute phase images;

[0031] Step 2: Use a 3×3 filter kernel (e.g., Figure 4 As shown, the absolute phase map is smoothed and filtered to smooth and denoise the absolute phase values ​​in the epipolar direction.

[0032] Step 3: Examine the smoothed left and right phase maps. Use the points on the left absolute phase map as the base points for stereo matching, and the points on the right absolute phase map that are on the same polar line as the left map as the target points to be matched.

[0033] Step 4: Take a point on the left absolute phase map and obtain the difference ΔP0 between two adjacent pixels on the smoothed horizontal epipolar line; use... Figure 5 A 3×3 neighborhood matching kernel is used, where (1,1) is the base point for stereo matching on the left absolute phase map, and the remaining points are the neighborhood points of the base point. The absolute phase values ​​of the base point and its 8 neighborhood points are recorded.

[0034] Step 5: On the same polar line where the base point is located on the right absolute phase map and the left absolute phase map, select a point as the target point to be matched, and use... Figure 4 The 3×3 neighborhood scheme is used, where (1,1) is the point, and the absolute phase value between the point and its 8 neighboring points is recorded.

[0035] Step 6: Perform difference processing on the absolute phase values ​​of the 9 points recorded in the left absolute phase map and the absolute phase values ​​of the 9 points recorded in the right absolute phase map;

[0036] Step 7: Use formulas (1) to (5) to judge the data after difference processing. Abs() represents taking the absolute value of the value in parentheses, ΔP(0,0) represents the difference of the phase values ​​at position (0,0) on the left and right absolute phase diagrams, and so on.

[0037] Abs(ΔP(0,0)+ΔP(0,1)+ΔP(0,2)) < 3ΔP0 Formula (1)

[0038] Abs(ΔP(1,0)) <ΔP0 Formula (2)

[0039] Abs(ΔP(1,1)) <ΔP0 / 2 Formula (3)

[0040] Abs(ΔP(1,2)) <ΔP0 Formula (4)

[0041] Abs(ΔP(2,0)+ΔP(2,1)+ΔP(2,2)) < 3ΔP0 Formula (5)

[0042] Step 8: If a point on the selected right absolute phase map satisfies the above 5 judgment formulas, then the point is considered a matching point. Record the coordinates of the points on the left and right phase maps, and repeat steps 4-7.

[0043] Step 9: If a point on the selected right absolute phase diagram does not satisfy the above 5 judgment formulas, repeat steps 5-7. When all points on the corresponding epipolar line of the right absolute phase diagram have been selected and none of them satisfy the above judgment formulas, discard the point on the left absolute phase diagram and consider it an invalid point, and start executing step 4 again.

[0044] Step 10: Once all points on the left absolute phase map have undergone the above steps, terminate the stereo matching and output the matching result.

[0045] In step 3 above, points on the right absolute phase map can also be selected as base points for stereo matching, and points on the left absolute phase map that lie on the same epipolar line as the left map can be selected as target points to be matched. In this case, the left absolute phase map in steps 4-10 is replaced by the right absolute phase map, and the right absolute phase map is replaced by the left absolute phase map. The stereo matching method of the present invention is as follows:

[0046] Step 1: Input the left and right absolute phase images;

[0047] Step 2: Use a 3×3 filter kernel (e.g., Figure 4 As shown, the absolute phase map is smoothed and filtered to smooth and denoise the absolute phase values ​​in the epipolar direction.

[0048] Step 3: Examine the smoothed left and right phase maps. Use the points on the right absolute phase map as the base points for stereo matching, and the points on the left absolute phase map that are on the same polar line as the left map as the target points to be matched.

[0049] Step 4: Take a point on the right absolute phase map and obtain the difference ΔP0 between two adjacent pixels on the smoothed horizontal epipolar line; use... Figure 5 A 3×3 neighborhood matching kernel is used, where (1,1) is the base point for stereo matching on the right absolute phase map, and the remaining points are the neighborhood points of the base point. The absolute phase values ​​of the base point and its 8 neighborhood points are recorded.

[0050] Step 5: On the same polar line where the base points on the left and right absolute phase maps are located, select a point as the target point to be matched, and use... Figure 4 The 3×3 neighborhood scheme is used, where (1,1) is the point, and the absolute phase value between the point and its 8 neighboring points is recorded.

[0051] Step 6: Perform difference processing on the absolute phase values ​​of the 9 points recorded in the right absolute phase map and the absolute phase values ​​of the 9 points recorded in the left absolute phase map;

[0052] Step 7: Use formulas (1) to (5) to judge the data after difference processing. Abs() represents taking the absolute value of the value in parentheses, ΔP(0,0) represents the difference of the phase values ​​at position (0,0) on the left and right absolute phase diagrams, and so on.

[0053] Abs(ΔP(0,0)+ΔP(0,1)+ΔP(0,2)) < 3ΔP0 Formula (1)

[0054] Abs(ΔP(1,0)) <ΔP0 Formula (2)

[0055] Abs(ΔP(1,1)) <ΔP0 / 2 Formula (3)

[0056] Abs(ΔP(1,2)) <ΔP0 Formula (4)

[0057] Abs(ΔP(2,0)+ΔP(2,1)+ΔP(2,2)) < 3ΔP0 Formula (5)

[0058] Step 8: If a point on the selected right absolute phase map satisfies the above 5 judgment formulas, then the point is considered a matching point. Record the coordinates of the points on the left and right phase maps, and repeat steps 4-7.

[0059] Step 9: If a point on the selected right absolute phase diagram does not satisfy the above 5 judgment formulas, repeat steps 5-7. When all points on the corresponding epipolar line of the right absolute phase diagram have been selected and none of them satisfy the above judgment formulas, discard the point on the left absolute phase diagram and consider it an invalid point, and start executing step 4 again.

[0060] Step 10: Once all points on the left absolute phase map have undergone the above steps, terminate the stereo matching and output the matching result.

[0061] In summary, this invention employs a smoothing filter method on noise and phase jump points in the absolute phase map to remove noisy points and reduce the impact of phase jump points on stereo matching. Figure 3 The filtering kernel uses the phase values ​​of neighboring pixels to control the phase value of the center pixel. After smoothing filtering, no jump points appear in the epipolar direction, and the phase value increase becomes smoother. Simultaneously, this invention incorporates neighboring points of the left and right points to be matched into the matching process, using the distance of neighboring points from the point to be matched and whether they are on the epipolar line for differential judgment processing. The judgment conditions differ for points in different neighboring locations. On the left and right absolute phase maps, if the absolute phase values ​​of the actual pair of points to be matched differ, the phase matching of neighboring points will perform certain judgment compensation, ensuring that the two points with the highest similarity in absolute phase in the local region are successfully matched, thereby improving matching accuracy.

[0062] The above description is merely an embodiment of the present invention and does not constitute any limitation on the technical scope of the present invention. Therefore, any minor modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A stereo matching method based on smoothing filtering and neighborhood pixel assistance, characterized in that: The method includes the following steps: Step 1: Input the left and right absolute phase images; Step 2: Use a 3×3 filter kernel to perform smoothing filtering on the absolute phase map to smooth and remove noise from the absolute phase values ​​in the epipolar direction; Step 3: Examine the smoothed left and right phase maps. Use the points on the left absolute phase map as the base points for stereo matching, and the points on the right absolute phase map that are on the same polar line as the left map as the target points to be matched; or, use the points on the right absolute phase map as the base points for stereo matching, and the points on the left absolute phase map that are on the same polar line as the left map as the target points to be matched. Step 4: Take a point on the left or right absolute phase map and obtain the difference ΔP0 between two adjacent pixels on the smoothed horizontal epipolar line in the direction of the horizontal epipolar line; use a 3×3 neighborhood matching kernel for the selected point, where (1,1) is the base point for stereo matching on the left or right absolute phase map, and the other points are the neighborhood points of the base point. Record the absolute phase values ​​of the base point and its 8 neighborhood points. Step 5: On the same polar line where the base point is located on the right absolute phase map and the left absolute phase map, take a point as the target point to be matched; or on the same polar line where the base point is located on the left absolute phase map and the right absolute phase map, take a point as the target point to be matched; use a 3×3 neighborhood matching kernel for the target point to be matched, where (1,1) is the point, and record the absolute phase values ​​of the point and its 8 neighboring points; Step 6: Perform difference processing on the absolute phase values ​​of the 9 points recorded in the left absolute phase map and the absolute phase values ​​of the 9 points recorded in the right absolute phase map; Step 7: Use formulas (1) to (5) to judge the data after the difference processing; where Abs() represents taking the absolute value within the parentheses, ΔP(0,0) represents the difference between the phase values ​​at position (0,0) of the left and right absolute phase diagrams, and so on; Abs(ΔP(0,0)+ΔP(0,1)+ΔP(0,2)) < 3ΔP0 Formula (1) Abs(ΔP(1,0)) <ΔP0 Formula (2) Abs(ΔP(1,1)) <ΔP0 / 2 Formula (3) Abs(ΔP(1,2)) <ΔP0 Formula (4) Abs(ΔP(2,0)+ΔP(2,1)+ΔP(2,2)) < 3ΔP0 Formula (5) Step 8: If a point on the selected right absolute phase map or left absolute phase map satisfies the above 5 judgment formulas, then the point is considered a matching point. Record the coordinates of the points on the left and right phase maps, and repeat steps 4-7. Step 9: If a point on the selected right absolute phase map or left absolute phase map does not satisfy the above 5 judgment formulas, repeat steps 5-7. When all points on the corresponding epipolar line of the right absolute phase map or left absolute phase map are taken and none of them satisfy the above judgment formulas, discard the point on the left absolute phase map or right absolute phase map, consider it an invalid point, and start executing step 4 again. Step 10: Once all points on the left or right absolute phase map have undergone the above steps, terminate the stereo matching and output the matching result.

Images