A real-time binocular stereo matching method suitable for pure CPU and related products

Abstract

The application relates to the technical field of computer vision, and provides a real-time binocular stereo matching method suitable for a pure CPU and related products, which comprises the following steps: taking a left camera image as a reference, combining a right camera image, performing feature point extraction and sparse matching, obtaining a sparse disparity map, and performing interpolation to generate a dense initial disparity; performing cost aggregation of a semi-global block matching algorithm in a cost aggregation range for each pixel to obtain a left disparity map; the cost aggregation range is an interval with the dense initial disparity value as a center and a deviation as a radius; and based on the left disparity map, re-projection is performed to obtain a three-dimensional point cloud and a depth map. The precision is unchanged, the calculation amount is greatly reduced, the CPU time consumption is obviously reduced, and no additional hardware such as a GPU and an FPGA is needed.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and in particular relates to a real-time binocular stereo matching method and related products applicable to pure CPU. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Binocular stereo matching calculates depth by measuring the disparity between the left and right images, and is one of the core methods for robot 3D perception. Semi-Global Block Matching (SGBM) has become the mainstream choice in industry due to its balance of accuracy and ease of implementation. However, SGBM is computationally intensive, requiring multi-directional dynamic programming aggregation for each pixel across the entire row. Its complexity is H×W×D×N_paths, where H represents the image height, W represents the image width, D represents the disparity search range (the maximum number of disparities the algorithm searches for in the horizontal direction), and N_paths refers to the number of cost aggregation directions (typically 4 or 8 directions). At 640×480 resolution, 96 disparities, and 4 / 8 directions, a single-core CPU typically takes 80-150ms, which is insufficient to meet the 30fps real-time requirement. Existing acceleration methods include GPU or FPGA (Programmable Gate Array) hardware acceleration schemes, as well as end-to-end disparity estimation methods based on neural networks.

[0004] In terms of hardware acceleration, existing solutions utilize heterogeneous computing platforms (such as a collaborative architecture of FPGA and ARM (Reduced Instruction Set Computing)) to design low-power real-time stereo matching systems, and combine them with inertial measurement units for synchronous data acquisition to enhance environmental perception capabilities. Other solutions leverage the parallel hardware logic resources of FPGAs to pipeline the stereo matching process to meet real-time requirements. Furthermore, methods such as StereoSGM (a semi-global stereo matching algorithm) based on CUDA (Unified Computing Device Architecture), the OpenCL (Open Computing Language) parallel framework, and dedicated FPGA accelerators can significantly reduce algorithm latency, but these methods incur additional hardware costs and power consumption, making them unsuitable for large-scale, low-cost deployment of unmanned platforms.

[0005] Regarding algorithm optimization, some studies have proposed a multi-scale real-time stereo matching method based on group distance. This method extracts features from input image pairs at multiple scales using a feature extractor based on block convolution, obtaining feature maps at different resolutions. Subsequently, an initial coarse disparity map is generated at the lowest resolution, and the disparity estimation is gradually refined through multiple stages of residual correction. The disparity maps output from each stage are finally upsampled and output. Meanwhile, while classic stereo matching networks (such as PSMNet and RAFT-Stereo) perform well in terms of accuracy, they are computationally intensive and require high-performance hardware such as GPUs (graphics processing units) for real-time inference. Furthermore, their training process relies on a large amount of labeled data, making deployment more challenging.

[0006] In summary, there is a lack of a universal solution that can accelerate CPU-side SGBM to ≥30fps without additional hardware, without sacrificing accuracy. Summary of the Invention

[0007] To address the technical problems mentioned above, this invention provides a real-time binocular stereo matching method and related products suitable for pure CPUs. It proposes a sparse-dense hybrid disparity prediction mechanism: sparse feature matching → interpolation → local SGBM refinement, which maintains the accuracy, greatly reduces the computational load, significantly reduces CPU time consumption, and eliminates the need for additional hardware such as GPUs and FPGAs.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides a real-time stereo matching method for pure CPU, comprising: Obtain the left and right images; Using the left image as a reference and combining it with the right image, feature point extraction and sparse matching are performed to obtain a sparse disparity map. Interpolation is then performed to generate a dense initial disparity. For each pixel, a cost aggregation algorithm of the semi-global block matching algorithm is run within the cost aggregation range to obtain the left disparity map. The cost aggregation range is an interval centered on the dense initial disparity value and with the deviation as the radius. Based on the left disparity map, it is reprojected into a 3D point cloud to obtain a depth map.

[0009] Furthermore, the feature point extraction includes: calculating the gradient value of each pixel in the left image using an isogradient operator; dividing the left image into several regions, and calculating the average gradient of each region based on the gradient values ​​of the pixels contained in each region; linearly mapping the average gradient to the detection threshold of the fast corner detection algorithm; after determining the detection threshold for each region, extracting corner points using the fast corner detection algorithm, and generating a binary feature vector for each corner point using the directional fast corner and rotation briefing descriptor.

[0010] Furthermore, the sparse matching includes: for each feature point in the left image, traversing all feature points in the epipolar line of the right image, calculating the matching confidence between two feature points using Hamming distance, and retaining matching points with high matching confidence.

[0011] Furthermore, it also includes: before interpolation, using a parabolic fitting method to refine the sub-pixel quality of the matching points in the sparse disparity map.

[0012] Furthermore, the method for determining the deviation is as follows: the left disparity map is compared pixel by pixel with the real disparity map, a set of different allowable error values ​​from small to large is set, and the proportion of pixels whose difference between the left disparity map and the real disparity map does not exceed the allowable error value under each allowable error value is counted. If there is an allowable error value that makes the proportion of qualified pixels reach the set level, then this allowable error value is determined as the deviation.

[0013] Furthermore, it also includes: calculating the boundary saturation rate R_boundary=(N_boundary) / (N_total)×100% based on the dense initial disparity and the precise disparity; where N_total is the total number of pixels participating in dense matching; N_boundary is the number of effective pixels that satisfy |D_final-D0|≥δ-0.5, D0 is the dense initial disparity, D_final is the precise disparity, and δ is the window size; if the boundary saturation rate exceeds the threshold, a semi-global block matching algorithm with no limit on the cost aggregation range is used as a fallback.

[0014] Furthermore, it also includes post-processing the left disparity map before reprojecting it into a 3D point cloud, including: using the right image as a reference, generating a right disparity map using a right matcher, performing a left-right consistency check, filtering the left disparity map after the left-right consistency check, and setting a pixel threshold to remove mismatches.

[0015] A second aspect of the present invention provides a real-time binocular stereo matching system suitable for pure CPU use, comprising: The image acquisition module is configured to acquire left and right images. The disparity calculation module is configured to: use the left image as a reference and combine it with the right image to perform feature point extraction and sparse matching to obtain a sparse disparity map, and then perform interpolation to generate a dense initial disparity; for each pixel, run the cost aggregation algorithm of the semi-global block matching algorithm within the cost aggregation range to obtain the left disparity map; the cost aggregation range is the interval centered on the dense initial disparity value and with the deviation as the radius; The matching module is configured to reproject the left disparity map into a 3D point cloud to obtain a depth map.

[0016] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a real-time binocular stereo matching method applicable to a pure CPU as described above.

[0017] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps of a real-time binocular stereo matching method applicable to a pure CPU as described above.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This invention proposes a sparse-dense hybrid disparity prediction mechanism: sparse feature matching → interpolation → local SGBM refinement, which maintains the same accuracy, greatly reduces the amount of computation, significantly reduces CPU time, and does not require additional hardware such as GPU or FPGA. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0020] Figure 1 This is a flowchart of a real-time binocular stereo matching method applicable to a pure CPU according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0022] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] Example 1 This embodiment provides a real-time binocular stereo matching method suitable for pure CPU.

[0024] Current stereo matching algorithms do not comprehensively consider issues such as high precision, real-time performance, low cost, and ease of deployment. This embodiment provides a real-time stereo matching method suitable for pure CPU. The core technical problem to be solved is to reduce the single matching time of SGBM from ≥80ms to ≤30ms under the premise of pure CPU, no additional hardware, and no reduction in resolution, while maintaining the depth accuracy RMSE <0.5 pixels.

[0025] This embodiment provides a real-time binocular stereo matching method suitable for pure CPUs. Based on the "sparse-prediction-dense" framework, it accelerates the traditional SGBM algorithm to real-time level on the CPU while ensuring matching accuracy. It can be widely used in online 3D perception or obstacle avoidance systems of computing-constrained platforms such as drones, unmanned forklifts, mobile robots, AR (augmented reality) / VR (virtual reality).

[0026] This embodiment provides a real-time stereo matching method suitable for pure CPU, such as... Figure 1 As shown, it includes the following steps: Step 1: Image Acquisition.

[0027] Using a binocular vision camera, camera parameters were set, and 1280 images were acquired. 480p, 30fps, left and right binocular stitched images, and the images are split into 640p. The left and right images are 480.

[0028] Step 2, Image Correction.

[0029] Using the intrinsic and extrinsic parameters obtained from offline calibration, distortion correction and epipolar alignment are performed on the left and right images, respectively, and grayscale images are output.

[0030] Step 3: Using the left image as a reference, feature point extraction and sparse matching are performed to obtain a sparse disparity map DS. Then, a parabolic fitting method is used to refine the sub-pixel of the matching points in the sparse disparity map.

[0031] Step 301: Use FAST+ORB to extract 3,000-5,000 feature points from the left image.

[0032] The number of feature points is automatically adjusted according to the image gradient. A high threshold is used in areas with rich texture to avoid over-density, while a low threshold is used in areas with weak texture to ensure detection. At the same time, spatial uniformity is maintained through non-maximum suppression.

[0033] Preferably, the steps for feature point extraction are as follows: (1) Use gradient operators such as Sobel to calculate the gradient value of each pixel in the left image.

[0034] (2) Calculate the average gradient of the entire left-side image based on the gradient value of each pixel. The calculation formula is as follows; then... Number of linear mappings to target feature points (3k-5k) serves as a reference for the number of global feature points; ; in, Indicates the image height. Indicates the image width. and Representing pixels The gradient values ​​in the x and y directions are calculated from step (1).

[0035] (3) For the left-hand image, according to 16 The 16-square grid is divided into several regions. Based on the gradient values ​​of the pixels contained in each region, the average gradient M of each region is calculated.

[0036] (4) Linearly map the average gradient M to the threshold T=α extracted by the Fast corner detection algorithm. M+β, where α and β are empirical parameters, and when determining α and β, the values ​​calculated in step (2) are used. For global constraints: By adjusting the values ​​of α and β, the total number of feature points detected in all regions of the entire image is made to approach a certain value. Candidate corner points are extracted using the FAST algorithm based on the detection threshold T for each region.

[0037] (5) For all candidate corner points detected in step (4), perform non-maximum suppression: In the 3×3 neighborhood of each candidate corner point, compare the FAST response values ​​of each candidate point (i.e., the sum of the absolute values ​​of the gray difference between the center pixel and the surrounding pixels), retain only the candidate point with the largest response value, and eliminate other weaker responses in the neighborhood, so as to ensure that the feature points are evenly distributed in the image space.

[0038] (6) Use ORB (Oriented Fast Corner and Rotation Briefing Descriptor) to generate a binary feature vector for each corner point for subsequent feature point matching.

[0039] Step 302: Perform epipolar search + Hamming distance matching on the right camera for each feature point, retain feature points with high confidence, and calculate the sparse disparity map DS.

[0040] The confidence score is used to filter matching points: for each feature point in the left image, all feature points are traversed in the epipolar line of the right image, and the Hamming distance is used to calculate the matching confidence score between two feature points. Each pair of matching points corresponds to a matching confidence score, and matching points with high matching confidence scores are retained.

[0041] The confidence level can be set as needed.

[0042] Step 303: Use the parabolic fitting method to refine the matching points in the sparse disparity map to sub-pixels with an accuracy of 0.1 pixels.

[0043] Step 4: Parallax prediction.

[0044] Using the sparse disparity map DS refined from subpixels as control points, Delaunay triangulation and linear interpolation of the centroid coordinates are employed to generate a dense initial disparity D0. This method is time-efficient on the CPU and provides good smoothing and edge preservation in areas with weak textures.

[0045] Step 5: Dense matching.

[0046] For each pixel, SGBM cost aggregation is rerun only within the cost aggregation range (dense initial disparity D0±δ) to obtain the accurate disparity (left disparity map).

[0047] The search range is changed from 96 to 9, which greatly reduces the amount of computation. The 8-directional aggregation is maintained, and the accuracy is consistent with the full range. The single-threaded complexity is 640×480×9×8≈29M, and the actual test result of Intel i7-12700H is 22–25ms.

[0048] The method for determining the deviation δ is as follows: run the dataset (e.g., KITTI (autonomous driving scenario dataset) / Middlebury (controlled static indoor environment dataset)) once, and count the proportion of predicted values ​​(left disparity map) that deviate from the true value by ≤k pixels, where k can be 2, 4, 6, or 8, and plot the curve; if 90% of the points fall within the value of k=4, the local SGBM window can be locked at ±4, i.e., δ=4 pixels, and the speedup ratio will be stable at more than 4 times.

[0049] Step 6: Quick rollback strategy.

[0050] Preserve the "fast rollback" path: When the predicted dense initial disparity D0 deviates significantly from the true disparity, directly roll back to the original full-range SGBM to avoid a large number of erroneous disparity outputs due to the failure of local search windows, while also preventing the overall frame rate from being lowered in extreme scenarios.

[0051] Specifically, after cost aggregation is completed in step 5 within the limited search range [D0-δ, D0+δ], the final output disparity D_final for each valid pixel is physically forced to be limited to this range. If the true disparity is much greater than D0+δ or much less than D0-δ, the algorithm tends to "truncate" the disparity results and pile them up on the two boundaries of the search window. Accordingly, this step defines the boundary saturation rate R_boundary, which is calculated as follows: R_boundary=(N_boundary) / (N_total)×100%; Where N_total is the total number of valid pixels in the left disparity map (i.e., the pixels participating in dense matching in step 3); N_boundary is the number of these valid pixels that satisfy |D_final-D0|≥δ-0.5 (i.e., the final disparity falls near the boundary of the search window, close to D0±δ). In actual engineering, given that the disparity map is floating-point data, the boundary judgment condition can be simplified to D_final≥D0+δ-0.5 or D_final≤D0-δ+0.5.

[0052] When R_boundary exceeds 40%~50%, it indicates that the dense initial disparity D0 has shifted globally from the true disparity surface, and the current preset local search range [D0-δ, D0+δ] cannot cover the true disparity of most pixels. At this point, if local SGBM is continued to be used, the disparity values ​​of nearly half of the pixels will be forcibly truncated at the boundary, resulting in a systematic error and severely contaminating the left-right consistency check and WLS filtering effect in the subsequent step 7.

[0053] Therefore, when R_boundary is detected to exceed the preset threshold (the preferred threshold in this embodiment is 50%), the fallback mechanism is immediately triggered: the local SGBM refinement process of the current frame is terminated, and the original SGBM algorithm with no limit on the cost aggregation range (i.e., the full disparity range [0, D_max]) is directly switched to rematch, so as to ensure the global correctness of the output disparity map in extreme scenarios.

[0054] Step 7: Post-processing.

[0055] Step 701: Use the right matcher to generate the right disparity DR, perform left-right consistency check, and remove the left disparity map of mismatched points.

[0056] Steps 3-6 involve calculating the matching disparity of each pixel in the left image on the right image, resulting in the precise disparity, which is the left disparity. The right matcher, conversely, uses the right image as a reference to calculate the cost on the left image, thus obtaining the right disparity. Theoretically, the sum of the left and right disparities should be infinitely close to 0. Step 701 verifies the accuracy of the left disparity by removing mismatched points where the difference between the left and right disparities is greater than 1, marking the disparity as 0, and finally obtaining the left disparity map after removing mismatched points.

[0057] Step 702: For the left disparity map that has passed the left-right consistency test, use WLS (Weighted Least Squares) filtering (λ=6000, σ=2.0) to smooth the edges. The filtering parameters are set as λ=6000 and σ=2.0; λ is the global smoothing intensity regularization parameter, controlling the overall smoothness of the disparity map; a larger value results in stronger smoothing. σ is the gradient sensitivity parameter of the guide map (original left-view grayscale image), controlling the sensitivity to edge preservation; a smaller value indicates greater sensitivity to grayscale changes and stronger edge preservation ability.

[0058] Step 703: For the filtered left disparity map, set a certain pixel threshold to eliminate mismatches.

[0059] Among them, pixel threshold elimination of mismatches refers to setting a disparity threshold (e.g., 32 pixels) to eliminate isolated outliers that remain after steps 701 and 702 and have excessive disparity differences from the neighborhood.

[0060] Step 704: For the left disparity map after mismatch removal, sequentially take each valid pixel, and perform 3D reprojection of its pixel coordinates (u,v) and corresponding disparity value d with the Q matrix obtained from offline calibration to calculate the 3D spatial coordinates (X,V) of the pixel in the world coordinate system. 3D ,Y 3D Z 3D The calculation formula is as follows: a 3D point cloud is constructed from the 3D coordinates corresponding to all valid pixels, and the Z coordinate value of each pixel is calculated. 3D The depth value is output as a depth map.

[0061] ; ; ; ; The Q matrix is ​​calibrated for the stereo camera and provides the camera's geometric projection relationship (focal length, baseline, principal point, etc.); X, Y, Z, and W represent intermediate calculation results in homogeneous coordinates, which are normalized by dividing by W to eliminate the projection scale factor.

[0062] This embodiment provides a real-time binocular stereo matching method suitable for pure CPU, proposing a sparse-dense hybrid disparity prediction mechanism: first sparse feature matching → TPS interpolation → local SGBM refinement, which greatly reduces the amount of computation, maintains the accuracy, and significantly reduces CPU time consumption.

[0063] This embodiment provides a real-time binocular stereo matching method suitable for pure CPUs, proposing a method for dynamically generating local search windows: adjusting the minimum and maximum disparity in real time based on the TPS prediction value ±δ, without the need for a global search.

[0064] This embodiment provides a real-time binocular stereo matching method suitable for pure CPU, proposing an adaptive feature point density strategy: automatically adjusting the FAST threshold according to the image gradient to ensure that control points are evenly distributed in weak texture areas.

[0065] This embodiment provides a real-time stereo matching method for pure CPU, which can be implemented in real time on a pure CPU: single thread 30fps@640×480, without the need for additional hardware such as GPU or FPGA.

[0066] Example 2 This embodiment provides a real-time binocular stereo matching system suitable for pure CPU use, comprising: The image acquisition module is configured to acquire left and right images. The disparity calculation module is configured to: use the left image as a reference and combine it with the right image to perform feature point extraction and sparse matching to obtain a sparse disparity map, and then perform interpolation to generate a dense initial disparity; for each pixel, run the cost aggregation algorithm of the semi-global block matching algorithm within the cost aggregation range to obtain the left disparity map; the cost aggregation range is the interval centered on the dense initial disparity value and with the deviation as the radius; The matching module is configured to reproject the left disparity map into a 3D point cloud to obtain a depth map.

[0067] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0068] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a real-time binocular stereo matching method applicable to a pure CPU as described in Embodiment 1 above.

[0069] Example 4 This embodiment provides a computer device, such as... Figure 2 As shown, the system includes a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. The processor 1001, communication interface 1002, and computer-readable storage medium 1003 can be connected via a bus or other means. The communication interface 1002 is used to receive and transmit data. When the processor 1001 executes the program, it implements the steps of the real-time binocular stereo matching method suitable for a pure CPU as described in Embodiment 1 above.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A real-time binocular stereo matching method suitable for pure CPU, characterized in that, include: Obtain the left and right images; Based on the left image, combined with the right image, feature points are extracted and sparse matching is performed to obtain a sparse disparity map, and interpolation is performed to generate a dense initial disparity. For each pixel, the cost aggregation of the semi-global block matching algorithm is run within the cost aggregation range, and the resulting accurate disparity is the left disparity map; the cost aggregation range is the interval centered on the dense initial disparity value and with the deviation as the radius; Based on the dense initial disparity and the precise disparity, the boundary saturation rate R_boundary = (N_boundary) / (N_total) × 100% is calculated; where N_total is the total number of pixels participating in dense matching; N_boundary is the number of effective pixels that satisfy |D_final-D0|≥δ-0.5, D0 is the dense initial disparity, D_final is the precise disparity, and δ is the window size; if the boundary saturation rate exceeds the threshold, a semi-global block matching algorithm with no limit on the cost aggregation range is used as a fallback. Based on the left disparity map, it is reprojected into a 3D point cloud to obtain a depth map; The feature point extraction includes: calculating the gradient value of each pixel in the left image using the Sobel gradient operator; and calculating the average gradient of the entire left image based on the gradient value of each pixel. Then Number of linear mappings to target feature points The number of global feature points is used as a reference. For the left image, it is divided into several regions. Based on the gradient values ​​of the pixels contained in each region, the average gradient M of each region is calculated. The average gradient M is then linearly mapped to the threshold T=α extracted by the high-speed corner detection algorithm. M+β, where α and β are empirical parameters, and when determining α and β, we use... As a global constraint, by adjusting the values ​​of α and β, the total number of feature points detected in all regions of the entire image is made to approach a certain value. After determining the detection threshold for each region, corner points are extracted using the high-speed corner detection algorithm. A binary feature vector is generated for each corner point using the directional fast corner point and rotation briefing descriptor for subsequent feature point matching. The method for determining the deviation is as follows: compare the dense initial disparity map with the real disparity map pixel by pixel, set a set of different allowable error values ​​from small to large, and count the proportion of pixels whose difference between the dense initial disparity map and the real disparity map does not exceed the allowable error value under each allowable error value. If there is an allowable error value that makes the proportion of qualified pixels reach the set level, then this allowable error value is determined as the deviation.

2. The real-time stereo matching method suitable for pure CPU as described in claim 1, characterized in that, The sparse matching includes: for each feature point in the left image, traversing all feature points in the epipolar line of the right image, calculating the matching confidence between two feature points using Hamming distance, and retaining matching points with high matching confidence.

3. The real-time stereo matching method for pure CPU as described in claim 1, characterized in that, Also includes: Before interpolation, a parabolic fitting method is used to refine the sub-pixel quality of the matching points in the sparse disparity map.

4. The real-time stereo matching method suitable for pure CPU as described in claim 1, characterized in that, It also includes post-processing the left disparity map before reprojecting it into a 3D point cloud, including: using the right image as a reference, generating a right disparity map with a right matcher, performing a left-right consistency check, filtering the left disparity map after the left-right consistency check, and setting a pixel threshold to remove mismatches.

5. A real-time binocular stereo matching system suitable for pure CPU, characterized in that, include: The image acquisition module is configured to acquire left and right images. The disparity calculation module is configured to: use the left image as a reference, combine it with the right image, extract feature points and perform sparse matching to obtain a sparse disparity map, and then perform interpolation to generate a dense initial disparity; For each pixel, a cost aggregation algorithm of the semi-global block matching algorithm is run within the cost aggregation range to obtain the left disparity map; the cost aggregation range is the interval centered on the dense initial disparity value and with the deviation as the radius; Based on the dense initial disparity and the precise disparity, the boundary saturation rate R_boundary = (N_boundary) / (N_total) × 100% is calculated; where N_total is the total number of pixels participating in dense matching; N_boundary is the number of effective pixels that satisfy |D_final-D0|≥δ-0.5, D0 is the dense initial disparity, D_final is the precise disparity, and δ is the window size; if the boundary saturation rate exceeds the threshold, a semi-global block matching algorithm with no limit on the cost aggregation range is used as a fallback. The matching module is configured to reproject the left disparity map into a 3D point cloud to obtain a depth map. The feature point extraction includes: calculating the gradient value of each pixel in the left image using the Sobel gradient operator; and calculating the average gradient of the entire left image based on the gradient value of each pixel. Then Number of linear mappings to target feature points The number of global feature points is used as a reference. For the left image, it is divided into several regions. Based on the gradient values ​​of the pixels contained in each region, the average gradient M of each region is calculated. The average gradient M is then linearly mapped to the threshold T=α extracted by the high-speed corner detection algorithm. M+β, where α and β are empirical parameters, and when determining α and β, we use... As a global constraint, by adjusting the values ​​of α and β, the total number of feature points detected in all regions of the entire image is made to approach a certain value. After determining the detection threshold for each region, corner points are extracted using the high-speed corner detection algorithm. A binary feature vector is generated for each corner point using the directional fast corner point and rotation briefing descriptor for subsequent feature point matching. The method for determining the deviation is as follows: compare the dense initial disparity map with the real disparity map pixel by pixel, set a set of different allowable error values ​​from small to large, and count the proportion of pixels whose difference between the dense initial disparity map and the real disparity map does not exceed the allowable error value under each allowable error value. If there is an allowable error value that makes the proportion of qualified pixels reach the set level, then this allowable error value is determined as the deviation.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in a real-time stereo matching method suitable for a pure CPU as described in any one of claims 1-4.

7. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the real-time binocular stereo matching method applicable to a pure CPU as described in any one of claims 1-4.

Images