3D road edge detection and distance estimation method based on binocular stereo vision, electronic equipment and readable storage medium

Abstract

The invention belongs to the technical field of unmanned system environment perception, and particularly discloses a 3D road edge detection and distance estimation method based on binocular stereoscopic vision, electronic equipment and a readable storage medium, and the method comprises the steps: S1, constructing a binocular vision geometric model, and carrying out the preprocessing of left and right view images obtained by a binocular camera; s2, constructing a multi-feature fusion stereo matching model based on the preprocessed image, and obtaining a high-precision disparity map; s3, generating a U / V disparity map according to the high-precision disparity map for dynamic adaptive ROI extraction; s4, converting the U / V disparity map into a 3D point cloud to obtain a continuous road edge point cloud; s5, introducing a conditional neural process model, complementing the occlusion region, and outputting a complete 3D road edge curve; s6, dynamic correction is carried out based on the distance mean value of the 3D road edge curve feature points, and road edge distance errors are corrected; and S7, updating the road edge core parameters based on the Kalman filtering tracking model, and obtaining an updated road edge distance detection result.

Description

Technical Field

[0001] This invention belongs to the field of environmental perception technology for unmanned systems, specifically relating to a 3D curb detection and distance estimation method based on binocular stereo vision, an electronic device, and a readable storage medium. Background Technology

[0002] As the physical boundary of a road, the three-dimensional shape and distance information of the curb are core environmental perception data for intelligent mobile platforms to perform path planning, obstacle avoidance, and lane keeping, directly affecting driving safety and control accuracy.

[0003] Currently, curb detection and distance estimation technologies are mainly divided into three categories: monocular vision solutions, lidar solutions, and traditional binocular stereo vision solutions. However, all three types of solutions have obvious technical shortcomings, specifically:

[0004] 1. Monocular vision solution: This solution can only detect the contour of the two-dimensional road edge. The distance estimation of the road edge depends on the preset road width, road surface smoothness and other prior parameters. It cannot adapt to complex scenarios such as road surface undulation and road edge shape changes. The actual distance measurement error between this solution and the road edge is generally greater than 10%, and the robustness is extremely poor.

[0005] 2. LiDAR solution: This solution can directly acquire 3D point cloud data and realize roadside detection and ranging. However, the hardware cost of the LiDAR solution is high. The price of a single LiDAR is usually 10 to 20 times that of a binocular camera. In addition, this solution has blind spots at close range when in actual use. The point cloud density in the roadside area is easily affected by vehicle body and vegetation. It is difficult to meet the application needs of large-scale industrialization.

[0006] 3. Traditional binocular stereo vision solution: This solution generates a disparity map through stereo matching to achieve 3D reconstruction. However, this solution has three major pain points: First, stereo matching often relies on single features, such as color or gradient, which can easily lead to matching failures in complex scenes such as backlighting and repetitive textures, resulting in low accuracy of the disparity map. Second, the extraction parameters of the region of interest (ROI) are fixed and cannot adapt to different road widths and shooting angles. In actual testing, it will include a lot of background interference or missed road edges. Third, it lacks an effective completion mechanism for occluded road edges, and the distance calculation does not take into account the influence of road slope and camera installation height deviation, resulting in a high road edge detection miss rate (≥15%) and a large distance measurement error (≥8%). Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a 3D curb detection and distance estimation method, electronic device, and readable storage medium based on binocular stereo vision.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0009] The first aspect of this invention provides a 3D curb detection and distance estimation method based on binocular stereo vision, comprising the following steps:

[0010] S1. Construct a binocular vision geometric model and preprocess the left and right view images obtained by the binocular camera.

[0011] S2. Based on the preprocessed image, construct a multi-feature fusion stereo matching model with Census transform, truncated gradient and color features to obtain a high-precision disparity map.

[0012] S3. Generate a U / V disparity map based on the high-precision disparity map for dynamic adaptive ROI extraction.

[0013] S4. Convert the U / V disparity map into a 3D point cloud to obtain a continuous roadside point cloud.

[0014] S5. Introduce a conditional neural process model to complete the occluded area and output a complete 3D curb curve.

[0015] S6. Dynamically correct the roadside distance error based on the average distance of feature points of the 3D roadside curve.

[0016] S7. Update the core parameters of the roadside based on the Kalman filter tracking model to obtain the updated roadside distance detection results.

[0017] According to the above-mentioned 3D curb detection and distance estimation method based on binocular stereo vision, preferably, in step S1, the step of constructing the binocular vision geometric model is as follows: obtain the intrinsic and extrinsic parameters of the binocular camera based on the Zhang Zhengyou calibration method. The intrinsic parameters include the focal length f and principal point coordinates (u0, v0) of the binocular camera, and the extrinsic parameters include the rotation matrix R and translation matrix T of the binocular camera. Establish the binocular vision geometric model based on the intrinsic and extrinsic parameters of the binocular camera.

[0018] The preprocessing steps for the left and right view images are as follows:

[0019] CLAHE image enhancement is performed on the left and right view images obtained by the binocular camera based on the binocular vision geometric model. The CLAHE image enhancement parameters are:

[0020]

[0021] Where, σ noise The mean gray-level variance of the 5×5 sliding window of the image before preprocessing is σ0 = 20;

[0022] The denoised and enhanced image is obtained after median filtering and Gaussian filtering.

[0023] The kernel size of the median filter is 3×3 (for low noise levels) or 5×5 (for high noise levels); the standard deviation of the Gaussian filter is... Wherein, σ0 represents the baseline gray-scale variance (a preset reference value used to normalize the noise level).

[0024] According to the above-mentioned 3D road edge detection and distance estimation method based on binocular stereo vision, preferably, in step S2, the step of obtaining a high-precision disparity map using a multi-feature fusion stereo matching model includes:

[0025] First, calculate the cost:

[0026] Cost = w C ·C+w G ·G+w Color • Color;

[0027] Where Cost represents the overall matching cost, used to measure the degree of matching of pixels in a binocular image (the smaller the value, the higher the matching degree); C represents the Census transform cost (or a similar local matching cost), used to characterize the structural similarity of the pixel neighborhood; G represents the gradient feature cost, a matching metric based on image gradient information, enhancing the matching robustness of edge regions; Color represents the color feature cost, a matching metric based on pixel color information, adapted to textured regions; w C w G w Color Let w represent the weights of the multi-feature fusion. C ∈[0.3,0.5]、w G ∈[0.2,0.4]、w Color ∈[0.2,0.3], w C w G w Color The weights sum to 1; experiments have shown that this range improves matching accuracy by ≥20% compared to other weight combinations.

[0028] Then perform stage gradient calculation:

[0029] T = k·μ;

[0030] Where T represents the gradient threshold, used for truncation control in gradient calculation; μ is the average gray value of the image, used to solve the gradient saturation problem under backlight; k represents the adjustment coefficient, k∈[1.0,1.5] used to dynamically adjust the threshold and solve the matching error problem caused by gradient value saturation in backlight scenes.

[0031] Then, multi-scale split tree aggregation is performed: the weights of the s-th layer are:

[0032]

[0033] Where i represents the index of the i-th level of the multi-scale split tree.

[0034] Nodes are generated by Euclidean distance clustering of 4×4 pixel blocks, which improves the efficiency by 25% compared with uniform weight aggregation; after left-right consistency detection and weighted median filtering, a high-precision disparity map is output.

[0035] The deviation threshold for the left-right consistency detection is:

[0036]

[0037] Where τ is the deviation threshold; d max This represents the maximum disparity value.

[0038] According to the above-described 3D road edge detection and distance estimation method based on binocular stereo vision, preferably, in step S3, the step of generating a U / V disparity map based on a high-precision disparity map includes:

[0039] S301. Obtaining the road surface slope: The road surface slope is obtained by fitting the pixel distribution of the V disparity map.

[0040] S302. Vanishing point detection: Three sets of intersection points of parallel road edges are obtained through Hough transform. If the ordinate of the intersection point is lower than 1 / 2 of the image height, it is corrected to 1 / 2 of the image height (based on normal scene statistics). This correction is used to avoid unreasonable vanishing point positions and ensure the accuracy of subsequent region analysis.

[0041] S303, ROI parameter optimization: The left boundary ratio of the ROI region is 0.6 to 0.9 times the ratio of the vanishing point x-coordinate to the image width. The vertical angle of the ROI region is compensated according to the road slope (90°±15°). The top cropping height of the ROI region is the minimum value of the longitudinal distribution along the road edge minus 10 to 30 pixels. This design reduces the ROI region by 30% and decreases the false negative rate by 50%.

[0042] According to the above-mentioned 3D road edge detection and distance estimation method based on binocular stereo vision, preferably, in step S4, the 3D point cloud conversion step is as follows:

[0043] Based on a binocular vision geometric model, a high-precision disparity map is converted into a 3D point cloud (X, Y, Z) (the conversion relationship conforms to the basic principles of binocular vision); hierarchical clustering and RANSAC fitting are performed on the 3D point cloud; the hierarchical clustering of the 3D point cloud is based on the road surface plane, and is divided into layers according to the height difference (2cm for continuous curbs, 5cm for stepped curbs), and the point cloud density retained in the hierarchical clustering is ≥3 points / cm. 2 The candidate layer for roadside points; the RANSAC fitting of the 3D point cloud requires 1000 iterations, and the inlier threshold for RANSAC fitting is 2cm, to obtain a continuous roadside point cloud.

[0044] According to the above-mentioned 3D curb detection and distance estimation method based on binocular stereo vision, preferably, in step S5, the step of optimizing the 3D curb morphology using the conditional neural process model (CNP) is as follows:

[0045] S501, Training Data: Based on 5000 pairs of left and right view images with 10% to 50% occlusion, the visible point cloud and the actual occlusion contour of the LiDAR are labeled.

[0046] S502. Input 3D curb features: 3D curb features include local curb slope (fitted by a 5-point sliding window), curb curve curvature (calculated based on the slope derivative) and Euclidean distance between adjacent points.

[0047] S503, Smooth Fusion: The conditional neural process model adopts a structure of 3-layer convolutional encoder (3×3 kernels, 64→128→256 channels) + 2-layer fully connected decoder, which reduces the completion error by 35% compared with the traditional model; the inferred contour and visible point cloud are processed by Gaussian process regression (RBF kernel) to optimize the 3D curb morphology and output a complete 3D curb curve.

[0048] The conditional neural process model was trained using mean squared error loss, iterated for 50 rounds, with a learning rate of 0.001, and the validation set completion error after convergence was ≤3cm.

[0049] According to the above-mentioned 3D curb detection and distance estimation method based on binocular stereo vision, preferably, in step S6, feature points are selected uniformly along the length direction of the 3D curb curve, the adjacent spacing of the feature points is ≤20cm, and the number of feature points is ≥5 (≥10 when longer than 5m); the initial distance of the feature point is selected as: the average Euclidean distance from the feature point to the camera origin; after obtaining the average distance of the feature point, the average distance of the feature point is dynamically corrected: the average distance of the feature point is corrected according to the road slope and the installation height of the binocular camera.

[0050] According to the above-mentioned 3D road edge detection and distance estimation method based on binocular stereo vision, preferably, in step S7, the Kalman filter tracking model is initialized according to state variables and iteratively updated based on state transition and observation equations to ensure inter-frame stability, thereby obtaining the updated road edge distance detection result; the state variables include road edge slope, road edge curve curvature, and corrected road edge distance; the initialization is performed based on a preset initial covariance matrix, and the detection result of the first frame of the initialization is the initial state; the iterative update is performed based on the state transition and observation equations, and the process noise and observation noise in the iterative update are set based on static data statistics, with a tracking error ≤5%.

[0051] The state transition equation is:

[0052] X k =A·X k-1 +wk ;

[0053] Among them, X k Let X represent the state vector at time k. k-1 Let w represent the state vector at time k-1. k It is process noise.

[0054] The observation equation is:

[0055] Z k =H·X k +v k ;

[0056] Among them, Z k Let v represent the observation vector at time k. k To observe noise, A is a diagonal matrix (elements 0.95-0.99), and H is an identity matrix, used to ensure smooth transition of parameters between frames.

[0057] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any step of the 3D curb detection and distance estimation method based on binocular stereo vision as described in the first aspect.

[0058] A third aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a computer processor, implements any step in the 3D curb detection and distance estimation method based on binocular stereo vision as described in the first aspect.

[0059] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0060] 1. This invention addresses the problems of insufficient robustness in detection, missed detection of occluded road edges, and large ranging errors in existing technologies. First, it establishes a binocular visual geometric model using the Zhang Zhengyou calibration method, and performs adaptive enhancement and denoising preprocessing on the left and right view images of the binocular camera. Second, it constructs a multi-feature fusion stereo matching model based on Census transform, truncated gradients, and color features, and optimizes the disparity map through dynamic weight allocation and multi-scale aggregation. Then, it combines vanishing point detection and road slope fitting to achieve dynamic adaptive ROI extraction, reducing background interference. Next, it generates a 3D point cloud based on binocular geometric principles, and obtains a continuous road edge point cloud through hierarchical clustering and RANSAC fitting. It introduces a conditional neural process model (CNP) to complete the occluded area by combining the 3D geometric features of the road edge (local slope, curvature, and distance between adjacent points), and optimizes the 3D road edge morphology through Gaussian process regression. Based on the dynamic correction of the mean distance between feature points, road slope, and camera height, it achieves high-precision ranging. Finally, it uses Kalman filtering to track the core parameters of the road edge, ensuring inter-frame stability.

[0061] 2. Experimental verification shows that the method of the present invention has a detection accuracy of ≥92% in backlight, repetitive texture, and roadside occlusion scenarios, an occlusion completion error of ≤3cm, a ranging error of ≤3%, and an inter-frame tracking error of ≤5%. It can be adapted to the road boundary three-dimensional shape perception and spatial distance accurate measurement scenarios of intelligent mobile platforms such as unmanned vehicles, intelligent inspection robots, and agricultural machinery navigation. It has the advantages of strong robustness, high accuracy, and low cost.

[0062] 3. This invention has the advantages of strong robustness, high detection accuracy, and controllable cost. It can solve the shortcomings of existing technologies in terms of adaptability to complex scenarios, occlusion handling, and ranging accuracy, and meet the actual application needs of intelligent mobile platforms. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the process of the present invention in Example 1;

[0064] Figure 2 This is a schematic diagram of the overall logic of the present invention in Embodiment 1;

[0065] Figure 3 This is a before-and-after comparison diagram of the point cloud completion effect for the occluded roadside in Example 1;

[0066] Figure 4 U / V parallax in Example 1 Figure 2 A diagram illustrating the effect of value-based conversion. Detailed Implementation

[0067] The present invention will be further described in detail below through specific embodiments, but this does not limit the scope of the present invention.

[0068] Example 1

[0069] A 3D curb detection and distance estimation method based on binocular stereo vision, such as Figure 1-4 As shown, it includes the following steps:

[0070] S1. Construct a binocular vision geometric model and preprocess the left and right view images obtained by the binocular camera.

[0071] In step S1, the steps for constructing the binocular vision geometric model are as follows: the intrinsic and extrinsic parameters of the binocular camera are obtained based on the Zhang Zhengyou calibration method. The intrinsic parameters include the focal length f and principal point coordinates (u0, v0) of the binocular camera, and the extrinsic parameters include the rotation matrix R and translation matrix T of the binocular camera. The binocular vision geometric model is then established based on the intrinsic and extrinsic parameters of the binocular camera.

[0072] The preprocessing steps for the left and right view images are as follows:

[0073] CLAHE image enhancement is performed on the left and right view images obtained by the binocular camera based on the binocular vision geometric model. The CLAHE image enhancement parameters are:

[0074]

[0075] Where, σ noise The mean gray-level variance of the 5×5 sliding window of the image before preprocessing is σ0 = 20;

[0076] The denoised and enhanced image is obtained after median filtering and Gaussian filtering.

[0077] The kernel size of the median filter is 3×3 (for low noise levels) or 5×5 (for high noise levels); the standard deviation of the Gaussian filter is... Wherein, σ0 represents the baseline gray-scale variance (a preset reference value used to normalize the noise level).

[0078] S2. Based on the preprocessed image, construct a multi-feature fusion stereo matching model with Census transform, truncated gradient and color features to obtain a high-precision disparity map.

[0079] In step S2, the steps of obtaining a high-precision disparity map using a multi-feature fusion stereo matching model include:

[0080] First, calculate the cost:

[0081] Cost = w C ·C+w G ·G+w Color • Color;

[0082] Where Cost represents the overall matching cost, used to measure the degree of matching of pixels in a binocular image (the smaller the value, the higher the matching degree); C represents the Census transform cost (or a similar local matching cost), used to characterize the structural similarity of the pixel neighborhood; G represents the gradient feature cost, a matching metric based on image gradient information, enhancing the matching robustness of edge regions; Color represents the color feature cost, a matching metric based on pixel color information, adapted to textured regions; w C w G w Color Let w represent the weights of the multi-feature fusion. C ∈[0.3,0.5]、w G ∈[0.2,0.4]、w Color ∈[0.2,0.3], w C w G w Color The weights sum to 1; experiments have shown that this range improves matching accuracy by ≥20% compared to other weight combinations.

[0083] Then perform stage gradient calculation:

[0084] T = k·μ;

[0085] Where T represents the gradient threshold, used for truncation control in gradient calculation; μ is the average gray value of the image, used to solve the gradient saturation problem under backlight; k represents the adjustment coefficient, k∈[1.0,1.5] used to dynamically adjust the threshold and solve the matching error problem caused by gradient value saturation in backlight scenes.

[0086] Then, multi-scale split tree aggregation is performed: the weights of the s-th layer are:

[0087]

[0088] Where i represents the index of the i-th level of the multi-scale split tree.

[0089] Nodes are generated by Euclidean distance clustering of 4×4 pixel blocks, which improves the efficiency by 25% compared with uniform weight aggregation; after left-right consistency detection and weighted median filtering, a high-precision disparity map is output.

[0090] The deviation threshold for the left-right consistency detection is:

[0091]

[0092] Where τ is the deviation threshold; d max This represents the maximum disparity value.

[0093] The optimal weight for multi-feature fusion is w C The weight of the Census transformation cost is 0.4, w G The weights for the gradient truncation cost are 0.3 and w. Color The weight of color feature cost is 0.3, the matching accuracy is 92% for backlit scenes and 88% for repetitive texture scenes, which is ≥15% higher than other combinations.

[0094] S3. Generate a U / V disparity map based on the high-precision disparity map for dynamic adaptive ROI extraction.

[0095] In step S3, the step of generating a U / V disparity map based on the high-precision disparity map includes:

[0096] S301. Obtaining the road surface slope: The road surface slope is obtained by fitting the pixel distribution of the V disparity map.

[0097] S302. Vanishing point detection: Three sets of intersection points of parallel road edges are obtained through Hough transform. If the ordinate of the intersection point is lower than 1 / 2 of the image height, it is corrected to 1 / 2 of the image height (based on normal scene statistics). This correction is used to avoid unreasonable vanishing point positions and ensure the accuracy of subsequent region analysis.

[0098] S303, ROI parameter optimization: The left boundary ratio of the ROI region is 0.6 to 0.9 times the ratio of the vanishing point x-coordinate to the image width. The vertical angle of the ROI region is compensated according to the road slope (90°±15°). The top cropping height of the ROI region is the minimum value of the longitudinal distribution along the road edge minus 10 to 30 pixels. This design reduces the ROI region by 30% and reduces the false negative rate by 50%.

[0099] S4. Convert the U / V disparity map into a 3D point cloud to obtain a continuous roadside point cloud.

[0100] In step S4, the 3D point cloud conversion steps are as follows:

[0101] Based on a binocular vision geometric model, a high-precision disparity map is converted into a 3D point cloud (X, Y, Z) (the conversion relationship conforms to the basic principles of binocular vision); hierarchical clustering and RANSAC fitting are performed on the 3D point cloud; the hierarchical clustering of the 3D point cloud is based on the road surface plane, and is divided into layers according to the height difference (2cm for continuous curbs, 5cm for stepped curbs), and the point cloud density retained in the hierarchical clustering is ≥3 points / cm. 2 The candidate layer for roadside points; the RANSAC fitting of the 3D point cloud requires 1000 iterations, and the inlier threshold for RANSAC fitting is 2cm, to obtain a continuous roadside point cloud.

[0102] S5. Introduce a conditional neural process model to complete the occluded area and output a complete 3D curb curve.

[0103] In step S5, the steps of optimizing the 3D curb morphology using the Conditional Neural Process Model (CNP) are as follows:

[0104] S501, Training Data: Based on 5000 pairs of left and right view images with 10% to 50% occlusion, the visible point cloud and the actual occlusion contour of the LiDAR are labeled.

[0105] S502. Input 3D curb features: 3D curb features include local curb slope (fitted by a 5-point sliding window), curb curve curvature (calculated based on the slope derivative) and Euclidean distance between adjacent points.

[0106] S503, Smooth Fusion: The conditional neural process model adopts a structure of 3-layer convolutional encoder + 2-layer fully connected decoder; Gaussian process regression is used to process and infer the contour and visible point cloud, optimize the 3D curb morphology, and output a complete 3D curb curve.

[0107] The conditional neural process model was trained using mean squared error loss, iterated for 50 rounds, with a learning rate of 0.001, and the validation set completion error after convergence was ≤3cm.

[0108] S6. Dynamically correct the roadside distance error based on the average distance of feature points of the 3D roadside curve.

[0109] In step S6, feature points are first uniformly selected along the length of the 3D road curve. The spacing between adjacent feature points is ≤20cm, and the number of feature points is ≥5 (≥10 when the length is greater than 5m). The initial distance of the feature points is selected as the average Euclidean distance from the feature point to the camera origin. After obtaining the average distance of the feature points, the average distance of the feature points is dynamically corrected: the average distance of the feature points is corrected according to the road slope and the installation height of the binocular camera (the greater the slope and the greater the height deviation, the greater the correction amount), and the final distance error is ≤3%.

[0110] S7. Update the core parameters of the roadside based on the Kalman filter tracking model to obtain the updated roadside distance detection results.

[0111] In step S7, the Kalman filter tracking model is initialized based on state variables and iteratively updated based on state transition and observation equations to ensure inter-frame stability, thereby obtaining the updated roadside distance detection result. The state variables include roadside slope, roadside curve curvature, and corrected roadside distance. The initialization is performed based on a preset initial covariance matrix, and the detection result of the first frame of the initialization is the initial state. The iterative update is performed based on state transition and observation equations, and the process noise and observation noise in the iterative update are set based on static data statistics.

[0112] The state transition equation is:

[0113] X k =A·X k-1 +w k ;

[0114] Among them, X k Let X represent the state vector at time k. k-1 Let w represent the state vector at time k-1. k It is process noise.

[0115] The observation equation is:

[0116] Z k =H·X k +v k ;

[0117] Among them, Z k Let v represent the observation vector at time k. k To observe noise, A is a diagonal matrix (elements 0.95-0.99), and H is an identity matrix, used to ensure smooth transition of parameters between frames.

[0118] Example 2

[0119] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any step of the 3D curb detection and distance estimation method based on binocular stereo vision as described in Embodiment 1.

[0120] Furthermore, the 3D curb detection and distance estimation method based on binocular stereo vision described in Embodiment 1 can be implemented as a computer software program. For example, this embodiment includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method. In such an embodiment, the computer program can be downloaded and installed from a network, and / or installed from a removable medium. When the computer program is executed by a processor, it performs the functions defined in the method of this application.

[0121] Example 3

[0122] A computer-readable storage medium storing a computer program that, when executed by a processor, implements any step of a method for implementing a 3D curb detection and distance estimation based on binocular stereo vision as described in Embodiment 1.

[0123] The computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0124] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Python and C++, as well as conventional procedural programming languages ​​or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0125] In this embodiment, the computer-readable storage medium can be accelerated using hardware such as a GPU. The parallel computing advantage of the GPU is used to accelerate any step in the implementation of a 3D curb detection and distance estimation method based on binocular stereo vision as described in Embodiment 1.

[0126] In summary, this invention effectively overcomes the shortcomings of the prior art and has high industrial applicability. The above embodiments are intended to illustrate the substantive content of this invention, but are not intended to limit the scope of protection of this invention. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this invention without departing from the essence and scope of protection of this invention.

[0127] The above embodiments are specific implementations of the present invention, but the implementation of the present invention is not limited to the above embodiments. Any other combination, change, modification, substitution, or simplification that does not exceed the design concept of the present invention shall fall within the protection scope of the present invention.

Claims

1. A 3D curb detection and distance estimation method based on binocular stereo vision, characterized in that, Includes the following steps: S1. Construct a binocular vision geometric model and preprocess the left and right view images obtained by the binocular camera; S2. Construct a multi-feature fusion stereo matching model based on the preprocessed image to obtain a high-precision disparity map; S3. Generate a U / V disparity map based on the high-precision disparity map for dynamic adaptive ROI extraction; S4. Convert the U / V disparity map into a 3D point cloud to obtain a continuous roadside point cloud; S5. Introduce a conditional neural process model to complete the occluded area and output a complete 3D curb curve; S6. Dynamically correct the roadside distance error based on the average distance of feature points of the 3D roadside curve. S7. Update the core parameters of the roadside based on the Kalman filter tracking model to obtain the updated roadside distance detection results.

2. The 3D curb detection and distance estimation method based on binocular stereo vision according to claim 1, characterized in that, In step S1, the steps for constructing the binocular vision geometric model are as follows: the intrinsic and extrinsic parameters of the binocular camera are obtained based on the Zhang Zhengyou calibration method. The intrinsic parameters include the focal length f and principal point coordinates (u0, v0) of the binocular camera, and the extrinsic parameters include the rotation matrix R and translation matrix T of the binocular camera. The binocular vision geometric model is then established based on the intrinsic and extrinsic parameters of the binocular camera. The preprocessing steps for the left and right view images are as follows: CLAHE image enhancement is performed on the left and right view images obtained by the binocular camera based on the binocular vision geometric model. The CLAHE image enhancement parameters are: Where, σ noise The mean of the grayscale variance of the image before preprocessing; The denoised and enhanced image is obtained after median filtering and Gaussian filtering. The kernel size of the median filter is 3×3 or 5×5; the standard deviation of the Gaussian filter is... Where σ0 represents the baseline grayscale variance.

3. The 3D curb detection and distance estimation method based on binocular stereo vision according to claim 1, characterized in that, In step S2, the steps of obtaining a high-precision disparity map using a multi-feature fusion stereo matching model include: First, calculate the cost: Cost=w C ·C+w G ·G+w Color ·Color; Where Cost represents the overall matching cost, used to measure the degree of matching of pixels in a binocular image; C represents the Census transform cost, used to characterize the structural similarity of the pixel neighborhood; G represents the gradient feature cost, a matching metric based on image gradient information, enhancing the matching robustness of edge regions; Color represents the color feature cost, a matching metric based on pixel color information, adapted to textured regions; w C w G w Color Let w represent the weights of the multi-feature fusion. C ∈[0.3,0.5]、w G ∈[0.2,0.4]、w Color ∈[0.2,0.3], w C w G w Color The weights sum to 1; Then perform stage gradient calculation: T = k·μ; Where T represents the gradient threshold, used for truncation control in gradient calculation; μ is the average gray value of the image, used to solve the gradient saturation problem under backlighting; k represents the adjustment coefficient, k∈[1.0,1.5] used to dynamically adjust the threshold; Then, multi-scale split tree aggregation is performed: the weights of the s-th layer are: Where i represents the index of the i-th level of the multi-scale splitting tree; Nodes are generated through Euclidean distance clustering; after left-right consistency detection and weighted median filtering, a high-precision disparity map is output. The deviation threshold for the left-right consistency detection is: Where τ is the deviation threshold; d max This represents the maximum disparity value.

4. The 3D curb detection and distance estimation method based on binocular stereo vision according to claim 1, characterized in that, In step S3, the step of generating a U / V disparity map based on the high-precision disparity map includes: S301. Obtaining the road surface slope: The road surface slope is obtained by fitting the pixel distribution of the V disparity map; S302. Vanishing point detection: The intersection of parallel road edges is obtained through Hough transform. If the ordinate of the intersection is lower than 1 / 2 of the image height, it is corrected to 1 / 2 of the image height. This correction is used to avoid unreasonable vanishing point positions and ensure the accuracy of subsequent region analysis. S303, ROI parameter optimization: The left boundary ratio of the ROI region is 0.6 to 0.9 times the ratio of the vanishing point x-coordinate to the image width. The vertical angle of the ROI region is compensated according to the road slope. The top cropping height of the ROI region is the minimum value of the longitudinal distribution along the road edge minus 10 to 30 pixels.

5. The 3D curb detection and distance estimation method based on binocular stereo vision according to claim 1, characterized in that, In step S4, the 3D point cloud conversion steps are as follows: A high-precision disparity map is converted into a 3D point cloud based on a binocular vision geometric model; hierarchical clustering and RANSAC fitting are then performed on the 3D point cloud; the hierarchical clustering of the 3D point cloud is based on the road surface plane and is divided into layers according to the height difference, and the point cloud density retained in the hierarchical clustering is ≥3 points / cm. 2 The candidate layer for roadside points; the RANSAC fitting of the 3D point cloud requires 1000 iterations, and the inlier threshold for RANSAC fitting is 2cm, to obtain a continuous roadside point cloud.

6. The 3D curb detection and distance estimation method based on binocular stereo vision according to claim 1, characterized in that, In step S5, the steps for optimizing the 3D curb morphology using the conditional neural process model are as follows: S501, Training data: Based on left and right view images with 10% to 50% occlusion, the visible point cloud and the actual occlusion contour of the LiDAR are labeled. S502, Input 3D curb features: 3D curb features include local curb slope, curb curve curvature, and Euclidean spacing between adjacent points; S503, Smooth Fusion: The conditional neural process model adopts a structure of 3-layer convolutional encoder + 2-layer fully connected decoder; Gaussian process regression is used to process and infer the contour and visible point cloud, optimize the 3D curb morphology, and output a complete 3D curb curve.

7. The 3D curb detection and distance estimation method based on binocular stereo vision according to claim 1, characterized in that, In step S6, feature points are selected uniformly along the length of the 3D roadside curve, with the adjacent spacing of feature points ≤20cm and the number of feature points ≥5. The initial distance of the feature points is selected as the average Euclidean distance from the feature point to the camera origin. After obtaining the average distance of the feature points, the average distance of the feature points is dynamically corrected: the average distance of the feature points is corrected according to the road slope and the installation height of the binocular camera.

8. The 3D curb detection and distance estimation method based on binocular stereo vision according to claim 1, characterized in that, In step S7, the Kalman filter tracking model is initialized based on state variables and iteratively updated based on state transition and observation equations to ensure inter-frame stability, thereby obtaining the updated roadside distance detection result; the state variables include roadside slope, roadside curve curvature, and corrected roadside distance; The initialization is performed based on a preset initial covariance matrix, and the detection result of the first frame of the initialization is the initial state. The iterative update is performed based on the state transition and observation equations, and the process noise and observation noise in the iterative update are set based on static data statistics. The state transition equation is: X k =A·X k-1 +w k ; Among them, X k Let X represent the state vector at time k. k-1 Let w represent the state vector at time k-1. k It is process noise; The observation equation is: Z k =H·X k +v k ; Among them, Z k Let v represent the observation vector at time k. k For observing noise, A is a diagonal matrix and H is an identity matrix, used to ensure smooth transition of parameters between frames.

9. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements any step in the 3D curb detection and distance estimation method based on binocular stereo vision as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer processor, implements any step in the 3D curb detection and distance estimation method based on binocular stereo vision as described in any one of claims 1-8.

Images