Stereoscopic vision depth estimation method for drainage wire live working robot
By combining Gaussian weighted SAD and improved Census transformation methods, combined with cost aggregation of guide filtering, the depth estimation error problem of drainage line live-operated work robots in weak texture and light-sensitive areas is solved, achieving higher precision depth perception and job safety.
Patent Information
- Application Number
- CN202510337339.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-21
- Publication Date
- 2025-07-08
AI Technical Summary
In the prior art, in the drainage line live operation robot, the depth perception algorithm based on binocular stereo vision has high error and instability problems in weak texture areas and light-sensitive areas, which affects the work accuracy and safety.
The Gaussian-weighted SAD algorithm is combined with the improved Census transformation, combined with the cost aggregation method based on guide filtering, and the accuracy and stability of the initial matching cost are improved by performing calculations and parallax calculations in the HSV color space. The winner is the king algorithm is used to generate the parallax result, and finally the depth estimation result is restored through the binocular stereoscopic visual depth calculation formula.
The depth estimation accuracy of the drainage line live operation robot in large areas of weak texture and light change areas is improved, and the safety, reliability and efficiency of the operation are enhanced.
Smart Images
Figure CN120279074A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of substation operations, and more specifically, to a method for stereo vision depth estimation of a live working robot for bypass conductors. Background Art
[0002] In order to enable the live working robot for bypass conductors to complete operation tasks more intelligently and safely, it is necessary to have the ability to accurately perceive the depth level of the surrounding environment in real time. The stereo vision depth perception system meets the operation requirements of the live working robot for bypass conductors due to its high accuracy and stable algorithm. Currently, the depth perception algorithm based on binocular stereo vision has also developed to a certain extent; among them:
[0003] A stereo matching algorithm based on improved Census transform and feature fusion uses the neighborhood pixels of the transformation window to replace the central pixel to solve the problem that the traditional Census transform overly relies on the central pixel window. However, in weak texture areas and areas where noise is easily interfered, it still shows instability.
[0004] A stereo matching algorithm based on improved Census transform and adaptive parameter guided filtering is used to solve the problem of high matching error of the stereo matching depth estimation algorithm in areas sensitive to noise and light changes. However, in large areas of weak texture areas, it still shows a high error, which brings interference to the stereo vision depth estimation of the live working robot for bypass conductors.
[0005] In view of this, the present invention provides a method for stereo vision depth estimation of a live working robot for bypass conductors to solve the above problems. Summary of the Invention
[0006] In order to overcome the problems in the prior art, the present invention proposes a method for stereo vision depth estimation of a live working robot for bypass conductors, which overcomes the problem of high depth estimation error in large areas of weak texture areas, while maintaining the algorithm stability in light-sensitive areas and ensuring the algorithm robustness in depth-discontinuous areas.
[0007] In a first aspect, the present invention provides a method for stereo vision depth estimation of a live working robot for bypass conductors, including the following steps:
[0008] S1: Obtain operation scene images by using a binocular stereo camera carried by the live working robot for bypass conductors, and the operation scene images include a target left image and a target right image;
[0009] S2: Calculate the operation scene images in the HSV space by using the Gaussian weighted SAD algorithm and combine it with an improved Census transform cost calculation method to obtain an initial matching cost;
[0010] S3: Aggregate the initial matching cost obtained in step S2 using a cost aggregation method based on guided filtering to obtain an aggregated matching cost;
[0011] S4: Use the winner-takes-all algorithm to calculate the disparity for the aggregated matching cost obtained after processing in step S3 to obtain a matching disparity result;
[0012] S5: Calculate the matching disparity result obtained in step S4 using the depth calculation formula of binocular stereo vision, and recover the depth estimation result of the working scene of the live working robot for the drainage line.
[0013] As a preferred technical solution of the first aspect of the present invention, the acquisition logic of the initial matching cost:
[0014] S21: Use an image processing library to convert the working scene image obtained by the binocular camera from the RGB color space to the HSV color space;
[0015] S22: In the HSV color space, calculate the SAD absolute difference for the pixel region corresponding to the working scene image, and use Gaussian weighting to assign different weights to the pixels in the local neighborhood window to obtain a Gaussian-weighted SAD cost;
[0016] S23: In the HSV color space, apply an improved Census transform to the left and right images respectively to generate Census feature descriptors;
[0017] S24: Weightedly fuse the Gaussian-weighted SAD cost and the improved Census transform cost to generate an initial matching cost. By combining the SAD algorithm and the Census transform algorithm, the matching accuracy of both texture-rich regions and weak-texture regions can be taken into account.
[0018] As a preferred technical solution of the first aspect of the present invention, for each pixel point of the working scene image, define a local neighborhood window, calculate the SAD absolute difference of the HSV components of each pixel within the local neighborhood window; use a Gaussian kernel function to assign weights to the pixels within the window; sum the weighted absolute differences to obtain a Gaussian-weighted SAD cost.
[0019] As a preferred technical solution of the first aspect of the present invention, for each pixel point of the working scene image, define a local neighborhood window, compare the HSV components of each pixel within the local neighborhood window with the HSV components of the central pixel. If the HSV component of the pixel is greater than that of the central pixel, it is 1; otherwise, it is 0.
[0020] As a preferred technical solution of the first aspect of the present invention, normalize the Gaussian-weighted SAD cost and the Census transform cost.
[0021] As a preferred technical solution of the first aspect of the present invention, the acquisition logic of the cost aggregation matching cost is as follows:
[0022] Step 31: Prepare a guiding image. Taking the left image as the guiding image, convert the left image from the RGB color space to a grayscale image.
[0023] Step 32: Define a local neighborhood window, and calculate the local mean and variance of the guiding image within the window.
[0024] Step 33: Calculate the local linear coefficient. The local linear coefficient includes the local ratio, local difference, and regularization parameter within each local window, where:
[0025] The local ratio is the ratio of the covariance of the local information of the guiding image within the local window to the initial matching cost to the variance of the guiding image; the local difference is the difference between the local mean of the initial matching cost and the product of the local mean of the guiding image and the ratio; the regularization parameter is used to prevent overfitting and ensure the stability of the filtering result.
[0026] Step 34: Calculate the aggregated matching cost. For the multiple local windows corresponding to each pixel point, weight and average the results corresponding to the multiple local windows to obtain the aggregated cost aggregation matching cost.
[0027] As a preferred technical solution of the first aspect of the present invention, the acquisition logic of the matching disparity result is as follows:
[0028] Step 41: Based on the cost aggregation matching cost in step S3, traverse all possible disparity values for each pixel point.
[0029] Step 42: For each pixel point, select the disparity value corresponding to the minimum matching cost, and the disparity map corresponding to the combination of the disparity values of all pixel points is the matching disparity result.
[0030] As a preferred technical solution of the first aspect of the present invention, the acquisition logic of the depth estimation result is as follows:
[0031] S51: Calculate the depth value by using the depth calculation formula of binocular stereo vision for the matching disparity result.
[0032] S52: Mark the depth map corresponding to the combination of the depth values of all pixel points as the depth estimation result.
[0033] As a preferred technical solution of the first aspect of the present invention, the depth calculation formula of binocular stereo vision is based on the principle of triangulation, and the formula is depth value = focal length of the camera * baseline distance of the binocular camera / disparity value.
[0034] The specific advantages of the present invention are as follows:
[0035] The present invention can improve the depth estimation accuracy of the live working robot for bypass conductors in the operation process, and improve the working efficiency and safety reliability of the live working robot for bypass conductors. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a flowchart of the visual depth estimation method of the present invention; DETAILED DESCRIPTION OF THE EMBODIMENTS
[0037] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0038] Embodiment 1
[0039] Please refer to Figure 1 , the present invention provides a technical solution: a stereo vision depth estimation method for a live working robot for bypass conductors, including the following steps:
[0040] S1: Use the binocular stereo camera carried by the live working robot for bypass conductors to obtain the operation scene images, and the operation scene images include the target left image and the target right image;
[0041] S2: Use the SAD algorithm with Gaussian weighting to calculate the operation scene images in the HSV space, and combine it with the improved Census transform cost calculation method to obtain the initial matching cost;
[0042] It should be noted that the SAD (Sum of Absolute Difference) algorithm can optimize the weights of the pixels in the local neighborhood window. Combining the SAD algorithm with Gaussian weighting and the improved Census transform, and calculating in the HSV color space can effectively improve the calculation accuracy of the initial matching cost of the live working robot for bypass conductors in weak texture regions, depth discontinuous regions, and illumination change regions. This method takes into account the matching requirements of both texture-rich regions and weak texture regions, laying a solid foundation for subsequent cost aggregation and depth estimation.
[0043] Specifically, the acquisition logic of the initial matching cost:
[0044] S21: Use an image processing library to convert the job scene image obtained by the binocular camera from the RGB color space to the HSV color space. This can solve the problem that the RGB color space is sensitive to light changes, while the HSV color space separates the hue, saturation, and value of the image, enabling better handling of light changes and weak texture areas.
[0045] S22: In the HSV color space, calculate the SAD absolute difference for the pixel region corresponding to the job scene image, and use Gaussian weighting to assign different weights to the pixels in the local neighborhood window to obtain the Gaussian-weighted SAD cost. The Gaussian weighting method can assign weights to the pixels in the local neighborhood window to optimize the accuracy of the absolute difference and SAD (Sum of Absolute Difference), improving the matching accuracy for large areas of weak texture regions. On this basis, calculating the cost of the Gaussian-weighted SAD (Sum of Absolute Difference) algorithm in the HSV (Hue, Saturation, Value) color space, that is, the hue, saturation, and bright color space, can effectively improve the robustness to noise interference in large areas of weak texture regions and depth discontinuity regions. Different from the traditional SAD algorithm that assigns the same weight to all pixels in the local window and is easily affected by noise interference, Gaussian weighting can assign a higher weight to the central pixel and reduce the influence of noise. The specific means are as follows:
[0046] For each pixel point in the job scene image, define a local neighborhood window (such as 5x5 or 7x7).
[0047] Calculate the SAD absolute difference of the HSV components (H, S, V) of each pixel in the local neighborhood window.
[0048] Use a Gaussian kernel function to assign weights to the pixels in the window, with the highest weight for the central pixel and gradually decreasing weights for the edge pixels.
[0049] Sum the weighted absolute differences to obtain the Gaussian-weighted SAD cost.
[0050] S23: In the HSV color space, apply the improved Census transform to the left image and the right image respectively to generate Census feature descriptors. Different from the traditional Census transform that overly relies on the central pixel, the improved Census transform reduces the dependence on the central pixel by utilizing the statistical characteristics of neighboring pixels, improving the stability in weak texture regions. Specifically:
[0051] For each pixel point in the operation scene image, a local neighborhood window is defined. The HSV components of each pixel within the local neighborhood window are compared with those of the central pixel to generate a binary code (e.g., 1 if the pixel value is greater than the central pixel, otherwise 0).
[0052] The binary codes are concatenated to form a Census feature descriptor; the Hamming distance between the Census feature descriptors of corresponding pixels in the left image and the right image is calculated as the matching cost of the Census transform.
[0053] S24: The Gaussian-weighted SAD cost and the improved Census transform cost are weighted and fused to generate an initial matching cost. By combining the SAD algorithm and the Census transform algorithm, the matching accuracy in both texture-rich regions and weak-texture regions can be taken into account. Specifically:
[0054] The Gaussian-weighted SAD cost and the Census transform cost are normalized.
[0055] They are combined using weighting coefficients (such as α and β) to generate the initial matching cost:
[0056] Initial matching cost = α × SAD cost + β × Census cost Initial matching cost = α × SAD cost + β × Census cost
[0057] The weighting coefficients can be adjusted according to the specific scenario to balance the contributions of SAD and Census.
[0058] In this embodiment, in weak-texture regions, the statistical characteristics of the Census transform can provide more stable matching results, while the Gaussian-weighted SAD algorithm reduces noise interference through weighting; in depth-discontinuous regions, the Gaussian-weighted SAD algorithm can retain edge information, while the Census transform enhances the matching accuracy of edges through the comparison of neighboring pixels; the Gaussian-weighted SAD (Sum of Absolute Difference) algorithm calculates the cost for the input image in the HSV color space and combines it with the improved Census transform cost calculation method, overcoming the drawback of poor accuracy of existing algorithms in large-area weak-texture regions. Calculating SAD and Census costs in the HSV color space can effectively reduce the influence of illumination changes on the matching results.
[0059] S3: The cost aggregation method based on guided filtering is used to perform cost aggregation on the initial matching cost obtained in step S2 to obtain the cost-aggregated matching cost;
[0060] It should be noted that there is noise or inconsistency in the stereo image matching cost, and the guided filter can smooth the cost while retaining edge information; the matching cost is filtered by a reference image (such as the left image) to retain edge details while smoothing the noise.
[0061] Specifically, the acquisition logic of the cost aggregation matching cost is as follows:
[0062] Step 31: Prepare a guidance image. Taking the left image as the guidance image, convert the left image from the RGB color space to a grayscale image (if the guided filter is performed in the grayscale space).
[0063] Step 32: Define a local neighborhood window, and calculate the local mean and variance of the guidance image within the window.
[0064] Step 33: Calculate the local linear coefficients, which include the local ratio, local difference, and regularization parameter within each local window; where:
[0065] The local ratio is the ratio of the covariance of the local information of the guidance image and the initial matching cost within the local window to the variance of the guidance image; the local difference is the difference between the local mean of the initial matching cost and the product of the local mean of the guidance image and the ratio; the regularization parameter is used to prevent overfitting and ensure the stability of the filtering result.
[0066] The local ratio reflects the local linear relationship between the guidance image and the initial matching cost. If the guidance image and the initial matching cost are highly correlated within the local window, the value of akak will be larger, indicating that the guidance image has a stronger influence on the matching cost; the local difference is an offset used to adjust the filtered matching cost to keep it consistent with the initial matching cost within the local window; through the local linear coefficients, the guided filter can smooth the noise in the initial matching cost, especially in weakly textured regions and regions with illumination changes; in regions with depth discontinuities (such as object edges), the guided filter can retain the edge information in the guidance image to ensure a sharper transition of the matching cost in these regions.
[0067] Step 34: Calculate the aggregated matching cost. For multiple local windows corresponding to each pixel point, weight and average the results corresponding to the multiple local windows to obtain the aggregated cost aggregation matching cost.
[0068] Through the cost aggregation method based on the guided filter, the noise in the initial matching cost can be effectively smoothed while retaining the edge information in the depth discontinuity regions. This method shows good performance in weakly textured regions, depth discontinuity regions, and regions with illumination changes, providing high-quality matching costs for subsequent disparity calculation and depth estimation.
[0069] S4: Use the winner-takes-all algorithm to calculate the disparity for the cost aggregation matching cost obtained after the processing in step S3, and obtain the matching disparity result;
[0070] It should be noted that for the winner-takes-all algorithm (WTA, Winner Takes All), for each pixel, the algorithm selects the disparity value corresponding to the minimum matching cost as the final disparity of the pixel, assuming that the disparity of each pixel is unique and the disparity corresponding to the minimum matching cost is optimal; at each pixel position, the disparity value corresponding to the minimum matching cost is selected to generate a disparity map.
[0071] Specifically, the acquisition logic of the matching disparity result is as follows:
[0072] Step 41: Based on the cost aggregation matching cost in step S3, traverse all possible disparity values for each pixel;
[0073] Step 42: For each pixel, select the disparity value corresponding to the minimum matching cost, and the disparity map corresponding to the combination of the disparity values of all pixels is the matching disparity result.
[0074] S5: Use the depth calculation formula of binocular stereo vision to calculate the matching disparity result obtained in step S4, and restore the depth estimation result of the live working robot operation scene of the drainage line.
[0075] It should be noted that the disparity information is converted into actual depth information to provide accurate environmental perception data for the robot. According to the baseline distance and focal length of the binocular cameras, the disparity value is converted into a depth value.
[0076] Specifically, the acquisition logic of the depth estimation result:
[0077] S51: Calculate the depth value by using the depth calculation formula of binocular stereo vision for the matching disparity result;
[0078] The depth calculation formula of binocular stereo vision is based on the principle of triangulation, and the formula is depth value (i.e., the distance from the object to the camera) = focal length of the camera (in pixels) * baseline distance of the binocular cameras (i.e., the horizontal distance between the left and right cameras) divided by the disparity value (i.e., the horizontal displacement of the corresponding pixels in the left and right images).
[0079] S52: Mark the depth map obtained by combining the depth values of all pixels as the depth estimation result.
[0080] In this embodiment, the disparity map is converted into actual depth information through the depth calculation formula of binocular stereo vision, providing accurate environmental perception data for the robot. The depth map can be used to reconstruct the three-dimensional structure of the operation scene, assisting the robot in path planning and obstacle avoidance. In the substation operation environment, it is applied to the operation technology field of live working robots for bypass conductors in the field of intelligent robot operations, improving the operation efficiency and safety of live working robots for bypass conductors, and being able to provide effective technical guarantees for the safety maintenance and repair of substations.
[0081] To improve the quality of the depth map, post-processing of the depth map can be performed, such as using median filtering or bilateral filtering to remove noise, or filling in missing depth values through interpolation methods. For complex scenes, multi-scale depth estimation methods can be adopted to calculate the depth map at different resolutions and combine multi-scale information to generate more accurate depth estimation results.
[0082] In summary, this embodiment uses the Gaussian-weighted SAD (Sum of Absolute Difference) algorithm to calculate the input image in the HSV color space and combines it with an improved Census transform cost calculation method to calculate the initial matching cost between the left and right images obtained by the stereo vision of the live working robot for bypass conductors. The Gaussian-weighted method can assign weights to the pixels in the local neighborhood window to optimize the accuracy of the sum of absolute differences (SAD), improving the matching accuracy for large areas of weak texture regions. On this basis, calculating the cost of the Gaussian-weighted SAD algorithm in the HSV (Hue, Saturation, Value) color space, that is, the hue, saturation, and bright color space, can effectively improve the robustness to noise interference in large areas of weak texture regions and depth discontinuity regions. Secondly, this method uses a cost aggregation algorithm based on guided filtering for cost aggregation, which can further improve the matching accuracy in the scene edge region, that is, improve the depth estimation performance in the depth discontinuity region. Using the method of this embodiment, more accurate depth estimation results can be calculated in large areas of weak texture regions and depth discontinuity regions in the substation operation environment of the live working robot for bypass conductors.
[0083] As described above, the above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claimed rights.
[0084] Finally, the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.
Claims
1. A method for stereo vision depth estimation of a live working robot for drainage lines, characterized in that, It includes the following steps: S1: Use the binocular stereo camera carried by the live working robot for the drainage line to obtain the operation scene images, which include the target left image and the target right image; S2: Use the Gaussian-weighted SAD algorithm to calculate the operation scene images in the HSV color space, and combine it with the improved Census transform cost calculation method to obtain the initial matching cost; S3: Use the cost aggregation method based on guided filtering to perform cost aggregation on the initial matching cost obtained in step S2 to obtain the cost aggregation matching cost; S4: Use the winner-takes-all algorithm to calculate the disparity of the cost aggregation matching cost obtained after the processing in step S3 to obtain the matching disparity result; S5: Use the depth calculation formula of binocular stereo vision to calculate the matching disparity result obtained in step S4, and restore the depth estimation result of the operation scene of the live working robot for the drainage line.
2. The method for stereo vision depth estimation of a live working robot for drainage lines according to claim 1, wherein The acquisition logic of the initial matching cost: S21: Use the image processing library to convert the operation scene images obtained by the binocular camera from the RGB color space to the HSV color space; S22: In the HSV color space, calculate the SAD absolute difference of the pixel region corresponding to the operation scene images, and use Gaussian weighting to assign different weights to the pixels in the local neighborhood window to obtain the Gaussian-weighted SAD cost; S23: In the HSV color space, apply the improved Census transform to the left image and the right image respectively to generate the Census feature descriptors; S24: Perform weighted fusion on the Gaussian-weighted SAD cost and the improved Census transform cost to generate the initial matching cost. By combining the SAD algorithm and the Census transform algorithm, the matching accuracy of both the texture-rich region and the weak-texture region can be taken into account.
3. The stereo vision depth estimation method for a live working robot of a drainage line according to claim 2, characterized in that For each pixel point of the operation scene images, define a local neighborhood window, calculate the SAD absolute difference of the HSV components of each pixel in the local neighborhood window; use the Gaussian kernel function to assign weights to the pixels in the window; sum the weighted absolute differences to obtain the Gaussian-weighted SAD cost.
4. A method for stereo vision depth estimation of a live working robot for drainage lines according to claim 3, characterized in that, For each pixel point of the operation scene images, define a local neighborhood window, compare the HSV components of each pixel in the local neighborhood window with the HSV components of the central pixel. If the HSV component of the pixel is greater than the HSV component of the central pixel, it is 1, otherwise it is 0.
5. A method for stereo vision depth estimation of a live working robot for drainage lines according to claim 4, characterized in that, Normalize the Gaussian-weighted SAD cost and the Census transform cost.
6. A method for stereo vision depth estimation of a live working robot for a drainage line, according to claim 5, characterized in that The acquisition logic of the cost aggregation matching cost: Step 31: Prepare the guidance image. Use the left image as the guidance image and convert the left image from the RGB color space to a grayscale image; Step 32: Define the local neighborhood window and calculate the local mean and variance of the guidance image within the window; Step 33: Calculate the local linear coefficients, where the local linear coefficients include the local ratio, local difference, and regularization parameter within each local window, where: The local ratio is the ratio of the covariance of the local information of the guidance image within the local window to the initial matching cost to the variance of the guidance image; the local difference is the difference between the local mean of the initial matching cost and the product of the local mean of the guidance image and the ratio; the regularization parameter is used to prevent overfitting and ensure the stability of the filtering result; Step 34: Calculate the aggregated matching cost. For the multiple local windows corresponding to each pixel point, weight and average the results corresponding to the multiple local windows to obtain the aggregated cost, the aggregated matching cost.
7. A stereo vision depth estimation method for a live working robot of a drainage line, according to claim 6, characterized in that, The acquisition logic of the matching disparity result is as follows: Step 41: Based on the cost-aggregated matching cost in step S3, traverse all possible disparity values for each pixel point; Step 42: For each pixel point, select the disparity value corresponding to the minimum matching cost, and the disparity map corresponding to the combination of the disparity values of all pixel points is the matching disparity result.
8. A stereo vision depth estimation method for a live working robot of a drainage line, according to claim 7, characterized in that The acquisition logic of the depth estimation result: S51: Calculate the depth value of the matching disparity result using the depth calculation formula of binocular stereo vision; S52: Mark the depth map formed by combining the depth values of all pixel points as the depth estimation result.
9. A stereo vision depth estimation method for a live working robot of a drainage line, according to claim 8, wherein The depth calculation formula of the binocular stereo vision is based on the principle of triangulation, and the formula is depth value = focal length of the camera * baseline distance of the binocular camera divided by the disparity value.