Local stereo matching method based on adaptive window and edge features

Abstract

The invention provides a local stereo matching method based on a self-adaptive window and edge features, and the method comprises the steps: automatically adjusting the size of a matching window according to a gray mean value and a gradient mean value around a matching pixel point, determining the pixel point participating in the calculation of a gray cost value through a region growing method, and forming a self-adaptive matching window; acquiring left and right images of the binocular camera, and calculating by using a Canny operator to obtain left and right edge feature maps; in the adaptive matching window, performing logical operation on corresponding pixel points of the left and right edge feature maps, and calculating edge feature matching cost; performing cost aggregation on the SAD matching cost and the edge feature matching cost to obtain a matching cost value; and traversing all matched candidate points in the left image and the right image to obtain a parallax value of the whole image. The invention aims to improve the performance of a local stereo matching algorithm in a weak texture region and improve the robustness of the algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a local stereo matching method based on adaptive windows and edge features. Background Art

[0002] Stereo matching is a widely used technology in computer vision. Its core goal is to recover the depth information of a real scene from a two-dimensional image. This process typically involves finding matching pairs of points in two or more images of the same scene and then determining the disparity between these points. Once the disparity information is accurately obtained, the depth of the physical point in space corresponding to the point pair can be calculated using triangulation principles.

[0003] As a core step in key technologies such as 3D reconstruction, stereo navigation, and non-contact ranging, stereo matching obtains depth information by matching two or more images. Due to its simple implementation and low cost, stereo matching technology has shown great application potential in many fields. In the field of industrial production automation, stereo matching technology can achieve precise positioning and measurement of objects on the production line; in assembly line control, it helps to improve production efficiency and accuracy; in the field of autonomous vehicles, stereo matching is an important cornerstone for achieving ranging and navigation functions; in security monitoring, stereo matching technology can enhance the intelligent recognition and analysis capabilities of monitoring systems; in remote sensing image analysis, it helps to more accurately extract terrain information; and in the field of intelligent robot control, stereo matching can enhance the robot's environmental perception and autonomous navigation capabilities.

[0004] However, existing local stereo matching algorithms face some challenges in their application. These algorithms mainly use the grayscale information within the region to calculate the matching similarity, but this method is easily affected by illumination and noise. Especially in weak texture areas, due to the insignificant changes in grayscale information, the matching effect of the local stereo matching algorithm is poor, and it is difficult to accurately obtain disparity information. In addition, the robustness of the local stereo matching algorithm is also a key factor affecting the accuracy of depth calculation. In practical applications, factors such as changes in lighting conditions, noise interference, and differences in image resolution may affect the robustness of the algorithm, resulting in deviations in the depth calculation results. Traditional SAD matching algorithms use a fixed-size matching window. If a small window is used, the information of the surrounding pixels cannot be fully utilized in the weak texture area, affecting the matching accuracy. If a large window is used, the computational complexity will increase and a lot of interference information will be introduced.

[0005] Therefore, in order to address the problems of existing local stereo matching algorithms, it is of great significance to develop a stereo matching algorithm that can maintain good matching results in weak texture areas and has strong robustness. Summary of the Invention

[0006] In view of the above shortcomings of the prior art, the present invention provides a local stereo matching method based on adaptive windows and edge features, aiming to improve the performance of the local stereo matching algorithm in weak texture areas and enhance the robustness of the algorithm.

[0007] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0008] A local stereo matching method based on adaptive window and edge features comprises the following steps:

[0009] Set the initial matching window with the current matching point as the center;

[0010] The matching window size is automatically adjusted according to the grayscale mean and gradient mean around the matching pixel point, and the pixel points involved in the grayscale cost calculation are determined using the region growing method to form an adaptive matching window.

[0011] Use a binocular camera to obtain the left and right images to be matched, and apply the Canny operator to perform edge detection respectively to obtain the left and right edge feature maps containing edge information;

[0012] In the adaptive matching window, logical operations are performed on the corresponding pixels of the left and right edge feature maps to calculate the edge feature matching cost;

[0013] Design a matching cost aggregation function that integrates grayscale features and edge features, aggregate the SAD matching cost and edge feature matching cost to obtain a matching cost value;

[0014] Traverse all matching candidate points in the left and right images, select the right / left matching candidate point with the smallest matching cost as the right / left matching point that matches the left / right matching point, and use the position difference between the left and right matching points at this time as the disparity to obtain the disparity value of the entire image.

[0015] According to a local stereo matching method based on an adaptive window and edge features provided by the present invention, the adjustment process of the adaptive matching window includes:

[0016] Calculate the grayscale variance of the pixels within the matching window;

[0017] Determine whether the grayscale variance is greater than the set variance threshold and dynamically adjust the size of the matching window based on the judgment result.

[0018] According to a local stereo matching method based on adaptive window and edge features provided by the present invention, the grayscale variance S is calculated according to the following formula: 2 (p,q):

[0019]

[0020] Where N is the number of pixels in the matching window; (x, y) is the pixel in the matching window; W(p, q, s) is the matching window with (p, q) as the center and side length s; S 2 (p,q) is the grayscale variance within the matching window; I(x,y) is the pixel grayscale value; is the average gray value in the matching window.

[0021] According to a local stereo matching method based on adaptive windows and edge features provided by the present invention, the calculated grayscale variance is compared with a preset variance threshold S. If the grayscale variance is less than the variance threshold S, it is determined that the size of the current matching window is insufficient to fully reflect the texture features. The length and width of the matching window are each increased by 1 pixel unit, and the grayscale variance calculation process is repeated until the grayscale variance is greater than or equal to the variance threshold S. At this time, it is determined that the window meets the requirements of a strong texture area and is set as the matching window W.

[0022] Within the determined matching window W, the current matching point is used as the initial seed point to start the region growing process. This process is achieved by comparing the pixel value of the seed point with the pixel values of the adjacent pixels. If the difference between the pixel value of the adjacent pixel and the pixel value of the seed point is greater than the preset threshold T, the adjacent pixel is added to the current growing region.

[0023] The newly added pixel point is used as a new seed point and the region growing process is continued until no new pixel point can be added or the growing region reaches the boundary of the matching window W. The region obtained at this time is the grayscale matching region R.

[0024] According to a local stereo matching method based on an adaptive window and edge features provided by the present invention, the step of calculating the edge feature matching cost includes:

[0025] Normalize the pixel values of the left and right edge feature maps, convert the pixel values at the edge from 255 to 1, and keep the pixel values at non-edges as 0;

[0026] In the normalized left and right edge images, according to the size and position of the adaptive matching window W determined previously, an "AND" logical operation is performed on the pixels of the left and right edge images within the corresponding matching window. That is, only when the pixel values of the left and right edge images at the same position are both 1, that is, both are edge pixels, the output of this position is 1, otherwise it is 0, thus obtaining the edge matching cost map, which is used to reflect the consistency of the edge features of the left and right images within the matching window;

[0027] A bitwise sum operation is performed on all pixel values in the edge matching cost map, and the sum result is used as the edge feature matching cost. This cost value quantifies the similarity of the edge features of the left and right images within the matching window.

[0028] According to a local stereo matching method based on adaptive window and edge features provided by the present invention, the calculation formula of the edge feature matching cost is expressed as the following formula:

[0029]

[0030] Among them, C Edge (p,q,d) is the edge feature matching cost when the disparity is d; W(p,q,s) is the matching window centered at (p,q) and with a size of s; E l (x, y) is the pixel value in the left edge image matching window, which is 0 or 1; & is the 'and' logical operation; E r (x,y+d) is the pixel value within the right edge image matching window, which is 0 or 1.

[0031] According to a local stereo matching method based on adaptive window and edge features provided by the present invention, the matching cost aggregation function integrating the SAD matching cost and the edge feature matching cost is expressed as the following formula:

[0032]

[0033] in, is the absolute value matching cost of the grayscale difference within the grayscale matching area R of the binocular image; is the average gray value in the matching window W; C Edge is the edge feature matching cost.

[0034] According to a local stereo matching method based on adaptive window and edge features provided by the present invention, when performing cost aggregation and generating a disparity map, the method includes the following steps:

[0035] a) For each pixel in the left image, it is considered as a left matching point, and a right matching point with an initial position is set on the corresponding row in the right image;

[0036] b) For the current left matching point, move the right matching point from left to right pixel by pixel on its row. After each move, calculate the SAD matching cost between the current left matching point and the right matching point within the adaptive matching window W, and calculate the edge feature matching cost in the corresponding edge matching cost map;

[0037] c) According to the preset matching cost aggregation function, the calculated SAD matching cost and edge feature matching cost are aggregated to obtain the comprehensive matching cost at the current right matching position;

[0038] d) Repeat steps b) and c) until the right matching point moves to the column position corresponding to the preset maximum disparity range. Record the comprehensive matching costs of all right matching positions during this process, and select the position with the lowest comprehensive matching cost as the best matching position for the current left matching point. The difference between this position and the column position of the left matching point is the desired disparity.

[0039] e) Repeat steps a) to d) for all pixels in the left image. After the traversal is completed, a disparity value map of the entire image is generated based on the disparity of each left matching point.

[0040] According to a local stereo matching method based on an adaptive window and edge features provided by the present invention, when calculating the SAD matching cost between the current left matching point and the right matching point within the adaptive matching window W, the method includes the following steps:

[0041] For a left matching point in the left image, determine a corresponding right matching point in the right image, and obtain a grayscale matching area R centered on the two matching points and determined according to the adaptive window W;

[0042] In the grayscale matching area R, for each pixel in the left image, calculate the grayscale value difference between it and the corresponding pixel in the right image, and take the absolute value of the difference to obtain the SAD value of the pixel;

[0043] Add the SAD values of all pixels in the grayscale matching area R to obtain the total SAD matching cost of the left and right matching points in the area R;

[0044] When the right matching point moves pixel by pixel in the right image, the SAD matching costs of the left and right matching points in region R under different disparities are calculated.

[0045] According to a local stereo matching method based on adaptive windows and edge features provided by the present invention, when processing large-scale image data, the method further includes the following steps to utilize a distributed computing framework for parallel processing:

[0046] Image data segmentation: Split large-scale image data into multiple image blocks, each containing a portion of the original image data. The size of each image block is optimized based on the node processing power and network bandwidth of the distributed computing framework.

[0047] Task allocation: Utilize the task scheduling system of the distributed computing framework Hadoop or Spark to assign the stereo matching task of each image block to different computing nodes in the framework for processing;

[0048] Parallel computing: On each computing node, stereo matching calculations are performed independently on each image block to obtain the disparity value map or related matching results of each image block;

[0049] Result merging: After all computing nodes complete the stereo matching calculations for their respective image blocks, the data merging mechanism of the distributed computing framework is used to merge the disparity value maps or matching results calculated by each node to obtain the final stereo matching result of the entire large-scale image data.

[0050] It can be seen that the present invention proposes a local stereo matching algorithm based on adaptive window and edge features, which has significant beneficial effects compared with traditional stereo matching algorithms, specifically reflected in the following aspects:

[0051] 1. Significantly Improved Matching Accuracy: By introducing an adaptive matching window, the present invention automatically adjusts the matching window size based on the grayscale and gradient mean values surrounding the matching pixel. In areas with weak texture, the algorithm uses a larger matching window to fully utilize the information from surrounding pixels, thereby improving matching accuracy. In areas with strong texture, a smaller matching window is used to reduce computational effort and avoid introducing excessive interference. This adaptive matching window strategy enables the algorithm to maintain high matching accuracy across a wide range of texture conditions.

[0052] 2. Enhanced algorithm robustness: This paper proposes a matching cost aggregation function that fuses grayscale and edge features. Edge features are less susceptible to illumination variations than grayscale features. Therefore, by fusing these two features, the algorithm can better cope with complex environmental conditions such as illumination variations, thereby improving its robustness. Furthermore, the fused matching cost function effectively integrates grayscale and edge information within the matching area, further improving matching accuracy.

[0053] 3. Improved Computational Efficiency: The use of an adaptive matching window not only improves matching accuracy but also improves computational efficiency to a certain extent. In areas with weak texture, although a larger matching window is used, the information of surrounding pixels is fully utilized, reducing the possibility of mismatches and thus reducing the computation required for subsequent optimization and correction steps. In areas with strong texture, using a smaller matching window directly reduces computational complexity.

[0054] In summary, the present invention significantly improves the matching accuracy and robustness of the local stereo matching algorithm, and enhances the computational efficiency and application scope by proposing innovative points such as an adaptive matching window and a matching cost aggregation function that fuses grayscale features and edge features.

[0055] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flowchart of an embodiment of a local stereo matching method based on adaptive windows and edge features of the present invention.

[0057] Figure 2 The present invention is a flowchart of a local stereo matching method based on adaptive windows and edge features for performing local stereo matching on left and right images in an embodiment of the present invention.

[0058] Figure 3 This is a principle diagram of obtaining a grayscale matching region R by region growing in an embodiment of a local stereo matching method based on an adaptive window and edge features of the present invention.

[0059] Figure 4 This is a diagram showing the calculation principle of the edge feature matching cost in an embodiment of a local stereo matching method based on an adaptive window and edge features of the present invention. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0061] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0062] See also Figures 1 to 4 This embodiment provides a local stereo matching method based on an adaptive window and edge features, the method comprising the following steps:

[0063] Step S1, set the initial matching window (size 5*5) with the current matching point (p,q) as the center;

[0064] Step S2: automatically adjust the matching window size according to the grayscale mean and gradient mean around the matching pixel point, and use the region growing method to determine the pixel points involved in the grayscale cost calculation to form an adaptive matching window;

[0065] Step S3: Use a binocular camera to obtain the left and right images to be matched, and apply the Canny operator to perform edge detection on each image to obtain left and right edge feature maps containing edge information;

[0066] Step S4: performing logical operations on corresponding pixel points of the left and right edge feature maps within the adaptive matching window to calculate the edge feature matching cost;

[0067] Step S5: design a matching cost aggregation function that integrates grayscale features and edge features, aggregate the SAD matching cost and the edge feature matching cost to obtain a matching cost value;

[0068] Step S6, traverse all matching candidate points in the left and right images, select the right / left matching candidate point with the smallest matching cost as the right / left matching point that matches the left / right matching point, and use the position difference between the left and right matching points at this time as the disparity to obtain the disparity value of the entire image.

[0069] In the above step S2, the process of adjusting the adaptive matching window includes:

[0070] Calculate the grayscale variance S of the pixels in the matching window 2 (p,q), expressed as the following formula:

[0071]

[0072] Where N is the number of pixels in the matching window; (x, y) is the pixel in the matching window; W(p, q, s) is the matching window with (p, q) as the center and side length s; S 2 (p,q) is the grayscale variance within the matching window; I(x,y) is the pixel grayscale value; is the average gray value in the matching window.

[0073] Determine whether the grayscale variance is greater than the set variance threshold and dynamically adjust the size of the matching window based on the judgment result.

[0074] As can be seen, this embodiment proposes an adaptive matching window that automatically adjusts based on the image grayscale variance. The image grayscale variance reflects the image contrast. The larger the grayscale variance, the greater the image contrast and the more dramatic the pixel grayscale changes. The smaller the grayscale variance, the lower the image contrast and the more gradual the pixel grayscale changes. In weak texture areas, the grayscale value changes are small, so the grayscale variance is small.

[0075] In this embodiment, the calculated grayscale variance is compared with a preset variance threshold S. If the grayscale variance is less than the variance threshold S, it is determined that the size of the current matching window is insufficient to fully reflect the texture features. The length and width of the matching window are each increased by 1 pixel unit, and the grayscale variance calculation process is repeated until the grayscale variance is greater than or equal to the variance threshold S. At this time, it is determined that the window meets the requirements of the strong texture area and is set as the final matching window W.

[0076] Within the determined matching window W, the current matching point is used as the initial seed point to start the region growing process. This process is achieved by comparing the pixel value of the seed point with the pixel values of the adjacent pixels. If the difference between the pixel value of the adjacent pixel and the pixel value of the seed point is greater than the preset threshold T, the adjacent pixel is added to the current growing region.

[0077] The newly added pixel point is used as a new seed point and the region growing process is continued until no new pixel point can be added or the growing region reaches the boundary of the matching window W. The region obtained at this time is the grayscale matching region R.

[0078] The two images from a binocular camera may have different lighting conditions, so the SAD algorithm may fail to match based on grayscale similarity. However, the edge features of an image are not easily affected by lighting and have better robustness than pixel grayscale features. In the left and right images of a binocular camera, there should be similar edge features within the corresponding matching windows. When a mismatch occurs, the edges are located in completely different positions within the matching windows. In order to measure the similarity of edge features within the matching window, this embodiment designs an edge feature matching cost, which is calculated as follows:

[0079]

[0080] Among them, C Edge (p,q,d) is the edge feature matching cost when the disparity is d; W(p,q,s) is the matching window centered at (p,q) and with a size of s; E l (x, y) is the pixel value in the left edge image matching window, which is 0 or 1; & is the 'and' logical operation; E r (x,y+d) is the pixel value within the right edge image matching window, which is 0 or 1.

[0081] like Figure 2 As shown, Figure 2 The calculation principle of the edge feature matching cost is shown in the figure. In the figure, 1 represents an edge pixel and 0 represents a non-edge pixel. In the above step S4, the calculation steps of the edge feature matching cost include:

[0082] Use a binocular camera to obtain the left and right images to be matched, and apply the Canny operator to perform edge detection respectively to obtain the edge maps of the left and right images containing edge information, where the pixel value at the edge is set to 255 and the pixel value at the non-edge is set to;

[0083] Normalize the pixel values of the left and right edge feature maps, convert the pixel values at the edge from 255 to 1, and keep the pixel values at non-edges as 0;

[0084] In the normalized left and right edge images, according to the size and position of the adaptive matching window W determined previously, an "AND" logical operation is performed on the pixels of the left and right edge images within the corresponding matching window. That is, only when the pixel values of the left and right edge images at the same position are both 1, that is, both are edge pixels, the output of this position is 1, otherwise it is 0, thus obtaining the edge matching cost map, which is used to reflect the consistency of the edge features of the left and right images within the matching window;

[0085] A bitwise sum operation is performed on all pixel values in the edge matching cost map, and the sum result is used as the edge feature matching cost. This cost value quantifies the similarity of the edge features of the left and right images within the matching window.

[0086] In the above step S5, the SAD matching cost and the edge feature matching cost have different meanings and orders of magnitude, so they cannot be directly added or subtracted during the matching cost aggregation. This embodiment proposes a cost aggregation function that integrates SAD and edge features as shown in the following formula:

[0087]

[0088] in, is the absolute value matching cost of the grayscale difference within the grayscale matching area R of the binocular image; is the average gray value in the matching window W; C Edge is the edge feature matching cost.

[0089] In the above step S6, when performing cost aggregation and generating a disparity map, the following steps are included:

[0090] a) For each pixel in the left image, it is considered as a left matching point, and a right matching point with an initial position is set on the corresponding row in the right image;

[0091] b) For the current left matching point, move the right matching point from left to right pixel by pixel on its row. After each move, calculate the SAD matching cost between the current left matching point and the right matching point within the adaptive matching window W, and calculate the edge feature matching cost in the corresponding edge matching cost map;

[0092] c) According to the preset matching cost aggregation function, the calculated SAD matching cost and edge feature matching cost are aggregated to obtain the comprehensive matching cost at the current right matching position;

[0093] d) Repeat steps b) and c) until the right matching point moves to the column position corresponding to the preset maximum disparity range. Record the comprehensive matching costs of all right matching positions during this process, and select the position with the lowest comprehensive matching cost as the best matching position for the current left matching point. The difference between this position and the column position of the left matching point is the desired disparity.

[0094] e) Repeat steps a) to d) for all pixels in the left image. After the traversal is completed, a disparity value map of the entire image is generated based on the disparity of each left matching point.

[0095] Specifically, when calculating the SAD matching cost between the current left matching point and the right matching point within the adaptive matching window W, the following steps are included:

[0096] For a left matching point in the left image, determine a corresponding right matching point in the right image, and obtain a grayscale matching area R centered on the two matching points and determined according to the adaptive window W;

[0097] In the grayscale matching area R, for each pixel in the left image, calculate the grayscale difference between it and the pixel at the corresponding position in the right image (i.e., the corresponding position when the disparity is d), and take the absolute value of the difference to obtain the SAD value of the pixel;

[0098] Add the SAD values of all pixels in the grayscale matching area R to obtain the total SAD matching cost of the left and right matching points in the area R;

[0099] When the right matching point moves pixel by pixel in the right image, the SAD matching costs of the left and right matching points in region R under different disparities are calculated for subsequent cost aggregation and disparity determination.

[0100] In this embodiment, the specific steps of starting the region growing process and further determining the grayscale matching region R are:

[0101] In the determined adaptive matching window W, the current matching point is selected as the initial seed point;

[0102] Compare the pixel value of the seed point with the pixel values of its adjacent pixels in the adaptive matching window W. If the difference between the pixel value of the adjacent pixel and the pixel value of the seed point is less than or equal to the preset pixel value difference threshold T, then mark the adjacent pixel as a candidate pixel.

[0103] Add all marked candidate pixels to the current growth area and use them as new seed points;

[0104] The pixels that meet the conditions among the adjacent pixels of the newly added seed point are continuously added to the current growth area until there are no new pixels to be added or the growth area is extended to the boundary of the adaptive matching window W;

[0105] The final growth area is determined as the grayscale matching area R, which contains pixels with high grayscale similarity to the initial matching points and is used for subsequent SAD matching cost calculation.

[0106] In this embodiment, when processing large-scale image data, the following steps are further included to utilize a distributed computing framework for parallel processing:

[0107] Image data segmentation: Split large-scale image data into multiple image blocks, each containing a portion of the original image data. The size of each image block is optimized based on the node processing power and network bandwidth of the distributed computing framework.

[0108] Task allocation: Utilize the task scheduling system of the distributed computing framework Hadoop or Spark to assign the stereo matching task of each image block to different computing nodes in the framework for processing;

[0109] Parallel computing: On each computing node, stereo matching calculations are performed independently on each image block to obtain the disparity value map or related matching results of each image block;

[0110] Result merging: After all computing nodes complete the stereo matching calculations for their respective image blocks, the data merging mechanism of the distributed computing framework is used to merge the disparity value maps or matching results calculated by each node to obtain the final stereo matching result of the entire large-scale image data.

[0111] In actual applications, the binocular camera installed on the AGV captures the left and right images of the surrounding environment in real time, performs Canny edge detection on the acquired left and right images, extracts the edge features in the image, and provides a basis for subsequent edge matching cost calculation.

[0112] For each matching point in the left image, an adaptive window W is automatically calculated according to the grayscale changes around it, and the pixel points involved in the grayscale cost calculation are determined using the region growing method to form a grayscale matching region R.

[0113] In the same row of the right image, the right matching point is moved pixel by pixel, and at each position, the SAD matching cost of the left and right matching points within the grayscale matching region R is calculated. At the same time, the edge matching cost within the adaptive window W is calculated. The SAD matching cost and the edge matching cost are aggregated to form the total matching cost.

[0114] For each left matching point, find the position difference of the right matching point with the minimum matching cost as the disparity of the point, repeatedly traverse all points in the left image, and obtain the disparity value of the entire image.

[0115] Using parallax information and the camera's internal and external parameters, triangulation is used to calculate the depth of the AGV's surroundings. The AGV system uses this extracted depth information to update the AGV's position and posture in real time. Combined with pre-defined path planning and obstacle information, the AGV can make precise navigation and positioning decisions, achieving autonomous navigation and obstacle avoidance.

[0116] In summary, the present invention introduces an adaptive matching window, which automatically adjusts the size of the matching window based on the grayscale mean and gradient mean around the matching pixel. In areas with weak texture, the algorithm uses a larger matching window to fully utilize the information from surrounding pixels, thereby improving matching accuracy. In areas with strong texture, a smaller matching window is used to reduce computational effort and avoid introducing excessive interference. This adaptive matching window strategy enables the algorithm to maintain high matching accuracy under various texture conditions.

[0117] Furthermore, the present invention proposes a matching cost aggregation function that fuses grayscale and edge features. Edge features are less susceptible to illumination variations than grayscale features. Therefore, by fusing these two features, the algorithm can better cope with complex environmental conditions such as illumination variations, thereby improving its robustness. Furthermore, the fused matching cost function effectively integrates grayscale and edge information within the matching area, further improving matching accuracy.

[0118] Furthermore, the significant improvements in matching accuracy and robustness achieved by this invention enable its wider application in a variety of scenarios requiring precise depth information extraction. For example, in the automated logistics AGV industry, driverless cars, security monitoring, and other fields, this invention can provide more accurate and reliable depth information support, helping to improve the level of intelligence and overall performance in these fields.

[0119] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0120] The above embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and replacements made by technicians in this field on the basis of the present invention fall within the scope of protection required by the present invention.

Claims

1. A local stereo matching method based on adaptive window and edge features, characterized in that: The following steps are involved: Set the initial matching window with the current matching point as the center; The matching window size is automatically adjusted according to the grayscale mean and gradient mean around the matching pixel point, and the pixel points involved in the grayscale cost calculation are determined using the region growing method to form an adaptive matching window. Use a binocular camera to obtain the left and right images to be matched, and apply the Canny operator to perform edge detection respectively to obtain the left and right edge feature maps containing edge information; In the adaptive matching window, logical operations are performed on the corresponding pixels of the left and right edge feature maps to calculate the edge feature matching cost; Design a matching cost aggregation function that integrates grayscale features and edge features, aggregate the SAD matching cost and edge feature matching cost to obtain a matching cost value; Traverse all matching candidate points in the left and right images, select the right / left matching candidate point with the smallest matching cost as the right / left matching point that matches the left / right matching point, and use the position difference between the left and right matching points at this time as the disparity to obtain the disparity value of the entire image.

2. The method according to claim 1, characterized in that The process of adjusting the adaptive matching window includes: Calculate the grayscale variance of the pixels within the matching window; Determine whether the grayscale variance is greater than the set variance threshold and dynamically adjust the size of the matching window based on the judgment result.

3. The method according to claim 2, wherein: The grayscale variance S is calculated according to the following formula 2 (p,q): Where N is the number of pixels in the matching window; (x, y) is the pixel in the matching window; W(p, q, s) is the matching window with (p, q) as the center and side length s; S 2 (p,q) is the grayscale variance within the matching window; I(x,y) is the pixel grayscale value; is the average gray value in the matching window.

4. The method according to claim 3, wherein: The calculated grayscale variance is compared with the preset variance threshold S. If the grayscale variance is less than the variance threshold S, it is determined that the size of the current matching window is insufficient to fully reflect the texture features. The length and width of the matching window are increased by 1 pixel each, and the grayscale variance calculation process is repeated until the grayscale variance is greater than or equal to the variance threshold S. At this time, it is determined that the window meets the requirements of the strong texture area and is set as the matching window W; Within the determined matching window W, the current matching point is used as the initial seed point to start the region growing process. This process is achieved by comparing the pixel value of the seed point with the pixel values of the adjacent pixels. If the difference between the pixel value of the adjacent pixel and the pixel value of the seed point is greater than the preset threshold T, the adjacent pixel is added to the current growing region. The newly added pixel point is used as a new seed point and the region growing process is continued until no new pixel point can be added or the growing region reaches the boundary of the matching window W. The region obtained at this time is the grayscale matching region R.

5. The method according to claim 4, characterized in that The step of calculating the edge feature matching cost includes: Normalize the pixel values of the left and right edge feature maps, convert the pixel values at the edge from 255 to 1, and keep the pixel values at non-edges as 0; In the normalized left and right edge images, according to the size and position of the adaptive matching window W determined previously, an "AND" logical operation is performed on the pixels of the left and right edge images within the corresponding matching window. That is, only when the pixel values of the left and right edge images at the same position are both 1, that is, both are edge pixels, the output of this position is 1, otherwise it is 0, thus obtaining the edge matching cost map, which is used to reflect the consistency of the edge features of the left and right images within the matching window; A bitwise sum operation is performed on all pixel values in the edge matching cost map, and the sum result is used as the edge feature matching cost. This cost value quantifies the similarity of the edge features of the left and right images within the matching window.

6. The method according to claim 5, characterized in that: The calculation formula of edge feature matching cost is expressed as the following formula: Among them, C Edge (p,q,d) is the edge feature matching cost when the disparity is d; W(p,q,s) is the matching window centered at (p,q) and with a size of s; E l (x, y) is the pixel value in the left edge image matching window, which is 0 or 1; & is the 'and' logical operation; E r (x,y+d) is the pixel value within the right edge image matching window, which is 0 or 1.

7. The method according to claim 1, wherein: The matching cost aggregation function that integrates the SAD matching cost and the edge feature matching cost is expressed as the following formula: in, is the absolute value matching cost of the grayscale difference within the grayscale matching area R of the binocular image; is the average gray value in the matching window W; C Edge is the edge feature matching cost.

8. The method according to claim 1, characterized in that When performing cost aggregation and disparity map generation, the following steps are included: a) For each pixel in the left image, it is considered as a left matching point, and a right matching point with an initial position is set on the corresponding row in the right image; b) For the current left matching point, move the right matching point from left to right pixel by pixel on its row. After each move, calculate the SAD matching cost between the current left matching point and the right matching point within the adaptive matching window W, and calculate the edge feature matching cost in the corresponding edge matching cost map; c) According to the preset matching cost aggregation function, the calculated SAD matching cost and edge feature matching cost are aggregated to obtain the comprehensive matching cost at the current right matching position; d) Repeat steps b) and c) until the right matching point moves to the column position corresponding to the preset maximum disparity range. Record the comprehensive matching costs of all right matching positions during this process, and select the position with the lowest comprehensive matching cost as the best matching position for the current left matching point. The difference between this position and the column position of the left matching point is the desired disparity. e) Repeat steps a) to d) for all pixels in the left image. After the traversal is completed, a disparity value map of the entire image is generated based on the disparity of each left matching point.

9. The method according to claim 8, characterized in that When calculating the SAD matching cost between the current left matching point and the right matching point within the adaptive matching window W, the following steps are included: For a left matching point in the left image, determine a corresponding right matching point in the right image, and obtain a grayscale matching area R centered on the two matching points and determined according to the adaptive window W; In the grayscale matching area R, for each pixel in the left image, calculate the grayscale value difference between it and the corresponding pixel in the right image, and take the absolute value of the difference to obtain the SAD value of the pixel; Add the SAD values of all pixels in the grayscale matching area R to obtain the total SAD matching cost of the left and right matching points in the area R; When the right matching point moves pixel by pixel in the right image, the SAD matching costs of the left and right matching points in region R under different disparities are calculated.

10. The method according to any one of claims 1 to 9, characterized in that When processing large-scale image data, the following steps are also included to utilize a distributed computing framework for parallel processing: Image data segmentation: Split large-scale image data into multiple image blocks, each containing a portion of the original image data. The size of each image block is optimized based on the node processing power and network bandwidth of the distributed computing framework. Task allocation: Utilize the task scheduling system of the distributed computing framework Hadoop or Spark to assign the stereo matching task of each image block to different computing nodes in the framework for processing; Parallel computing: On each computing node, stereo matching calculations are performed independently on each image block to obtain the disparity value map or related matching results of each image block; Result merging: After all computing nodes complete the stereo matching calculations for their respective image blocks, the data merging mechanism of the distributed computing framework is used to merge the disparity value maps or matching results calculated by each node to obtain the final stereo matching result of the entire large-scale image data.

Images