Stereo matching method based on improved Census transformation and fusion cost

Abstract

The invention discloses a stereo matching method based on improved Census transformation and fusion cost. The method comprises the following steps: designing a cost function based on SAD and MSD fusion; searching a median by using an accumulative histogram acceleration method to replace a central pixel value to calculate the improved Census cost; the SAD cost, the MSD cost and the Census cost are further fused, and the fusion cost under the normalized scale is calculated; constructing a cross domain around the target pixel, and performing cost aggregation in the upper, lower, left and right directions of the central pixel; performing scanning line optimization in the upper, lower, left and right directions of the pixel points; calculating and storing the parallax of the left and right views; and carrying out multi-step parallax optimization on the obtained parallax value and outputting a final parallax image. According to the method, the defect of insufficient robustness to illumination and noise in a traditional stereo matching method is overcome to a certain extent. The accuracy of stereo matching in the environment with obvious illumination change and multi-intensity noise is improved, the mismatching rate is reduced, and the quality of the disparity map is improved. Compared with some traditional stereo matching methods, the method provided by the invention shows stronger robustness and higher precision.

Description

Technical Field

[0001] The invention relates to image processing, stereo matching and other technologies, and specifically designs a stereo matching method based on improved Census transformation and fusion cost. Background Art

[0002] Stereo matching is an important research area in computer vision, and its main goal is to recover 3D information from two or more images with overlapping fields of view. This technology is crucial in many applications, such as robot navigation, binocular ranging, 3D reconstruction, virtual reality, augmented reality, autonomous driving, etc.

[0003] Traditional stereo matching methods mainly rely on calculating similarity measures between corresponding pixels in two images. However, these methods often perform poorly when dealing with situations with repetitive textures, noise, illumination changes, occlusions, etc. In real-world application scenarios, changes in lighting conditions and image noise are inevitable factors. Illumination changes can cause the same object to present different grayscale values ​​in different images, which affects the accurate identification of matching points. In addition, image noise can interfere with the matching process and lead to incorrect matching of matching points. These problems seriously affect the accuracy and robustness of stereo matching algorithms. Summary of the invention

[0004] In view of the shortcoming that the current stereo matching is not robust to noise and illumination, the purpose of the present invention is to provide a stereo matching method based on improved Census transformation and fusion cost. By fusing SAD cost, MSD cost and improved Census cost, a stereo matching method with stronger robustness to illumination and noise and higher matching accuracy is proposed.

[0005] The main technical solutions adopted in the present invention are:

[0006] Firstly, a stereo matching method based on improved Census transformation and fusion cost includes the following steps:

[0007] Step S1: design and calculate the cost function based on the fusion of the sum of absolute differences (SAD) and the mean square difference (MSD);

[0008] Step S2: By using the cumulative histogram acceleration method to find the median instead of the central pixel value to calculate the improved Census cost and further fuse the SAD cost, MSD cost, and Census cost to calculate the fusion cost under the normalized scale;

[0009] Step S3: construct a cross domain around the target pixel, and perform cost aggregation in four directions of up, down, left, and right of the center pixel;

[0010] Step S4: Optimize the scan line from the four directions of up, down, left, and right of the pixel point;

[0011] Step S5: Calculate and store the disparity between the left and right views, perform multi-step disparity optimization on the obtained disparity values, and output a final disparity map.

[0012] The specific steps of designing and calculating the cost function based on the fusion of the sum of absolute differences (SAD) and the mean square deviation (MSD) in step S1 are as follows:

[0013] Step S1.1: Select two left and right view images taken by a binocular camera as input. These images are usually the same scene taken from different viewpoints.

[0014] Step S1.2: With each pixel in the input image as the center, determine a 9*7 pixel window. For each pair of corresponding pixel windows, calculate their pixel values ​​I in the three RGB channels in the left image and the right image respectively. l,c and I r,c The mean absolute difference, that is, the calculation of SAD cost Cost SAD , the formula is as follows:

[0015]

[0016] I l,c Represents the pixel value of the left image in the RGB three channels, I r,c Represents the pixel values ​​of the left image in the three RGB channels;

[0017] Step S1.3: Similarly, calculate the mean square error cost (MSD) between the corresponding pixel windows of size 9*7, that is, the pixel values ​​I on the three RGB channels in the two pixel blocks l,c and I r,c Subtract the squares one by one and then find the average, which is to calculate the MSD cost. MSD , the formula is as follows:

[0018]

[0019] Step S1.4: Calculate the fusion cost Cost using the designed fusion cost function SA-MSD , the formula is as follows:

[0020]

[0021] In step S2, the improved Census cost is calculated by using the cumulative histogram acceleration method to find the median instead of the central pixel value and further fuse the SAD cost, MSD cost, and Census cost. The specific steps for calculating the fusion cost under the normalized scale are as follows:

[0022] Step S2.1: Initialize a histogram to record the number of occurrences of each gray level in the image.

[0023] Step S2.2: Traverse each pixel of the image and process it using a 9*7 window. For each pixel

[0024] Update the histogram and record the number of occurrences of each gray level in the 9*7 window.

[0025] Step S2.3: Calculate the cumulative histogram from the histogram. Find the median from the cumulative histogram.

[0026] Step S2.4: A binary code is obtained by comparing the value of each pixel in the pixel window with the median. The codes of the corresponding pixel positions in the left and right input images are T and L and T R ,By calculating the Hamming distance of each pair of codes, the Census cost corresponding to each pixel is obtained, and the formula is as follows:

[0027] Cost Census =Hamming(T L ,T R )

[0028] Step S2.5: Combine the previously calculated Cost SAD and Cost MSD , calculate the final cost Cost under normalized scale normal , the formula is as follows:

[0029]

[0030] Among them, λ SAD , MSD , Census These are empirical values, which are set to 10, 65, and 30 respectively in the present invention.

[0031] The beneficial effect of the present invention is that the present invention provides a stereo matching method based on improved Census transformation and fusion cost, which can well cope with the influence of illumination and noise on stereo matching and improve the accuracy of stereo matching.

[0032] The method proposed in the present invention is different from the previous stereo matching algorithms. On the one hand, the mean square error cost is combined in the design of the cost function, which improves the sensitivity of the algorithm to noise, making the algorithm robust to noises of different intensities. On the other hand, it is proposed to use the median instead of the central pixel to calculate the Census code. This method has a certain robustness to noises of various intensities, and uses the cumulative histogram to accelerate the Census transformation, which reduces the time consumption of cost calculation while ensuring robustness, and improves the execution efficiency of the algorithm. Combining the above two points, the method of the present invention has better performance than traditional algorithms for stereo matching in scenes affected by illumination and noise, and can improve matching accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Stereo matching process based on improved Census transformation and fusion cost;

[0034] Figure 2 Comparison of Census transformation before and after improvement;

[0035] Figure 3 Cross-domain cost aggregation algorithm;

[0036] Figure 4 Disparity map output after multi-disparity optimization.

[0037] Specific measures

[0038] The following drawings illustrate in detail the various details involved in the technical solution of the present invention.

[0039] A stereo matching method based on improved Census transformation and fusion cost, such as Figure 1 As shown, the specific steps are as follows:

[0040] Step S1: design and calculate the cost function based on the fusion of the sum of absolute differences (SAD) and the mean square difference (MSD);

[0041] Step S2: By using the cumulative histogram acceleration method to find the median instead of the central pixel value to calculate the improved Census cost, and further integrate the SAD cost, MSD cost, and Census cost to calculate the fusion cost under the normalized scale. Figure 2 As shown in the figure, in the traditional Census transformation, if the central pixel is affected by noise, there is a large difference between the Census code obtained in the noise-free case and the Census code obtained in the noise-free case, which affects the subsequent matching results. However, the Census code obtained by calculating the median instead of the central pixel value has almost no error, which can well reduce the matching error caused by noise and has better robustness to noises of different intensities.

[0042] Step S3: construct a cross domain around the target pixel and perform cost aggregation in the four directions of the center pixel, such as Figure 3 As shown;

[0043] Step S4: Optimize the scan line from the four directions of up, down, left, and right of the pixel point;

[0044] Step S5: Calculate and store the disparity between the left and right views, perform multi-step disparity optimization on the obtained disparity values ​​and output the final disparity map, such as Figure 4 shown.

[0045] The specific steps of designing and calculating the cost function based on the fusion of the sum of absolute differences (SAD) and the mean square error (MSD) in step S1 are as follows:

[0046] Step S1.1: Select two left and right view images taken by a binocular camera as input. These images are usually the same scene taken from different viewpoints.

[0047] Step S1.2: With each pixel in the input image as the center, determine a 9*7 pixel window. For each pair of corresponding pixel windows, calculate their pixel values ​​I in the three RGB channels in the left image and the right image respectively. l,c and I r,c The mean absolute difference, that is, the calculation of SAD cost Cost SAD , the formula is as follows:

[0048]

[0049] Step S1.3: Similarly, calculate the mean square error cost (MSD) between the corresponding pixel windows of size 9*7, that is, the pixel values ​​I on the three RGB channels in the two pixel blocks l,c and I r,c Subtract the squares one by one and then find the average, which is to calculate the MSD cost. MSD , the formula is as follows:

[0050]

[0051] Step S1.4: Calculate the fusion cost Cost using the designed fusion cost function SA-MSD , the formula is as follows:

[0052]

[0053] In step S2, the improved Census cost is calculated by finding the median instead of the central pixel value by using the cumulative histogram acceleration method and further fusing the SAD cost, MSD cost, and Census cost. The specific steps for calculating the fusion cost under the normalized scale are as follows:

[0054] Step S2.1: Initialize a histogram to record the number of occurrences of each gray level in the image.

[0055] Step S2.2: Traverse each pixel of the image and process it using a 9*7 window. For each pixel

[0056] Update the histogram and record the number of occurrences of each gray level in the 9*7 window.

[0057] Step S2.3: Calculate the cumulative histogram from the histogram. Find the median from the cumulative histogram.

[0058] Step S2.4: A binary code is obtained by comparing the value of each pixel in the pixel window with the median. The codes of the corresponding pixel positions in the left and right input images are T and L and T R ,By calculating the Hamming distance of each pair of codes, the Census cost corresponding to each pixel is obtained, and the formula is as follows:

[0059] Cost Census =Hamming(T L ,T R )

[0060] Step S2.5: Combine the previously calculated Cost SAD and Cost MSD , calculate the final cost Cost under normalized scale normal , the formula is as follows:

[0061]

[0062] Among them, λ SAD , MSD , Census These are empirical values, which are set to 10, 65, and 30 respectively in the present invention.

Claims

1. A stereo matching method based on improved Census transformation and fusion cost, characterized in that: The following steps are involved: Step S1: design and calculate the cost function based on the fusion of SAD and MSD; Step S2: The improved Census cost is calculated by using the cumulative histogram acceleration method to find the median instead of the central pixel value, and the SAD cost, MSD cost, and Census cost are further integrated to calculate the fusion cost under the normalized scale; Step S3: construct a cross domain around the target pixel, and perform cost aggregation in four directions of up, down, left, and right of the center pixel; Step S4: Optimize the scan line from the four directions of up, down, left, and right of the pixel point; Step S5: Calculate and store the disparity between the left and right views, perform multi-step disparity optimization on the obtained disparity values, and output a final disparity map.

2. The stereo matching method based on improved Census transformation and fusion cost as claimed in claim 1, characterized in that: The specific steps of designing and calculating the cost function based on the fusion of SAD and MSD in step S1 are as follows: Step S1.1: Select two left and right view images taken by a binocular camera as input, wherein the images are of the same scene taken from different viewpoints; Step S1.2: With each pixel in the input image as the center, determine a 9*7 pixel window. For each pair of corresponding pixel windows, calculate their pixel values ​​I in the three RGB channels in the left image and the right image respectively. l,c and I r,c The mean absolute difference, that is, the calculation of SAD cost Cost SAD , the formula is as follows: I l,c Represents the pixel value of the left image in the RGB three channels, I r,c Represents the pixel values ​​of the left image in the three RGB channels; Step S1.3: Calculate the MSD between the corresponding pixel windows of size 9*7, that is, the pixel values ​​I on the three RGB channels in the two pixel blocks l,c and I r,c Subtract the squares one by one and then find the average, which is to calculate the MSD cost. MSD , the formula is as follows: Step S1.4: Calculate the fusion cost Cost using the designed fusion cost function SA-MSD , the formula is as follows:

3. The stereo matching method based on improved Census transformation and fusion cost as claimed in claim 2, characterized in that: In step S2, the improved Census cost is calculated by finding the median instead of the central pixel value by using the cumulative histogram acceleration method, and the SAD cost, MSD cost, and Census cost are further integrated to calculate the fusion cost under the normalized scale. The specific steps are as follows: Step S2.1: Initialize a histogram to record the number of occurrences of each gray level in the image; Step S2.2: traverse each pixel of the image and process it using a 9*7 window; update the histogram for each pixel and record the number of occurrences of each gray level in the 9*7 window; Step S2.3: Calculate the cumulative histogram based on the histogram, and find the median from the cumulative histogram; Step S2.4: A binary code is obtained by comparing the value of each pixel in the pixel window with the median. The codes of the corresponding pixel positions in the left and right input images are T and L and T R ,By calculating the Hamming distance of each pair of codes, the Census cost corresponding to each pixel is obtained, and the formula is as follows: Cost Census =Hamming(T L ,T R ) Step S2.5: Combine the Cost calculated in claim 2 SAD Cost MSD , calculate the final cost Cost under normalized scale normal , the formula is as follows: Among them, λ SAD , MSD , Census For experience value.

4. The stereo matching method based on improved Census transformation and fusion cost as claimed in claim 3, characterized in that The lambda SAD , MSD , Census The values ​​are set to 10, 65, and 30 respectively.

Images