A Fast Stereo Matching Method for Aggregate Reconstruction Based on Grayscale

Abstract

The present invention discloses a fast stereo matching method for aggregate reconstruction based on grayscale, which reduces the computational amount by narrowing the disparity search interval during matching and reducing the number of matching pixel points, so as to achieve the purpose of fast matching. The method for reducing the number of matching pixel points is to only obtain the ROI pixels for matching. The ROI is a pixel region containing aggregate information. Compared with the feature-based stereo matching method, the present invention can obtain a more accurate and denser disparity map without interpolation processing while ensuring real-time performance. Compared with the grayscale-based stereo matching method, the present invention can greatly reduce the computational amount of the algorithm, shorten the calculation time and improve the efficiency on the premise of ensuring high-precision reconstruction.

Description

Technical Field

[0001] The present invention relates to the field of stereo vision, and particularly to a fast stereo matching method for aggregate reconstruction based on grayscale. Background Art

[0002] Aggregates are very important raw materials in projects such as high-rise building and bridge construction. Their three-dimensional morphological parameters are crucial for the final quality of concrete. Initially, the method of manual screening and detection was used to detect the three-dimensional morphological parameters of aggregates, but it has problems such as long cycle, low accuracy, and large subjective influence. With the rise of computer digital image technology, many scholars began to use two-dimensional images of aggregates to measure the three-dimensional morphological parameters of aggregates. However, even with the development of digital image technology to date, even the relatively mature two-dimensional image detection technology is difficult to overcome its inherent defect, that is, the two-dimensional image detection technology cannot obtain the height information of aggregates in the third dimension. To address these problems, three-dimensional detection technology based on digital images emerged. The specific steps are to reconstruct the aggregates, and then calculate their three-dimensional morphological parameters for the reconstructed aggregates. The stereo matching described in the present invention is the most important step in three-dimensional reconstruction.

[0003] The stereo matching method for aggregate reconstruction involved in the present invention requires generating a dense and high-precision disparity map, and only the grayscale-based stereo matching method can be selected. In the prior art, although the grayscale-based stereo matching method can generate a dense disparity map, the problems brought at the same time are large computational amount and long calculation time, and it is difficult to meet the real-time requirement. In the industrial three-dimensional detection of aggregates, the most important thing is the time cost.

[0004] Patent with application publication number CN104867133A discloses a fast distributed stereo matching method. This method proposes to use the mean shift algorithm to perform color image segmentation on the images to be matched, and form a disparity constraint based on image segmentation with segmentation regions of any size and shape as support windows, and then perform stereo matching based on a fixed window. Although this method can theoretically achieve stereo matching based on a fixed segmentation window, to a certain extent, it limits the disparity search range, thereby reducing the computational amount of disparity matching and shortening the matching time. However, first, when the mean shift algorithm performs segmentation, since the number of segmentation regions cannot be limited, there will be many noisy regions in the segmented images, especially for aggregate images with low surface color differentiation, and a good segmentation effect cannot be obtained. Second, even if relevant optimization can be carried out to obtain a good segmentation effect, the cost calculation of each segmented image block using the zero-mean normalized cross-correlation metric function described in the text and obtaining the disparity value d when the cost is the largest (the correlation degree of the matching image pair to the segmentation region is the largest) sAs the estimated disparity of all pixel points in the segmentation region, although such a method can estimate the disparity to a certain extent, it also greatly increases the computational complexity. Finally, only restricting the disparity search range reduces the computational complexity to a certain extent, but it is still difficult to achieve real-time matching. Summary of the Invention

[0005] The object of the present invention is to propose a fast stereo matching method based on grayscale for aggregate reconstruction to solve the problems of large computational complexity and long computational time of existing grayscale-based stereo matching in view of the above problems.

[0006] In the aggregate reconstruction described in the present invention, the aggregate is the reconstruction object, and the background that does not need to be reconstructed occupies most of the image area. Therefore, the technical idea adopted in the present invention is to perform high-precision reconstruction only on the area containing aggregate information. The technical solution is: a fast stereo matching method based on grayscale for aggregate reconstruction, which reduces the computational complexity by reducing the number of matching pixel points while narrowing the disparity search range during matching, so as to achieve the purpose of fast matching. The method for reducing the number of matching pixel points is to only obtain ROI pixels for matching. The ROI is the pixel region containing aggregate information. It mainly includes the following steps:

[0007] Step 1: Acquisition and preprocessing of the left and right images of the measured aggregate;

[0008] After calibrating the internal and external parameters of the acquisition device, the images of the measured aggregate are acquired to obtain the left and right image pairs I p , I q . Using the calibrated internal and external parameters, the distortion correction and epipolar correction are performed on the aggregate images to obtain the corrected aggregate images I′ p , I′ q ;

[0009] Step 2: Obtain the effective disparity range through the first stereo matching;

[0010] For the corrected aggregate images I′ p , I′ q with the size of h×w in Step 1, downsample by n times. After sampling, the aggregate images I″ p , I″ q are obtained, and the image size is , and the number of pixels is I′ p , I′ q of Subsequently, perform stereo matching on I″ p , I″ q to obtain the initial disparity map D of the aggregate, eliminate the mismatched points and invalid matching points, and then obtain the maximum disparity and minimum disparity of the initial disparity map D to form the initial effective disparity range d e :

[0011] d e = [min(d x,y ): max(d x,y )); (x, y) ∈ D

[0012] where d x,y is the disparity value of the pixel with coordinates (x, y) (the pixel value in the disparity map is equal to the disparity value), and (x, y) are the pixel coordinates of the disparity map D;

[0013] Step 3: Image segmentation to obtain the ROI of the measured aggregate image;

[0014] The I' p , I' q obtained in Step 1 is segmented by the segmentation algorithm to obtain the binary segmentation image pair S p , S q , and then color inversion processing is performed to obtain S' p , S' q . To ensure that the edge information of the aggregate is fully retained, morphological erosion operation needs to be performed on S' p , S' q to expand the ROI according to actual needs to obtain S'' p , S'' q . Subsequently, S'' p , S'' q is pixel - superimposed with the corrected image pair I' p , I' q . The formula is as follows. S'' i (x, y)+I' i (x, y) are the corresponding pixels, that is, the sum of the pixel values of the points with the same pixel coordinates:

[0015]

[0016] where R i represents the ROI image, S'' i (x, y) represents the pixel value of the pixel with coordinates (x, y) in the binary image after the morphological erosion operation, I' i (x, y) is the pixel value of the pixel with coordinates (x, y) in the corrected aggregate image, and in (p, q), p represents the left image and q represents the right image; for the pixels with pixel values less than 255 after pixel superposition, the pixel values remain the original pixel values, and for the pixels with pixel values greater than or equal to 255, the pixel values are set to 255, so as to obtain the ROI image pair R p , R q , where the area with pixel values less than 255 is the ROI;

[0017] Step 4: Second stereo matching of ROI and disparity filling of non - ROI;

[0018] Perform stereo matching on the ROI image pair R p , R q , and perform disparity filling for non-ROI areas; First, magnify the effective disparity range d e obtained in step 2 by the same multiple as downsampling to get the disparity range d' e to adapt to the disparity search of the original resolution image:

[0019] d' e = n × [min(d x,y ): max(d x,y )]; n ∈ R + ; (x, y) ∈ D

[0020] To reduce the error in matching, appropriately expand the effective disparity range, and set the disparity to d'' e , and take the integer value of the obtained value, where R + represents a positive integer:

[0021] d'' e = n × [min(d x,y ) - 1: max(d x,y ) + 1]; n ∈ R + ; (x, y) ∈ D

[0022] The matching images are R p , R q obtained in step 3. In this process, only stereo matching disparity calculation is performed on the ROI, and the disparity of the non-ROI is directly obtained from the disparity map D obtained in the first stereo matching in step 2; The specific filling steps are as follows: First, apply upsampling with the same downsampling multiple to the initial disparity map D, and interpolate to obtain a disparity map D' with the same resolution as the corrected aggregate image pair I' p , I' q in step 1. For the non-ROI pixel points in R p , R q , find the corresponding points in D', that is, the pixel points with the same pixel coordinates, and then obtain their pixel values to get the final disparity map D''.

[0023] Furthermore, in step 1, the distortion correction is a process of obtaining the distortion coefficients of the camera through calibration, and then using the distortion coefficients to correct the distortion introduced by the deviation of the camera lens due to manufacturing accuracy and assembly process.

[0024] Further, in step 1, the epipolar line correction is to calibrate and obtain the internal and external parameters of the camera, and then use the internal and external parameters to correct two images that are actually non-coplanar row-aligned into coplanar row-aligned images, and correct the actual binocular system into an ideal binocular system; the coplanar row alignment means that the image planes of the two cameras are on the same plane, and when the same point is projected onto the image planes of the two cameras, it should be on the same row of the two pixel coordinate systems.

[0025] Further, in step 2, the methods for removing mismatched points and invalid points include the left-right consistency method, removing small connected regions, and uniqueness detection.

[0026] Further, in step 3, the segmentation algorithms include the clustering segmentation algorithm, the watershed segmentation algorithm, the region growing segmentation algorithm, and the neural network segmentation algorithm.

[0027] Further, in step 3, for the binary segmented images S p , S q the pixel values of the ROI are 255 and the pixel values of the non-ROI are 0. After performing an inverse color processing, S' p , S' q is obtained, where the pixel values of the ROI are 0 and the pixel values of the non-ROI are 255.

[0028] Further, in step 3, the morphological erosion operation is used for binary images. By calculating the minimum pixel value of the local n×n matrix and then assigning the minimum pixel value to each element of the n×n matrix, the black regions (i.e., regions with pixel value 0) in the image are expanded in this way.

[0029] Further, in step 4, for the second stereo matching, only the ROI needs to calculate the disparity, and the background regions of the non-ROI directly use the disparities of the corresponding points obtained from the first stereo matching for filling; the second stereo matching of the ROI and the disparity filling of the non-ROI are carried out simultaneously, that is, when traversing the ROI pixels, the stereo matching disparity is calculated, and when traversing the non-ROI pixels, the disparity filling is carried out.

[0030] Further, in step 4, the disparity map D' is an upsampled and interpolated disparity map with the same resolution as I' p , I' q for low-precision disparity. The disparity of the non-ROI is obtained from this, and the disparity of the ROI is the high-precision disparity calculated by performing stereo matching on the original resolution image.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] 1) Compared with the feature-based stereo matching method, a more accurate and denser disparity map can be obtained while ensuring real-time performance, and no interpolation processing is required. Compared with the gray-based stereo matching method, the computational complexity of the algorithm can be greatly reduced, the calculation time can be shortened, and the efficiency can be improved on the premise of ensuring high-precision reconstruction.

[0033] 2) The initial disparity of the image needs to be calculated and estimated based on the geometric relationship between the camera and the object to be measured. This disparity estimation method has strong subjective consciousness and a large estimation range. The downsampling estimation method described in the present invention has no subjective consciousness and an accurate estimation range.

[0034] 3) By extracting the ROI of the image to be matched and downsampling the image to obtain the initial disparity value, the computational complexity is greatly reduced. The corresponding point disparity filling is directly performed on the non-ROI, and the implementation is simple. It can be applied to the stereo matching of high-resolution images that only require accuracy for the ROI. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0036] Figure 1 Schematic diagram of the brief steps of the method of the present invention;

[0037] Figure 2 Schematic diagram of the detailed process for implementing this method;

[0038] Figure 3 Original image of the aggregate used in the embodiment of this method;

[0039] Figure 4 Schematic diagram of aggregate segmentation; where (a) is the binary image of aggregate segmentation in the embodiment of this method; (b) is the binary image obtained by inverting the color of (a) in the embodiment of this method; (c) is the binary image obtained by performing morphological erosion on (b) in the embodiment of this method; (d) is the aggregate image obtained by pixel superposition of (a) and (c) in the embodiment of this method;

[0040] Figure 5 Disparity map finally obtained in the embodiment of this method. DETAILED DESCRIPTION OF THE INVENTION

[0041] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] As Figure 1 - Figure 2 shown, the embodiment is a preferred embodiment of the present invention. The specific implementation methods of each step are as follows. Please refer to the attached Figures 3 - 5 to understand this embodiment:

[0043] Step 1: Acquisition and preprocessing of left and right images of the aggregate to be measured

[0044] Before collecting the images of the aggregate to be measured, it is necessary to calibrate the internal and external parameters of the acquisition device first. During the calibration process, 15-30 sets of calibration plate photos need to be taken from different angles and distances while ensuring that the relative positions of the two cameras remain unchanged. Subsequently, the calibration photos are imported into the computer and binocular stereo calibration is performed using the checkerboard calibration method to obtain the internal and external calibration parameters and distortion parameters. When the binocular cameras are placed, there is no absolute coplanarity between the left and right cameras. Sometimes, in order to expand the common field of view of the left and right cameras, the placement angles of the two cameras are intentionally controlled within 180°. After completing the calibration work, the image acquisition work of the aggregate to be measured is started, and the left and right image pairs I p , I q (image size is 4096×3000) are obtained. After that, the collected images are transmitted into the computer, and the distortion correction and epipolar correction of the aggregate images are performed using the previously calibrated parameters to obtain the corrected left and right image pairs I' p , I' q , as shown in the attached Figure 3 (any pixel point in the picture and the corresponding pixel point in the other picture are on the same horizontal epipolar line). The distortion correction is the process of obtaining the distortion coefficient of the camera through calibration, and then using the distortion coefficient to correct the distortion introduced by the deviation of the camera lens due to manufacturing accuracy and assembly process. The epipolar correction is to obtain the internal and external parameters of the camera through calibration, and then use the internal and external parameters to correct the two images that are actually non-coplanar row-aligned into coplanar row-aligned, and correct the actual binocular system into an ideal binocular system; the coplanar row-alignment means that the image planes of the two cameras are on the same plane, and when the same point is projected onto the image planes of the two cameras, it should be on the same row of the two pixel coordinate systems.

[0045] Step 2: First stereo matching to obtain the effective disparity range

[0046] After obtaining the corrected image pairs I' p , I' q , it is necessary to downsample them. In this embodiment, the downsampling factor is 8, and the sampled image pairs I″ p , I″q The resolution is 512×375, and the number of pixels is 1 / 64 of the original image. Subsequently, for I″ p , I″ q perform stereo matching to obtain the disparity map D. Due to the rough texture on the surface of the aggregate, a large number of mismatched points will be generated. Therefore, it is necessary to remove the mismatched points and invalid points. The methods for removing mismatched points and invalid points include the left-right consistency method, removing small connected regions, and uniqueness detection; the left-right consistency method is based on the uniqueness constraint of disparity, that is, each pixel has at most one correct disparity; the specific steps are to swap the positions of the left and right images, that is, the left image becomes the right image, and the right image becomes the left image, and then perform stereo matching again to obtain another disparity map. Since each value in the disparity map reflects the corresponding relationship between two pixels, according to the uniqueness constraint of disparity, through the disparity map of the left image, find the corresponding point pixel of each pixel in the right image and the disparity value corresponding to this pixel. If the difference between these two disparity values is less than a certain threshold, it satisfies the uniqueness constraint and is retained; otherwise, it does not satisfy the uniqueness constraint and is removed; removing small connected regions means removing the extremely small connected regions in the disparity map, and the difference between the disparity within the same connected region and the neighboring disparity is less than the set threshold; uniqueness detection means calculating the values of the minimum cost and the second minimum cost for each pixel. If the relative difference between the two is less than a certain threshold, it is removed. Retrieve the pixel values of the disparity map D, and the maximum disparity obtained is rounded up to 144, and the minimum disparity is rounded down to 124, forming the initial valid disparity interval d e : [124:144].

[0047] d e = [min(d x,y ): max(d x,y )]; (x, y) ∈ D

[0048] where d x,y is the disparity value of the pixel with coordinates (x, y) (the pixel value in the disparity map is equal to the disparity value), and (x, y) is the pixel coordinate of the disparity map D.

[0049] Step 3: Obtain the effective ROI of the measured aggregate image through image segmentation;

[0050] After obtaining the valid disparity interval d eAfter that, the ROI of the image of the aggregate to be measured needs to be extracted. The ROI refers to the area with aggregate pixel information. In this embodiment, a U-Net neural network is used to train a binary segmentation model for the aggregate. As the aggregate is a small granular object with single picture information, the segmentation effect is better when using the segmentation algorithm, and the extraction of the ROI is also more convenient. The segmentation algorithm is not limited to the segmentation algorithm of the neural network, and any segmentation algorithm can be used on the premise of ensuring effective segmentation, including clustering segmentation algorithm, watershed segmentation algorithm, region growing segmentation algorithm, and neural network segmentation algorithm, etc. After obtaining the optimal model, I′ p and I′ q are input into the model to obtain the binary segmentation image pair S p and S q , as shown in a of the appendix Figure 4 (the picture with pixel values only 0 and 255, the ROI pixel value is 255, and the non-ROI pixel value is 0). After inverse color processing, S′ p and S′ q are obtained, as shown in b of the appendix Figure 4 (the pixel value of 0 becomes 255, and the pixel value of 255 becomes 0). Since the edge information of the aggregate is obvious and the edge depth has mutability, more adjacent surrounding pixels are needed as support information for matching. If pixel superposition is carried out without morphological processing, some edge information will be lost, and these information can be fully retained after morphological processing. To ensure that the edge information of the aggregate can be fully retained, the segmentation image pair S′ p and S′ q will also be subjected to morphological erosion operation to appropriately expand the ROI according to actual needs to obtain the image pair S″ p and S″ q . In this embodiment, the size of the convolution kernel used is 7*7, as shown in c of the appendix Figure 4 (the black part in the figure has a larger range than that in b of Figure 4 ). Subsequently, S″ p and S″ q are pixel-superposed with I′ p and I′ q . The formula is as follows. S″ i (x,y)+I′ i is the pixel value of the same-name pixel points, that is, the sum of the pixel values of the points with the same pixel coordinates:

[0051]

[0052] Among them, R i represents the ROI image, S″ i (x,y) represents the pixel value of the pixel with pixel coordinates (x,y) in the binary image after morphological erosion operation, and I′ i(x, y) is the pixel value of the pixel with pixel coordinates (x, y) in the corrected aggregate image. In (p, q), p represents the left image and q represents the right image; the pixel value of the pixel point with a pixel value greater than or equal to 255 is set to 255, and the pixel value of the pixel point with a pixel value less than 255 is the original pixel value, so as to obtain the ROI image pair R with ROI information p , R q , as shown Figure 4 in d shown in the appendix (ROI pixels are less than 255, and non-ROI pixels are 255).

[0053] Step 4: Second stereo matching of ROI and disparity filling of non-ROI

[0054] The ROI containing aggregate information has been extracted. The second stereo matching only needs to calculate the disparity for the ROI, and the background area of the non-ROI is directly filled with the disparity of the corresponding points obtained by the first stereo matching; this step includes performing stereo matching on the ROI image pair R p , R q and disparity filling of non-ROI. The second stereo matching of ROI and disparity filling of non-ROI are carried out simultaneously, that is, when traversing the ROI pixels, stereo matching disparity calculation is performed, and when traversing the non-ROI pixels, disparity filling is performed; first, obtain the effective disparity range d e :[124:144] obtained in Step 2, and then perform magnification by the same multiple as downsampling to obtain the disparity range d′ e :[992:1152] to adapt to the disparity search of the original resolution image;

[0055] d′ e =n×[min(d x,y ):max(d x,y )]; n ∈ R + ; (x, y) ∈ D

[0056] To reduce the matching error, appropriately expand the effective disparity range, and the disparity range is taken as d″ e :[984:1160], R + represents a positive integer.

[0057] d″ e =n×[min(d x,y ) - 1:max(d x,y ) + 1]; n ∈ R + ; (x, y) ∈ D

[0058] Subsequently, perform stereo matching on the ROI image pair R p , R q , and the matching image is R p , R q, this process only performs stereo matching disparity calculation on the ROI, and the disparity of non-ROI is directly obtained from the disparity map D acquired in the first stereo matching in step 2. The specific matching algorithm process is as follows:

[0059] Image R to be matched p Such as Figure 4 shown as d in (the target matching image is R p corresponding right-view image R q ), starting from the upper left corner of the ROI image R p , the pixel values of each pixel of the entire image are taken and relevant calculations are performed with the right-view image R q . When the ROI is traversed, stereo matching disparity calculation is performed. When the non-ROI area in the image (i.e., Figure 4 the white part of d) is traversed, no disparity calculation is performed, and the disparity is directly obtained from the disparity map D' obtained by upsampling the disparity map D obtained in step 2. The resolution of the disparity map D is 512×375, and the resolution of the disparity map D' is 4096×3000. The disparity map D' is an upsampled and interpolated disparity map with the same resolution as I' p , I' q . The disparity of the same resolution is low-precision disparity, and the non-ROI disparity is obtained from this. The disparity of the ROI is high-precision disparity obtained by performing stereo matching calculation on the original resolution image. The disparity value (the disparity value is equal to the pixel value) is obtained by finding the corresponding pixels of the same name in the disparity map D', and then the disparity value is assigned to the corresponding pixels of the same name in the disparity map D'' to complete the disparity filling. The finally obtained disparity map D'' is as shown in Appendix Figure 5 (the pixel value of each pixel in the figure carries the depth information of that point).

[0060] After the above steps, a disparity image D'' with the same high resolution as the rectified image pair I' p , I' q will be obtained, which greatly shortens the calculation time while ensuring high precision of the ROI.

[0061] The matching algorithm used in this embodiment is the AD-Census algorithm. Without any hardware acceleration, the time taken for normal matching of the downsampled 8-fold resolution image pair I'' p , I'' q is 2.771 s (the disparity estimation interval is selected as [100:160] according to the geometric relationship between the camera and the measured object). The time taken for normal matching of the image pair I' p , I' q with the original resolution of 4096×3000 is 3313.276 s (the disparity estimation interval is selected as [800:1280]). When using the method of this embodiment, for the image pair I' p , I'q The time taken for matching is 73.127 s, and the speedup ratio is 45.3.

[0062] The disparity map D″ has the same high resolution as the rectified image pair I′ p , I′ q , and only complex calculations of pixel-by-pixel matching are performed in the ROI with high precision requirements. The disparity search range is also restricted to the smallest effective disparity interval, while in the non-ROI with low precision requirements, the disparity filling of corresponding points is directly performed. This greatly shortens the time-consuming of the high-resolution aggregate image in stereo matching calculation. At the same time, this method can be applied to stereo matching with only precision requirements for the ROI.

[0063] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A fast stereo matching method for aggregate reconstruction based on grayscale, characterized in that, The method includes the following steps: Step 1: Acquisition and preprocessing of the left and right images of the aggregate to be measured; After calibrating the internal and external parameters of the acquisition device, the images of the measured aggregate are acquired to obtain the left and right image pairs of the measured aggregate The internal and external parameters obtained by calibration are used to perform distortion correction and epipolar correction on the aggregate images to obtain the corrected aggregate images ; Step 2: Obtaining the effective disparity range through the first stereo matching; The size of the corrected image in step 1 is of the aggregate image is downsampled by n times, and the resulting aggregate image after sampling is , the image size is , and the number of pixels is of . Subsequently, stereo matching is performed on to obtain the initial disparity map of the aggregate . The mismatched points and invalid matching points are removed. The maximum disparity and the minimum disparity of the obtained initial disparity map form the initial valid disparity interval : ； wherein, is the disparity value at the pixel coordinate ), and the pixel value in the disparity map is equal to the disparity value, ) is the pixel coordinate of the disparity map ; Step 3: Image segmentation to obtain the ROI of the aggregate image to be measured; The one obtained in step 1 After image segmentation using the segmentation algorithm, a binary segmentation image pair is obtained , and then color inversion processing is performed to obtain . To ensure that the edge information of the aggregate is fully retained, it is necessary to perform morphological erosion operation to expand the ROI according to actual needs to obtain . Subsequently, is pixel-overlaid with the calibration image pair . The formula is as follows is the same-name pixel point, that is, the sum of the pixel values of points with the same pixel coordinates ， ； Among them, represents the ROI image, represents the pixel value at the pixel coordinate in the binary image after the morphological erosion operation, is the pixel value at the pixel coordinate in the aggregate image after correction, where p represents the left image and q represents the right image; after pixel superposition, the pixel values of the pixel points with pixel values less than 255 remain the original pixel values, and the pixel values of the pixel points greater than or equal to 255 are set to 255, so as to obtain the ROI image pair with ROI information, where the area with pixel value less than 255 is the ROI; Step 4: Second stereo matching of the ROI and disparity filling of the non-ROI; For the ROI image pair Perform stereo matching and disparity filling for non-ROI areas; first, for the valid disparity range obtained in step 2 Perform upsampling by the same factor as the downsampling to obtain the disparity range To adapt to the disparity search of the original resolution image: ； To reduce errors, appropriately expand the effective parallax range, and the parallax value is , and the obtained value is rounded. represents a positive integer: ； The matching image is the one obtained in Step 3 , and only the stereo matching disparity calculation is performed on the ROI in this process. The disparity of the non-ROI is directly obtained from the disparity map obtained in the first stereo matching in Step 2; the specific filling steps are as follows: First, upsample the initial disparity map with the same upsampling factor, and interpolate to obtain a disparity map with the same resolution as the corrected aggregate image pair in Step 1 . For the non-ROI pixel points in , find the corresponding points in , that is, the pixel points with the same pixel coordinates, and then obtain their pixel values to get the final disparity map .

2. The fast stereo matching method for aggregate reconstruction based on grayscale according to claim 1, characterized in that, In Step 1, the distortion correction is a process of obtaining the distortion coefficients of the camera through calibration, and then using the distortion coefficients to correct the distortion introduced by the deviation of the manufacturing accuracy and assembly process of the camera lens.

3. A fast stereo matching method for aggregate reconstruction based on grayscale according to claim 1, characterized in that, In Step 1, the epipolar correction is to obtain the internal and external parameters of the camera through calibration, and then use the internal and external parameters to correct the two images with actually non-coplanar rows aligned into coplanar rows aligned, and correct the actual binocular system into an ideal binocular system; the coplanar row alignment means that the image planes of the two cameras are on the same plane, and when the same point is projected onto the image planes of the two cameras, it should be on the same row of the two pixel coordinate systems.

4. A fast stereo matching method based on gray scale for aggregate reconstruction according to claim 1, characterized in that In Step 2, the methods for removing mismatched points and invalid matching points include the left-right consistency method, removing small connected regions, and uniqueness detection.

5. A fast stereo matching method for aggregate reconstruction based on grayscale according to claim 1, characterized in that, In Step 3, the segmentation algorithms include the clustering segmentation algorithm, the watershed segmentation algorithm, the region growing segmentation algorithm, and the neural network segmentation algorithm.

6. The fast stereo matching method for aggregate reconstruction based on gray scale according to claim 1, wherein, In step 3, the binary segmentation image pair has an ROI pixel value of 255 and a non-ROI pixel value of 0. After performing color inversion processing, we obtain where the ROI pixel value is 0 and the non-ROI pixel value is 255.

7. A fast stereo matching method for aggregate reconstruction based on grayscale according to claim 1, characterized in that, In Step 3, the morphological erosion operation is used for binary images. By calculating the minimum pixel value of the local n×n matrix and then assigning the minimum pixel value to each element of the n×n matrix, the black region in the image, that is, the region with a pixel value of 0, is expanded.

8. A fast stereo matching method based on grayscale for aggregate reconstruction according to claim 1, characterized in that, In Step 4, the second stereo matching only needs to calculate the disparity for the ROI, and the background region of the non-ROI directly uses the disparity of the corresponding points obtained by the first stereo matching for filling; the second stereo matching of the ROI and the disparity filling of the non-ROI are carried out simultaneously, that is, when traversing the ROI pixels, the stereo matching disparity is calculated, and when traversing the non-ROI pixels, the disparity filling is carried out.

9. A fast stereo matching method for aggregate reconstruction based on grayscale according to claim 1, characterized in that In step 4, the disparity map obtained by upsampling interpolation is the disparity map with the same resolution as the low-precision disparity. The non-ROI disparity is obtained from this, and the disparity of the ROI is the high-precision disparity obtained by performing stereo matching calculation on the original-resolution image.

Images