Coal mine underground image stereo matching method based on threshold and weight census transform

Abstract

The application discloses a kind of coal mine underground image stereo matching methods based on threshold and weight Census transformation, comprising the following steps: S1, image information is collected by two monocular camera modules binocular holder;S2, the gray value of all pixels in support window is thresholded;S3, improved center point pixel calculation method obtains matching generation value;S4, improved dynamic cross-domain obtains cost aggregation value;S5, disparity value is obtained using WTA strategy, S6, the overall process is verified on the visual system of underground unmanned auxiliary transport vehicle above-mentioned.The application applies threshold and weight Census transformation method to coal mine underground perception, realizes the autonomous obstacle avoidance and visual reconnaissance function of coal mine underground unmanned auxiliary transport vehicle, reduces the influence of factors such as dust, unstable illumination conditions on stereo matching, improves the accuracy of stereo matching.

Description

Technical Field

[0001] This invention belongs to the field of coal mine safety production technology, and more specifically, relates to a stereo matching method for underground coal mine images based on threshold and weighted Census transform. Background Technology

[0002] In the underground coal mine environment, the computational cost of stereo matching is easily affected by coal dust and particulate matter, unstable lighting, and long-term exposure of the vision sensor, resulting in salt-and-pepper noise during image acquisition, which significantly reduces the accuracy of stereo matching. Therefore, reducing the computational cost of matching is crucial for the vision system of unmanned assisted transport vehicles in underground coal mines.

[0003] By comparing the matching performance of different matching cost calculation methods under salt-and-pepper noise, it was found that methods based on pixel gray values, such as the sum of absolute gray-level differences, are greatly affected by salt-and-pepper noise. The method based on Census transform, however, exhibits strong robustness to salt-and-pepper noise, which is significant for selecting the matching cost calculation method in the stereo matching of unmanned vehicles in coal mines. However, the traditional Census transform relies too heavily on the center pixel of the transform window, making it susceptible to noise. To address this deficiency, some researchers have combined the absolute gray-level difference with the Census transform. During cost calculation, the cost of the analog-to-digital (AD) transformation and the cost of the Census transform are normalized and added together. This approach considers not only the absolute values ​​of the three color differences but also performs a Census transform on the center pixel, fully utilizing the cross-correlation between pixels. This method shows some robustness to salt-and-pepper noise, but outliers still exist when the salt-and-pepper noise is severe, leading to incorrect disparity estimation. Some researchers have improved the Census algorithm by employing a mean-based Census transform. When calculating the matching cost, the average grayscale value of each pixel within the window is used instead of the grayscale value of the center pixel. This method avoids excessive reliance on the center pixel, but it is highly dependent on pixels within the window. In the case of salt-and-pepper noise, outliers can significantly impact the mean calculation, leading to an increase in false matches. In the underground coal mine environment, salt-and-pepper noise increases the false match rate in stereo matching, disrupts the continuity of disparity, and reduces the stability and robustness of matching. Therefore, researching and improving stereo matching algorithms to address the impact of salt-and-pepper noise is crucial.

[0004] Therefore, given the above problems, it is necessary to propose a stereo matching algorithm based on threshold and weighted Census transform, so as to effectively reduce the influence of salt-and-pepper noise on stereo matching, reduce matching error, improve matching accuracy, and provide reliable perception information for intelligent decision-making systems of unmanned assisted transport vehicles. Summary of the Invention

[0005] This invention discloses a stereo matching algorithm for underground coal mine images based on threshold and weighted Census transform. It can provide a better stereo matching algorithm for the visual perception of unmanned assisted transport vehicles in the face of factors such as high coal dust and unstable lighting in underground coal mines, thereby reducing matching errors and improving matching accuracy.

[0006] To address at least one of the aforementioned technical problems, according to one aspect of the present invention, a method for stereo matching of underground coal mine images based on threshold and weighted Census transform is provided, comprising the following steps:

[0007] S1. Image information is acquired using a binocular pan-tilt unit consisting of two monocular cameras;

[0008] S2. Threshold the grayscale values ​​of all pixels within the support window;

[0009] S3. Use an improved center point pixel calculation method to obtain the matching cost;

[0010] S4. Obtain the cost aggregation value using the improved dynamic cross-domain;

[0011] S5. Use the WTA strategy to obtain disparity values, and then use left-right consistency checks, improved disparity filling, median filtering, and sub-pixel thinning methods to optimize the disparity map and disparity values.

[0012] Preferably, in step S2, thresholding is performed on the grayscale values ​​of all pixels within the support window, including the following steps:

[0013] S21. Traverse the grayscale values ​​of all pixels in the window and find the maximum and minimum values;

[0014] S22. Compare the maximum and minimum values ​​with the set maximum and minimum thresholds, and select the maximum and minimum values ​​after comparison as the thresholds;

[0015] S23. Compare the obtained threshold with all pixels in the window, and remove pixels in the window that are greater than or equal to the maximum threshold and less than or equal to the minimum threshold.

[0016] Preferably, in S3, the improved center point pixel calculation method for obtaining the matching cost includes the following steps:

[0017] S31. Select the four diagonal pixels corresponding to the center point of the pixel in the window after processing in S2, namely the top left, bottom left, top right, and bottom right.

[0018] S32. Take 12 pixels from each of the four diagonal lines according to their distance from the center pixel.

[0019] S33. Assign weights of 0.7, 0.2, and 0.1 to the 12 pixels with step sizes of 1, 2, and 3 respectively. Do not assign weights to pixels within the window that have undergone thresholding and are located on the diagonal. The specific formula is as follows:

[0020]

[0021] In the formula, I(a) represents the gray value of the pixel on the diagonal line; x, y represent the coordinates of the center pixel of the window; h, w represent the coordinates of the pixel on the diagonal line;

[0022] S34. Multiply each of the 12 valid pixels by its corresponding weight, and then divide by the sum of the valid weights to obtain the weighted center pixel value. The specific formula is as follows:

[0023] I(p1)=(0.7*I(x±1,j±1)+0.2*I(x±2,j±2)+0.1*I(x±3,j±3)) / 4 (2)

[0024] In the formula, I(p1) represents the weighted gray value of the center point, and I(x±i,j±j) represents the gray value on the diagonal line.

[0025] S35. Compare the calculated value of the center pixel with other pixels within the window to generate a binary string. The specific formula is as follows:

[0026]

[0027]

[0028] In the formula, T Census (p) represents the binary string generated by window transformation of the center pixel p; N p Then I(p) represents the neighborhood of pixel p; I(p) is the gray value of the center pixel p after weighted fusion processing; I(q) is the gray value of pixel q in the neighborhood. Indicates a bitwise concatenation operator;

[0029] Perform an XOR operation on the binary strings of corresponding pixels in the left and right images, count the number of 1s in the resulting binary string, and obtain the Hamming distance. This gives the calculated matching cost, as shown in the specific formula:

[0030] C Census (p,q)=Hamming(T(p),T(q)) (4)

[0031] In the formula, T(p) is the binary string generated by the left figure, T(q) is the binary string generated by the right figure, and the Hamming distance is the number of corresponding bits that are different between the two binary strings;

[0032] S36. To verify the superiority of the threshold- and weight-based Census algorithm in matching cost calculation, comparative experiments were conducted on the traditional Census, mean Census, and the Census method used in this paper. Four sets of images Cones, Teddy, Dolls, and Aloe from the Middlebury dataset were used to compare the initial matching cost maps and standard disparity maps obtained by the traditional Census, mean Census, and the Census method used in this paper. The mismatch rate of all regions and the mismatch rate of non-occluded regions were also analyzed.

[0033] S37. Compare the algorithm in this paper with the traditional Census algorithm and the mean-based Census algorithm, calculate the matching cost under different noise levels, and eliminate the interference of cost aggregation and disparity optimization.

[0034] Preferably, the specific steps of S4 are as follows:

[0035] S41. By setting three different color and distance thresholds, three dynamic cross-shaped regions of different sizes are generated for the same pixel.

[0036] S42. Calculate the average of the cost aggregation values ​​of the three dynamic cross-domains corresponding to the pixel as the final cost aggregation value of the pixel.

[0037] Preferably, the specific steps of S5 are as follows:

[0038] S51. The cost aggregation value in S4 is used to select the feature with the smallest disparity value for each pixel using the WTA strategy.

[0039] S52. The left-right consistency detection method is used to detect mismatched disparity points. Outliers are detected by comparing the disparity values ​​of matching disparity points in the left and right images. The calculation process is as follows:

[0040] |d l (p)-d r (p+d)|>α (5)

[0041] Where, d l (p) represents the disparity value of point p in the left matching map, d r (p+d) is the disparity value corresponding to point p in the right matching graph, and α is the decision threshold;

[0042] S53. The mismatched regions obtained after the left-right consistency check are filled with disparity using an improved 8-way interpolation method. The average of the ten most recent valid values ​​for each method is taken as the disparity value of that method, and then the average of the 8 directions is taken as the disparity value of the mismatched region.

[0043] S54. A 3×3 sliding window is used to perform median filtering and smoothing on the disparity map to remove some isolated outliers in the disparity map and obtain the final disparity map.

[0044] S55. Sub-pixel refinement is performed on the disparity values ​​obtained through WTA, and the disparity values ​​represented by the minimum points of the quadratic curve are taken as pixel disparity values.

[0045] S56. To verify the performance of this algorithm, the disparity maps generated by the three algorithms are compared.

[0046] Preferably, the specific steps of S6 are as follows:

[0047] S61. Use the binocular camera gimbal of the underground unmanned assisted transport vehicle to acquire black and white checkerboard images and perform camera calibration of the binocular camera gimbal.

[0048] S62. Select 25 pairs of clear, multi-angle images for calibration; use the obtained parameters to correct the images, and obtain the intrinsic parameter matrix L of the left camera, the intrinsic parameter matrix R of the right camera, the rotation matrix Q, and the translation matrix T through calibration, i.e.:

[0049]

[0050]

[0051]

[0052] T = [-309.467 11.985 -22.362] (6);

[0054] S63. Apply the stereo matching algorithm described above to the calibrated left and right images to obtain the corresponding disparity maps.

[0055] According to another aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for stereo matching of coal mine underground images based on threshold and weighted Census transform of the present invention.

[0056] According to another aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for stereo matching of coal mine underground images based on threshold and weight Census transform of the present invention.

[0057] Compared with the prior art, the present invention has at least the following beneficial effects:

[0058] This invention applies a stereo matching algorithm based on threshold and weighted Census transform to the visual perception of unmanned assisted transport vehicles in coal mines. It enables environmental perception and obstacle localization of the unmanned assisted transport vehicles, thereby planning safe driving paths. It can perform stereo matching more accurately in coal mines, making up for the shortcomings of traditional stereo matching algorithms in coal mines and improving the accuracy of stereo matching.

[0059] This invention effectively reduces the proportion of salt-and-pepper noise within the window through thresholding, thus mitigating its impact on stereo matching. This helps improve the accuracy and stability of the matching results. Simultaneously, thresholding addresses the influence of outliers on weighted fusion, preventing them from misleading the matching results. This enhances the algorithm's robustness to factors such as high dust levels and unstable lighting conditions in underground coal mine environments. Furthermore, by employing a weighting strategy, larger weights are allocated to reliable pixels, avoiding the over-reliance on the center point of the Census transform window found in traditional algorithms. This contributes to improved matching accuracy and reduced false matching rates.

[0060] Compared to global matching algorithms, this invention has lower computational complexity and can perform stereo matching more efficiently. This makes the algorithm more feasible for real-time applications and large-scale data processing. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of the present invention and are not intended to limit the present invention.

[0062] Figure 1 This is a flowchart of the method of the present invention;

[0063] Figure 2 For the original Figure 3 Comparison chart of cost calculations for the two algorithms;

[0064] Figure 3 The bar chart shows the mismatch rate of the three algorithms in the whole region under salt-and-pepper noise (the horizontal axis is the noise density and the vertical axis is the mismatch rate of the whole region).

[0065] Figure 4 The bar chart shows the mismatch rate of the three algorithms in the unoccluded area under salt-and-pepper noise (the horizontal axis is the noise density and the vertical axis is the mismatch rate in the unoccluded area).

[0066] Figure 5 Comparison of experimental results under 10% salt-and-pepper noise;

[0067] Figure 6 For reprojection error diagram;

[0068] Figure 7To correct the left and right images before and after;

[0069] Figure 8 The corrected left and right images;

[0070] Figure 9 The following are parallax maps of coal mines using three different algorithms. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention.

[0072] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0073] like Figure 1-9 As shown,

[0074] Example 1:

[0075] like Figure 1 The stereo matching algorithm for underground coal mine images based on threshold and weighted Census transform, as shown, includes the following steps:

[0076] S1. Image information is acquired using a binocular pan-tilt unit consisting of two monocular cameras;

[0077] S2. Threshold the grayscale values ​​of all pixels within the support window, including the following steps:

[0078] S21. Traverse the grayscale values ​​of all pixels in the window and find the maximum and minimum values;

[0079] S22. Compare the maximum and minimum values ​​with the set maximum and minimum thresholds, and select the maximum and minimum values ​​after comparison as the thresholds;

[0080] S23. Compare the obtained threshold with all pixels in the window, and remove pixels in the window that are greater than or equal to the maximum threshold and less than or equal to the minimum threshold.

[0081] S3. An improved method for calculating the center point pixel to obtain the matching cost value includes the following steps:

[0082] S31. Select the four diagonal pixels corresponding to the center point of the pixel in the window after processing in S2, namely the top left, bottom left, top right, and bottom right.

[0083] S32. Take 12 pixels from each of the four diagonal lines according to their distance from the center pixel.

[0084] S33. Assign weights of 0.7, 0.2, and 0.1 to these 12 pixels with step sizes of 1, 2, and 3 respectively. Pixels within the window that have undergone thresholding and are located on the diagonal do not require weighting. The specific formula is as follows:

[0085]

[0086] In the formula, I(a) represents the gray value of the pixel on the diagonal line; x,y represent the coordinates of the center pixel of the window; and h,w represent the coordinates of the pixel on the diagonal line.

[0087] S34. Multiply each of the 12 valid pixels by its corresponding weight, and then divide by the sum of the valid weights to obtain the weighted center pixel value. The specific formula is as follows:

[0088] I(p1)=(0.7*I(x±1,j±1)+0.2*I(x±2,j±2)+0.1*I(x±3,j±3)) / 4 (2)

[0089] In the formula, I(p1) represents the weighted gray value of the center point, and I(x±i,j±j) represents the gray value on the diagonal line.

[0090] S35. Compare the calculated value of the center pixel with other pixels within the window to generate a binary string. The specific formula is as follows:

[0091]

[0092]

[0093] In the formula, T Census (p) represents the binary string generated by window transformation of the center pixel p; N p Then I(p) represents the neighborhood of pixel p; I(p) is the gray value of the center pixel p after weighted fusion processing; I(q) is the gray value of pixel q in the neighborhood. This indicates a bitwise concatenation operator.

[0094] Perform an XOR operation on the binary strings of corresponding pixels in the left and right images, count the number of 1s in the resulting binary string, and obtain the Hamming distance. This gives the calculated matching cost, as shown in the specific formula:

[0095] C Census (p,q)=Hamming(T(p),T(q)) (4)

[0096] In the formula, T(p) is the binary string generated by the left figure, T(q) is the binary string generated by the right figure, and the Hamming distance is the number of corresponding bits that are different between the two binary strings.

[0097] S36. To verify the superiority of the threshold- and weight-based Census algorithm in matching cost calculation, comparative experiments were conducted on the traditional Census, mean Census, and the Census method used in this paper. Four sets of images (Cones, Teddy, Dolls, and Aloe) from the Middlebury dataset were used. The initial matching cost maps and standard disparity maps obtained by the three algorithms were compared, and the mismatch rate for all regions and the mismatch rate for non-occluded regions were analyzed. The experimental results are as follows: Figure 2 As shown, Figure 2 The images are, in order: the original left image, the true disparity map, the traditional Census algorithm, the mean-based Census algorithm, and the Census algorithm used in this paper. Table 1 shows the error rates for all regions of the three algorithms, and Table 2 shows the mismatch rates for non-occluded regions.

[0098] S37. To further verify the noise resistance of the algorithm in this paper, a comparative experiment was conducted with the traditional Census algorithm and the mean-based Census algorithm. The matching cost under different noise levels was calculated to eliminate the interference of cost aggregation and disparity optimization.

[0099] In the experiment, salt-and-pepper noise with densities of 0.01, 0.05, 0.1, 0.15, and 0.2 was added to the test images. Three algorithms were used to calculate the initial matching cost maps of the four images under different noise levels. The obtained initial matching cost maps were compared with the standard disparity maps, and the mismatch rate of each algorithm in all regions and non-occluded regions of the four images under each noise level was calculated. The results are shown in Tables 3 and 4.

[0100] S4. The improved dynamic cross-domain obtains the cost aggregation value, including the following steps:

[0101] S41. By setting three different color and distance thresholds, three dynamic cross-shaped domains of different sizes are generated for the same pixel.

[0102] S42. Calculate the average of the cost aggregation values ​​of the three dynamic cross-domains corresponding to the pixel as the final cost aggregation value of the pixel.

[0103] S5. Obtain disparity values ​​using the WTA strategy, and then optimize the disparity map and disparity values ​​using left-right consistency checks, improved disparity filling, median filtering, and sub-pixel thinning methods, including the following steps:

[0104] S51. The cost aggregation value in S4 is used to select the feature with the smallest disparity value for each pixel using the WTA strategy.

[0105] S52. The left-right consistency detection method is used to detect mismatched disparity points. Outliers are detected by comparing the disparity values ​​of matching disparity points in the left and right images. The calculation process is as follows:

[0106] |d l (p)-d r (p+d)|>α (5)

[0107] Where, d l (p) represents the disparity value of point p in the left matching map, d r (p+d) is the disparity value corresponding to point p in the right matching graph, and α is the decision threshold.

[0108] S53. The mismatched regions obtained after the left-right consistency check are filled with disparity using an improved 8-direction interpolation method. The average of the ten most recent valid values ​​for each method is taken as the disparity value of that method, and then the average of the 8 directions is taken as the disparity value of the mismatched regions.

[0109] S54. A 3×3 sliding window is used to perform median filtering and smoothing on the disparity map to remove some isolated outliers in the disparity map and obtain the final disparity map.

[0110] S55. Sub-pixel refinement is performed on the disparity values ​​obtained through WTA, and the disparity value represented by the minimum point of the quadratic curve is taken as the pixel disparity value.

[0111] S56. To verify the performance of this algorithm, the disparity maps generated by the three algorithms are compared, and the results are shown in the figure below. Figure 5 As shown, from top to bottom, the images are: the original left image, the true disparity map, the left image with 0.1 salt-and-pepper noise, the initial matching cost map of the traditional Census algorithm under noise, the initial matching cost map of the mean-based Census algorithm under noise, the initial matching cost map of the algorithm under noise, the final disparity map of the algorithm under noise, and the mismatched pixel map of the algorithm under noise.

[0112] S6. Verify the above process on the vision system of the unmanned underground transport vehicle.

[0113] S61. Use the binocular camera gimbal of the underground unmanned assisted transport vehicle to acquire black and white checkerboard images and perform camera calibration on the binocular camera gimbal. Use a black and white checkerboard as the calibration plate, with each checkerboard square being 41mm in size, and obtain the camera's internal and external parameters through calibration.

[0114] S62. Select 25 pairs of clear, multi-angle images for calibration. Use Matlab software to calibrate the reprojection error as follows: Figure 6 As shown, the average error is 0.22, which meets practical standards. Using the obtained parameters to correct the image yields the following results: Figure 7 , Figure 8 As shown. Through calibration, the intrinsic parameter matrix L of the left camera, the intrinsic parameter matrix R of the right camera, the rotation matrix Q, and the translation matrix T are obtained, i.e.:

[0115]

[0116]

[0117]

[0118] T = [-309.467 11.985 -22.362] (6)

[0120] S63. Apply the stereo matching algorithm described above to the calibrated left and right images to obtain the corresponding disparity maps, as shown in the image. Figure 9 As shown. From left to right, the images are: the left image obtained using a binocular camera pan-tilt unit, the traditional Census disparity map, the mean-based Census disparity map, and the disparity map obtained using the algorithm presented in this paper.

[0121] Example 2:

[0122] The computer-readable storage medium of this embodiment stores a computer program that, when executed by a processor, implements the steps in the stereo matching method for underground coal mine images based on threshold and weighted Census transform of Embodiment 1.

[0123] The computer-readable storage medium in this embodiment can be an internal storage unit of the terminal, such as the terminal's hard disk or memory; the computer-readable storage medium in this embodiment can also be an external storage device of the terminal, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc. equipped on the terminal; furthermore, the computer-readable storage medium can include both the terminal's internal storage unit and external storage devices.

[0124] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0125] Example 3:

[0126] The computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the stereo matching method for underground coal mine images based on threshold and weight Census transform of Embodiment 1.

[0127] In this embodiment, the processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The memory can include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0128] Those skilled in the art will understand that the content disclosed in the embodiments can be provided as a method, system, or computer program product. Therefore, this solution can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this solution can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage) containing computer-usable program code.

[0129] This solution is described with reference to flowchart illustrations and / or block diagrams of methods and computer program products according to embodiments of this solution. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0132] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0133] The examples described herein are merely preferred embodiments of the invention and are not intended to limit the concept and scope of the invention. Any modifications and improvements made by those skilled in the art to the technical solutions of the invention without departing from the design concept of the invention should fall within the protection scope of the invention.

Claims

1. A stereo matching method for underground coal mine images based on threshold and weighted Census transform, characterized in that, Includes the following steps: S1. Image information is acquired using a binocular pan-tilt unit consisting of two monocular cameras; S2. Threshold the grayscale values ​​of all pixels within the support window; S3. An improved method for calculating the center point pixel to obtain the matching cost; S4. Improved dynamic cross-domain for obtaining cost aggregation value; S5. Use the WTA strategy to obtain disparity values, and then use left-right consistency check, improved disparity filling, median filtering and sub-pixel thinning methods to optimize the disparity map and disparity values; S6. Verify the entire process described above on the vision system of the unmanned underground assisted transport vehicle. The specific steps of S2 are as follows: S21. Traverse the grayscale values ​​of all pixels in the window and find the maximum and minimum values; S22. Compare the maximum and minimum values ​​with the set maximum and minimum thresholds, and select the maximum and minimum values ​​after comparison as the thresholds; S23. Compare the obtained threshold with all pixels in the window, and remove pixels in the window that are greater than or equal to the maximum threshold and less than or equal to the minimum threshold. The specific steps for S3 are as follows: S31. Select the four diagonal pixels corresponding to the center point of the pixel in the window after processing in S2, namely the top left, bottom left, top right, and bottom right. S32. Take 12 pixels from each of the four diagonal lines according to their distance from the center pixel. S33. Assign weights of 0.7, 0.2, and 0.1 to the 12 pixels with step sizes of 1, 2, and 3 respectively. Do not assign weights to pixels within the window that have undergone thresholding and are located on the diagonal. The specific formula is as follows: （1） In the formula This represents the grayscale value of the pixels on the diagonal line; Indicates the coordinates of the center pixel of the window; Indicates the coordinates of the pixel on the diagonal line; S34. Multiply each of the 12 valid pixels by its corresponding weight, and then divide by the sum of the valid weights to obtain the weighted center pixel value. The specific formula is as follows: （2） In the formula This represents the weighted gray value of the center point. This represents the grayscale value on the diagonal line; S35. Compare the calculated value of the center pixel with other pixels within the window to generate a binary string. The specific formula is as follows: ； （3） In the formula, Represents the center pixel The binary string generated by window transformation; This represents a pixel. The neighborhood; The center pixel after weighted fusion processing grayscale value; For pixels in the neighborhood grayscale value; Indicates a bitwise concatenation operator; Perform an XOR operation on the binary strings of corresponding pixels in the left and right images, count the number of 1s in the resulting binary string, and obtain the Hamming distance. This gives the calculated matching cost, as shown in the specific formula: （4） In the formula, The binary string generated in the left image. The binary string generated in the image on the right. Distance is the number of corresponding bits that are different between two binary strings; S36. To verify the advancement of the threshold- and weight-based Census algorithm in matching cost calculation, comparative experiments were conducted on the traditional Census, mean Census, and the Census method used in this paper. Four sets of images Cones, Teddy, Dolls, and Aloe from the Middlebury dataset were used to compare the initial matching cost maps and standard disparity maps obtained by the traditional Census, mean Census, and the Census method used in this paper. The mismatch rate of all regions and the mismatch rate of non-occluded regions were also analyzed. S37. Compare the algorithm in this paper with the traditional Census algorithm and the mean-based Census algorithm, calculate the matching cost under different noise levels, and eliminate the interference of cost aggregation and disparity optimization. The specific steps for S4 are as follows: S41. By setting three different color and distance thresholds, three dynamic cross-shaped domains of different sizes are generated for the same pixel. S42. Calculate the average of the cost aggregation values ​​of the three dynamic cross-domains corresponding to the pixel as the final cost aggregation value of the pixel.

2. The method according to claim 1, characterized in that, The specific steps for S5 are as follows: S51. The cost aggregation value in S4 is used to select the feature with the smallest disparity value for each pixel using the WTA strategy. S52. The left-right consistency detection method is used to detect mismatched disparity points. Outliers are detected by comparing the disparity values ​​of matching disparity points in the left and right images. The calculation process is as follows: （5） in, For the left matching point in the graph The disparity value, To match the points in the right graph The corresponding disparity value, To determine the threshold; S53. The mismatched regions obtained after the left-right consistency check are filled with disparity using an improved 8-way interpolation method. The average of the ten most recent valid values ​​for each method is taken as the disparity value of that method, and then the average of the 8 directions is taken as the disparity value of the mismatched regions. S54. A 3×3 sliding window is used to perform median filtering and smoothing on the disparity map to remove some isolated outliers in the disparity map and obtain the final disparity map. S55. Sub-pixel refinement is performed on the disparity values ​​obtained through WTA, and the disparity values ​​represented by the minimum points of the quadratic curve are taken as pixel disparity values. S56. To verify the performance of this algorithm, the disparity maps generated by the three algorithms are compared.

3. The method according to claim 2, characterized in that, The specific steps for S6 are as follows: S61. Use the binocular camera gimbal of the underground unmanned assisted transport vehicle to acquire black and white checkerboard images and perform camera calibration of the binocular camera gimbal. S62. Select 25 pairs of clear, multi-angle images for calibration; use the obtained parameters to correct the images, and obtain the intrinsic parameter matrix L of the left camera, the intrinsic parameter matrix R of the right camera, the rotation matrix Q, and the translation matrix T through calibration, i.e.: ； ； ； （6）； S63. Apply the above stereo matching algorithm to the calibrated left and right images to obtain the corresponding disparity map.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the steps in the method for stereo matching of coal mine underground images based on threshold and weight Census transform as described in any one of claims 1 to 3.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for stereo matching of coal mine underground images based on threshold and weight Census transform as described in any one of claims 1 to 3.

Images