Anchor drilling automatic positioning system and method based on binocular vision

Abstract

The present invention relates to an anchor drilling automatic positioning system and method based on binocular vision, belonging to the technical field of automatic drilling positioning, and includes an anchor drilling robot, a binocular vision component and a controller. The binocular vision component photographs a mining support steel belt arranged on a tunnel roof or a side wall to obtain a steel belt image pair, and the controller processes the steel belt image pair to obtain the spatial three-dimensional coordinates of the center of the anchor drilling hole on the mining support steel belt, and controls the anchor drilling robot to work according to the spatial three-dimensional coordinates to complete the drilling process of the anchor drilling hole, thereby being able to automatically locate the anchor drilling position and automatically drill the hole, so as to solve the problems of high labor intensity, low efficiency and poor safety in the drilling process of the existing manually operated anchor drilling machine or anchor drilling vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic drilling positioning, and in particular to an automatic anchor drilling positioning system and method based on binocular vision. Background Art

[0002] The automation and intelligence level of coal mine tunnels in my country is low. Tunnel formation mainly includes tunnel excavation and tunnel support. Tunnel excavation is completed by comprehensive excavation equipment, accounting for 30% of the tunnel formation time. Tunnel support is mainly completed by manually carrying single drilling rigs, accounting for more than 60% of the tunnel formation time. The workers have high labor intensity, low efficiency and high risk. Therefore, improving the efficiency of tunnel support has become a difficult problem that needs to be solved urgently in the rapid development of automation and intelligence of comprehensive excavation working faces. In order to improve the excavation efficiency and system safety of comprehensive excavation working faces, the research on automatic positioning technology of anchor drilling holes is of great significance. Summary of the invention

[0003] The purpose of the present invention is to provide an automatic anchor drilling positioning system and method based on binocular vision, which can automatically locate the anchor drilling position and automatically drill holes to solve the problems of high labor intensity, low efficiency and poor safety in the drilling process of existing manually operated anchor drilling rigs or anchor drilling vehicles.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] An anchor drilling automatic positioning system based on binocular vision, the automatic positioning system comprises an anchor drilling robot, a binocular vision component and a controller; the binocular vision component is fixedly installed on the drilling arm of the anchor drilling robot; the controller is respectively connected to the anchor drilling robot and the binocular vision component for communication;

[0006] The binocular vision component is used to photograph the mining support steel belt arranged on the top plate or the side wall of the tunnel to obtain a pair of steel belt images; the pair of steel belt images includes a first steel belt image and a second steel belt image;

[0007] The controller is used to process the steel belt image pair to obtain the spatial three-dimensional coordinates of the center of the anchor hole on the mining support steel belt, and control the drilling and anchoring robot to work according to the spatial three-dimensional coordinates to complete the drilling process of the anchor hole.

[0008] An automatic anchor drilling positioning method based on binocular vision, the automatic positioning method comprising:

[0009] Receiving a pair of steel strip images obtained by photographing a mining support steel strip by a binocular vision component; the pair of steel strip images includes a first steel strip image and a second steel strip image;

[0010] Processing the steel belt image pair to obtain the spatial three-dimensional coordinates of the center of the anchor drill hole on the mining support steel belt;

[0011] The anchor drilling robot is controlled to work according to the three-dimensional spatial coordinates to complete the drilling process of the anchor hole.

[0012] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0013] The present invention is used to provide an anchor drilling automatic positioning system and method based on binocular vision, including an anchor drilling robot, a binocular vision component and a controller. The binocular vision component photographs a mining support steel belt arranged on a tunnel roof or a side wall to obtain a steel belt image pair, and the controller processes the steel belt image pair to obtain the spatial three-dimensional coordinates of the center of the anchor drilling hole on the mining support steel belt, and controls the anchor drilling robot to work according to the spatial three-dimensional coordinates to complete the drilling process of the anchor drilling hole, thereby being able to automatically locate the anchor drilling position and automatically drill the hole, so as to solve the problems of high labor intensity, low efficiency and poor safety in the drilling process of the existing manually operated anchor drilling machine or anchor drilling vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A schematic diagram of the structure of the automatic positioning system provided in Example 1 of the present invention;

[0016] Figure 2 This is a schematic diagram of the installation of the binocular vision component provided in Example 1 of the present invention;

[0017] Figure 3 A schematic diagram of the structure of a binocular vision component provided in Example 1 of the present invention;

[0018] Figure 4 A method flow chart of the automatic positioning method provided by Embodiment 2 of the present invention;

[0019] Figure 5 A flowchart of automatic anchor drilling positioning provided by Embodiment 2 of the present invention;

[0020] Figure 6 A schematic diagram of the positions of various coordinate systems provided in Example 2 of the present invention.

[0021] Explanation of symbols:

[0022] 1-controller; 2-drilling and anchoring robot; 3-drilling arm; 4-binocular vision component; 5-mine support steel belt; 6-anchor drilling hole; 41-camera mounting bracket; 42-fill light device; 43-mine explosion-proof camera. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The purpose of the present invention is to provide an automatic anchor drilling positioning system and method based on binocular vision, which can automatically locate the anchor drilling position and automatically drill holes to solve the problems of high labor intensity, low efficiency and poor safety in the drilling process of existing manually operated anchor drilling rigs or anchor drilling vehicles.

[0025] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Embodiment 1:

[0027] This embodiment is used to provide an automatic anchor drilling positioning system based on binocular vision. Figure 1 As shown, the automatic positioning system includes a drilling and anchoring robot 2, a binocular vision component 4 and a controller 1, and the controller 1 can be an explosion-proof computer. The binocular vision component 4 is fixedly installed on the drilling arm 3 of the drilling and anchoring robot 2, and the controller 1 can be fixedly installed on the drilling and anchoring robot 2 or fixedly installed at other locations outside, but it is necessary to ensure that the controller 1 is communicated with the drilling and anchoring robot 2 and the binocular vision component 4 respectively.

[0028] The binocular vision component 4 is used to shoot the mining support steel belt 5 set on the top plate or side wall of the tunnel to obtain a pair of steel belt images, which includes a first steel belt image and a second steel belt image. Before anchoring, the mining support steel belt 5 is placed on the temporary support device of the drilling and anchoring robot, against the top plate or side wall of the tunnel, and circular holes (i.e., anchor holes 6) are evenly arranged on the mining support steel belt 5. By drilling the anchor holes 6, the top plate anchor net and the anchor rod are anchored together to play a connecting role, so as to be anchored and connected with the top plate or side wall of the tunnel. The purpose of this embodiment is to automatically find the position of the anchor hole 6 and automatically complete the drilling process.

[0029] The controller 1 is connected to the binocular vision component 4 via an Ethernet cable, and the steel belt image transmitted by the binocular vision component 4 is processed in real time to obtain the spatial three-dimensional coordinates of the center of the anchor hole 6 on the mining support steel belt 5 in the spatial coordinate system, and the drilling and anchoring robot 2 is controlled to work according to the spatial three-dimensional coordinates to perform automatic hole finding operations and complete the drilling process of the anchor hole 6. The controller 1 feeds back the acquired spatial three-dimensional coordinates to the drilling and anchoring robot 2, controls the drilling arm 3 of the drilling and anchoring robot 2 to adjust to the position directly below the center of each anchor hole, and then the drill rod completes the drilling operation.

[0030] Specifically, Figure 2 and Figure 3 As shown, the binocular vision component 4 used in this embodiment may include a camera mounting frame 41, a binocular camera and a fill light device 42. The bottom of the camera mounting frame 41 is fixedly mounted on the front end of the drill arm 3 of the drilling and anchoring robot 2 by bolts. The camera mounting frame 41 provides an installation position for the binocular camera. The binocular camera can be installed on the front end of the drill arm 3 of the drilling and anchoring robot 2. The optical axis direction of the binocular camera is parallel to the axis direction of the drill arm 3 and faces the top direction of the drill arm 3. The binocular camera includes a first camera and a second camera. Both the first camera and the second camera can be selected from a mining explosion-proof camera 43. The first camera and the second camera are fixed in parallel on both sides of the camera mounting frame 41 by bolts. The first camera is used to shoot the mining support steel belt 5 to obtain a first steel belt image, and the second camera is used to shoot the mining support steel belt 5 to obtain a second steel belt image. The first steel belt image and the second steel belt image constitute a steel belt image pair. The fill light device 42 can be fixedly set at the baseline center of the binocular camera component to provide the required lighting conditions for the binocular camera. It is used to fill light for the mining support steel belt 5 during the shooting process of the binocular camera, so as to fill light under low illumination conditions in the tunnel and improve the quality of the collected image.

[0031] When the system is working, the drill arm 3 of the drilling and anchoring robot 2 approaches the position of the mining support steel belt 5 according to the set program. When approaching the mining support steel belt 5, the binocular camera of the binocular vision component 4 collects the steel belt image pair through the image acquisition card, and transmits the steel belt image pair to the controller 1 for real-time processing, obtains the spatial three-dimensional coordinates of the center of the anchor hole 6 on the mining support steel belt 5 in the spatial coordinate system, and controls the drill arm 3 of the drilling and anchoring robot 2 to move to the bottom of the anchor hole 6 according to the spatial three-dimensional coordinates, completing the automatic hole finding operation and subsequent drilling operations. When approaching the first mining support steel belt 5, the position of the drill arm 3 can be manually adjusted. When approaching the subsequent mining support steel belt 5, the drill arm 3 is directly controlled to move automatically according to the distance between the two mining support steel belts 5. For example, when approaching the second mining support steel belt 5, the drill arm 3 is controlled to move automatically according to the distance between the first mining support steel belt 5 and the second mining support steel belt 5.

[0032] It should be noted that in order to be able to comprehensively photograph the entire mining support steel belt 5, this embodiment will continuously move the position of the drill arm 3 when photographing a mining support steel belt 5, and collect a steel belt image pair of the mining support steel belt 5 at each position. All steel belt image pairs need to cover the entire mining support steel belt 5, so as to determine the spatial three-dimensional coordinates of all anchor holes 6 on the mining support steel belt 5, and drill holes according to the spatial three-dimensional coordinates to complete the automatic positioning and automatic drilling process. The drilling process can be carried out synchronously with the shooting process of the steel belt image pair, that is, after all the anchor holes 6 of a steel belt image pair are identified, the anchor holes 6 are drilled according to the spatial three-dimensional coordinates of the anchor holes 6. After the drilling is completed, continue shooting. The drilling process can also be located after the shooting process, that is, after all the anchor holes 6 on the mining support steel belt 5 are identified, the anchor holes 6 are drilled in sequence.

[0033] The binocular vision positioning method is used to realize the automatic positioning of the anchor drill hole 6, obtain the three-dimensional coordinates of the center space of the anchor drill hole 6, and then automatically locate the position of the anchor drill hole 6 and automatically drill the hole, so as to solve the problems of high labor intensity, low efficiency and poor safety in the drilling process of the existing manually operated anchor drilling machine or anchor drilling vehicle. When shooting the steel belt image, the binocular camera can be placed closer to the steel belt, so that the obtained steel belt image quality is better and the accuracy is higher, so that the accurate automatic positioning of the anchor drill hole 6 is realized.

[0034] Embodiment 2:

[0035] This embodiment is used to provide an automatic anchor drilling positioning method based on binocular vision, which controls the automatic positioning system described in Example 1 to work, processes the steel strip image pairs collected by the binocular camera, and realizes automatic anchor drilling positioning and drilling. Figure 4 and Figure 5 As shown, the automatic positioning method includes:

[0036] S1: receiving a pair of images of a mining support steel strip obtained by photographing the steel strip with a binocular vision component; the pair of images of the steel strip includes a first steel strip image and a second steel strip image;

[0037] S2: Processing the steel belt image pair to obtain the spatial three-dimensional coordinates of the center of the anchor drill hole on the mining support steel belt;

[0038] The steps of processing the steel strip image pair in this embodiment include image enhancement, image denoising, stereo correction, stereo matching, edge detection and contour fitting, and the automatic acquisition process of the center coordinates of the anchor drilling hole is completed.

[0039] Image enhancement is mainly used to improve image contrast. The input is a pair of steel strip images collected by a binocular camera, and the output is an enhanced image pair obtained after image enhancement processing.

[0040] Image denoising is mainly used to remove image noise. The input is an enhanced image pair obtained after image enhancement processing, and the output is a denoised image pair obtained after image denoising processing.

[0041] Stereo correction has two main functions. First, due to reasons such as the camera manufacturing process, the camera-collected images have radial distortion and tangential distortion. Stereo correction can eliminate the impact of these adverse factors on the results. Second, the two cameras of a binocular camera are not necessarily ideal models (the ideal model means that the camera imaging planes are parallel and collinear). Stereo correction can correct them to the ideal model. The input of stereo correction is the denoised image pair obtained after image denoising, and the output is the corrected image pair obtained after stereo correction.

[0042] The function of stereo matching is to stereo match the corrected image pair to ensure the matching of corresponding points. The input is the corrected image pair obtained after stereo correction processing, and the output is the matched image pair obtained after stereo matching processing.

[0043] The function of edge detection is to obtain the edge point information of the circular anchor drilling hole on the steel belt. The input is the matching image pair obtained after stereo matching processing, and the output is the edge image pair obtained after edge detection processing.

[0044] The function of contour fitting is to fit the anchor hole contour according to the edge detection result and obtain the center coordinates of the anchor hole on the steel belt. The input is the edge image pair obtained after edge detection processing, and the output is the center coordinates of the anchor hole on the steel belt obtained after contour fitting processing.

[0045] Specifically, S2 may include:

[0046] (1) performing image enhancement, image denoising and stereo correction on the first steel strip image and the second steel strip image respectively to obtain a first corrected image and a second corrected image;

[0047] Among them, this embodiment performs image enhancement on the first steel strip image and the second steel strip image respectively based on the retinex method, improves the image contrast, solves the problem of unclear target contour in the image under low illumination environment, and obtains the first enhanced image and the second enhanced image. The first enhanced image and the second enhanced image constitute an enhanced image pair.

[0048] Among them, this embodiment selects median filtering for denoising according to the noise characteristics in the image to reduce the influence of image noise on the automatic positioning accuracy. For each pixel in the first enhanced image and the second enhanced image, the following formula is used to perform denoising respectively to obtain a first denoised image and a second denoised image, which constitute a denoised image pair.

[0049] g(x,y)=median[f(xi,yi),...,f(x,y),...f(x+i,y+i)], i∈Z; (1)

[0050] In formula (1), g(x, y) is the pixel value of the pixel at the coordinate (x, y) in the first denoised image (second denoised image); the output of median[.] is the middle value after arranging the values ​​in [.] in descending order; f(x, y) is the pixel value of the pixel at the coordinate (x, y) in the first enhanced image (second enhanced image); Z is an integer, which represents a certain window area, (x, y) represents the center position of the window area, and the size of the window area is determined by Z. For example, if Z is 1, the window area is a 3*3 neighborhood range centered on (x, y). At this time, the output of median[.] is to arrange the pixel values ​​of the nine pixels in the 3*3 neighborhood range from large to small and take the middle value.

[0051] Among them, the stereo correction is to make the left and right image planes on the same plane, and the same point is projected onto the two camera image planes on the same row of the two pixel coordinate systems, that is, to achieve coplanar row alignment of the two images, and to correct the actual binocular system to an ideal binocular system (i.e., an ideal model). In this embodiment, any existing stereo correction method can be used to correct the first denoised image and the second denoised image respectively to obtain a first corrected image and a second corrected image, and the first corrected image and the second corrected image constitute a corrected image pair.

[0052] (2) performing stereo matching on the first corrected image and the second corrected image respectively to obtain a first matching image and a second matching image;

[0053] Stereo matching refers to stereo matching of enhanced, denoised and corrected image pairs. The matching objects are anchor holes on steel belts. A feature-based matching algorithm is used to perform stereo matching of the corrected image pairs.

[0054] Specifically, the stereo matching process includes:

[0055] 1) extracting first feature points in the first corrected image using the SIFT algorithm to obtain a first point set; extracting second feature points in the second corrected image using the SIFT algorithm to obtain a second point set;

[0056] 2) Using SAD (Sum of Absolute Difference) as the matching cost function, perform preliminary matching on the first point set and the second point set to obtain an initial matching set;

[0057] 3) Using the RANSAC (Random Sample Consensus) algorithm, incorrect matching point pairs in the initial matching set are eliminated to obtain a matching set, and a first matching image and a second matching image are obtained based on the matching set. The first matching image and the second matching image constitute a matching image pair.

[0058] This embodiment uses the SIFT algorithm to extract feature points, uses SAD as the matching cost function to perform preliminary matching of the corrected image, and uses a random sampling consistency algorithm to eliminate mismatched point pairs to obtain better feature point matching results, thereby completing the stereo matching process and ensuring that the same pixel point corresponds in the matching image pair.

[0059] (3) performing edge detection on the first matching image and the second matching image respectively to obtain a first edge image and a second edge image;

[0060] During edge detection, the Canny operator is used to perform image edge detection to extract the edge of the anchor hole contour. Specifically, the edge detection process may include:

[0061] For the first matching image, the following steps are performed: Gaussian filtering is performed on the first matching image to obtain a first filtered image; the edge gradient and gradient direction of each first pixel in the first filtered image are calculated using a Sobel operator; non-maximum suppression is performed on the first filtered image according to the edge gradient and gradient direction of each first pixel to obtain a first initial edge point in the first filtered image; double-threshold screening is performed on the first initial edge point to determine the first edge point and obtain a first edge image;

[0062] For the second matching image, the following steps are performed: Gaussian filtering is performed on the second matching image to obtain a second filtered image; the edge gradient and gradient direction of each second pixel in the second filtered image are calculated using the Sobel operator; non-maximum suppression is performed on the second filtered image according to the edge gradient and gradient direction of each second pixel to obtain a second initial edge point in the second filtered image; double-threshold screening is performed on the second initial edge point to determine the second edge point to obtain a second edge image.

[0063] Among them, the main purpose of Gaussian filtering is to reduce noise and make the image smooth (blurred), but it may also increase the width of the edge.

[0064] The steps of calculating the edge gradient and gradient direction include: for the Gaussian filtered image, using the Sobel operator in the horizontal direction to calculate the horizontal gradient G x , the Sobel operator is used in the vertical direction to calculate the vertical gradient G y , and then according to the horizontal gradient G xand the vertical gradient G y Calculate the edge gradient and gradient direction of each pixel.

[0065] The horizontal gradient G x and the vertical gradient G y The calculation method is: use two 3*3 matrix operators to convolve with the original image A (i.e. the image after Gaussian filtering):

[0066]

[0067]

[0068] The calculation formula of edge gradient G is:

[0069]

[0070] The calculation formula for the gradient direction θ is:

[0071]

[0072] After the above operations, the edge gradient and gradient direction of each pixel in the Gaussian filtered image can be obtained.

[0073] The steps of non-maximum suppression include: for each pixel point in the Gaussian filtered image, checking whether it is the local maximum value among its neighboring pixels in the gradient direction, that is, for the same gradient direction, selecting the pixel point with the largest edge gradient value in the gradient direction as the edge point. The role of the local maximum value is to screen out the maximum value in the pixel's neighboring range. As a result, the obtained thicker edge is processed, and only the one with the largest change in pixel gradient value is retained, so as to perform a comprehensive scan on the Gaussian filtered image, remove irrelevant pixels that do not constitute an edge, and obtain a binary image with a thin edge. The binary image means that the values ​​of the edge point and the non-edge point are different.

[0074] The steps of double threshold screening include: setting two thresholds, the minimum threshold minVal and the maximum threshold maxVal. For each edge point, if the edge gradient of the edge point is greater than the maximum threshold maxVal, the edge point is a true edge point; if the edge gradient of the edge point is less than the minimum threshold minVal, the edge point is a false edge point and will be classified as a non-edge point; if the edge gradient of the edge point is between the maximum threshold maxVal and the minimum threshold minVal, the edge point is an undetermined edge point, and the continuity of the undetermined edge point is judged. If the undetermined edge point is adjacent to the real edge point, the undetermined edge point is a real edge point, otherwise the undetermined edge point is a false edge point and is classified as a non-edge point. The judgment process of whether the undetermined edge point is adjacent to the real edge point refers to: judging whether there is an edge point in the 8-neighborhood of the undetermined edge point. If so, the undetermined edge point is adjacent to the real edge point, and the 8-neighborhood refers to the pixel points adjacent to the undetermined edge point. Through the above process, the edge points after non-maximum suppression are further judged to determine the real edge points, which constitute the edge image.

[0075] (4) performing contour fitting on the first edge image and the second edge image respectively to obtain a first coordinate of the center of the anchor drill hole on the mining support steel belt in the first edge image and a second coordinate of the center of the anchor drill hole on the mining support steel belt in the second edge image;

[0076] Contour fitting is based on Hough transform circle fitting to achieve circle contour fitting and obtain the image coordinates of the circle center point. In this embodiment, anchor hole circle fitting is achieved by Hough circle detection method to determine the circle center coordinates and radius. The principle is as follows:

[0077] The formula for a circle is: (xa) 2 +(yb) 2 =r 2 ; (6)

[0078] In formula (6), x, y are the coordinates of the edge point; (a, b) are the coordinates of the center of the circle; and r is the radius.

[0079] Assume there is an edge point (x, y), then the circle passing through the edge point satisfies (xa) 2 +(yb) 2 =r 2 , and there are countless circles that meet this condition. Since a, b, and r are all variables, they can be expressed in a three-dimensional parameter space abr (a is the horizontal coordinate, b is the vertical coordinate, and r is the vertical coordinate):

[0080] When r=0, (xa) 2 +(yb) 2 =r 2We can get a=x, b=y, and (a, b, r) represents a point in the three-dimensional parameter space;

[0081] When r=1, (xa) 2 +(yb) 2 =r 2 A set of (a,b) values ​​can be calculated, which all satisfy (xa) 2 +(yb) 2 =r 2 , and form a circle in the three-dimensional parameter space;

[0082] Similarly, for different r values, different circles can be obtained, which form a conical surface when drawn in the parameter space.

[0083] Different edge points will result in different conical surfaces. If two conical surfaces intersect at one point, then the point represents a circle passing through two edge points at the same time. The center coordinates a, b and the radius r can be determined based on the coordinates of the point. This embodiment sets a preset threshold. If the number of conical surfaces intersecting at one point exceeds the preset threshold, then the set of parameters is considered to be a circle.

[0084] The above principle can also be expressed as: circle detection is performed using a voting mechanism. For each edge point, the parameters of the circle passing through the edge point are fitted using formula (6), and countless circles can be fitted. Each circle corresponds to a set of (a, b, r) parameters. The above processing is performed on all edge points to obtain all possible (a, b, r) parameters. Set up a ballot box containing all possible (a, b, r) parameters. When a set of (a, b, r) parameters appears once, the number of votes for the set (a, b, r) is increased by 1. After all edge points are processed, check the ballot box and select the (a, b, r) parameters with a number of votes greater than the preset threshold. They are considered to be circles, and the a and b they include are the coordinates of the center of the circle. Otherwise, it is considered that it does not constitute a circle. At this time, the collected steel belt image pair does not include the anchor drilling hole.

[0085] However, since there are countless circles passing through a point, the efficiency of detection using the above principle is low. Generally, OpenCV uses the Hough gradient method for circle detection, which is mainly divided into two stages: the first stage detects the center of the circle, and the second stage uses the center of the circle to deduce the radius of the circle. Principle of circle center detection: The center of the circle is the intersection of the normal directions of each point on the circumference. The center of the circle passing through the detection point must be in the normal direction. A threshold is set. If the intersection point on the intersecting straight line at a certain point is greater than the threshold, the intersection point is considered to be the center of the circle. Principle of circle radius detection: The distance from the center of the circle to the circumference is the same. A threshold is determined. As long as the number of identical distances is greater than the threshold, the distance is considered to be the radius of the center of the circle.

[0086] This embodiment uses the HoughCircles operator in OpenCV to detect circles, and its function form is: HoughCircles(InputArray image, OutputArray circles, int method, double dp, doubleminDist, double param1, double param2, int minRadius, int maxRadius). Among them, InputArray: input image, the data type can generally be Mat type, which needs to be an 8-bit single-channel grayscale image. In this embodiment, it is the first edge image and the second edge image. OutputArray: stores the output vector of the detected circle, in this embodiment, it is the (a, b, r) parameters filtered by the preset threshold and the preset radius range. Method: the detection method used. Currently, OpenCV only has one method, the Hough gradient method, and this parameter can be filled in HOUGH_GRADIENT (under opencv 4.1.0). dp: The inverse of the ratio of the resolution of the accumulator image used to detect the center of the circle to the resolution of the input image. This parameter allows the creation of an accumulator with a lower resolution than the input image. For example, if dp=1, the accumulator and the input image have the same resolution; if dp=2, the accumulator width and height are half the width and height of the input image. minDist: The minimum distance between the centers of circles detected by the Hough transform. param1: The corresponding parameter of the detection method set by the third parameter method. For the current only method Hough gradient method HOUGH_GRADIENT, it represents the high threshold passed to the canny edge detection operator. The low threshold of the canny edge detection operator is half of the high threshold. param2: The corresponding parameter of the detection method set by the third parameter method. For the current only method Hough gradient method HOUGH_GRADIENT, it represents the accumulator threshold of the center of the circle in the detection stage. The smaller it is, the more circles that do not exist at all can be detected. The larger it is, the closer the circle that can pass the detection is to a perfect circle. This accumulator threshold is the preset threshold set in this embodiment. minRadius: The minimum radius of the circle. maxRadius: The maximum radius of the circle.

[0087] The obtained first edge image and second edge image are respectively input as InputArray to the HoughCircles operator in OpenCV, and the (a, b, r) parameters filtered by the preset threshold and the preset radius range (determined by the minimum circle radius minRadius and the maximum circle radius maxRadius) can be obtained, and the first coordinates of the center of the anchor hole on the mining support steel belt in the first edge image and the second coordinates of the center of the anchor hole on the mining support steel belt in the second edge image can be directly output. It should be noted that the number of (a, b, r) parameters filtered by the preset threshold and the preset radius range is the number of anchor holes included in this steel belt image pair, and the first coordinates and second coordinates of each anchor hole in the steel belt image pair can be determined according to the (a, b, r) parameters.

[0088] (5) Calculate the three-dimensional coordinates of the center of each anchor hole based on the first coordinate and the second coordinate.

[0089] Before calculating the spatial three-dimensional coordinates of the center of the anchor hole according to the first coordinate and the second coordinate, the automatic positioning method of this embodiment also includes calibrating the binocular camera in the binocular vision component to obtain the internal and external parameters of the first camera and the second camera included in the binocular camera, as well as the rotation matrix and translation matrix between the first camera and the second camera. The calibration process of the binocular camera includes the calibration of the internal and external parameters of the camera and the binocular stereo calibration. The calibration process can adopt the existing calibration method, such as the Zhang Zhengyou calibration method. The specific calibration process is that the camera collects images of the checkerboard calibration plate at different positions, and performs calibration processing on them in matlab to obtain the internal and external parameters of the camera and the rotation matrix and translation matrix between the two cameras.

[0090] The camera external parameters include the rotation matrix and translation matrix between the world coordinate system and the camera coordinate system, which are used to construct the mapping relationship between the world coordinate system and the camera coordinate system. The camera internal parameters include the focal length of the camera, the main pixel coordinates, and the image distortion parameters, which are used to construct the mapping relationship between the camera coordinate system and the pixel coordinate system. The internal and external parameters of the camera jointly construct the mapping relationship between the world coordinate system and the pixel coordinate system. The rotation matrix and the translation matrix between the first camera and the second camera indicate the spatial relationship between the two cameras. The rotation matrix can be used as an input for stereo correction to complete the stereo correction process.

[0091] After the calibration is completed, the mapping relationship between the real physical world and the virtual camera coordinate system is constructed based on the calibrated external parameter results, and then the corresponding relationship between the physical world and the image is constructed using the calibrated internal parameter results to complete the spatial mapping between a point in space and a point on the image, such as Figure 6 As shown, Figure 6 In, O W -X W Y W ZW is the world coordinate system, describing the camera position, in units of m; O C -X C Y C Z C is the camera coordinate system, the optical center is the far point, the unit is m; o-xy is the image coordinate system, the origin is the midpoint of the imaging plane, the unit is mm; o-uv is the pixel coordinate system, the origin is the upper left corner of the imaging plane, the unit is pixel. W , Y W , Z W ) represents a point in space, p(x, y) represents the imaging point of P in the image, with coordinates (x, y) in the image coordinate system and (u, v) in the pixel coordinate system.

[0092] The world coordinate system is converted to the camera coordinate system: the conversion from 3D point to 3D point includes the camera external parameter rotation matrix R and translation matrix T. The conversion equation used is:

[0093]

[0094] In formula (7), (x c ,y c , z c ) represents the coordinates of a point in space in the camera coordinate system; (x w ,y w , z w ) represents the coordinates of a point in space in the world coordinate system.

[0095] The camera coordinate system is converted to the pixel coordinate system: the conversion from a three-dimensional point to a two-dimensional point, including the camera intrinsic focal length f and the principal point coordinates u0 and v0.

[0096] The conversion from camera coordinate system to pixel coordinate system should include the conversion process from camera coordinate system to image coordinate system to pixel coordinate system. When converting from image coordinate system to pixel coordinate system, it is necessary to establish image coordinate system oxy and pixel coordinate system ouv. The conversion equation used to convert a point p(x, y) in the image coordinate system to the pixel coordinate system is: u=x / dx+u0; v=y / dy+v0.

[0097] The conversion equation used to convert the camera coordinate system to the pixel coordinate system is:

[0098]

[0099] In formula (8), Z C It is expressed as a scale factor (not 0); (u, v) represents the coordinates of a point in space in the pixel coordinate system; dx represents the physical size of each pixel on the u-axis; dy represents the physical size of each pixel on the v-axis.

[0100] Therefore, the conversion equation from the coordinates of a point in space in the world coordinate system to the coordinates of a point in space in the pixel coordinate system is:

[0101]

[0102] The above formula (9) constructs the mapping relationship from a point in space to a point on the image.

[0103] After the above calibration process, this embodiment can construct a binocular camera spatial positioning model according to the binocular camera calibration result, and calculate the three-dimensional coordinates of the spatial point based on the first coordinate and the second coordinate according to the triangulation principle. Then, the formula used to calculate the spatial three-dimensional coordinates of the center of the anchor hole according to the first coordinate and the second coordinate is:

[0104]

[0105] In formula (10), (X, Y, Z) is the three-dimensional coordinate of the center of the anchor hole; (u l ,v l ) is the first coordinate; (u r ,v r ) is the second coordinate; d is the distance between the optical centers of the two cameras, which is obtained through the translation matrix between the two cameras; f is the focal length of the camera.

[0106] S3: Controlling the anchor drilling robot to work according to the three-dimensional spatial coordinates to complete the drilling process of the anchor drilling hole.

[0107] According to the spatial three-dimensional coordinates of the center of the anchor drilling hole, the accurate automatic positioning of the anchor drilling hole is achieved, and the drill arm is controlled to move to the bottom of the anchor drilling hole and complete the drilling operation.

[0108] As an optional implementation, before S1, the automatic positioning method of this embodiment also includes: controlling the drilling and anchoring robot to move around the mining support steel belt, and can move to a predetermined distance from the mining support steel belt. The predetermined distance is a manually set value and can be set according to actual needs.

[0109] When the binocular camera of this embodiment approaches the steel belt, it collects the image of the steel belt and transmits it to the controller for processing, thereby obtaining the spatial three-dimensional coordinates of the center of the anchor hole in the spatial coordinate system. The spatial three-dimensional coordinates of the center of the anchor hole can be automatically determined in the tunnel, and the drill arm can be controlled to move to the bottom of the hole accordingly to complete subsequent drilling and anchoring work. The automatic positioning of the anchor hole in the tunnel can be realized, which effectively improves the automation level of the drilling and anchoring robot, improves the efficiency of support operations, and improves the safety of support.

[0110] Each embodiment in this specification focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referenced to each other.

[0111] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An automatic anchor drilling positioning method based on binocular vision, characterized in that: The automatic positioning method comprises: Receiving a pair of steel strip images obtained by photographing a mining support steel strip by a binocular vision component; the pair of steel strip images includes a first steel strip image and a second steel strip image; Processing the steel belt image pair to obtain the spatial three-dimensional coordinates of the center of the anchor drill hole on the mining support steel belt; Controlling the anchor drilling robot to work according to the three-dimensional spatial coordinates to complete the drilling process of the anchor drilling hole; The processing of the steel belt image pair to obtain the spatial three-dimensional coordinates of the center of the anchor drill hole on the mining support steel belt specifically includes: Performing image enhancement, image denoising and stereo correction on the first steel strip image and the second steel strip image respectively to obtain a first corrected image and a second corrected image; Performing stereo matching on the first corrected image and the second corrected image respectively to obtain a first matching image and a second matching image; Performing edge detection on the first matching image and the second matching image respectively to obtain a first edge image and a second edge image; Performing contour fitting on the first edge image and the second edge image respectively to obtain a first coordinate of the center of the anchor drill hole on the mining support steel belt in the first edge image and a second coordinate of the center of the anchor drill hole on the mining support steel belt in the second edge image; Calculate the spatial three-dimensional coordinates of the center of the anchor drilling hole according to the first coordinates and the second coordinates; The performing stereo matching on the first corrected image and the second corrected image respectively to obtain the first matching image and the second matching image specifically includes: Extracting first feature points in the first corrected image using a SIFT algorithm to obtain a first point set; extracting second feature points in the second corrected image using a SIFT algorithm to obtain a second point set; Using SAD as a matching cost function, preliminarily matching the first point set and the second point set to obtain an initial matching set; Using the RANSAC algorithm to eliminate mismatched point pairs in the initial matching set to obtain a matching set, and obtaining a first matching image and a second matching image according to the matching set; The performing edge detection on the first matching image and the second matching image respectively to obtain the first edge image and the second edge image specifically comprises: For the first matching image, perform the following steps: Performing Gaussian filtering on the first matching image to obtain a first filtered image; Calculating the edge gradient and gradient direction of each first pixel in the first filtered image using a Sobel operator; Performing non-maximum suppression on the first filtered image according to the edge gradient and the gradient direction of each first pixel point to obtain a first initial edge point in the first filtered image; Performing double threshold screening on the first initial edge point to determine the first edge point and obtain a first edge image; For the second matching image, perform the following steps: Performing Gaussian filtering on the second matching image to obtain a second filtered image; Calculating the edge gradient and gradient direction of each second pixel in the second filtered image using a Sobel operator; Performing non-maximum suppression on the second filtered image according to the edge gradient and the gradient direction of each second pixel point to obtain a second initial edge point in the second filtered image; The second initial edge points are screened using a double threshold value to determine second edge points and obtain a second edge image.

2. The automatic positioning method according to claim 1, characterized in that: The step of performing contour fitting on the first edge image and the second edge image respectively to obtain the first coordinate of the center of the anchor drill hole on the mining support steel belt in the first edge image and the second coordinate of the center of the anchor drill hole on the mining support steel belt in the second edge image specifically includes: Taking the first edge image as input, using the HoughCircles operator in OpenCV to determine the first coordinates of the center of the anchor drill hole on the mining support steel belt in the first edge image; The second edge image is used as input and the HoughCircles operator in OpenCV is used to determine the second coordinates of the center of the anchor drill hole on the mining support steel belt in the second edge image.

3. The automatic positioning method according to claim 1, characterized in that: Before calculating the spatial three-dimensional coordinates of the center of the anchor hole based on the first coordinates and the second coordinates, the automatic positioning method also includes calibrating the binocular camera in the binocular vision component to obtain the intrinsic and extrinsic parameters of the first camera and the second camera included in the binocular camera, as well as the rotation matrix and translation matrix between the first camera and the second camera.

4. The automatic positioning method according to claim 1, characterized in that: Before receiving the pair of images of the mining support steel belt obtained by photographing the mining support steel belt by the binocular vision component, the automatic positioning method further includes: controlling the drilling and anchoring robot to move around the mining support steel belt.

5. An automatic anchor drilling positioning system based on binocular vision, working based on the automatic positioning method according to any one of claims 1 to 4, characterized in that: The automatic positioning system comprises a drilling and anchoring robot, a binocular vision component and a controller; the binocular vision component is fixedly mounted on the drilling arm of the drilling and anchoring robot; the controller is respectively connected to the drilling and anchoring robot and the binocular vision component for communication; The binocular vision component is used to photograph the mining support steel belt arranged on the top plate or the side wall of the tunnel to obtain a pair of steel belt images; the pair of steel belt images includes a first steel belt image and a second steel belt image; The controller is used to process the steel belt image pair to obtain the spatial three-dimensional coordinates of the center of the anchor hole on the mining support steel belt, and control the drilling and anchoring robot to work according to the spatial three-dimensional coordinates to complete the drilling process of the anchor hole.

6. The automatic positioning system according to claim 5, characterized in that: The binocular vision component includes a camera mounting frame and a binocular camera; the bottom of the camera mounting frame is fixedly installed on the front end of the drill arm; the binocular camera includes a first camera and a second camera; the first camera and the second camera are fixed in parallel on both sides of the camera mounting frame; the first camera is used to shoot the mining support steel belt to obtain a first steel belt image; the second camera is used to shoot the mining support steel belt to obtain a second steel belt image.

7. The automatic positioning system according to claim 6, characterized in that: The binocular vision component also includes a fill light device; the fill light device is used to fill light to the mining support steel belt during the shooting process of the binocular camera.

Images