Mine roadway inspection robot high-precision positioning method based on image marking
The integration of image marking and advanced algorithms in mine tunnel inspection robots addresses navigation challenges, achieving millimeter-level precision and orientation accuracy for accurate detection.
Patent Information
- Application Number
- CN202410991372.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-12-06
AI Technical Summary
Existing mine tunnel inspection robots face challenges in achieving high-precision navigation and orientation due to complex tunnel environments, dust, water vapor, and blocked radio signals, which affect navigation accuracy and stability, making it difficult to meet the requirements for accurate positioning and orientation needed for precise inspections.
A high-precision positioning method for mine tunnel inspection robots using image marking, integrating laser radar SLAM mapping, structured light cameras, and image markers, combined with the GFTT and EPnP algorithms, to achieve millimeter-level positioning and orientation accuracy.
The method enables millimeter-level positioning accuracy and orientation better than 2°, ensuring accurate detection of tunnel features and deformation, overcoming the limitations of traditional navigation methods in mine tunnels.
Smart Images

Figure CN119085645A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of positioning of underground mine inspection equipment, and in particular to a high-precision positioning method of a mine tunnel inspection robot based on image marking. Background Art
[0002] Manual inspections in mines are high-risk, inefficient, and fatigue of inspectors can affect the quality of inspections. Autonomous navigation robots equipped with detection equipment can be used in mine tunnels to replace manual inspections, but some detection equipment requires high-precision positioning and orientation to the target location (mine exits in through-vein tunnels, chutes in mining areas, return air shafts, mine exits of buckets, and tunnel tops, especially for deformation detection of tunnel supports and chute openings) to carry out precise inspections. In addition, in order to ensure the accuracy of the comparison of instrument monitoring data before and after, the mine tunnel inspection robot is required to accurately dock at a fixed target location each time and keep the docking direction angle unchanged.
[0003] At present, the main problems of using robots for automatic inspection in mine tunnels are as follows:
[0004] (1) High-precision navigation in mine tunnels is difficult: The environment in mine tunnels is very complex. There is a lot of dust, water vapor, insufficient light, etc. that interfere with navigation sensors and reduce navigation accuracy. Radio signals are blocked and attenuated by walls and obstacles, affecting the stability and accuracy of the navigation system, making it impossible to meet the requirements of accurate positioning in mine tunnels.
[0005] (2) Mine tunnels have complex terrain, limited space, and some characteristics of tunnels. Traditional lidar slam map navigation is difficult to achieve a positioning accuracy of 10 cm in tunnels, and the directional angle error is within 15°. In order to complete the robot's autonomous navigation to multiple mine exits and adjust the position to facilitate the work of the on-board detection equipment, the on-board detection equipment requires the navigation positioning accuracy to be within 2 cm and the directional angle error to be less than 2°. Some equipment requires the positioning accuracy to be no more than 5 mm and the directional angle error to be no more than 1°. Summary of the invention
[0006] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art, and to provide a high-precision positioning method for a mine tunnel inspection robot based on image marking in complex and similar scenarios in mine tunnels, so as to achieve millimeter-level high-precision positioning of the inspection platform and an orientation accuracy better than 2° for accurate detection by the carried detection equipment.
[0007] The technical solution adopted by the present invention is as follows:
[0008] A high-precision positioning method for a mine tunnel inspection robot based on image marking, the method comprising the following steps:
[0009] Step S1: Use laser radar to perform SLAM mapping navigation and preliminarily locate the target location, which is a mine exit precision positioning tunnel with an image mark set on the top;
[0010] Step S2: The structured light camera acquires an RGB image of the image marker;
[0011] Step S3: Detecting corner points and feature points of the marked graphics in the image mark based on the GFTT algorithm;
[0012] Step S4: Reversely solve the coordinates (x, y, z) and attitude information (yaw, pitch, roll) of the target position in the current world coordinate system of the structured light camera based on the EPnP algorithm;
[0013] Step S5: finely adjust the slide rail and the pan / tilt of the two-dimensional slide according to the target position, so that the carried equipment can reach a position with a positioning error of 0.05 mm and an adjustment error of 0.2°;
[0014] Step S6: taking photos of the lane;
[0015] Step S7: Go to the next target location;
[0016] The inspection robot comprises a fixed robot body, a chassis 1 fixed on the robot body, a frame 2 fixed on the chassis, a two-dimensional slide 3 arranged on the chassis, a pan-tilt 4 installed on a slider of the two-dimensional slide, a structured light camera 5 installed on the pan-tilt, and a laser radar 6 fixed on the frame. Other detection equipment or instruments can also be carried on the robot according to detection needs;
[0017] The image tag includes an RFID card and an icon (71) recording each icon index, wherein the icon is composed of a blue background, a red L-shaped (72) set on the blue background, and black and white chessboards (73) distributed on the three corners of the blue background.
[0018] Furthermore, the GFTT algorithm is as follows: the autocorrelation matrix of each pixel is calculated from the grayscale gradient information in its neighborhood, and the matrix can describe the grayscale change in the neighborhood of the pixel. The corner point response value at the pixel is obtained by calculating the eigenvalue and eigenvector of the autocorrelation matrix, and whether the point is a corner point is determined; the pixel neighborhood of the autocorrelation matrix response value is expressed as:
[0019]
[0020] Will Performing Taylor expansion, we can get the autocorrelation matrix A:
[0021]
[0022] in,
[0023]
[0024] Formula (3) represents x i Image gradients everywhere;
[0025] Before detecting the image feature points, the GFTT detector needs to convolve the image with a Gaussian weighted kernel w to obtain an estimate of the local shape of the autocorrelation function at each pixel:
[0026]
[0027] In order to obtain the response value of the autocorrelation matrix, a characteristic analysis is performed on it, in which the change of the local shape can be represented by the eigenvalue, so the two eigenvalues of the autocorrelation matrix are used to represent the point features. Since the corresponding amount of GFTT depends on the smaller eigenvalue, GFTT uses the smaller eigenvalue to represent the response value R of the corner point:
[0028] R=min(λ1,λ2) (5)
[0029] After obtaining the corner points of the marked figure, the accurate two-dimensional coordinates of the corner points of the marked figure are used, and combined with the three-dimensional coordinates in the world reference system, the position coordinates (x, y, z) and attitude orientation information (yaw, pitch, roll) of the structured light camera in the current world coordinate system are inverted based on the EPnP algorithm.
[0030] The 3D coordinates of the target in the world coordinate system are The non-homogeneous coordinates of the virtual control point in the target's world coordinate system can be expressed as The relationship between the two coordinates is as follows (6):
[0031]
[0032] The 3D coordinates in the target world coordinate system can be expressed by linear weighted control points (n≥5), a ij is the weighting coefficient,
[0033] The relationship between the 3D coordinates of the target and the coordinates of the control points in the camera coordinate system is as follows (7):
[0034]
[0035] The relationship between the virtual control points in the two coordinate systems is as follows:
[0036]
[0037] Assume that the projection coordinates of the target's 3D coordinates in the pixel coordinate system are (u i ,v i), then the relationship between the projection coordinates and the 3D virtual control points is as follows:
[0038]
[0039] In formula (9), Z c is the projection depth of the coordinate point, K is the camera internal parameter matrix, f x , f y , u0, v0 are the internal parameters of the camera;
[0040] By establishing the relationship between the 3D virtual control points and pixel coordinates of the target, the 3D points are matched one by one with the 2D points, and then the EPnP algorithm is used to linearly weight the 3D virtual control points to solve the target pose, and the rotation matrix R and translation matrix t of the target are obtained;
[0041] Furthermore, the two-dimensional slide can move in X and Y directions, and is driven by a servo motor, a reducer and a ball screw guide, with a positioning accuracy of 0.02mm, a stroke of ±150mm and self-locking capability.
[0042] Furthermore, the gimbal adopts a high-precision gimbal with two degrees of freedom: horizontal 0-360°, pitch ±60°, and positioning accuracy of 0.1-0.2°, and a load capacity of 10kg.
[0043] Furthermore, the frame is a rectangular frame, and the rectangular frame is horizontally fixed on the chassis.
[0044] The present invention has at least the following advantages:
[0045] (1) The mine tunnel has complex terrain and tunnel-like features, so it is impossible to use laser radar to directly perform precise positioning. The present invention adopts an image marking method for precise positioning. The positioning method meets the requirements of the vehicle-mounted detection equipment for navigation positioning accuracy within 2 cm, and the direction angle error is less than 2°. The positioning accuracy required by some detection equipment does not exceed 5 mm, and the direction angle error is not greater than 1°. The detection equipment carried can achieve accurate detection, providing guarantee for the safe production and accident prevention and control of mines; it overcomes the traditional laser radar slam map navigation + IMU inertial navigation fusion method, which is difficult to achieve 10 cm positioning accuracy in mine tunnels, and the directional angle with an error within 15° cannot be accurately positioned.
[0046] (2) Since accurate measurement of the deformation of the eyebrow beam in a mine requires an accuracy of 1 cm, the positioning method of the present invention enables the millimeter-wave radar to achieve a positioning error of less than 5 mm and an angle error of less than 1° in the tunnel to complete the detection task.
[0047] (3) The present invention adopts an underground navigation and positioning mode that integrates an integrated laser radar slam and image marker multi-strategy fusion, and a four-degree-of-freedom (XY two-dimensional slide + 2-degree-of-freedom pan / tilt) vehicle-mounted high-precision image marker self-positioning adjustment system to achieve millimeter-level high-precision positioning of the mine tunnel inspection robot and an orientation accuracy better than 2°. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 is a flow chart of the positioning method of the present invention;
[0049] Figure 2 This is a schematic diagram of the structure of the robot of the present invention (equipped with main inspection equipment);
[0050] Figure 3 It is a schematic diagram of image marking of the present invention;
[0051] Figure 4 This is a schematic diagram of the precise positioning and inspection of the mine exits 1 to 8 by the inspection robot according to an embodiment of the present invention in a tunnel. DETAILED DESCRIPTION
[0052] The content of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0053] See also Figure 1-Figure 3 A high-precision positioning method for a mine tunnel inspection robot based on image marking, the inspection robot used in the method includes a chassis 1 fixed on the robot body, a frame 2 fixed on the chassis, the frame is a rectangular frame, the rectangular frame is horizontally fixed on the chassis, a two-dimensional slide 3 is arranged on the chassis, the slide rail of the two-dimensional slide is horizontally installed on the rectangular frame, a pan-tilt 4 is installed on the slider 31 of the two-dimensional slide, a structured light camera 5 installed on the pan-tilt, and a laser radar fixed on the frame; the two-dimensional slide can move in two directions of X and Y, and is driven by a servo motor, a reducer and a ball screw guide rail. The positioning accuracy is 0.02mm, the stroke is ±150mm and it has self-locking ability. The two-dimensional slide used in this embodiment is made of aluminum alloy profile: model MF90-1610-500-40, the motor is side-mounted, and the MF90-1610-500-60 motor is side-mounted. The servo adopts a 48V DC drive special motor to match the mobile robot lithium battery power supply scenario requirements; the gimbal adopts a high-precision gimbal with two degrees of freedom, horizontal 0~360°, pitch ±60°, and positioning accuracy of 0.1~0.2°, with a load capacity of 3kg, and a customized worm gear drive gimbal, model CH-W004YT.
[0054] The positioning method of this embodiment includes the following steps:
[0055] See also Figure 4 , Step S1: Use laser radar for SLAM mapping navigation and preliminarily locate the target location. The target location is Figure 4 The mine exit 1 is shown;
[0056] Step S2: The structured light camera acquires an RGB image of the image marker 1;
[0057] See also Figures 1 to 3 , Step S3: Detecting the corner points of the marked graphics in the image mark based on the GFTT algorithm;
[0058] The pixel neighborhood of the autocorrelation matrix response value is expressed as:
[0059]
[0060] Will Performing Taylor expansion, we can get the autocorrelation matrix A:
[0061]
[0062] in,
[0063]
[0064] Formula (3) represents x i The image gradient at ;
[0065] Before detecting the image feature points, the GFTT detector needs to convolve the image with a Gaussian weighted kernel w to obtain an estimate of the local shape of the autocorrelation function at each pixel:
[0066]
[0067] In order to obtain the response value of the autocorrelation matrix, a feature analysis is performed on it, where the change of the local shape can be represented by the eigenvalue, so the two eigenvalues of the autocorrelation matrix are used to represent the point features. Since the corresponding amount of GFTT depends on the smaller eigenvalue, GFTT uses the smaller eigenvalue to represent the response value R of the corner point:
[0068] R = min(λ1,λ2) (5) After obtaining the corner points of the marked figure, the accurate two-dimensional coordinates of the corner points of the marked figure are used, combined with the three-dimensional coordinates in the world reference system, to inversely solve the position coordinates (x, y, z) and attitude information (yaw, pitch, roll) of the structured light camera in the current world coordinate system based on the EPnP (Efficient Perspective-n-Point) algorithm.
[0069] Step S4: Decode the coordinates (x, y, z) of the target position in the current world coordinate system of the structured light camera based on the EPnP algorithm
[0070] and attitude information (yaw, pitch, roll);
[0071] The EPnP algorithm is as follows:
[0072] The 3D coordinates of the target in the world coordinate system are The non-homogeneous coordinates of the virtual control point in the target's world coordinate system can be expressed as The relationship between the two coordinates is as follows (6):
[0073]
[0074] The 3D coordinates in the target world coordinate system can be expressed by linear weighted control points (n≥5), a ij is the weighting coefficient.
[0075] The relationship between the 3D coordinates of the target and the coordinates of the control points in the camera coordinate system is as follows (7):
[0076]
[0077] The relationship between the virtual control points in the two coordinate systems is as follows:
[0078]
[0079] Assume that the projection coordinates of the target's 3D coordinates in the pixel coordinate system are (u i ,v i ), then the relationship between the projection coordinates and the 3D virtual control points is as follows:
[0080]
[0081] In formula (9), Z c is the projection depth of the coordinate point, K is the camera internal parameter matrix, f x , f y , u0, v0 are the internal parameters of the camera.
[0082] By establishing the relationship between the 3D virtual control points and pixel coordinates of the target, the 3D points are matched one by one with the 2D points, and then the EPnP algorithm is used to linearly weight the 3D virtual control points to solve the target pose, and the rotation matrix R and translation matrix t of the target are obtained;
[0083] Reverse the coordinates (x, y, z) and attitude information (yaw, pitch, roll) of the target position of the structured light camera in the current world coordinate system
[0084] Step S5: fine-tune the slide rail and pan-tilt platform of the two-dimensional slide according to the target position coordinates, and adjust the posture to a positioning error of 0.01 mm and an adjustment error of 0.1°, so that the camera can reach the predetermined position, to ensure that the vehicle-mounted detection equipment that moves with the camera can always be in the specified position and specified angle to shoot the alleyway, so as to detect the deformation of the alleyway.
[0085] Step S6: taking photos of the lane;
[0086] Step S7: Go to the next target location, the mine outlet 2, and inspect the mine outlet 2;
[0087] See also Figure 4 , the same steps as above are followed to complete the high-precision positioning method and inspection of the mine outlets 2 to 8 in sequence.
[0088] The above-described embodiments are merely descriptions of preferred embodiments of the present invention, and are not limitations on the protection scope of the present invention. Without departing from the design spirit of the present invention, various equivalent modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field all fall within the protection scope of the present invention.
Claims
1. A high-precision positioning method for a mine tunnel inspection robot based on image marking, characterized in that: The method comprises the following steps: Step S1: Use laser radar to perform SLAM mapping navigation and preliminarily locate the target location, which is a mine exit precision positioning tunnel with an image mark set on the top; Step S2: The structured light camera acquires an RGB image of the image marker; Step S3: Detecting corner points and feature points of the marked graphics in the image mark based on the GFTT algorithm; Step S4: Reversely solve the coordinates (x, y, z) and attitude information (yaw, pitch, roll) of the target position in the current world coordinate system of the structured light camera based on the EPnP algorithm; Step S5: According to the calculated target position, the position is finely adjusted using the slide rail and the pan / tilt of the two-dimensional slide table, so that the carried equipment can reach a position with a positioning error of 0.05 mm and an adjustment error of 0.2°; Step S6: taking photos of the lane; Step S7: Go to the next target location; The mine tunnel inspection robot comprises a robot body, a chassis (1) fixed on the robot body, a frame (2) fixed on the chassis, a two-dimensional slide (3) arranged on the chassis, a pan-tilt platform (4) installed on the two-dimensional slide, a structured light camera (5) installed on the pan-tilt platform, and a laser radar (6) fixed on the frame; The image tag includes an RFID card and an icon (71) recording each icon index, wherein the icon is composed of a blue background, a red L-shaped (72) set on the blue background, and black and white chessboards (73) distributed on the three corners of the blue background.
2. The high-precision positioning method of a mine tunnel inspection robot based on image marking according to claim 1 is characterized in that: The GFTT algorithm is as follows: the autocorrelation matrix of each pixel is calculated from the grayscale gradient information in its neighborhood. The matrix can describe the grayscale changes in the neighborhood of the pixel. The corner point response value at the pixel is obtained by calculating the eigenvalue and eigenvector of the autocorrelation matrix, and whether the point is a corner point is determined. The pixel neighborhood of the autocorrelation matrix response value is expressed as: Will Performing Taylor expansion, we can get the autocorrelation matrix A: in, Formula (3) represents x i Image gradients everywhere; Before detecting the image feature points, the GFTT detector needs to convolve the image with a Gaussian weighted kernel w to obtain an estimate of the local shape of the autocorrelation function at each pixel: In order to obtain the response value of the autocorrelation matrix, a characteristic analysis is performed on it, in which the change of the local shape can be represented by the eigenvalue, so the two eigenvalues of the autocorrelation matrix are used to represent the point features. Since the corresponding amount of GFTT depends on the smaller eigenvalue, GFTT uses the smaller eigenvalue to represent the response value R of the corner point: R=min(λ1,λ2) (5) After obtaining the corner points of the marked figure, the accurate two-dimensional coordinates of the corner points of the marked figure are used, and combined with the three-dimensional coordinates in the world reference system, the position coordinates (x, y, z) and attitude orientation information (yaw, pitch, roll) of the structured light camera in the current world coordinate system are inverted based on the EPnP algorithm.
3. The high-precision positioning method of a mine tunnel inspection robot based on image marking according to claim 1 is characterized in that: The EPnP algorithm uses the result of linear weighting of virtual control points to represent the coordinates of feature points on the real object in the camera coordinate system. The EPnP algorithm uses the properties of absolute quadratic curves and absolute quadratic surfaces to transform the PnP problem into a linear problem, and quickly and stably solves the camera posture.
4. The high-precision positioning method of a mine tunnel inspection robot based on image marking according to claim 3 is characterized in that: The EPnP algorithm is specifically: The 3D coordinates of the target in the world coordinate system are The non-homogeneous coordinates of the virtual control point in the target's world coordinate system can be expressed as The relationship between the two coordinates is as follows (6): The 3D coordinates in the target world coordinate system can be expressed by linear weighted control points (n≥5), a ij is the weighting coefficient, The relationship between the 3D coordinates of the target and the coordinates of the control points in the camera coordinate system is as follows (7): The relationship between the virtual control points in the two coordinate systems is as follows: Assume that the projection coordinates of the target's 3D coordinates in the pixel coordinate system are (u i ,v i ), then the relationship between the projection coordinates and the 3D virtual control points is as follows: In formula (9), Z c is the projection depth of the coordinate point, K is the camera internal parameter matrix, f x , f y , u0, v0 are the internal parameters of the camera; By establishing the relationship between the 3D virtual control points and pixel coordinates of the target, the 3D points are corresponded one by one with the 2D points. Then the EPnP algorithm is used to perform the linear weighted summation of the 3D virtual control points to solve the target's pose and obtain the target's rotation matrix R and translation matrix t.
5. The high-precision positioning method of a mine tunnel inspection robot based on image marking according to claim 1 is characterized in that: The two-dimensional slide can move in X and Y directions, and is driven by a servo motor, a reducer and a ball screw guide. The positioning accuracy is 0.02mm, the stroke is ±150mm and it has self-locking capability.
6. The high-precision positioning method of a mine tunnel inspection robot based on image marking according to claim 1 is characterized in that: The pan / tilt head adopts a high-precision pan / tilt head with two degrees of freedom, horizontal 0-360°, pitch ±60°, and positioning accuracy of 0.1-0.2°, and has a load capacity of 10kg.
7. A high-precision positioning method for a mine tunnel inspection robot based on image marking according to any one of claims 1 to 6, characterized in that: The frame is a rectangular parallelepiped frame, and the rectangular parallelepiped frame is horizontally fixed on the chassis.
