A binocular vision-based measurement system and method for an attitude angle of an advanced hydraulic support

Abstract

The application discloses a kind of measurement system and method of advance hydraulic support posture angle based on binocular vision, including the connection binocular camera of explosion-proof terminal, the process of real-time measurement advance hydraulic support operation;Before supporting operation, the initial posture angle of advance hydraulic support is measured;During supporting process, the posture angle of advance hydraulic support change at different time is obtained by Euclidean transformation and posture angle algorithm;Innovation optimization is carried out to the feature detection and stereo matching algorithm of target during measurement process.The efficient measurement of advance hydraulic support posture angle is realized by Euclidean transformation and innovative posture angle algorithm, and the solving algorithm is fast, and the measurement time is short;Continuous optimization target feature detection and target stereo matching algorithm, improve the accuracy and robustness of solution result.The application can effectively reduce the measurement error of advance hydraulic support posture angle, complete real-time non-contact online accurate measurement in mine roadway.

Description

Technical Field

[0001] The invention relates to a method for measuring the attitude angle of an advanced hydraulic support, in particular to a system and method for measuring the attitude angle of an advanced hydraulic support based on binocular vision. Background Art

[0002] With the development of intelligent fully mechanized mining faces and in-depth research into computer control systems, visual measurement technology has gradually been applied to modern coal mining. Mine advance support is an auxiliary measure for advanced excavation to ensure stable mining at the working face. Specifically, this involves the advanced hydraulic support equipment group. As an important support equipment for the working face, it plays a key role in miner safety and the normal operation of the coal mining equipment group. Real-time measurement of the attitude angle of the advanced hydraulic support based on binocular vision and feedback on working deviations is of great significance for reducing the downtime rate of support equipment and improving the automation and intelligence level of auxiliary equipment in the two lanes of the working face.

[0003] Currently, most advanced hydraulic support systems rely on the subjective judgment of miners for support. Hydraulic system failures or machine process disruptions can prevent the advanced hydraulic supports from reaching their designated support positions or cause collisions and interference between the support top beam and the surrounding rock of the tunnel, significantly reducing support effectiveness or even rendering the support ineffective. Therefore, a binocular vision-based system and method for measuring the attitude angle of advanced hydraulic supports is urgently needed. Summary of the Invention

[0004] The present invention aims to provide a binocular vision-based system and method for measuring the attitude angle of an advanced hydraulic support. This system utilizes Euclidean transformation and an innovative attitude angle solution algorithm to achieve efficient measurement of the attitude angle of an advanced hydraulic support. The solution is fast and measurement time is short. Continuous optimization of the target feature detection and target stereo matching algorithms improves the accuracy and robustness of the solution results. This system effectively reduces measurement errors of the attitude angle of an advanced hydraulic support, enabling real-time, non-contact, online, and precise measurement within mine tunnels.

[0005] In order to achieve the above objectives, the technical solutions adopted by the present invention are:

[0006] A binocular vision-based advanced hydraulic support attitude angle measurement system and method includes the following steps:

[0007] S01: The explosion-proof terminal is connected to a binocular camera to measure the process of the advanced hydraulic support support operation in real time;

[0008] S02: Before the support operation, measure the initial attitude angle of the advance hydraulic support;

[0009] S03: During the support process, the attitude angles of the leading hydraulic support at different times are obtained through Euclidean transformation and attitude angle solution algorithm;

[0010] S04: In the measurement process, the feature detection and stereo matching algorithm of the target is innovatively optimized.

[0011] In the preferred technical solution, the step of connecting the binocular camera to the explosion-proof terminal in S01 comprises the following steps:

[0012] S11: The explosion-proof terminal is installed with Pycharm software under Windows system, combined with Python language and Opencv database, to write a program for solving the attitude angle of the advance hydraulic support.

[0013] S12: The posture angle is measured more clearly by installing an infrared binocular camera in the dark environment of the mine crossheading tunnel section; the infrared binocular camera is installed horizontally at the top beam of the advance hydraulic support to measure the posture angle at different times in real time.

[0014] In the preferred technical solution, the step of measuring the initial posture angle in S02 comprises the following steps:

[0015] S21: The top beam coordinate system (X, Y, Z), the camera coordinate system (X, Y, Z), and the roadway roof coordinate system (X, Y, Z) are respectively established on the visual measurement model of the advance hydraulic support. x M O M y M z M x c O c y c z c x N O N y N z N

[0016] S22: The advance hydraulic support is pushed forward, and the initial posture angle of the top beam relative to the roadway roof is measured before the supporting operation; three groups of coordinates (X, Y, Z), (X, Y, Z), and (X, Y, Z) are taken on the plane of the roadway roof. M 1 x 1 y 1 z 1 M 2 x 2 y ​​​​​​​​​2 , z 2 ), M 3 ( x 3 , y 3 , z 3 ), construct the spatial plane equation of the upper surface of the tunnel roof, and calculate the plane normal vector ;

[0017] S23: Since the top beam coordinate system is parallel to the camera coordinate system, the plane normal vector Relative to the camera coordinate system ( x c O c y c z c ) The angle between the three axes is the initial attitude angle of the top beam relative to the roadway roof. Ψ 0 , θ 0 , φ 0 .

[0018] In a preferred technical solution, measuring the changed attitude angle in S03 includes the following steps:

[0019] S31: The attitude angle is calculated by obtaining the position relationship between the top beam and the tunnel roof at different times. Since the binocular camera is arranged horizontally on the top beam, the three axes of the camera coordinate system and the top beam coordinate system are parallel, and only the coordinate origin is different. Calculate the position change from the top beam coordinate system to the camera coordinate system;

[0020] S32: The roof of the tunnel in the mine is fixed. After the advance hydraulic support starts working, the posture change of the top beam relative to the tunnel roof can be simply regarded as the rotation of the top beam coordinate system around the three axes of the tunnel roof coordinate system. Ψ , θ , φ Angle, the rotation matrix is:

[0021] (1);

[0022] The total rotation matrix of the top beam relative to the tunnel roof is:

[0023] (2)

[0024] In the preferred technical solution, the rotation matrix of the top beam in S32 relative to the roof of the roadway comprises:

[0025] The top beam coordinate system is transformed to the position of the camera coordinate system through a translation matrix The rotation matrix of the top beam coordinate system relative to the roof coordinate system of the roadway after the position is transformed to the camera coordinate system is equivalent to the rotation matrix of the camera coordinate system relative to the roof coordinate system of the roadway :

[0026] (3);

[0027] The rotation is converted into a changed attitude angle , , :

[0028] (4);

[0029] The measured initial attitude angle and the changed attitude angle are added to obtain the calculation formula of the attitude angle of the advanced hydraulic support:

[0030] (5);

[0031] The calculation of the attitude angle does not consider the non-commutative error, and the attitude angle of the advanced hydraulic support in the mine changes within the range of [-30, 30] degrees, so that the angle singular value does not appear.

[0032] In the preferred technical solution, the innovation optimization of target detection in S04 comprises:

[0033] The ORB feature detection is performed on the target, and errors occur in the feature point detection due to the interference of obstacles, environment and other factors. The detection distance threshold and the corner point intensity threshold are set to eliminate the wrong detection points in the image, the detection accuracy is improved, and the detection speed is reduced; the GPU acceleration module of "cv2.cuda" is implanted in the program, and the "multiprocessing" module is used to optimize the feature extraction speed of the ORB algorithm, so that the rapid and accurate detection of the target is realized.

[0034] In the preferred technical solution, the innovation optimization of target matching in S04 comprises:

[0035] The Gaussian pyramid method is added to reduce the discontinuity of the target stereo matching. Firstly, a Gaussian filter is applied to smooth the original advanced hydraulic support image; then the down-sampling is performed on the smoothed image; finally, the above steps are repeated until the desired effect is achieved; the median filter method is used to remove image noise points, the edge enhancement method is used to enhance image edges, and the hole filling method is used to repair the missing areas of the image, so that the depth image is further optimized.

[0036] Compared with the prior art, the present application has the following advantages:

[0037] At present, the measurement of the attitude angle of the advanced hydraulic support generally adopts manual measurement or inclination sensor measurement, which has the defects of large number of sensors, high cost, poor precision and insufficient visibility, and cannot meet the requirements of automatic, accurate and real-time measurement of the advanced support equipment in the intelligent mine. The present application is a measurement method of the attitude angle of the advanced hydraulic support based on binocular vision, which measures the initial attitude angle and accurately calculates the changing attitude angle in the working process through the binocular camera, has the advantages of fast data update rate and short-time accurate measurement. At the same time, the target feature detection and target stereo matching algorithm are continuously optimized to ensure the accuracy of the attitude measurement of the advanced hydraulic support, further reduce the measurement error of the attitude angle, improve the accuracy and robustness of the calculation result, and complete the real-time non-contact online accurate measurement of the advanced hydraulic support in the mine roadway. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is the flow chart of the whole machine measurement system in the present application.

[0039] Figure 2 is the schematic diagram of measuring the initial attitude angle based on binocular vision in the present application.

[0040] Figure 3 is the schematic diagram of calculating the changing attitude angle based on binocular vision in the present application.

[0041] Figure 4 is the innovative optimization flow chart of the target feature detection algorithm in the present application.

[0042] Figure 5 is the innovative optimization flow chart of the target stereo matching algorithm in the present application. EMBODIMENT

[0043] The present application will be further described below.

[0044] As shown in Figure 1 , a measurement system and method of the attitude angle of the advanced hydraulic support based on binocular vision, comprising the following steps:

[0045] S01: The explosion-proof terminal connects the binocular camera to measure the process of the advanced hydraulic support support operation in real time;

[0046] S02: Before the support operation, the initial attitude angle of the advanced hydraulic support is measured;

[0047] S03: During the support process, the changing attitude angle of the advanced hydraulic support at different times is obtained through the Euclidean transformation and the attitude angle calculation method;

[0048] S04: During the measurement process, the target feature detection and stereo matching algorithm are innovatively optimized.

[0049] Further, the connecting binocular camera in the explosion-proof terminal in step S01 comprises:

[0050] S11: The explosion-proof terminal is installed with Pycharm programming software under Windows system, based on Python language and Opencv database, and a forward hydraulic support attitude angle calculation program is written;

[0051] S12: The environment of the mine crossheading tunnel section is dim, and the infrared binocular camera is installed to measure the attitude angle more clearly; the infrared binocular camera is horizontally installed at the top beam of the forward hydraulic support, and the attitude angle at different times is measured in real time.

[0052] Further, as shown in Figure 2 , the measuring initial attitude angle in step S02 comprises:

[0053] S21: A top beam coordinate system (X, Y, Z) is established on the visual measurement model of the forward hydraulic support, a camera coordinate system (X', Y', Z') is established, and a roadway roof coordinate system (X'', Y'', Z'') is established; x M O M y M z M x c O c y c z c x N O N y N z N

[0054] S22: The forward hydraulic support is pushed forward, and the initial attitude angle of the top beam relative to the roadway roof is measured before the supporting operation; three groups of coordinates (X, Y, Z), (X', Y', Z') and (X'', Y'', Z'') are taken on the plane of the roadway roof, M 1 x 1 y 1 z 1 M 2 x 2 y 2 z 2 ​​​​​​​​​​​M 3 ( x 3 , y 3 , z 3 ) the spatial plane equation of the upper surface of the roadway roof is constructed, and a plane normal vector ;

[0055] S23: since the roof beam coordinate system is parallel to the camera coordinate system, the included angle of the plane normal vector with respect to the camera coordinate system x c O c y c z c is the initial attitude angle of the roof beam with respect to the roadway roof Ψ 0 , θ 0 , φ 0 .

[0056] Further, as shown in Figure 3 , the step S03 of measuring the changed attitude angle includes:

[0057] S31: the attitude angle is calculated by obtaining the position relationship of the roof beam with respect to the roadway roof at different times, since the binocular camera is horizontally arranged on the roof beam, the camera coordinate system is parallel to the three axes of the roof beam coordinate system, and only the coordinate origin is different, the position change of the roof beam coordinate system translated to the camera coordinate system is calculated by a translation matrix ;

[0058] S32: the roadway roof under the mine is fixed, after the advance hydraulic support starts to work, the attitude change of the roof beam with respect to the roadway roof can be simply regarded as the rotation of the roof beam coordinate system around the three axes of the roadway roof coordinate system Ψ , θ , φ angle, and the rotation matrix is:

[0059] (1);

[0060] The total rotation matrix of the roof beam with respect to the roadway roof is:

[0061] (2)

[0062] ) Specifically, the rotation matrix of the roof beam with respect to the roadway roof in step S32 includes:

[0063] The roof beam coordinate system is translated by a translation matrix After the position is transformed to the camera coordinate system, the rotation matrix of the roof beam coordinate system relative to the roadway roof coordinate system is equivalent to the rotation matrix of the camera coordinate system relative to the roadway roof coordinate system :

[0064] (3);

[0065] Get the rotation After that, it is converted into a changed attitude angle , , :

[0066] (4);

[0067] Add the measured initial attitude angle and the changed attitude angle to get the calculation formula of the attitude angle of the advanced hydraulic support:

[0068] (5);

[0069] The calculation of the attitude angle does not consider the non-commutative error, and the attitude angle of the advanced hydraulic support in the mine is always within the range of [-30, 30] degrees, and the angle singular value does not appear.

[0070] Further, as shown in Figure 4 , the innovative optimization of target feature detection in step S04 includes:

[0071] The target is detected by ORB feature detection, and the feature point detection has errors due to the interference of obstacles, environment and other factors. Set the detection distance threshold and the corner point intensity threshold to eliminate the wrong detection points in the image, improve the detection accuracy, and reduce the detection speed; implant the GPU acceleration module of "cv2.cuda" in the program, and use the "multiprocessing" module to optimize the feature extraction speed of the ORB algorithm, to realize the fast and accurate detection of the target.

[0072] Further, as shown in Figure 5 , the innovative optimization of target stereo matching in step S04 includes:

[0073] Add the Gaussian pyramid method to reduce the discontinuity of target stereo matching. First, apply a Gaussian filter to smooth the original advanced hydraulic support image; then, downsample the smoothed image; finally, repeat the above steps until the desired effect is achieved; use the median filter method to remove image noise points, the edge enhancement method to enhance image edges, and the hole filling method to repair missing areas of the image, to further optimize the depth image.

Claims

1. A system and method for measuring the attitude angle of an advanced hydraulic support based on binocular vision, characterized in that: The following steps are involved: S01: The explosion-proof terminal is connected to a binocular camera to collect image sequences of the advanced hydraulic support support operation in real time; S02: Before the support operation, measure the initial attitude angle of the advance hydraulic support; S03: During the support process, the attitude angles of the leading hydraulic support at different times are obtained through Euclidean transformation and attitude angle solution algorithm; S04: During the measurement process, innovative optimization of the target’s feature detection and stereo matching algorithms is performed; The measuring of the initial attitude angle in S02 includes: S21: Establish the top beam coordinate system (x M O M y M z M ), camera coordinate system (x c O c y c z c ), roadway roof coordinate system (x N O N y N z N ); S22: Before the support operation, measure the initial attitude angle of the top beam relative to the tunnel roof; take three sets of coordinates M1 (x1, y1, z1), M2 (x2, y2, z2), M3 (x3, y3, z3) on the tunnel roof plane, construct the spatial plane equation of the upper surface of the tunnel roof, and calculate the plane normal vector S23: Since the top beam coordinate system is parallel to the camera coordinate system, the plane normal vector Relative to the camera coordinate system (x c O c y c z c ) The angle between the three axes is the initial attitude angle of the top beam relative to the roadway roof. The step S03 of obtaining the changed attitude angle includes: S31: The attitude angle is calculated by obtaining the position relationship between the top beam and the tunnel roof at different times. Since the binocular camera is arranged horizontally on the top beam, the three axes of the camera coordinate system and the top beam coordinate system are parallel, and only the coordinate origin is different. Calculate the position change from the top beam coordinate system to the camera coordinate system; S32: The roof of the tunnel in the mine is fixed. After the advance hydraulic support starts working, the posture change of the top beam relative to the tunnel roof can be simply regarded as the top beam coordinate system rotating around the three axes of the tunnel roof coordinate system by ψ, θ, Angle, rotation matrix are: The total rotation matrix of the top beam relative to the tunnel roof is: The rotation matrix of the top beam relative to the roadway roof in S32 includes: The top beam coordinate system is obtained by translation matrix After transforming to the camera coordinate system, the rotation matrix of the top beam coordinate system relative to the tunnel roof coordinate system is Equivalent to the rotation matrix of the camera coordinate system relative to the tunnel roof coordinate system Get a Spin Then, it is converted into the changing attitude angle △ψ, △θ, Add the measured initial attitude angle and the changed attitude angle to obtain the solution formula for the attitude angle of the leading hydraulic support: The calculation of the attitude angle does not consider the non-commutative error, and the attitude angle of the advanced hydraulic support in the mine is within the range of [-30,30] degrees, and no angle singular value will occur.

2. The system and method for measuring the attitude angle of an advanced hydraulic support based on binocular vision according to claim 1, characterized in that: The explosion-proof terminal in S01 is connected to the binocular camera including: S11: Install Pycharm software under Windows system on the explosion-proof terminal, combine Python language and Opencv database to write the posture angle calculation program of the advanced hydraulic support; S12: The mine tunnel is dark, so an infrared binocular camera is installed to measure the attitude angle more clearly. The infrared binocular camera is installed horizontally on the top beam of the advance hydraulic support to measure the attitude angle at different times in real time.

3. The system and method for measuring the attitude angle of an advanced hydraulic support based on binocular vision according to claim 1, characterized in that: The innovative optimization of the target feature detection algorithm in S04 includes: When performing ORB feature detection on a target, errors may occur due to interference from obstacles, the environment, and other factors. Setting a detection distance threshold and a corner point intensity threshold can eliminate erroneous detection points in the image, improving detection accuracy and reducing detection speed. The "cv2.cuda" GPU acceleration module is embedded in the program, and the "multiprocessing" module is used to optimize the feature extraction speed of the ORB algorithm, achieving fast and accurate target detection.

4. The system and method for measuring the attitude angle of an advanced hydraulic support based on binocular vision according to claim 1, characterized in that: The innovative optimization of target stereo matching in S04 includes: The Gaussian pyramid method is added to reduce the discontinuity of target stereo matching. First, the original advanced hydraulic support image is smoothed by applying a Gaussian filter; then the smoothed image is downsampled; finally, the above steps are repeated until the desired ideal effect is achieved; the median filter method is used to remove image noise points, the edge enhancement method is used to enhance the image edges, and the hole filling method is used to repair the missing areas of the image, and the depth image is further optimized.

Images