Roadway deformation detection method and system based on image processing

Abstract

The invention discloses a roadway deformation detection method and system based on image processing, and relates to the technical field of coal mine roadway engineering. The method comprises the following specific steps: arranging angular points in a roadway; shooting angular points in the roadway to obtain a roadway image; performing corner detection on the roadway image through a Harris corner detection algorithm based on image recognition; carrying out superposition operation on the plurality of roadway images of which the angular points are detected according to different weights to obtain a superposed image; carrying out position coordinate solving on the identified angular points in the superposed image to obtain angular point coordinates; and comparing the position changes of the same angular point in the image at different moments to obtain the displacement of the angular point. According to the method, roadway deformation is detected through the Harris corner detection algorithm, high precision and stability are achieved, and the roadway deformation condition can be accurately detected in various complex mine environments.

Description

Technical Field

[0001] The invention relates to the technical field of coal mine tunnel engineering, and in particular to a tunnel deformation detection method and system based on image processing. Background Art

[0002] In the process of coal mining, tunnel deformation and damage is a serious safety hazard, which may bring many hazards. Tunnel deformation will directly threaten the life safety of miners and affect the normal production of coal mines. As the transportation and ventilation channel inside the coal mine, once the tunnel is deformed or damaged, it will lead to obstruction of transportation and ventilation, which will affect the mining efficiency and output of coal, bring economic losses to coal mining enterprises, and may also damage equipment and facilities, and increase maintenance and replacement costs. In addition, tunnel deformation may also cause secondary disasters, such as gas leakage, coal dust explosion, etc. Therefore, it is necessary to attach great importance to the maintenance and detection of tunnels, and take effective measures to prevent the occurrence of tunnel deformation and damage. The existing methods for detecting tunnel deformation are mainly: the first is the indirect measurement method, and the cross point method is a method commonly used for roof separation detection and stress detection in mine tunnels, but this method usually requires manual operation, which is time-consuming and laborious. Another method is direct measurement. The roof separation meter is a device specially used to detect roof separation (i.e., separation and displacement of roof rock layers) in coal mine tunnels. It is mainly used to evaluate the stability of the tunnel roof, prevent roof accidents, and ensure the safety of coal mine operations. However, the cost and maintenance cost of using the roof separation meter equipment are relatively high, which also brings certain difficulties to the tunnel deformation detection. Therefore, how to overcome the above defects and realize intelligent detection of tunnel deformation is a problem that technicians in this field need to solve urgently. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for detecting lane deformation based on image processing, so as to solve the problems raised in the background technology and realize accurate and efficient detection of lane deformation.

[0004] To achieve the above object, the present invention provides the following solution: a tunnel deformation detection method based on Harris corner point detection algorithm, the specific steps include the following:

[0005] Arrange corner points in the laneway;

[0006] Photograph the corner points in the lane to obtain the lane image;

[0007] Performing corner detection on the lane image by using a Harris corner detection algorithm based on image recognition;

[0008] Performing a superposition operation on the multiple lane images with corner points detected according to different weights to obtain a superimposed image;

[0009] Solving the position coordinates of the corner points identified in the superimposed image to obtain the coordinates of the corner points;

[0010] Compare the position changes of the same corner point in the image at different times to obtain the displacement of the corner point.

[0011] Preferably, reference corner points are arranged in a stable area section on the inner wall of the tunnel, and then corner points are arranged around the tunnel at fixed distances along the tunnel direction.

[0012] Preferably, a distortion-free camera is used to acquire the lane image at fixed time intervals.

[0013] Preferably, the superposition function for performing the superposition operation is:

[0014] g(x)=(1-α)f1(x)+αf2(x);

[0015] Among them, α is the weight, ranging from 0 to 1, f1(x) represents the first lane image, and f2(x) represents the second lane image.

[0016] Preferably, the initially detected corner points are refined by using the cv2.comerSubPix() function to obtain the precise coordinates of the corner points.

[0017] Preferably, the calculation process of the displacement of the corner point is:

[0018] Convert the pixel coordinates of the corner point (x0:y0) to image coordinates (x, y);

[0019] The image coordinates are converted into camera coordinates (X c : Y c :Z c );

[0020] Calculate the displacement of the corner point in the camera coordinate system.

[0021] Preferably, the displacement is obtained by Euclidean distance, and the calculation formula is:

[0022]

[0023] On the other hand, a lane deformation detection system based on the Harris corner point detection algorithm is provided, comprising a corner point setting module, an image acquisition module, a preliminary detection module, an overlay module, a coordinate acquisition module, and a displacement detection module; wherein,

[0024] The corner point setting module is used to arrange corner points in the lane;

[0025] The image acquisition module is used to photograph corner points in the lane to acquire lane images;

[0026] The preliminary detection module is used to perform corner detection on the lane image by using a Harris corner detection algorithm based on image recognition;

[0027] The superposition module is used to perform superposition operation on the multiple lane images with corner points detected according to different weights to obtain a superimposed image;

[0028] The coordinate acquisition module is used to solve the position coordinates of the corner points identified in the superimposed image to obtain the coordinates of the corner points;

[0029] The displacement detection module is used to compare the position changes of the same corner point in the image at different times to obtain the displacement of the corner point.

[0030] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0031] (1) By introducing the Harris corner detection algorithm, high-precision detection of tunnel deformation is achieved with a high degree of automation: the entire detection process does not require a large amount of manpower for on-site measurement, which reduces labor costs and labor intensity and improves detection efficiency.

[0032] (2) High precision and high reliability: By using high-resolution images and precise corner detection algorithms, accurate deformation data can be obtained, ensuring the reliability of the detection results.

[0033] (3) Strong real-time performance: images and detection data can be collected regularly or in real time to detect tunnel deformation in a timely manner and improve safety management. Through real-time detection, potential safety issues can be discovered and dealt with in the first place to avoid accidents.

[0034] (4) Easy equipment maintenance: The camera equipment and corner markers are relatively easy to maintain and replace, which reduces the difficulty and cost of equipment maintenance. The stability and maintainability of the equipment enable the detection system to operate reliably for a long time.

[0035] (5) The present invention provides an efficient, accurate and automated method for detecting lane deformation by combining image recognition technology and Harris corner detection algorithm, which solves many problems in traditional detection methods and provides strong support for lane safety management. This method not only improves the efficiency and accuracy of detection, but also reduces the cost and maintenance difficulty, which is of great significance to lane safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 is a flow chart of the method of the present invention;

[0038] Figure 2 It is a corner point pattern diagram of the present invention;

[0039] Figure 3 It is the layout diagram of the lane corner points of the present invention;

[0040] Figure 4 This is a flow chart of the key steps of the Harris corner detection algorithm of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The purpose of the present invention is to provide a lane deformation detection method based on Harris corner detection algorithm. Figure 1 As shown, the specific steps include the following:

[0043] S1. Arrange corner points in the lane;

[0044] S2, photographing the corner points in the lane to obtain a lane image;

[0045] S3, performing corner detection on the lane image by using the Harris corner detection algorithm based on image recognition;

[0046] S4, performing a superposition operation on the multiple lane images with corner points detected according to different weights to obtain a superimposed image;

[0047] S5, solving the position coordinates of the corner points identified in the superimposed image to obtain the coordinates of the corner points;

[0048] S6. Compare the position changes of the same corner point in the image at different times to obtain the displacement of the corner point.

[0049] Furthermore, in the process of coal mining, the types and forms of tunnels excavated are diverse, and many uneven areas often appear on the inner walls of the tunnels. These areas naturally form corner points. The definition of corner points is as follows: Corner points are usually the intersections between image contours. Even if the viewing angle changes in the same scene, the corner points still maintain stable features. In the area near the corner point, the gradient direction and gradient amplitude of the pixel point will change significantly, so the corner point is regarded as the extreme point of the gradient change. In the field of computer vision, corner points are key points used by computers to extract image features. In reality, corner points can be understood as the intersection of two straight lines.

[0050] In order to reduce the impact of the irregular shape of the tunnel wall, it is necessary to manually arrange corner points on the tunnel wall. This can ensure the consistency and identifiability of the corner points in different images. The style of the corner points is as follows: Figure 2 As shown. On the inner wall of the tunnel, according to the actual situation, the reference corner points are arranged in the stable area of ​​the tunnel. Usually, the number of corner points arranged in a tunnel section is more than 4 points, and then the corner points are arranged around the tunnel at a fixed distance along the tunnel direction. These manually arranged corner points will serve as the basis for image processing and detection, and provide reliable feature points for subsequent analysis.

[0051] After arranging corner points in the lane, Figure 3 As shown in the figure, different types of tunnels have different numbers of corner points. The number of these corner points determines the accuracy of the subsequent calculation of tunnel deformation data. To ensure the accuracy of the shooting results, the camera used should be a distortion-free camera to prevent the influence of radial distortion and tangential distortion, and the images should be taken at fixed time intervals.

[0052] After taking the image, it needs to be converted to grayscale, which can be done using the cv2.cvtColor() function. The Harris corner detection algorithm is then used to identify the corner position information on the image. The core idea of ​​the algorithm is to use a moving window to calculate the grayscale change value in the image, and then determine whether the area is a corner, edge, or smooth area based on its grayscale change difference. The key processes are as follows: Figure 4 As shown, it includes conversion to grayscale image, calculation of differential image, Gaussian smoothing, calculation of local extreme values, and confirmation of corner points. The above can be implemented using the cv2.comerHarris() function.

[0053] The mathematical formula is:

[0054]

[0055] Wherein, W(x, y) represents the moving window, I(x, y) represents the intensity of the pixel gray value, ranging from 0 to 255, and I(x+u, y+v) represents the intensity of the gray value after the pixel changes.

[0056] Furthermore, the image overlay operation is performed. The addWeighted() function is very suitable for synthesizing two different images in the image processing process. This function synthesizes a new image by applying a weighted method to two images, so that the synthesis result is not just a simple layer overlay, but can reflect the characteristics of both images at the same time. This weighted average method can make the content of one image more prominent by adjusting the respective weight ratios, while the other image serves as a background or supplement. The image overlay function is as follows:

[0057] g(x)=(1-α)f1(x)+αf2(x);

[0058] Among them, α is the weight, ranging from 0 to 1, f1(x) represents the first lane image, and f2(x) represents the second lane image.

[0059] After the two images are superimposed, the next step is to solve the position coordinates of the corner points identified in the superimposed image. This step is very important to ensure the accuracy of corner detection. To this end, the cv2.comerSubPix() function is used, which can further refine the position coordinates of the corner points, thereby improving the accuracy of detection. First, the Harris corner detection algorithm is used to identify the corner positions in the image. These initially detected corner positions are usually at the pixel level. The cv2.comerSubPix() function is used to refine the initially detected corner points to obtain accurate coordinates at the sub-pixel level.

[0060] Furthermore, after obtaining the position coordinates of the corner point, the displacement of the corner point can be obtained by comparing the position changes of the same corner point in the image at different times.

[0061] To calculate the actual deformation data of the roadway through pixel displacement, the camera needs to be calibrated first to obtain the camera's intrinsic parameter matrix K. Camera calibration usually uses the Zhang Zhengyou calibration method, which can be achieved by using the cv2.calibrateCamera() function in the OpenCV library.

[0062] The pixel coordinates (x0, y0) of a corner point are converted to image coordinates (x, y) using the following formula:

[0063]

[0064] Where W is the number of pixels arranged in the width of the image, and H is the number of pixels arranged in the height of the image.

[0065] The image coordinates are converted into camera coordinates (X c , Y c , Z c); can be calculated by the following formula:

[0066]

[0067] Among them, Z is the distance from the captured corner point to the camera, that is, the depth information, and the depth information can be obtained using a depth camera.

[0068] For two pixel points (x1, y1) and (x2, y2), they are transformed into camera coordinates respectively:

[0069]

[0070] Calculate the displacement of the corner point in the camera coordinate system:

[0071]

[0072] During the shooting process, the camera position remains fixed, so the camera coordinate system is consistent with the world coordinate system. The displacement in the camera coordinate system can be directly used to calculate the actual displacement. The actual movement distance can be obtained by calculating the Euclidean distance of the displacement vector:

[0073]

[0074] The present invention uses the Harris corner detection algorithm to detect tunnel deformation. The Harris corner detection algorithm is a computer vision technology that can effectively identify features such as corners and edges in images. In tunnel deformation detection, this method has high accuracy and stability and can work reliably in complex environments. Compared with existing tunnel deformation detection technologies, the use of the Harris corner detection algorithm has obvious advantages. First, it has good stability, is not easily disturbed by environmental noise, and can accurately detect tunnel deformation in various complex mining environments. Secondly, the algorithm is easy to operate and does not require complex equipment and professional knowledge. Ordinary mine workers can operate it after simple training. This greatly reduces the threshold and cost of use and improves the operability and practicality of detection.

[0075] On the other hand, a lane deformation detection system based on the Harris corner point detection algorithm is provided, including a corner point setting module, an image acquisition module, a preliminary detection module, an overlay module, a coordinate acquisition module, and a displacement detection module; wherein,

[0076] Corner point setting module, used to arrange corner points in the lane;

[0077] An image acquisition module is used to photograph corner points in the lane to acquire lane images;

[0078] A preliminary detection module is used to detect corners of the lane image using the Harris corner detection algorithm based on image recognition;

[0079] A superposition module is used to perform superposition operations on multiple lane images with corner points detected according to different weights to obtain a superimposed image;

[0080] A coordinate acquisition module is used to solve the position coordinates of the corner points identified in the superimposed image and obtain the coordinates of the corner points;

[0081] The displacement detection module is used to compare the position changes of the same corner point in the image at different times to obtain the displacement of the corner point.

[0082] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method and system for detecting lane deformation based on image processing, characterized in that: The specific steps include the following: Arrange corner points in the laneway; Photograph the corner points in the tunnel and obtain the tunnel image; Performing corner detection on the lane image by using a Harris corner detection algorithm based on image recognition; Performing a superposition operation on the multiple lane images with corner points detected according to different weights to obtain a superimposed image; Solving the position coordinates of the corner points identified in the superimposed image to obtain the coordinates of the corner points; Compare the position changes of the same corner point in the image at different times to obtain the displacement of the corner point.

2. The tunnel deformation detection method based on Harris corner detection algorithm according to claim 1, characterized in that: The reference corner points are arranged in a stable area section on the inner wall of the tunnel, and then the corner points are arranged around the tunnel at fixed distances along the tunnel direction.

3. The tunnel deformation detection method based on Harris corner detection algorithm according to claim 1, characterized in that: The tunnel image is acquired at fixed time intervals using a distortion-free camera.

4. The tunnel deformation detection method based on Harris corner detection algorithm according to claim 1, characterized in that: The superposition function for superposition operation is: g(x)=(1-α)f1(x)+αf2(x); Among them, α is the weight, ranging from 0 to 1, f1(x) represents the first lane image, and f2(x) represents the second lane image.

5. The method for detecting lane deformation based on Harris corner detection algorithm according to claim 1, characterized in that: The initially detected corner points are refined through the cv2.comerSubPix() function to obtain the precise coordinates of the corner points.

6. The method for detecting lane deformation based on Harris corner detection algorithm according to claim 1, characterized in that: The calculation process of the displacement of the corner point is: Convert the pixel coordinates (x0, y0) of the corner point to image coordinates (x, y); The image coordinates are converted into camera coordinates (X c , Y c , Z c ); Calculate the displacement of the corner point in the camera coordinate system.

7. A tunnel deformation detection method based on Harris corner detection algorithm according to claim 6, characterized in that: The displacement is obtained through the Euclidean distance, and the calculation formula is:

8. A tunnel deformation detection system based on Harris corner point detection algorithm, characterized in that: It includes a corner point setting module, an image acquisition module, a preliminary detection module, an overlay module, a coordinate acquisition module, and a displacement detection module; wherein, The corner point setting module is used to arrange corner points in the lane; The image acquisition module is used to photograph corner points in the lane to acquire lane images; The preliminary detection module is used to perform corner detection on the lane image by using a Harris corner detection algorithm based on image recognition; The superposition module is used to perform superposition operation on the multiple lane images with corner points detected according to different weights to obtain a superimposed image; The coordinate acquisition module is used to solve the position coordinates of the corner points identified in the superimposed image to obtain the coordinates of the corner points; The displacement detection module is used to compare the position changes of the same corner point in the image at different times to obtain the displacement of the corner point.

Images