A method for identifying tunnel surface defects

Abstract

The present invention discloses a method for identifying tunnel surface defects, which relates to the technical field of tunnel surface defect identification, and specifically comprises the following steps: firstly, preprocessing an input tunnel surface image to make the object edges in the tunnel surface image clearer; then, using a deep learning model, identifying tunnel surface defects on the preprocessed tunnel surface image; finally, using a point cloud data fitting algorithm to perform three-dimensional reconstruction on 3D point cloud data of the tunnel surface, and analyzing to obtain the deformation amount of the tunnel surface; the identification method is not affected by external light changes; for each frame of image, there is no need to pay too much attention to the size, type, color of foreign objects, nor to external information such as image exposure, and only the contrast of the target boundary needs to be improved; and for the same tunnel, two types of data are collected at one time, and there is no need to consider defect information too much. While detecting tunnel surface defects, the deformation of the tunnel is also detected, and the recognition rate is relatively high.

Description

Technical Field

[0001] The invention relates to the technical field of tunnel surface defect recognition, and in particular to a tunnel surface defect recognition method. Background Art

[0002] Current technologies mostly use convolutional neural network solutions to identify defects on the tunnel surface. This solution will train the corresponding model and identify defects on the tunnel surface. However, due to diffuse reflection, the recognition of the defect edge is not high, so the training samples of the model are difficult to obtain, resulting in the model's recognition rate and false alarm rate are not ideal, and the deformation of the tunnel surface cannot be detected. Summary of the invention

[0003] The purpose of the present invention is to provide a tunnel surface defect recognition method to solve the above problems, in view of the current tunnel surface defect recognition scheme, in which the recognition degree of the defect edge is not high due to diffuse reflection, the training samples of the model are difficult to obtain, resulting in the recognition rate and false alarm rate of the model being unsatisfactory, and the deformation of the tunnel surface cannot be detected.

[0004] The technical solution of the present invention is as follows:

[0005] A method for identifying tunnel surface defects comprises the following steps:

[0006] Step S1: preprocessing the input tunnel surface image to make the edges of objects in the tunnel surface image clearer;

[0007] Step S2: using a deep learning model to identify tunnel surface defects on the preprocessed tunnel surface image;

[0008] Step S3: Use a point cloud data fitting algorithm to perform three-dimensional reconstruction on the 3D point cloud data of the tunnel surface, and analyze and obtain the deformation of the tunnel surface.

[0009] Furthermore, the detailed steps of step S1 are:

[0010] Step S11: marking and extracting the edge of the object or the target shaped like the edge in the tunnel surface image to locate the edge of the object or the target shaped like the edge;

[0011] Step S12: sharpening the object edges or edge-like targets in the tunnel surface image, and outputting the preprocessed tunnel surface image.

[0012] Furthermore, the detailed steps in step S11 are: calculating the result matrix of Gabor filtering of each frame of the tunnel surface image, and mapping the result to the pre-processed tunnel surface image.

[0013] Furthermore, the detailed steps of step S12 are:

[0014] Use the Laplacian operator to sharpen each frame of the tunnel surface image;

[0015] The edge-type targets in the tunnel surface image are converted into foreground, and the remaining data are converted into background, so as to strengthen the edge of the object or the target shaped like an edge in the tunnel surface image.

[0016] Furthermore, the detailed steps of step S2 are:

[0017] Step S21: training a deep learning algorithm model according to the tunnel surface defect type;

[0018] Step S22: using the deep learning algorithm model trained in step S21 to perform defect detection on the preprocessed tunnel surface image;

[0019] Step S23: classify and label the defect detection results, and store the detection result images.

[0020] Furthermore, the 3D point cloud data of the tunnel surface is acquired by uniform sampling method.

[0021] Furthermore, the point cloud data fitting algorithm in step S3 is an ICP algorithm, which specifically includes:

[0022] According to the preset constraints, the nearest neighbor point is found in the target point cloud P and the source point cloud Q to be matched, and the minimum value of the error function is calculated, so as to iteratively calculate the distance between the adjacent points until the convergence condition is met.

[0023] Furthermore, the detailed steps of step S3 are:

[0024] Step S31: using a point cloud data fitting algorithm to perform three-dimensional reconstruction on the collected 3D point cloud data of the tunnel surface, and fit and construct a 3D model of the tunnel;

[0025] Step S32: performing structural geometry analysis on the three-dimensionally reconstructed tunnel 3D model to obtain structural geometry data of the tunnel surface;

[0026] Step S33: The deformation amount of the tunnel surface is calculated by using the structural geometric data of the tunnel surface obtained in step S32 and the structural geometric data of the tunnel surface in the tunnel 3D model template.

[0027] Furthermore, the specific steps of the ICP algorithm are as follows:

[0028] Step A: Take the point set pi∈P in the target point cloud P;

[0029] Step B: Find the corresponding point set qi∈Q in the source point cloud Q, so that ||qi-pi||=min;

[0030] Step C: Calculate the rotation matrix R and translation matrix t so that the error function Minimum;

[0031] Step D: Use the rotation matrix R and translation matrix t calculated in step C to perform rotation and translation transformation on pi, and obtain a new corresponding point set pi'={pi'=R*pi+t,pi∈P};

[0032] Step E: Calculate the average distance d between pi' and the corresponding point set qi;

[0033] Step F: If d is less than a given threshold or greater than a preset maximum number of iterations, stop the iterative calculation; otherwise, return to step B until the convergence condition is met.

[0034] Furthermore, the error function in step C The calculation formula is as follows:

[0035]

[0036] in:

[0037] n-the number of nearest neighbor pairs;

[0038] pi - a point in the target point cloud P;

[0039] qi-the closest point in the source point cloud Q corresponding to pi;

[0040] R - rotation matrix;

[0041] t- is the translation vector;

[0042] The calculation formula of the average distance d in step E is as follows:

[0043] .

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] A tunnel surface defect recognition method, first preprocessing an input tunnel surface image to make the object edges in the tunnel surface image clearer; then using a deep learning model to recognize tunnel surface defects on the preprocessed tunnel surface image; finally using a point cloud data fitting algorithm to perform three-dimensional reconstruction on the tunnel surface 3D point cloud data, and analyzing to obtain the deformation of the tunnel surface; the tunnel surface defect recognition method is not affected by external light changes, and is only related to the contrast between the target boundary and the background in the current image; for each frame of the image, because the data acquisition device will perform light compensation according to the on-site light conditions, there is no need to pay too much attention to the size, type, color of the foreign body, nor to external information such as image exposure, and only the contrast of the target boundary needs to be improved; and for the same tunnel, two types of data are collected at one time, without considering the defect information too much, and the deformation of the tunnel is also detected while detecting the tunnel surface defects, and the recognition rate is relatively high. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of a tunnel surface defect identification method;

[0047] Figure 2 This is an example of an input tunnel surface image;

[0048] Figure 3 This is an example of a sharpened tunnel surface image;

[0049] Figure 4 This is an example of Gabor filtering results;

[0050] Figure 5 This is an example of the preprocessed tunnel surface image. DETAILED DESCRIPTION

[0051] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0052] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0053] Embodiment 1

[0054] Current technologies mostly use convolutional neural network solutions to identify defects on the tunnel surface. This solution will train the corresponding model and identify defects on the tunnel surface. However, due to diffuse reflection, the recognition of the defect edge is not high, so the training samples of the model are difficult to obtain, resulting in the model's recognition rate and false alarm rate are not ideal, and the deformation of the tunnel surface cannot be detected.

[0055] This embodiment provides a tunnel surface defect identification method for the above-mentioned problem. Figure 1-5 , specifically including the following steps:

[0056] Step S1: preprocessing the input tunnel surface image to make the edges of objects in the tunnel surface image clearer; thereby making the edges of objects entering step S2 clearer;

[0057] Step S2: using a deep learning model to identify tunnel surface defects on the preprocessed tunnel surface image; the defects mainly include: cracks, breakage, water leakage, etc.;

[0058] Step S3: Use a point cloud data fitting algorithm to perform three-dimensional reconstruction on the 3D point cloud data of the tunnel surface, and analyze and obtain the deformation of the tunnel surface.

[0059] In this embodiment, specifically, the detailed steps of step S1 are:

[0060] Step S11: Annotate and extract the edge of the object or the target with a similar shape to the edge in the tunnel surface image to locate the edge of the object or the target with a similar shape to the edge; Specifically, the detailed steps of step S11 are: calculate the result matrix of Gabor filtering of each frame image in the input tunnel surface image through a Gabor filter, and map the result to the pre-processed tunnel surface image; Further, step S11 also provides a target positioning basis for the sharpening operation in the following step S12;

[0061] Step S12: sharpen the edges of objects or targets that resemble edges in the tunnel surface image, and output the preprocessed tunnel surface image; specifically, the detailed steps of step S12 are: use the Laplace operator to sharpen each frame of the tunnel surface image; convert the edge-type targets in the tunnel surface image into the foreground, and convert the remaining data into the background to strengthen the edges of objects or targets that resemble edges in the tunnel surface image; and provide data support for step S3, that is, provide data support for the foreground and background segmentation of the deep learning algorithm model in the following step S3 when processing the image; the purpose of using the Laplace operator is to strengthen and highlight the edges of objects or targets that resemble edges in the input tunnel surface image, that is, to mark the boundaries in the preprocessed tunnel surface image.

[0062] The mathematical expression of the two-dimensional Gabor function of the Gabor filter in step S11 is as follows:

[0063] Plural representation:

[0064]

[0065] Real part:

[0066]

[0067] Imaginary part:

[0068]

[0069] x＇ , y＇ The calculation formula is:

[0070]

[0071] in:

[0072] l -wavelength; represents the wavelength parameter of the cosine function in the Gabor kernel function; l The value of is specified in pixels and is usually greater than or equal to 2, but cannot be greater than 1 / 5 of the input image size;

[0073] i -direction; represents the direction of the parallel stripes in the Gabor filter kernel; the valid value is a real number from 0° to 360°;

[0074] ψ -Phase offset; represents the phase parameter of the cosine function in the Gabor kernel function; ψ The value range of is -180° to 180°; the equations corresponding to 0° and 180° are symmetrical with the origin, and the equations corresponding to -90° and 90° are centrally symmetrical with respect to the origin;

[0075] c -Aspect ratio; that is, the spatial aspect ratio, which determines the ellipticity of the Gabor function shape; when c =1, the shape is circular; when c <1, the shape is elongated along the parallel stripe direction, usually this value is c =0.5.

[0076] Half-response spatial frequency bandwidth of Gabor filter b and s / l The ratio of s Represents the standard deviation of the Gaussian factor of the Gabor function; the three have the following relationship:

[0077]

[0078] s The value of cannot be set directly; it varies only with bandwidth. b Transformation; the bandwidth value must be a positive real number, usually 1. In this case, the relationship between the standard deviation and the wavelength is s =0.56 l ; The smaller the bandwidth, the larger the standard deviation, the larger the Gabor shape, and the more parallel stripes are visible.

[0079] In this embodiment, the Laplace operator is used because the Laplace operator is more suitable for improving image blur caused by diffuse reflection of light; the Laplace operator method is a commonly used edge enhancement processing operator and is an isotropic second-order derivative.

[0080] The formula for Laplace operation is explained as follows:

[0081] For continuous binary functions we have:

[0082]

[0083] in:

[0084] -Pixels P The pixel coordinates in the image;

[0085] -Pixels P The Laplace value of .

[0086] After sorting:

[0087]

[0088] in:

[0089] -Pixels P The pixel coordinates in the image;

[0090] -Pixels P Gray value of

[0091] The preceding coefficients are the values ​​of the corresponding positions of the convolution kernel.

[0092] The Laplace operator template can be obtained as:

[0093] or

[0094] Among them, yes The convolution kernel.

[0095] In this embodiment, specifically, the detailed steps of step S2 are:

[0096] Step S21: training a deep learning algorithm model according to the type of tunnel surface defects; wherein the training method of the deep learning algorithm model is a prior art, and those skilled in the art are able to understand the training methods of different deep learning algorithm models, which will not be described in detail here;

[0097] Step S22: using the deep learning algorithm model trained in step S21 to perform defect detection on the preprocessed tunnel surface image;

[0098] Step S23: classify and annotate the defect detection results, and store the detection result image; preferably, the defect detection results are annotated in the form of a calibration frame on each input frame image.

[0099] In this embodiment, the 3D point cloud data of the tunnel surface in step S3 is acquired by uniform sampling method.

[0100] In this embodiment, the point cloud data fitting algorithm in step S3 specifically adopts the ICP algorithm, which specifically includes:

[0101] According to the preset constraints, the nearest neighbor point is found in the target point cloud P and the source point cloud Q to be matched, and the minimum value of the error function is calculated, so as to iteratively calculate the distance between the adjacent points until the convergence condition is met.

[0102] In this embodiment, specifically, the detailed steps of step S3 are:

[0103] Step S31: using a point cloud data fitting algorithm, three-dimensionally reconstructing the acquired 3D point cloud data of the tunnel surface, and fitting and constructing a 3D model of the tunnel; preferably, the 3D point cloud data of the tunnel surface is collected while using the captured image;

[0104] Step S32: performing structural geometry analysis on the three-dimensionally reconstructed tunnel 3D model to obtain structural geometry data of the tunnel surface;

[0105] Step S33: The deformation amount of the tunnel surface is calculated by using the structural geometric data of the tunnel surface obtained in step S32 and the structural geometric data of the tunnel surface in the tunnel 3D model template.

[0106] In this embodiment, it should be noted that the specific steps of the ICP algorithm are as follows:

[0107] Step A: Take the point set pi∈P in the target point cloud P;

[0108] Step B: Find the corresponding point set qi∈Q in the source point cloud Q, so that ||qi-pi||=min;

[0109] Step C: Calculate the rotation matrix R and translation matrix t so that the error function Minimum;

[0110] Step D: Use the rotation matrix R and translation matrix t calculated in step C to perform rotation and translation transformation on pi, and obtain a new corresponding point set pi'={pi'=R*pi+t,pi∈P};

[0111] Step E: Calculate the average distance d between pi' and the corresponding point set qi;

[0112] Step F: If d is less than a given threshold or greater than a preset maximum number of iterations, stop the iterative calculation; otherwise, return to step B until the convergence condition is met.

[0113] In this embodiment, specifically, the error function in step C The calculation formula is as follows:

[0114]

[0115] in:

[0116] n-the number of nearest neighbor pairs;

[0117] pi - a point in the target point cloud P;

[0118] qi-the closest point in the source point cloud Q corresponding to pi;

[0119] R - rotation matrix;

[0120] t- is the translation vector;

[0121] The calculation formula of the average distance d in step E is as follows:

[0122] .

[0123] Embodiment 2

[0124] Example 2 is a further explanation of Example 1. Figure 1-5 The deep learning algorithm model used in step S2 is the yolov5-v3.1 model, which has the advantages of faster detection rate and higher detection accuracy compared to the currently used convolutional neural network; specifically, the training method of the yolov5-v3.1 model and the recognition method based on the yolov5-v3.1 model are both prior arts, and those skilled in the art can understand the above-mentioned training method and recognition method, which will not be described in detail here.

[0125] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A method for identifying tunnel surface defects, characterized in that: The steps include: Step S1: preprocessing the input tunnel surface image to make the edges of objects in the tunnel surface image clearer; Step S2: using a deep learning model to identify tunnel surface defects on the preprocessed tunnel surface image; Step S3: reconstructing the 3D point cloud data of the tunnel surface using a point cloud data fitting algorithm, and analyzing and obtaining the deformation of the tunnel surface; The detailed steps of step S1 are: Step S11: marking and extracting the edge of the object or the target shaped like the edge in the tunnel surface image to locate the edge of the object or the target shaped like the edge; Step S12: sharpening the object edge or the target similar to the edge in the tunnel surface image, and outputting the preprocessed tunnel surface image; The detailed steps of step S12 are: Use the Laplacian operator to sharpen each frame of the tunnel surface image; The edge-type targets in the tunnel surface image are converted into foreground, and the remaining data are converted into background, so as to strengthen the edge of the object or the target shaped like an edge in the tunnel surface image.

2. A tunnel surface defect identification method according to claim 1, characterized in that: The detailed steps in step S11 are: calculating the result matrix of Gabor filtering of each frame of the tunnel surface image, and mapping the result to the pre-processed tunnel surface image.

3. A tunnel surface defect identification method according to claim 1, characterized in that: The detailed steps of step S2 are: Step S21: training a deep learning algorithm model according to the tunnel surface defect type; Step S22: using the deep learning algorithm model trained in step S21 to perform defect detection on the preprocessed tunnel surface image; Step S23: classify and label the defect detection results, and store the detection result images.

4. A tunnel surface defect identification method according to claim 1, characterized in that: The 3D point cloud data of the tunnel surface is acquired by uniform sampling method.

5. A tunnel surface defect identification method according to claim 1 or 4, characterized in that: The point cloud data fitting algorithm in step S3 is an ICP algorithm, which specifically includes: According to the preset constraints, the nearest neighbor point is found in the target point cloud P and the source point cloud Q to be matched, and the minimum value of the error function is calculated, so as to iteratively calculate the distance between the adjacent points until the convergence condition is met.

6. A tunnel surface defect identification method according to claim 5, characterized in that: The detailed steps of step S3 are: Step S31: using a point cloud data fitting algorithm to perform three-dimensional reconstruction on the collected 3D point cloud data of the tunnel surface, and fit and construct a 3D model of the tunnel; Step S32: performing structural geometry analysis on the three-dimensionally reconstructed tunnel 3D model to obtain structural geometry data of the tunnel surface; Step S33: The deformation amount of the tunnel surface is calculated by using the structural geometric data of the tunnel surface obtained in step S32 and the structural geometric data of the tunnel surface in the tunnel 3D model template.

7. A tunnel surface defect identification method according to claim 6, characterized in that: The specific steps of the ICP algorithm are as follows: Step A: Take the point set pi∈P in the target point cloud P; Step B: Find the corresponding point set qi∈Q in the source point cloud Q, so that ||qi-pi||=min; Step C: Calculate the rotation matrix R and translation matrix t so that the error function Minimum; Step D: Use the rotation matrix R and translation matrix t calculated in step C to perform rotation and translation transformation on pi, and obtain a new corresponding point set pi'={pi'=R*pi+t,pi∈P}; Step E: Calculate the average distance d between pi' and the corresponding point set qi; Step F: If d is less than a given threshold or greater than a preset maximum number of iterations, stop the iterative calculation; otherwise, return to step B until the convergence condition is met.

8. A tunnel surface defect identification method according to claim 7, characterized in that: The error function in step C The calculation formula is as follows: in: n-the number of nearest neighbor pairs; pi - a point in the target point cloud P; qi-the closest point in the source point cloud Q corresponding to pi; R - rotation matrix; t- is the translation vector; The calculation formula of the average distance d in step E is as follows: 。

Images