Tunnel multi-period change condition monitoring method based on linear array image data

Abstract

The invention belongs to the technical field of tunnel monitoring, and particularly relates to a tunnel multi-period change condition monitoring method based on linear array image data, which comprises the following steps: step 1, image registration; step 2, image change detection; and step 3, marking a change area. According to the method, image registration is carried out on the obtained two-stage images, then the change condition of each pixel is judged by using the SSIM index and the threshold value, the change area is obtained, and finally, labeling and re-checking of the change area are realized manually, so that timely monitoring of the long-term change condition is realized. Compared with the prior art, multi-stage image data of the same area are used for contrastive analysis, pixel-level detection of tunnel images can be achieved, small changes of a tunnel can be sensitively recognized, new small diseases can be found in time, the omission ratio is low, the development and change conditions of existing diseases can be continuously monitored, the conditions of equipment and facilities in the tunnel can be updated, and the efficiency is improved. And the method has good consistency for tunnels under different conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel monitoring, and in particular to a method for monitoring multi-period changes in tunnels based on linear array image data. Background Art

[0002] Tunnels are an important part of the transportation network. With the rapid development of my country's economy and the gradual improvement of urban construction, the total mileage of tunnels in my country is also increasing rapidly. In the daily operation of tunnels, they will be affected by a series of natural and human factors. Various diseases will inevitably appear in the tunnel environment, which seriously threatens the safety of the tunnel structure and the normal operation of the transportation system. Therefore, the investigation and maintenance of tunnel diseases are of vital importance to ensure the safe operation of tunnels and the normal operation of trains.

[0003] Traditional identification of tunnel surface defects is mainly based on manual visual identification methods, which have problems such as low detection efficiency, high risk, high labor cost, poor accuracy and consistency; the intensity images generated by mobile laser scanning technology can realize the identification of surface defects, but their resolution is low, only grayscale images can be generated, and detailed texture information will be lost due to different materials; area array cameras have high requirements for light sources, complex methods for generating full-section images, and low measurement frequency; linear array cameras can be used to obtain high-resolution full-section images of tunnels, with a resolution of 0.2mm or even higher, which meets the needs of tunnel surface defect detection.

[0004] The use of tunnel images for disease detection includes manual labeling methods and automatic identification methods. The manual labeling method still has problems such as low detection efficiency, while the use of deep learning and other methods to achieve automatic disease detection has problems of false detection and missed detection, and the effect of the model on tunnels in different situations varies. Summary of the invention

[0005] 1. Technical problems to be solved

[0006] The purpose of the present invention is to solve the problems of low detection efficiency in the manual labeling method in the prior art and false detection and missed detection using methods such as deep learning, and to propose a tunnel multi-period change monitoring method based on linear array image data.

[0007] 2. Technical solution

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] The method for monitoring multi-period changes of a tunnel based on linear array image data comprises the following steps:

[0010] Step 1: Image registration: Use image registration to register the images collected in the two phases to eliminate the displacement, rotation, scaling, bending and other changes of the image points caused by complex factors such as sensor orientation change, lens distortion, terrain undulation, uneven speed and irregular shaking of the rail vehicle during operation, so that the two phases of images have a consistent reference frame;

[0011] Step 2: Image change detection, using structural similarity (SSIM) as a comparison index. SSIM is an index to measure the similarity between two images. It compares the brightness, contrast, and structure of the two images, and then weights these three factors and expresses them as products. Suppose the two images are x and y, respectively. The brightness contrast function I(x, y), contrast contrast function c(x, y), and structure contrast function s(x, y) are shown in the formula:

[0012]

[0013] The constants C1, C2, and C3 are used to avoid instability when the denominator is 0. In particular, C1 = (K1L) 2 , C2=(K2L) 2 , L is the gray level of the image. For an 8-bit gray image, L = 255, and K1 < < 1, K2 < < 1,

[0014] μ x , μ x is the mean, σ x , σ y is the variance, σ xy is the covariance:

[0015]

[0016] The general formula of SSIM is:

[0017] SSIM(x,y)=[I(x,y)] α [c(x,t)] β [s(x,y)] γ (2.3)

[0018] Take α, β, γ = 1, C3 = C2 / 2, and get the simplified SSIM formula:

[0019]

[0020] Normally, the default values ​​are K1=0.01 and K2=0.03. There is a Gaussian weighted SSIM, which multiplies the weights when calculating the mean, variance, and covariance:

[0021]

[0022] The range of SSIM value is [-1, 1], SSIM(x, y) = 1 if and only if x = y.

[0023] Set the window size of Gaussian weights, open a search window at the corresponding position of image 2 for each pixel on image 1, calculate the maximum value of the SSIM value based on Gaussian weights of the corresponding points in the search window, obtain the set of changed pixels, use opening and closing operations to remove noise and fill holes, and finally obtain the outline of the changed area;

[0024] Step 3: Labeling of the changed area. After obtaining the changed area, manually label the properties of the changed area, store the labeling results, and use them as the original data for the next change detection to achieve timely monitoring of long-term changes.

[0025] Preferably, the image registration specifically comprises the following steps:

[0026] S1: Initial mileage alignment. The images collected in the two phases have inconsistent initial mileages, which may be quite different. It is known that each column of the image stores the image data of a section of the tunnel. Therefore, for the two phases of images, the data of the first x columns are selected, where x is determined by the actual situation, and feature matching is used for calculation;

[0027] S2: Feature matching. Image registration is based on the feature matching results of the two images. The initial matching columns are determined for the first image, and a fixed value of 5000 columns is used as the range of a single match. The initial columns corresponding to the first image are determined for the second image, and the search area is expanded to a certain extent based on the corresponding columns as the range of a single match. After each registration is completed, the registration results are used to update the corresponding initial columns of the two images, and the SURF operator is used to calculate the key points and descriptors of the images. Then, the Brute-Force feature matching method is used for feature matching, and the results are filtered using the threshold method to delete the mismatched points with too large difference values ​​and average difference values ​​in the X and Y directions.

[0028] S3: Construction of pixel coordinate transformation relationship. Linear array imaging has specific geometric parameters when imaging each column. The geometric relationship is too complicated. Therefore, a polynomial model and a local area correction model that avoid the spatial geometric relationship of imaging are used.

[0029] The polynomial model constructs a linear polynomial with multiple parameters, which directly simulates the image deformation itself mathematically, and regards the overall deformation of the image as the combined result of various basic deformations. Usually, the highest order is no more than 3. The effect of the polynomial model is weaker than that of the triangulated network model.

[0030]

[0031] The triangulated network model uses feature matching point sets to construct a triangulated network, finds the triangle corresponding to each pixel coordinate of the target image, and uses the triangle barycentric coordinate interpolation method to calculate the corresponding original x and y coordinates and resample them. In order to prevent some areas from having no corresponding triangles, the four endpoints of the image use the polynomial model to calculate the corresponding coordinates and add them to the point set when constructing the triangulated network. The interpolation formula is shown in Formula 1.2, where the coordinates of the three vertices of the triangle are (x A ,y A )、(x B ,y B )、(x C ,y C ), and their corresponding values ​​are V A 、V B 、V C ,

[0032]

[0033] Using the transformation relationship model, starting from the output image, that is, the blank image array, the position of each pixel in the original image is calculated in turn, and then the pixel value of the original image at that position is calculated, and then the calculated pixel value is assigned to the output image pixel. This method ensures that the pixels of the corrected image are evenly distributed in space. Grayscale resampling is required. The polynomial obtained by the polynomial model is used to bring the end column of this registration into the initial column corresponding to the next registration.

[0034] S4: Histogram normalization. The two images after reconstruction achieve consistency in pixel coordinates, but there is an obvious brightness difference between the two images. When collecting data in the same area, the brightness difference can be greatly improved by histogram normalization. In specific implementation, the image with large average brightness is used as a template.

[0035] Preferably, the initial mileage difference in S1 can also be directly obtained by a manual map recognition method.

[0036] Preferably, in S2, the effects of inaccurate image starting position, mismatch of left and right edges, and missing left and right edge features on the left and right edge matching results are taken into consideration, and the image is widened by 500 columns on each side. However, when the image registration results are finally stored, only the registration results of the middle 5000 columns are stored.

[0037] Preferably, in step 2, if the maximum value of the SSIM value is greater than a given threshold, it is considered that there is no change in the window. When the texture in some areas is too simple, a small pixel disturbance may cause a sharp change in the SSIM value. Therefore, if the maximum value is less than the threshold, the texture complexity of the area is evaluated. If the texture complexity is too low, it is considered that there may be a false detection. Otherwise, the coordinate value of the pixel is saved.

[0038] Preferably, the marking of regional properties in step 3 includes marking of newly emerged diseases, marking of changes in existing diseases, updating of facilities and equipment, and removal of false positives.

[0039] 3. Beneficial effects

[0040] Compared with the prior art, the advantages of the present invention are:

[0041] (1) In the present invention, by using multi-phase image data of the same area for comparative analysis, pixel-level detection of tunnel images can be achieved, and small changes in the tunnel can be sensitively identified, and new small diseases can be discovered in time with a low missed detection rate. The development and changes of existing diseases can be continuously monitored, and the equipment and facilities in the tunnel can be updated. In addition, the system has good consistency for tunnels in different situations, and the matching of two phases of data can be achieved without the need for precise mileage calibration.

[0042] (2) In the present invention, image registration is performed on the two acquired images so that the two images have a consistent reference frame. Then, the SSIM index and threshold are used to judge the change of each pixel and acquire the changed area. Finally, the changed area is manually labeled and reviewed to achieve timely monitoring of long-term changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A physical picture of the linear array image acquisition system proposed in the present invention;

[0044] Figure 2 A flowchart of multi-phase comparison of linear array images proposed in the present invention;

[0045] Figure 3 The present invention proposes two phases of collecting original images of the same position;

[0046] Figure 4 The triangulated network construction result diagram proposed in the present invention;

[0047] Figure 5 The result diagram of the changed area of ​​the solution proposed in the present invention. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0049] Embodiment 1:

[0050] Reference Figure 1-5 The method for monitoring multi-period changes of tunnels based on linear array image data comprises the following steps:

[0051] Step 1: Image registration. Use image registration to match and superimpose the images collected in the two phases to eliminate the displacement, rotation, scaling, bending and other changes of the image points caused by complex factors such as sensor orientation changes, lens distortion, terrain undulations, uneven speed and irregular shaking of the rail vehicle during operation, so that the two phases of images have a consistent reference frame. Specifically, the following steps are included:

[0052] S1: Initial mileage alignment. The images collected in the two phases have inconsistent initial mileages, and there may be large differences. It is known that each column of the image stores the image data of a section of the tunnel. For this reason, for the two phases of images, select the data of the first x columns (x is determined by the actual situation), use feature matching to calculate a set of same-name points, and calculate the average difference of the same-name points in the row direction as the initial mileage difference. This difference can also be directly obtained by manual image recognition.

[0053] S2: Feature matching, image registration is based on the feature matching results of the two images. The initial matching column is determined for the first image, and a fixed value of 5000 columns is used as the range of a single match; the initial column corresponding to the first image is determined for the second image, and the search area is expanded to a certain extent based on the corresponding column as the range of a single match; considering the impact of factors such as inaccurate image starting position, mismatch of left and right edges, and missing left and right edge features on the left and right edge matching results, the left and right images are widened by 500 columns, but only the registration results of the middle 5000 columns are stored when the image registration results are finally stored. After each registration is completed, the registration results are used to update the corresponding initial columns of the two images. The SURF operator is used to calculate the key points and descriptors of the image, and then the BF (Brute-Force) violent feature matching method is used for feature matching, and the results are filtered by the threshold method to delete the mismatched points with too large difference values ​​and average difference values ​​in the X and Y directions.

[0054] S3: Construction of pixel coordinate transformation relationship. Linear array images have specific geometric parameters when imaging each column. The geometric relationship is too complicated. Therefore, a polynomial model and a local area correction model (local triangulation) that avoids the spatial geometric relationship of imaging are used.

[0055] The polynomial model constructs a linear polynomial with multiple parameters, which directly simulates the image deformation itself mathematically, and regards the overall deformation of the image as the combined result of various basic deformations. Usually the highest order is no more than 3. The effect of the polynomial model is weaker than that of the triangulated network model.

[0056]

[0057] The triangulated network model uses the feature matching point set to construct a triangulated network, finds the triangle corresponding to each pixel coordinate of the target image, and uses the triangle barycentric coordinate interpolation method to calculate the corresponding original x and y coordinates and resample. In order to prevent some areas from having no corresponding triangles, the four endpoints of the image use the polynomial model to calculate the corresponding coordinates and add them to the point set when constructing the triangulated network. The interpolation formula is shown in Formula 1.2, where the coordinates of the three vertices of the triangle are (x A ,y A )、(x B ,y B )、(x C ,y C ), and their corresponding values ​​are V A 、V B 、V C .

[0058]

[0059] Using the transformation relationship model, starting from the output image, that is, the blank image array, the position of each pixel in the original image is calculated in turn, and then the pixel value of the original image at that position is calculated, and then the calculated pixel value is assigned to the output image pixel. This method ensures that the pixels of the corrected image are evenly distributed in space, and grayscale resampling is required. The polynomial obtained by the polynomial model is substituted into the end column of this registration to obtain the initial column corresponding to the next registration.

[0060] S4: Histogram regulation. The two images after reconstruction achieve consistency in pixel coordinates, but there is a significant brightness difference between the two images. When collecting the same area, the brightness difference can be greatly improved by histogram regulation. In specific implementation, the image with high average brightness is used as the template;

[0061] Step 2: Image change detection. Due to the influence of factors such as registration accuracy and resampling, direct comparison of registered images will produce a large number of contrast error pixels, which is mainly reflected in the detection of changes in unchanged areas. Using the Gaussian downsampling method to downsample the image can improve this problem. Directly subtracting the image will produce a large number of contrast error pixels, and the effect is poor. Here, structural similarity (SSIM) is used as the comparison indicator.

[0062] SSIM is an indicator to measure the similarity of two images. It compares the brightness, contrast, and structure of the two images, and then weights these three factors and expresses them as a product. Suppose the two images are x and y, respectively, the brightness contrast function I(x, y), the contrast contrast function c(x, y), and the structure contrast function s(x, y) are shown in the formula:

[0063]

[0064] The constants C1, C2, and C3 are used to avoid instability problems when the denominator is 0. In particular, C1 = (K1L) 2 , C2=(K2L) 2 , L is the grayscale level of the image, for 8-bit grayscale image, L = 255. And K1 << 1, K2 << 1.

[0065] μ x , μ y is the mean, σ x , σ y is the variance, σ xy is the covariance:

[0066]

[0067] The general formula of SSIM is:

[0068] SSIM(x,y)=[I(x,y)] α [c(x,y)] β [s(x,y)] γ (2.3)

[0069] Take α, β, γ = 1, C3 = C2 / 2, and get the simplified SSIM formula:

[0070]

[0071] Usually, the default K1 = 0.01, K2 = 0.03. There is a Gaussian weighted SSIM, which multiplies the weights when calculating the mean, variance, and covariance:

[0072]

[0073] The value range of SSIM is [-1, 1], and SSIM(x, y) = 1 if and only if x = y. Set the window size of Gaussian weights, and for each pixel on image 1, open a retrieval window at the corresponding position of image 2, and calculate the maximum value of the SSIM value based on the Gaussian weights for the corresponding points in the retrieval window. If the maximum value is greater than the given threshold, it is considered that there is no change in the window. When the texture in some areas is too simple, a small pixel disturbance may cause a sharp change in the SSIM value. Therefore, if the maximum value is less than the threshold, the texture complexity of the area is evaluated. If the texture complexity is too low, it is considered that there may be a false detection. Otherwise, the coordinate value of the pixel is saved. After obtaining the set of changed pixels, use opening and closing operations to remove noise and fill holes, and finally obtain the outline of the changed area;

[0074] Step 3: Labeling of the changed area. After obtaining the changed area, the nature of the changed area is manually labeled, including the labeling of new diseases, the labeling of changes in existing diseases, the updating of facilities and equipment, and the removal of false detections. The labeling results are stored and used as the original data for the next change detection to achieve timely monitoring of long-term changes.

[0075] In the present invention, by using the multi-period image data of the same area for comparative analysis, the pixel-level detection of tunnel images can be realized, the slight changes of the tunnel can be sensitively identified, and the newly appeared slight diseases can be discovered in time with a low missed detection rate. The development and changes of existing diseases can be continuously monitored, and the equipment and facilities in the tunnel can be updated, and the tunnels in different situations have good consistency. The matching of the two-period data can be realized without the need for precise mileage calibration.

[0076] In the present invention, image registration is performed on the two acquired images so that the two images have a consistent reference frame. Subsequently, the SSIM index and threshold are used to judge the change of each pixel and acquire the changed area. Finally, the changed area is manually labeled and reviewed to achieve timely monitoring of long-term changes.

[0077] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A tunnel multi-period change monitoring method based on linear array image data, characterized in that: The following steps are involved: Step 1: Image registration: Use image registration to match the images collected in the two phases to eliminate the displacement, rotation, scaling, bending and other changes of the image points caused by complex factors such as sensor orientation change, lens distortion, terrain undulation, uneven speed and irregular shaking of the rail vehicle during operation, so that the two phases of images have a consistent reference frame; Step 2: Image change detection, using structural similarity (SSIM) as a comparison index. SSIM is an index to measure the similarity between two images. It compares the brightness, contrast, and structure of the two images, and then weights these three factors and expresses them as products. Suppose the two images are x and y, respectively. The brightness contrast function I(x, y), contrast contrast function c(x, y), and structure contrast function s(x, y) are shown in the formula: The constants C1, C2, and C3 are used to avoid instability when the denominator is 0. In particular, C1 = (K1L) 2 , C2=(K2L) 2 , L is the gray level of the image. For an 8-bit gray image, L = 255, and K1 < < 1, K2 < < 1, μ x , μ y is the mean, σ x , σ y , is the variance, σ xy is the covariance: The general formula of SSIM is: SSIM(x,y)=[I(x,y)] α [c(x,y)] β [s(x,y)] γ (2.3) Take α, β, γ = 1, C3 = C2 / 2, and get the simplified SSIM formula: Normally, the default values ​​are K1=0.01 and K2=0.

03. There is a Gaussian weighted SSIM, which multiplies the weights when calculating the mean, variance, and covariance: The range of SSIM value is [-1, 1], SSIM(x, y) = 1 if and only if x = y. Set the window size of Gaussian weights, open a search window at the corresponding position of image 2 for each pixel on image 1, calculate the maximum value of the SSIM value based on Gaussian weights of the corresponding points in the search window, obtain the set of changed pixels, use opening and closing operations to remove noise and fill holes, and finally obtain the outline of the changed area; Step 3: Labeling of the changed area. After obtaining the changed area, manually label the properties of the changed area, store the labeling results, and use them as the original data for the next change detection to achieve timely monitoring of long-term changes.

2. The method for monitoring multi-period changes in tunnels based on linear array image data according to claim 1, characterized in that: The image registration specifically comprises the following steps: S1: Initial mileage alignment. The images collected in the two phases have inconsistent initial mileages, which may be quite different. It is known that each column of the image stores the image data of a section of the tunnel. Therefore, for the two phases of images, the data of the first x columns are selected, where x is determined by the actual situation, and feature matching is used for calculation; S2: Feature matching. Image registration is based on the feature matching results of the two images. The initial matching columns are determined for the first image, and a fixed value of 5000 columns is used as the range of a single match. The initial columns corresponding to the first image are determined for the second image, and the search area is expanded to a certain extent based on the corresponding columns as the range of a single match. After each registration is completed, the registration results are used to update the corresponding initial columns of the two images, and the SURF operator is used to calculate the key points and descriptors of the images. Then, the Brute-Force feature matching method is used for feature matching, and the results are filtered using the threshold method to delete the mismatched points with too large difference values ​​and average difference values ​​in the X and Y directions. S3: Construction of pixel coordinate transformation relationship. Linear array imaging has specific geometric parameters when imaging each column. The geometric relationship is too complicated. Therefore, a polynomial model and a local area correction model that avoid the spatial geometric relationship of imaging are used. The polynomial model constructs a linear polynomial with multiple parameters, which directly simulates the image deformation itself mathematically, and regards the overall deformation of the image as the combined result of various basic deformations. Usually, the highest order is no more than 3. The effect of the polynomial model is weaker than that of the triangulated network model. The triangulated network model uses feature matching point sets to construct a triangulated network, finds the triangle corresponding to each pixel coordinate of the target image, and uses the triangle barycentric coordinate interpolation method to calculate the corresponding original x and y coordinates and resample them. In order to prevent some areas from having no corresponding triangles, the four endpoints of the image use the polynomial model to calculate the corresponding coordinates and add them to the point set when constructing the triangulated network. The interpolation formula is shown in Formula 1.2, where the coordinates of the three vertices of the triangle are (x A ,y A )、(x B ,y B )、(x c ,y C ), and their corresponding values ​​are V A 、V B 、V C , Using the transformation relationship model, starting from the output image, that is, the blank image array, the position of each pixel in the original image is calculated in turn, and then the pixel value of the original image at that position is calculated, and then the calculated pixel value is assigned to the output image pixel. This method ensures that the pixels of the corrected image are evenly distributed in space. Grayscale resampling is required. The polynomial obtained by the polynomial model is used to bring the end column of this registration into the initial column corresponding to the next registration. S4: Histogram normalization. The two images after reconstruction achieve consistency in pixel coordinates, but there is an obvious brightness difference between the two images. When collecting data in the same area, the brightness difference can be greatly improved by histogram normalization. In specific implementation, the image with large average brightness is used as a template.

3. The method for monitoring multi-period changes in tunnels based on linear array image data according to claim 2, characterized in that: The initial mileage difference in S1 may also be directly obtained by manual map recognition.

4. The method for monitoring multi-period changes in tunnels based on linear array image data according to claim 2, characterized in that: In S2, the effects of inaccurate image starting position, mismatch of left and right edges, and missing left and right edge features on the left and right edge matching results are taken into consideration, and the left and right sides of the image are widened by 500 columns respectively. However, when the image registration results are finally stored, only the registration results of the middle 5000 columns are stored.

5. The method for monitoring multi-period changes in tunnels based on linear array image data according to claim 1, characterized in that: In step 2, if the maximum value of the SSIM value is greater than a given threshold, it is considered that there is no change in the window. When the texture in some areas is too simple, a small pixel disturbance may cause a sharp change in the SSIM value. Therefore, if the maximum value is less than the threshold, the texture complexity of the area is evaluated. If the texture complexity is too low, it is considered that there may be a false detection. Otherwise, the coordinate value of the pixel is saved.

6. The method for monitoring multi-period changes in tunnels based on linear array image data according to claim 1, characterized in that: The marking of regional properties in step 3 includes marking of newly emerged diseases, marking of changes in existing diseases, updating of facilities and equipment, and removal of false positives.

Images