Data registration calibration method based on tunnel laser scanning and high-definition images

Abstract

The data registration calibration method based on tunnel laser scanning and a high-definition image comprises the following steps: performing expansion projection on laser point cloud data according to a section contour line to obtain a laser image; obtaining a high-definition image feature map through the original visible light image map so as to obtain a high-definition visible light image spliced map; performing image registration on the laser image and high-definition visible light image spliced image to obtain a laser image and high-definition visible light image joint data set Union (Li, Ix laser, Iy laser, Pxj, Pyj, Refij, Sij, theta ij, Ix high definition, Iy high definition, Rij, Gij and Bij); the actual magnification Scij of the object is obtained on the basis of the joint data set, and a high-definition image accurate data set I (Li, Ix laser, Iy laser, Pxj, Pyj, Refij, Sij, theta ij, Ix high definition, Iy high definition, Rij, Gij, Bij and Scij) is obtained, so that a high-definition visible light image is obtained; the measurement precision of the disease characteristics in the high-definition visible light image is improved, and the actual state of the tunnel disease can be accurately mastered.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a data registration and calibration method based on tunnel laser scanning and high-definition images. Background Art

[0002] During the operation of urban rail transit tunnels, cracks, dislocation, water seepage, and structural deformation pose threats to the safety of the tunnel structure. Therefore, tunnel disease detection and maintenance work are very important for the safe operation of rail transit. In recent years, with the rapid development of automated detection technologies, mainly including laser scanning technology and photogrammetry technology, disease detection based on visible light cameras and lidar scanning equipment has become the main direction of tunnel disease detection, and many countries have developed tunnel detection systems based on visible light cameras and lidar scanning.

[0003] The tunnel detection system obtains two types of data: laser scanning point cloud data and visible light detection images. The laser scanning data has coordinate information and good measurability. However, due to technical principle limitations, the resolution of the image converted from the laser scanning point cloud is relatively low. The visible light camera image has the advantages of high clarity and rich texture information and can be used to identify tiny crack diseases. However, since the distance from the camera lens to the tunnel wall points is constantly changing due to the influence of the change in the tunnel cross-section contour shape during the tunnel shooting process, the actual magnification corresponding to each pixel point of the obtained image depends on the actual distance from the camera to the tunnel wall points. Therefore, the actual size of the disease cannot be accurately measured on the visible light image. Summary of the Invention

[0004] In view of this, aiming at the technical problem that the actual size of the disease cannot be accurately measured in the above-mentioned existing tunnel detection system, this application provides a data registration and calibration method based on tunnel laser scanning and high-definition visible light images, which calculates the actual magnification of the visible light image by using the actual distance from the linear array camera obtained by the laser cross-section scanning radar to the tunnel structure, and then accurately calculates the physical resolution of each pixel point, improves the measurement accuracy of image features with high accuracy requirements such as crack width, so as to obtain a more accurate size of the disease and more accurately and effectively control the current situation and change trend of tunnel diseases.

[0005] This application provides a data registration and calibration method based on tunnel laser scanning and high-definition images, and the method includes: Unfolding and projecting the laser point cloud data along the cross-section contour line to obtain a laser image; Obtaining a high-definition image feature map from the original visible light image to obtain a high-definition visible light image mosaic; Performing image registration on the laser image and the high-definition visible light image mosaic to obtain a joint dataset Union(L i, I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij , θ ij , I x高清 , I y高清 , R ij , G ij , B ij ); Obtain the actual magnification Sc of the object based on the combined dataset ij , and obtain the high-definition image precise dataset I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij , θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij ), to obtain a high-definition visible light image; Among them, L i is the tunnel mileage of the section point Point numbered (i, j) ij , (I x激光 , I y激光 ) are the row and column coordinates in the laser image calculated by projecting and designing according to the section contour line of this section point, (I x高清 , I y高清 ) are the row and column coordinates of this section point in the high-definition visible laser image mosaic, (P xj , P yj ) are the section coordinate system coordinates of the laser point cloud of this section point, △S ij is the section axis diameter difference, θ ij is the central angle, Ref ij is the laser reflection intensity value, (R ij , G ij , B ij ) are the RGB values of the high-definition visible light image of this section point, Sc ij is the actual magnification of the image of this section point, i is the section number obtained by the section scanning radar, and j is the section point number in each section.

[0006] Compared with the prior art, in the data registration and calibration method based on tunnel laser scanning and high-definition visible light images of the present application, images with different resolutions are obtained by two different means respectively, the main features on the two images are image-registered, and then two kinds of pixel correspondence relationships are obtained, and a registered joint data set is obtained. According to the joint data set, the visible light image is magnified by the lens magnification factor calculated based on the actual distance from each camera lens to the tunnel wall, so as to obtain a high-definition visible light image that is more accurate than before calibration. Furthermore, the measurement accuracy of disease characteristics can be further improved, especially for small sizes such as crack widths, which is convenient for staff to more effectively control the actual state and development trend of tunnel diseases and reasonably evaluate the safety status of tunnel structures.

[0007] Preferably, a laser radar is carried by an orbital inspection vehicle to scan the tunnel cross-section and obtain tunnel three-dimensional point cloud data (L i , P xj , P yj , Ref ij ).

[0008] Preferably, based on the point cloud data, the axial diameter difference △S i and the central angle θ ij are calculated according to the designed cross-section contour line for the cross-section point cloud with a cross-section mileage of L ij ; According to the cross-section contour line unfolding projection principle, the coordinates in the cross-section coordinate system are calculated to obtain the cross-section point cloud data set D 点云 (L i , P xj , P yj , Ref ij , △S ij , θ ij ); Among them, .

[0009] Preferably, according to the cross-section contour line unfolding projection principle, each cross-section point cloud is correspondingly converted into the laser image coordinate system to obtain the laser image coordinate system coordinates I 激光坐标 (I x , I y ); According to the value range mapping method, the laser reflection intensity value Ref ij is converted into a 0-255 level image gray value Grey 反射强度 , and based on I 激光坐标 (I x , I y ), the three-dimensional laser point cloud gray image unfolding diagram data set I 激 (I x , I y , Grey 反射强度, Ref ij , △S ij , θ ij ); Among them, I x and I y are respectively the horizontal and vertical coordinates of each pixel on the laser image; the absolute value of I y is equal to the unfolded length L j of the cross-sectional contour line from each point in the cross-sectional point cloud to the cross-sectional zero point, corresponding to the number of pixels; the cross-sectional zero point is the point in the cross-sectional point cloud where the abscissa P x of the cross-sectional coordinate system is 0 and P y > 0, I x = L i / physical resolution of column pixels, I y = -L j / physical resolution of row pixels.

[0010] Preferably, set the camera according to the sectional dimension of the tunnel design, collect and store the pixel line ln of a single frame of image to obtain the original image captured by the line array camera; Obtain the original images acquired by multiple line array cameras on the same cross-section collected by the track inspection vehicle; Use a feature extractor to detect and extract the feature mask of the bolt hole area in the high-definition original image, overlay the detection result on the original image, and set this area to blue to obtain the high-definition image feature map I 高清特征 (L i , I x高清 , I y高清 , R 特 , G 特 , B 特 ); Among them, the Mask R-CNN algorithm of the feature extractor uses the pre-trained convolutional neural network ResNet-50.

[0011] Preferably, according to the measured length of the bolt hole width, the measured length of the bolt hole height and the designed length in the high-definition image feature map I 高清特征 , calculate the scaling ratios Sc 宽 and Sc 高 of the width and height of the original image of the line array camera; Sc 宽 = measured width of the bolt hole in the high-definition feature map / designed width; Sc 高 = measured height of the bolt hole in the high-definition feature map / designed height; Perform image registration on the bolt hole features of multiple high-definition original images of the same cross-section corrected based on the scale to obtain an inter-ring mosaic image with a complete tunnel cross-section; Perform image stitching of the spliced images between rings in the mileage direction to obtain the dataset I of the high-definition image stitching of the linear array camera for the entire tunnel. 高清拼接 (I x , I y , R ij , G ij , B ij ).

[0012] Preferably, according to the value range mapping method, through the I 激 (I x , I y , Grey 反射强度 , Ref ij , △S ij , θ ij ) obtain the laser point cloud axial diameter difference depth map I 深 (I x , I y , Grey 轴径差 , R D , G D , B D ); Among them, the cross-sectional axial diameter difference △S ij is mapped from small to large to 0-255 levels of gray scale Grey 轴径差 , (R D , G D , B D ) is the cross-sectional axial diameter difference of the point on this cross-section expressed in RGB color value mode: the pixels with △S ij > 0.01 are set to blue (0, 0, 255), and the pixels with △S ij < -0.01 are set to yellow (255, 255, 0).

[0013] Preferably, align the initial positions of the tunnel mileage field L i in the laser image and the high-definition image dataset during registration; Through the image registration algorithm of SIFT, use the marked bolt hole areas in the laser point cloud axial diameter difference depth map I 深 and I 高清特征 as features for automatic image registration to obtain the joint dataset Union (L 高清拼接 , I i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij , θ ij , I x高清 , I y高清 , R ij, G ij , B ij ); Through this combined dataset, to obtain the linked query display, feature annotation, and dimension measurement calculation of the laser image and the high-definition image.

[0014] Preferably, according to the relative positional relationship between the cross-section scanning radar and the line array camera, obtain the cross-section coordinate system rotation angle φk of the center (Xc k , Yc k , Zc k ) of each line array camera and the center of the scanning radar (k = 1, 2,..., n is the number of the line array camera); Through the cross-section coordinates P ij (L i , P xj , P yj ) of the tunnel scanning laser point cloud, calculate the cross-section coordinate system coordinates P k ij (L i , P xj , P yj ) of the line array camera, and obtain the distance Dc from the tunnel cross-section point to the corresponding camera optical center ij ; P k xj = P xj cos∝ k - P yj sin∝ k + △x k ; P k yj = P xj sin∝ k - P yj cos∝ k + △y k ; ; According to the camera center distance Dc of each tunnel cross-section point ij , obtain the precise magnification Sc of the corresponding visible light image pixel points ij , and the high-definition image precise dataset I with the point-by-point precise magnification 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij , θ ij , I x高清 , I y高清 , R ij , Gij , B ij , Sc ij ); Among them, the object magnification Sc ij= Dc ij / f, where f is the camera focal length.

[0015] Preferably, the length, width, and area of the object in the visible light image are obtained through the high-definition image precise imaging data set; Among them, The area of the object feature = the sum of the areas of all pixels covered by the object feature; The area of a single pixel = the pixel size of the camera's photosensitive unit × magnification Sc ij ; The length of the object feature = the feature area ÷ representative width; The width of the object feature = the feature area ÷ representative length. Description of the Drawings

[0016] Figure 1 is the three-dimensional point cloud data obtained by three-dimensional laser scanning of a tunnel provided by an embodiment of the present application; Figure 2 is the schematic diagram of the unfolded projection of the cross-sectional contour line of the three-dimensional point cloud obtained by laser scanning provided by an embodiment of the present application; Figure 3 is the unfolded diagram of the cross-sectional contour of the laser-scanned point cloud provided by an embodiment of the present application; Figure 4 is the original high-definition tunnel image obtained by 9 linear array cameras provided by an embodiment of the present application; Figure 5 is the schematic diagram of the circumferential seam and intersection points automatically identified on the high-definition image provided by an embodiment of the present application; Figure 6 is the high-definition image feature map of the automatically extracted bolt hole area provided by an embodiment of the present application; Figure 7 is the schematic diagram of the scaling ratio correction of the original high-definition image provided by an embodiment of the present application; Figure 8 is the high-definition image after ratio correction and stitching provided by an embodiment of the present application; Figure 9 is an example diagram of the depth map of the laser-scanned point cloud image provided by an embodiment of the present application; Figure 10 is the high-definition image in the linkage query and measurement interface diagram provided by an embodiment of the present application; Figure 11 is Figure 10 the enlarged schematic diagram of the local A in Figure 12 isFigure 10 Schematic diagram of partial enlargement of B in Figure 13 is the three-dimensional laser point cloud map in the linkage query and measurement interface diagram provided by an embodiment of the present application; Figure 14 is the two-dimensional point cloud cross-section diagram in the linkage query and measurement interface diagram provided by an embodiment of the present application; Figure 15 is the schematic diagram of the coordinate system conversion principle of lidar and linear array camera provided by an embodiment of the present application; Figure 16 is the high-definition linear array image of the crack width standard card for inspecting the crack width provided by an embodiment of the present application; Figure 17 is the schematic diagram for calculating the scaling error effect provided by an embodiment of the present application. Detailed implementation manners

[0017] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the present disclosure will be described in detail, clearly and completely below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0018] In the description of the present application, if the first and the second are described only for the purpose of distinguishing technical features, they should not be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence of the indicated technical features.

[0019] Those skilled in the art should understand that in the disclosure of the present application, the orientation or positional relationships indicated by the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element indicated must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting the present application.

[0020] The following will further describe the present application in detail with reference to the accompanying drawings. See, for example, Figures 1 to 17 description.

[0021] The present application provides a data registration and calibration method based on tunnel laser scanning and high-definition images. By accurately calculating the physical resolution of each pixel point, it improves the measurement accuracy of disease characteristics such as crack width for small dimensions, so as to obtain high-definition images of the actual sizes of diseases in the tunnel structure, helping the staff to reasonably and accurately evaluate the safety status of the tunnel structure.

[0022] Specifically, the method includes the following specific steps: Step S1: Unfold and project the laser point cloud data along the cross-section contour line to obtain a laser image; Step S2: Obtain a high-definition image feature map from the original visible light image map to obtain a high-definition visible light image mosaic; Step S3: Perform image registration on the laser image and the high-definition visible light image mosaic to obtain a joint dataset Union (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij , θ ij , I x高清 , I y高清 , R ij , G ij , B ij ); Step S4: Obtain the actual magnification Sc of the object based on the joint dataset ij , and obtain a high-definition image precise dataset I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij , θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij ), to obtain a high-definition visible light image; Wherein, L i is the tunnel mileage of the cross-section point Point ij numbered (i, j), (I x激光 , I y激光 ) are the row and column coordinates in the laser image calculated by unfolding and projecting the cross-section point along the cross-section contour line, (I x高清 , I y高清 ) are the row and column coordinates of the cross-section point in the high-definition visible laser image mosaic, (P xj , P yj ) are the cross-section coordinate system coordinates of the laser point cloud of the cross-section point, △S ij is the cross-section axis diameter difference, Ref ij is the laser reflection intensity value, (R ij , G ij , B ij) are the RGB values of the high-definition visible light image of the cross-section point, Sc ij is the actual magnification of the image of the cross-section point, i is the cross-section number obtained by the cross-section scanning radar, and j is the cross-section point number in each cross-section.

[0023] In this embodiment, as Figure 2 、 Figure 3 shown, the laser point cloud data is obtained by scanning with a lidar in the tunnel, and after cross-section projection processing, a laser point cloud image development diagram (abbreviation: laser image) is obtained; as Figures 4 to 8 shown, the original image is obtained by shooting with a line array camera, and after splicing and size correction processing, a high-definition visible light image splicing diagram is obtained; as Figure 6 、 Figure 9 shown, then the laser image and the high-definition image splicing diagram are registered, so as to obtain a combined dataset of the laser image and the high-definition image, and after processing this dataset, the actual magnification of each object is obtained, so that a high-definition visible light image with higher measurement size accuracy can be directly obtained, so as to assist the staff to accurately draw and measure the accurate size of the detection target on the high-definition image. As Figures 10 to 14 shown, this combined dataset can support the linked query and accurate measurement of the laser point cloud and the high-definition image, making full use of the high-precision three-dimensional size characteristics of the laser point cloud and the high-clarity and high-restoration texture characteristics of the high-definition camera image.

[0024] On the basis of the above embodiment, step S1 is further expanded; among them, step S1 includes step S11, step S12 and step S13.

[0025] Specifically, step S11: as Figure 1 shown, a lidar is carried by an orbital inspection vehicle to scan and obtain the original data of the tunnel three-dimensional point cloud, and then the tunnel three-dimensional point cloud data (L i , P xj , P yj , Ref ij ) is obtained.

[0026] In this step, as Figure 15 shown, a laser scanning radar and n line array cameras are carried on the orbital inspection vehicle. The laser scanning radar is installed at the center of the base of the inspection vehicle, and the line array cameras are arranged in a ring on the base of the inspection vehicle. The installation plane of the cameras is parallel to the laser scanning cross-section; when the orbital inspection vehicle moves forward, while recording the traveling mileage, the laser scanning radar scans the tunnel cross-section, and the three-dimensional laser hits the inner wall of the tunnel and is reflected and then collected by the scanning radar, so as to obtain the tunnel mileage Li of the cross-section point and the coordinates (P xj , P yj ) of the cross-section coordinate system of the cross-section point and the laser reflection intensity value Ref ij, to assist in establishing a point cloud database. With such a structure, it is possible to make the pixel columns obtained by the line array camera and the cross-sectional point cloud obtained by the lidar lie on the same plane.

[0027] Among them, as Figure 17 shown, rollers are provided at the bottom of the base of the track inspection vehicle to facilitate the movement of the inspection vehicle; the line array cameras are distributed in an arc shape according to the Figure 17 shown method, and the scanning cross-section of the lidar should be parallel to the installation plane of the line array cameras, and there should be an overlapping field of view of not less than 20% between adjacent cameras, so as to improve the splicing and registration accuracy of the laser image after lidar scanning and the high-definition image obtained by the line array camera shooting. Further, a light source sheet distributed in an arc shape is provided outside the line array camera. The light source sheet is arranged in two rows, and the line array camera is located between the two rows of light source sheets to provide a better light source environment for the line array camera, improve the shooting illumination and image clarity; the inner diameter of the arc formed by the distribution of the light source sheet is larger than the inner diameter of the arc formed by the distribution of the line array camera, that is, the line array camera and the light source sheet are spaced apart.

[0028] It should be noted that in this application, the sectional laser scanner can scan 100 circles per second along the direction perpendicular to the platform movement direction, and the cross-sectional point cloud refers to the cross-sectional points of this circle collected by the laser scanner in 0.01 seconds.

[0029] In this application, the high-definition image refers to an image with a physical resolution of 0.2 mm per pixel, and this level of resolution can be used to identify crack diseases with a width of 0.2 mm.

[0030] After step S11, it further includes step S12: Based on the point cloud data, obtain the axial diameter difference △S i and the central angle θ ij calculated by expanding the cross-sectional point cloud with a cross-sectional mileage of L ij along the designed cross-sectional contour line; According to the cross-sectional contour line expansion projection principle, calculate the coordinates in the cross-sectional coordinate system to obtain the two-dimensional cross-sectional point cloud dataset D Figure 14 as shown in 点云 (L i , P xj , P yj , Ref ij , △S ij , θ ij ); Taking a circular tunnel with a cross-sectional radius of R as an example, .

[0031] In this embodiment, △S ijis the difference in shaft diameters of the cross-section. Its physical meaning is the difference between the actual distance from a cross-section point to the center point of the fitted cross-section and the theoretical distance from the cross-section point at that position to the cross-section center in the theoretical cross-section contour map. For example, if the theoretical cross-section is a circle with a radius of R, then ; θ ij is the angle between the cross-section point and the origin of the cross-section coordinate system; L is the cross-section mileage, i is the cross-section number obtained by the cross-section scanning radar, and j is the cross-section point number in each cross-section.

[0032] After step S12, it further includes step S13: According to the principle of unfolded projection of the cross-section contour line, each cross-section point cloud is correspondingly transformed into the simulated laser image coordinate system to obtain the laser image coordinate system coordinates I 激光坐标 (I x , I y ); Furthermore, according to the value range mapping method, the laser reflection intensity value Ref ij is converted into a 0-255 level image gray value Grey 反射强度 , and based on I 激光坐标 (I x , I y ), a three-dimensional laser point cloud gray image unfolded map dataset I 激 (I x , I y , Grey 反射强度 , Ref ij , △S ij , θ ij ) is obtained, and then the laser image in step S1 is obtained.

[0033] Among them, I x and I y are the horizontal and vertical coordinates of each pixel on the laser image map respectively; the absolute value of I y is equal to the number of pixels corresponding to the unfolded length L j from each point in the cross-section point cloud to the cross-section zero point; the cross-section zero point is the point in the cross-section point cloud where the cross-section coordinate system abscissa P x = 0 and P y > 0, I x = L i / the physical resolution of column pixels, I y = -L j / the physical resolution of row pixels.

[0034] In this embodiment, the definitions of steps i and j are the same as those in step S12.

[0035] On the basis of any of the above embodiments, step S2 is further expanded; such as Figure 3 、 Figures 10 to 17As shown, in step S2, it specifically includes step S21 and step S22.

[0036] Specifically, step S21: According to the sectional dimension of the tunnel design, set the pixel line ln of the single-frame image collected and stored by the camera to obtain the original image captured by the line array camera; as Figure 4 shown, these are 9 (9a~9i) original color high-definition images of the shield tunnel with ln = 5000 lines obtained by 9 cameras of this application. Among them, the original images are rotated 90 degrees clockwise; the height of the image is the advancing direction of the acquisition platform, so the height is ln, and the width of the image is the nominal resolution of the line array camera. In this application, it is exemplified as 8096 pixels obtained by a nominal resolution 8K camera.

[0037] Furthermore, obtain the original images acquired by multiple line array cameras at the same section collected by the track inspection vehicle; among them, use the Canny edge detection algorithm to detect all linear features, and use the Hough line detection algorithm in the linear feature results to detect the vertical pipe ring vertical seams and the inner-ring joints with a specific designed angle to the vertical seams in the advancing direction of the tunnel, and obtain the intersection points of the two types of joints as shown in Figure 5 . The red ones are the intersection points of the inter-ring joints, and the yellow ones are the intersection points of each block within the ring.

[0038] Among them, using the Canny edge detection algorithm is more suitable for extracting linear features such as tunnel ring seams under the interference of complex background noise, can detect the edges of ring seams with different thicknesses, and connect the disconnected edges through double thresholds, which is more suitable for extracting tunnel ring seam features. The Hough line detection algorithm can extract linear features within a specific angle range, and is especially suitable for the extraction of vertical seams and inner-ring joints here.

[0039] In this application, in accordance with the sectional dimension of the tunnel design, set the cameras according to the angle range of the tunnel section to be photographed and the field of view angle of each camera, so that the overlapping field of view angles of adjacent cameras are not less than 5°, and the field of view range composed of all cameras can cover the angle range of the tunnel section to be photographed.

[0040] Furthermore, use a feature extractor to detect and extract the feature mask of the bolt hole area in the high-definition original image, overlay the detection result on the original image, and set this area to blue (0, 0, 255) to obtain the high-definition image feature map I as shown in Figure 6 . 高清特征 (L i ,I x高清 ,I y高清 ,R 特 ,G 特 ,B 特 ); Among them, in the Mask R-CNN algorithm of the feature extractor, ResNet-50 is used as the backbone network to extract multi-level features of the input image, and then the features are used for subsequent tasks such as the Region Proposal Network (RPN) and mask generation. The specific process is as follows: 1) Image preprocessing, resizing to a fixed size and batch normalization; 2) Different from ResNet34, ResNet-50 is a deep convolutional neural network that downsamples the input image through convolutional and pooling operations to extract semantic information with different resolutions and levels; 3) Through the Feature Pyramid Network (FPN), multi-scale feature extraction is enhanced, and after feature fusion (fusion of low-level and high-level features, adding the channels of feature maps at different stages), the feature map is output; 4) Each layer of the FPN is input into the RPN, and through classification (predicting the category of candidate regions, such as bolt holes) and the regression branch (adjusting the position and size of candidate regions), candidate regions (Region Proposals) are generated, and fixed-size features are extracted through bilinear interpolation (RoIAlign) operations for object detection and mask generation.

[0041] It should be noted that the Mask R-CNN algorithm based on ResNet-50 can generate pixel-level masks, accurately locate the boundaries of bolt holes, and the FPN combined with the deep convolutional network can effectively extract multi-level features; the model can batch process large-scale tunnel data with strong robustness; this algorithm integrates object detection and segmentation, unifying the multi-functional classification and positioning of bolt holes; it can continuously optimize through data-driven, that is, as the samples increase, the prediction accuracy of the model is enhanced, and the labor cost is reduced.

[0042] After step S21, it also includes step S22: As Figure 7 shown, according to the measured length W of the bolt hole width, the measured length H of the height, and the designed length in the high-definition image feature map I 高清特征 calculate the scaling ratios Sc 宽 and Sc 高 of the width and height of the original image of the line array camera; Sc 宽 = Measured width of bolt hole in high-definition feature map / Designed width; Sc 高 = Measured height of bolt hole in high-definition feature map / Designed height; Furthermore, as Figure 8 shown, perform image registration on the bolt hole features of multiple high-definition original images of the same cross-section taken by the multi-line array camera after scaling correction based on the ratio to obtainFigure 8 The inter-ring stitching image with a complete tunnel cross-section shown in (a) in 高清拼接 (I x , I y , R ij , G ij , B ij ), and thus obtain Figure 8 the high-definition visible light image stitching map of multi-ring stitching shown in (b) in

[0043] Based on any of the above embodiments, step S3 is further expanded; step S3 includes step S31 and step S32.

[0044] Specifically, step S31: According to the value range mapping method, through the I 激 (I x , I y , Grey 反射强度 , Ref ij , △S ij , θ ij ), obtain the laser point cloud axial diameter difference depth map I 深 (I x , I y , Grey 轴径差 , R D , G D , B D ); Among them, the cross-section axial diameter difference △S ij is mapped from small to large to 0~255 levels of gray scale Grey 轴径差 , and the pixels with △S ij >0.01 are set to blue (0, 0, 255), and the pixels with △S ij <-0.01 are set to yellow (255, 255, 0), so as to clearly distinguish the laser point cloud axial diameter difference depth map I 深 (I x , I y , Grey 轴径差 , R D , G D , B D ), (R D , G D , B D ) is the cross-section axial diameter difference of this cross-section expressed in RGB color numerical mode, and D in R D represents depth; specifically, it can be as shown in Figure 9 .

[0045] Further, after step S31, it further includes step S32: Align the laser image with the tunnel mileage field L in the high-definition image dataset i at the initial position in registration; Through the image registration algorithm of SIFT, for the laser point cloud axial diameter difference depth map I 深 and I 高清特征 The marked bolt hole areas in are used as features for automatic image registration to obtain the joint dataset Union (L 高清拼接 , I i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij , θ ij , I x高清 , I y高清 , R ij , G ij , B ij ) of the laser image and the high-definition image mosaic I; Through this joint dataset, to obtain the linked query display, feature annotation, and dimension measurement calculation of the laser image and the high-definition image.

[0046] It should be noted that the high-definition image corresponding to this joint dataset is an orthophoto with measurable characteristics, and the measured distance dimension is the scaled value of the real distance dimension on the tunnel surface. This image not only corresponds point by point to the three-dimensional laser scanning point cloud (L i， P xj , P yj ), but also corresponds point by point to each scanned two-dimensional point cloud section in the tunnel mileage L i direction (P xj , P yj ); As Figures 10 to 14 shown, the high-definition image expansion diagram, the three-dimensional point cloud, and the two-dimensional section diagram are linked to query and display the accurate tunnel mileage and section coordinates of each point and perform length and area measurement.

[0047] Based on any of the above embodiments, step S4 is further expanded; step S4 includes step S41 and step S42; Specifically, step S41: As Figure 15 shown, according to the relative position relationship between the section scanning radar and the line array camera, obtain the cross-section coordinate system rotation angle φk of the center (Xc k , Yc k , Zc k ) of each line array camera and the center of the scanning radar (k = 1, 2,..., n is the number of the line array camera); Obtain the cross-section coordinates P of the laser point cloud through the tunnel scan ij (L i , P xj , P yj ), and calculate the coordinates P of the line array camera cross-section coordinate system k ij (L i , P xj , P yj ), and obtain the distance Dc from the tunnel cross-section point to the corresponding camera optical center ij ; Among them, i is the cross-section number obtained by the cross-section scanning radar, and j is the cross-section point number in each cross-section; P k xj = P xj cos∝ k -P yj sin∝ k +△x k ; P k yj = P xj sin∝ k -P yj cos∝ k +△y k ; .

[0048] Furthermore, according to the camera center distance Dc of each tunnel cross-section point ij , obtain the precise magnification factor Sc of the corresponding visible light image pixel points ij , and the high-definition image precise data set I with the point-by-point precise magnification factor 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij , θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij ); Among them, the object magnification factor Sc ij = Dc ij / f, where f is the camera focal length.

[0049] Further, after step S41, step S42 is further included: obtaining the length, width dimensions and area of an object in the visible light image through the high-definition image precise imaging data set; (1) First, calculate the area of the feature based on the total number of pixels of the extracted object feature. Among them, The area of the object feature = the sum of the areas of all pixels covered by the object feature; The area of a single pixel = the pixel size of the camera's photosensitive unit × magnification Sc ij ; (2) Measure the representative size of the feature, such as the length of the crack feature; (3) Then calculate the other size parameter of the object feature based on one of the measured size parameters of the planar object feature, such as length (or width), such as width (or length); among them, The length of the object feature = the feature area ÷ representative width; The width of the object feature = the feature area ÷ representative length.

[0050] It should be noted that in step S42, when measuring an object in the image, the size corresponding to a single pixel = the camera photosensitive pixel size * magnification Sc ij , and the sum of the sizes of all measured pixels is the object size. Sticking a crack standard width comparison card (such as Figure 16 ) in each tunnel segment can perform a correction accuracy test on the magnified and corrected image, so as to accurately calculate the physical resolution of each pixel point and improve the measurement accuracy of small sizes such as crack widths.

[0051] Among them, in step S42, for the object feature size measurement method, first calculate the pixel area of the feature, and then calculate the other feature size based on one of the representative feature sizes input by the user. This method controls the measurement of the object feature through a more accurate total area, which is more conducive to accurately perceiving the change amount of small features than the traditional method of directly measuring the number of pixels to calculate the length, such as the development of crack disease features.

[0052] Taking the most common circular shield subway tunnel with an inner diameter of 5500mm as an example, as Figure 17 shown, the maximum image scaling ratio error caused by the change in the distance from the camera lens to the tunnel wall due to the "substituting arc with chord" error of the arc-shaped segment imaging in the camera plane is approximately , and the maximum area error caused thereby is approximately That is, the processing method of the present application can improve the measurement accuracy of linear features by more than 3% and the measurement accuracy of planar features by more than 6%, which has high practical significance.

[0053] It should be noted that, in the case where the embodiments of the present application do not conflict with the solutions and the technical solutions can coexist, they can be arbitrarily combined into new embodiments.

[0054] The above has introduced the present application in detail. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the present application and its core idea. It should be pointed out that for those of ordinary skill in the art in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A data registration and calibration method based on tunnel laser scanning and high-definition imaging, characterized in that: The method comprises: Expand and project the laser point cloud data according to the cross-sectional contour line to obtain the laser image; Obtain a high-definition image feature map through the original visible light image map to obtain a high-definition visible light image mosaic map; The laser image is registered with the high-definition visible light image mosaic to obtain a joint dataset Union (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij ); The actual magnification Sc of the object is obtained based on the joint data set ij , obtain high-definition image accurate data set I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij ) to obtain high-definition visible light images; Among them, L i is the cross-section point numbered (i, j) ij Tunnel mileage, (I x激光 , I y激光 ) is the row and column coordinates of the cross-section point in the laser image calculated by unfolding the cross-section contour line, (I x高清 , I y高清 ) is the row and column coordinates of the cross-section point in the high-definition visible laser image mosaic, (P xj , P yj ) is the cross-sectional coordinate system coordinate of the laser point cloud of the cross-sectional point, △S ij is the cross-section axis diameter difference, θ ij is the central angle, Ref ij is the laser reflection intensity value, (R ij , G ij , B ij ) is the RGB value of the high-definition visible light image at the cross-section point, Sc ij is the actual magnification of the image of the section point, i is the section number obtained by the section scanning radar, and j is the section point number in each section.

2. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 1 is characterized in that: The laser radar is installed on the track inspection vehicle to scan the tunnel section and obtain the tunnel 3D point cloud data (L i , P xj , P yj , Ref ij ).

3. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 2 is characterized in that: Point Cloud Based on the point cloud data, the cross-sectional mileage L is obtained. i The cross-section point cloud is calculated according to the design cross-section contour line to calculate the shaft diameter difference △S after expansion ij and the central angle θ ij ; According to the cross-section contour line expansion projection principle, the cross-section coordinate system coordinates are calculated to obtain the cross-section point cloud data set D 点云 (L i , P xj , P yj , Ref ij , △S ij ,θ ij ); in, .

4. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 3 is characterized in that: According to the projection principle of the cross-section contour line, each cross-section point cloud is converted to the laser image coordinate system to obtain the laser image coordinate system coordinate I 激光坐标 (I x , I y ); According to the range mapping method, the laser reflection intensity value Ref ij Convert to 0~255 grayscale values ​​Grey 反射强度 , based on I 激光坐标 (I x , I y )Get the 3D laser point cloud grayscale image expansion map dataset I in the tunnel 激 (I x , I y , Grey 反射强度 , Ref ij , △S ij ,θ ij ); Among them, I x and I y are the horizontal and vertical coordinates of each pixel on the laser image; I y The absolute value of is equal to the expanded length L of the section contour line from each point in the section point cloud to the zero point of the section j The corresponding number of pixels; the cross-section zero point is the horizontal coordinate P of the cross-section coordinate system in the cross-section point cloud x = 0 and P y >0, I x =L i / column pixel actual resolution, I y =﹣L j / line pixel actual resolution.

5. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 4 is characterized in that: The camera is set according to the designed segment size of the tunnel, and the pixel line ln of the single frame image is collected and stored to obtain the original image taken by the line array camera; Acquire the original images acquired by multiple linear array cameras on the same section collected by the track inspection vehicle; The feature extractor is used to detect and extract the feature mask of the bolt hole area in the high-definition original image. The detection result is superimposed on the original image and the area is set to blue to obtain the high-definition image feature map I. 高清特征 (L i ,I x高清 ,I y高清 ,R 特 ,G 特 ,B 特 ); Among them, the Mask R-CNN algorithm of the feature extractor adopts the pre-trained convolutional neural network ResNet-50.

6. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 5 is characterized in that: According to the high-definition image feature map I 高清特征 The actual measured length of the bolt hole width, the actual measured length of the height and the designed length are used to calculate the scaling ratio Sc of the width and height of the original image of the linear array camera. 宽 and Sc 高 ; Sc 宽 = Actual width of bolt hole in HD feature image / design width; Sc 高 = Actual height of bolt hole in HD feature image / design height; The bolt hole features of multiple high-definition original images of the same section after scaling correction are registered to obtain an inter-ring stitching image with a complete tunnel section; The inter-ring stitched images are stitched in the mileage direction to obtain the high-definition image stitching dataset I of the entire tunnel line array camera. 高清拼接 (I x ,I y ,R ij ,G ij ,B ij ).

7. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 6 is characterized in that: According to the value range mapping method, through the I 激 (I x ,I y ,Grey 反射强度 ,Ref ij ,△S ij ,θ ij )Get the laser point cloud axis diameter difference depth map I 深 (I x ,I y ,Grey 轴径差 ,R D ,G D ,B D ); Among them, the cross-section shaft diameter difference △S ij Map from small to large to 0~255 grayscale Grey 轴径差 , (R D ,G D ,B D ) is the cross-sectional axis diameter difference of the cross-sectional point expressed in RGB color numerical form: △S ij Pixels > 0.01 are set to blue (0,0,255), △S ij Pixels < -0.01 are set to yellow (255,255,0).

8. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 7 is characterized in that: The laser image is compared with the tunnel mileage field L in the high-definition image dataset. i Initial position alignment in registration; Through the SIFT image registration algorithm, the laser point cloud axis diameter difference depth map I 深 and I 高清特征 The bolt hole area marked in the image is used as a feature for automatic image registration to obtain the laser image and high-definition image stitching I 高清拼接 The joint dataset Union (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij ); Through this joint data set, we can obtain the linked query display, feature annotation, and dimension measurement calculation of laser images and high-definition images.

9. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 8 is characterized in that: According to the relative position relationship between the cross-sectional scanning radar and the linear array camera, the center of each linear array camera (Xc k ,Yc k ,Zc k ) is the rotation angle φk of the cross-sectional coordinate system with the center of the scanning radar (k=1,2,…,n is the number of the linear array camera); Scan the laser point cloud cross-section coordinates P through the tunnel ij (L i ,P xj ,P yj ), calculate the coordinates of the cross-sectional coordinate system of the linear array camera P k ij (L i ,P xj ,P yj ), obtain the distance Dc from the tunnel section point to the corresponding camera optical center ij ; P k xj =P xj cos∝ k -P yj sin∝ k +△x k ; P k yj =P xj sin∝ k -P yj cos∝ k +△y k ; ； According to the camera center distance Dc of each tunnel section point ij , obtain the precise magnification Sc of the corresponding visible light image pixel ij , and a high-definition image precision dataset with point-by-point accurate magnification I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij ); where the object magnification Sc ij =Dc ij / f, f is the focal length of the camera.

10. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 9 is characterized in that: Obtaining the length, width and area of ​​objects in visible light images through the high-definition image precision imaging data set; Among them, the area of ​​the object feature = the sum of the areas of all pixels covered by the object feature; Single pixel area = camera photosensitive unit pixel size × magnification Sc ij ; Object characteristic length = characteristic area ÷ representative width; Object characteristic width = characteristic area ÷ representative length.

Images