A data registration and calibration method based on tunnel laser scanning and high-definition imaging

Abstract

The present invention discloses a data registration and calibration method based on tunnel laser scanning and high-definition images, comprising: projecting laser point cloud data according to cross-sectional contour lines to obtain laser images; obtaining high-definition image feature maps through original visible light image maps to obtain high-definition visible light image mosaics; performing image registration on the laser images and the high-definition visible light image mosaics to obtain a joint dataset of laser images and high-definition visible light images (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij ); Get the actual magnification of the object Sc based on the joint dataset ij , obtain high-definition image accurate data set I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij ) to obtain high-definition visible light images; improving the measurement accuracy of disease characteristics in high-definition visible light images can accurately grasp the actual status of tunnel diseases.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a data registration and calibration method based on tunnel laser scanning and high-definition imaging. Background Art

[0002] Cracks, misalignments, water leaks, and structural deformations that occur during the operation of urban rail transit tunnels threaten the safety of tunnel structures. Therefore, tunnel defect detection and maintenance are crucial for safe rail transit operations. In recent years, with the rapid development of automated inspection technologies, primarily laser scanning and photogrammetry, defect detection based on visible light cameras and lidar scanning equipment has become a major focus of tunnel defect inspection. Many countries have developed tunnel inspection systems based on these technologies.

[0003] The tunnel inspection system acquires two types of data: laser scanning point cloud data and visible light detection images. Laser scanning data carries coordinate information and has good measurability. However, due to technical limitations, the image resolution converted from the laser scanning point cloud is low. Visible light camera images have the advantages of high clarity and rich texture information, and can be used to identify tiny crack defects. However, since the camera is affected by the changes in the tunnel cross-sectional profile during the tunnel shooting process, the distance from the camera lens to the tunnel wall point is always changing. Therefore, the actual magnification corresponding to each pixel point in the image depends on the actual distance from the camera to the tunnel wall point. Therefore, the actual size of the defect cannot be accurately measured on the visible light image. Summary of the Invention

[0004] In view of this, in order to address the technical problem in the tunnel detection system in the above-mentioned prior art that the actual size of the defects cannot be accurately measured, the present application provides a data alignment and calibration method based on tunnel laser scanning and high-definition visible light imaging. It uses the actual distance from the linear array camera to the tunnel structure obtained by the laser cross-sectional scanning radar to calculate the actual magnification of the visible light image, and then accurately calculates the physical resolution of each pixel point, thereby improving the measurement accuracy of image features with higher precision requirements such as crack width, thereby obtaining a more accurate size of the defect and more accurately and effectively controlling the current status and changing trends of tunnel defects.

[0005] The present application provides a data registration and calibration method based on tunnel laser scanning and high-definition imaging, the method comprising:

[0006] Expand and project the laser point cloud data according to the cross-sectional contour line to obtain the laser image;

[0007] Obtain a high-definition image feature map through the original visible light image map to obtain a high-definition visible light image mosaic map;

[0008] The laser image is registered with the high-definition visible light image mosaic to obtain a joint dataset Union (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij );

[0009] The actual magnification Sc of the object is obtained based on the joint data set ij , obtain high-definition image accurate data set I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij ) to obtain high-definition visible light images;

[0010] Among them, L i Point is the cross-section point numbered (i, j) ij Tunnel mileage, (I x激光 , I y激光 ) is the row and column coordinates of the cross-section point in the laser image calculated by expanding the cross-section contour line projection design, (I x高清 , I y高清 ) is the row and column coordinates of the cross-section point in the high-definition visible laser image mosaic, (P xj , P yj ) is the cross-sectional coordinate system coordinate of the laser point cloud of the cross-sectional point, △S ij is the cross-section axis diameter difference, θ ij is the central angle, Ref ij is the laser reflection intensity value, (R ij , G ij , B ij ) is the RGB value of the high-definition visible light image at the cross-section point, Sc ij is the actual magnification of the image of the section point, i is the section number obtained by the section scanning radar, and j is the section point number in each section.

[0011] Compared to the prior art, the data registration and calibration method based on tunnel laser scanning and high-definition visible light imaging in this application uses two different methods to obtain images of different resolutions, and then performs image registration on the main features of the two images to obtain two pixel correspondences. A joint dataset is then obtained after registration. Based on the joint dataset, the visible light image is magnified according to the lens magnification calculated based on the actual distance from each camera lens to the tunnel wall to obtain a more accurate high-definition visible light image than before calibration. This can further improve the measurement accuracy of defect characteristics, especially small dimensions such as crack width, allowing personnel to more effectively understand the actual status and development trends of tunnel defects and rationally assess the safety of tunnel structures.

[0012] Preferably, a laser radar is mounted on a track inspection vehicle to scan the tunnel section and obtain the tunnel three-dimensional point cloud data (L i , P xj , P yj , Ref ij ).

[0013] Preferably, based on the point cloud data, the cross-sectional mileage L is obtained. i The cross-section point cloud is expanded according to the design cross-section contour line to calculate the shaft diameter difference △S ij and the central angle θ ij ;

[0014] According to the cross-section contour line expansion projection principle, the cross-section coordinate system coordinates are calculated to obtain the cross-section point cloud dataset D 点云 (L i , P xj , P yj , Ref ij , △S ij ,θ ij );

[0015] in, .

[0016] Preferably, according to the cross-sectional contour line expansion projection principle, each cross-sectional point cloud is converted to the laser image coordinate system to obtain the laser image coordinate system coordinate I 激光坐标 (I x激光 , I y激光 );

[0017] According to the range mapping method, the laser reflection intensity value Ref ij Convert to 0~255 level image grayscale value Grey 反射强度 , based on I 激光坐标 (I x激光 , I y激光) Obtain the 3D laser point cloud grayscale image expansion map dataset I in the tunnel 激 (I x激光 , I y激光 , Grey 反射强度 , Ref ij , △S ij ,θ ij );

[0018] Among them, I x激光 and I y激光 are the horizontal and vertical coordinates of each pixel on the laser image; I y激光 The absolute value of is equal to the expanded length L of the section contour line from each point in the section point cloud to the section zero point j The corresponding number of pixels; the cross-section zero point is the horizontal coordinate P of the cross-section coordinate system in the cross-section point cloud. x =0 and P y >0, I x激光 =L i / column pixel actual resolution, I y激光 =﹣L j / line pixel actual resolution.

[0019] Preferably, the camera is set according to the segment size of the tunnel design, and the pixel line ln of the single frame image is collected and stored to obtain the original image taken by the line array camera;

[0020] Obtaining original images captured by multiple linear array cameras on the same section of the track inspection vehicle;

[0021] Use the feature extractor to detect and extract the bolt hole area feature mask in the high-definition original image, superimpose the detection result on the original image, and set the area to blue to obtain the high-definition image feature map I 高清特征 (L i ,I x高清 ,I y高清 ,R 特 ,G 特 ,B 特 );

[0022] The Mask R-CNN algorithm of the feature extractor adopts the pre-trained convolutional neural network ResNet-50.

[0023] Preferably, according to the high-definition image feature map I 高清特征 The actual length of the bolt hole width, the actual length of the height and the designed length are used to calculate the scaling ratio Sc of the width and height of the original image of the linear array camera. 宽 and Sc 高 ;

[0024] Sc 宽= Actual width of bolt hole in HD feature image / designed width;

[0025] Sc 高 = Actual height of bolt hole in HD feature image / designed height;

[0026] The bolt hole features of multiple high-definition original images of the same section after scaling correction are registered to obtain an inter-ring mosaic image with a complete tunnel section.

[0027] The inter-ring stitching images are stitched in the mileage direction to obtain the high-definition image mosaic dataset of the entire tunnel line array camera I. 高清拼接 (I x高清 ,I y高清 ,R ij ,G ij ,B ij ).

[0028] Preferably, according to the range mapping method, through the I 激 (I x激光 ,I y激光 ,Grey 反射强度 ,Ref ij ,△S ij ,θ ij ) Obtain the laser point cloud axis diameter difference depth map I 深 (I x激光 ,I y激光 ,Grey 轴径差 ,R D ,G D ,B D );

[0029] Among them, the cross-section diameter difference △S ij Mapped from small to large to 0~255 grayscale Grey 轴径差 , (R D ,G D ,B D ) is the cross-section axis diameter difference of the cross-section point expressed in RGB color value: ij Pixels with values ​​> 0.01 are set to blue (0, 0, 255), △S ij Pixels with a value < -0.01 are set to yellow (255, 255, 0).

[0030] Preferably, the laser image is combined with the tunnel mileage field L in the high-definition image dataset. i Initial position alignment in registration;

[0031] Through the SIFT image registration algorithm, the laser point cloud axis diameter difference depth map I 深 and I 高清特征The bolt hole area marked in the image is used as a feature for automatic image registration to obtain the laser image and high-definition image stitching image I 高清拼接 The joint dataset Union (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij );

[0032] Through this joint dataset, we can obtain the linked query display, feature annotation, and dimension measurement calculation of laser images and high-definition images.

[0033] Preferably, the center of each linear array camera (Xc k ,Yc k ,Zc k ) and the cross-section coordinate system rotation angle α of the scanning radar center k (k=1,2,…,n is the number of the line scan camera);

[0034] Scan the laser point cloud cross-section coordinates P through the tunnel ij (L i ,P xj ,P yj ), calculate the coordinates P of the cross-section coordinate system of the linear array camera k ij (L i ,P xj ,P yj ), obtain the distance Dc from the tunnel section point to the corresponding camera optical center ij ;

[0035] P k xj =P xj cosα k -P yj sinα k +△x k ;

[0036] P k yj =P xj sin∝α k -P yj cosα k +△y k ;

[0037] ;

[0038] According to the camera center distance Dc of each tunnel section point ij , obtain the precise magnification Sc of the corresponding visible light image pixel ij , and a high-definition image precision dataset with point-by-point accurate magnification I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij );

[0039] Among them, the object magnification Sc ij= Dc ij / f, where f is the focal length of the camera.

[0040] Preferably, the length, width and area of ​​the object in the visible light image are obtained through the high-definition image precision imaging data set;

[0041] in,

[0042] The area of ​​an object feature = the sum of the areas of all pixels covered by the object feature;

[0043] Single pixel area = camera photosensitive unit pixel size × magnification Sc ij ;

[0044] Characteristic length of an object = characteristic area ÷ representative width;

[0045] Object characteristic width = characteristic area ÷ representative length. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The three-dimensional point cloud data obtained by three-dimensional laser scanning of a tunnel provided in an embodiment of the present application;

[0047] Figure 2 This is a schematic diagram of the projection principle of the cross-sectional contour line of a laser scanned three-dimensional point cloud provided in one embodiment of the present application;

[0048] Figure 3 This is an expanded view of the cross-sectional profile of a laser scanning point cloud provided in one embodiment of the present application;

[0049] Figure 4This is the original high-definition tunnel image acquired by nine linear array cameras provided in an embodiment of the present application;

[0050] Figure 5 This is a schematic diagram of annular seams and intersections automatically identified on a high-definition image provided by an embodiment of the present application;

[0051] Figure 6 This is a high-definition image feature map for automatically extracting bolt hole areas provided by an embodiment of the present application;

[0052] Figure 7 This is a schematic diagram of scaling correction of an original high-definition image provided by an embodiment of the present application;

[0053] Figure 8 This is a high-definition image that has been scaled and stitched, as provided in an embodiment of the present application;

[0054] Figure 9 This is an example diagram of a depth map of a laser scanning point cloud image provided by an embodiment of the present application;

[0055] Figure 10 This is a high-definition image in the linkage query and measurement interface diagram provided in one embodiment of the present application;

[0056] Figure 11 yes Figure 10 A magnified schematic diagram of the local area A in FIG;

[0057] Figure 12 yes Figure 10 A magnified schematic diagram of part B in FIG.

[0058] Figure 13 It is a three-dimensional laser point cloud image in the linkage query and measurement interface diagram provided in an embodiment of the present application;

[0059] Figure 14 It is a two-dimensional point cloud cross-section diagram in the linkage query and measurement interface diagram provided in one embodiment of the present application;

[0060] Figure 15 This is a schematic diagram of the coordinate system conversion principle of the laser radar and the linear array camera provided in one embodiment of the present application;

[0061] Figure 16 It is a high-definition linear image of a crack width standard card for testing crack width provided in an embodiment of the present application;

[0062] Figure 17 This is a schematic diagram of measuring the scaling error effect provided by an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the present disclosure is described in detail, clearly, and completely in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not intended to limit the present disclosure.

[0064] In the description of this application, if there is a description of first or second, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0065] Those skilled in the art should understand that, in the disclosure of this application, the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, which are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the above terms cannot be understood as limiting this application.

[0066] The present application will be described in further detail below with reference to the accompanying drawings. Figures 1 to 17 illustrate.

[0067] The present application provides a data registration and calibration method based on tunnel laser scanning and high-definition imaging. It improves the measurement accuracy of tiny-sized defect features such as crack width by accurately calculating the physical resolution of each pixel point, so as to obtain high-definition images of the actual size of defects in the tunnel structure, thereby helping staff to reasonably and accurately assess the safety status of the tunnel structure.

[0068] Specifically, the method includes the following specific steps:

[0069] Step S1: Projecting the laser point cloud data along the cross-sectional contour line to obtain a laser image;

[0070] Step S2: obtaining a high-definition image feature map through the original visible light image map to obtain a high-definition visible light image mosaic map;

[0071] Step S3: Perform image registration on the laser image and the high-definition visible light image mosaic to obtain the joint dataset Union (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , Iy高清 , R ij , G ij , B ij );

[0072] Step S4: Obtain the actual magnification Sc of the object based on the joint dataset ij , obtain high-definition image accurate data set I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij ) to obtain high-definition visible light images;

[0073] Among them, L i Point is the cross-section point numbered (i, j) ij Tunnel mileage, (I x激光 , I y激光 ) is the row and column coordinates of the cross-section point in the laser image calculated by expanding the cross-section contour line projection design, (I x高清 , I y高清 ) is the row and column coordinates of the cross-section point in the high-definition visible laser image mosaic, (P xj , P yj ) is the cross-sectional coordinate system coordinate of the laser point cloud of the cross-sectional point, △S ij is the cross-section shaft diameter difference, Ref ij is the laser reflection intensity value, (R ij , G ij , B ij ) is the RGB value of the high-definition visible light image at the cross-section point, Sc ij is the actual magnification of the image of the section point, i is the section number obtained by the section scanning radar, and j is the section point number in each section.

[0074] In this embodiment, if Figure 2 、 Figure 3 As shown in Figure 2, laser point cloud data is obtained by scanning the tunnel with a laser radar, and the laser point cloud image expansion diagram (referred to as laser image) is obtained after cross-sectional projection processing; Figures 4 to 8 As shown in the figure, the original image is obtained by shooting with a linear array camera, and after stitching and size correction, a high-definition visible light image stitching map is obtained; Figure 6 、 Figure 9As shown, the laser image and the high-definition image mosaic are then registered to obtain a joint dataset of laser images and high-definition images. This dataset is then processed to obtain the actual magnification of each object, thereby directly obtaining a high-definition visible light image with higher dimensional accuracy, thereby assisting personnel in accurately drawing and measuring the precise size of the inspection target on the high-definition image. Figures 10 to 14 As shown in the figure, the joint dataset can support the linked query and precise measurement of laser point clouds and high-definition images, making full use of the high-precision three-dimensional size characteristics of laser point clouds and the high-definition and high-reduction texture characteristics of high-definition camera images.

[0075] Based on the above embodiment, step S1 is further expanded; wherein step S1 includes step S11, step S12 and step S13.

[0076] Specifically, step S11: Figure 1 As shown, the laser radar is carried by the track inspection vehicle to scan and obtain the original data of the tunnel 3D point cloud, and then obtain the tunnel 3D point cloud data (L i , P xj , P yj , Ref ij ).

[0077] In this step, if Figure 15 As shown in the figure, the track inspection vehicle is equipped with a laser scanning radar and n linear array cameras. The laser scanning radar is installed at the center of the base of the inspection vehicle, and the linear array cameras are arranged in a ring on the base of the inspection vehicle. The installation plane of the cameras is parallel to the laser scanning section. When the track inspection vehicle moves forward, it records the travel distance while scanning the tunnel section with the laser radar. The three-dimensional laser hits the inner wall of the tunnel and is reflected by the scanning radar, thereby obtaining the tunnel mileage Li of the section point and the section coordinate system coordinates of the section point (P xj , P yj ) and laser reflection intensity value Ref ij , to assist in establishing a point cloud database. Through this structure, the pixel columns acquired by the line array camera and the cross-sectional point cloud acquired by the laser scanning radar can be on the same plane.

[0078] Among them, such as Figure 17 As shown, the bottom of the track inspection vehicle base is equipped with rollers to facilitate the movement of the inspection vehicle; the linear array camera is in an arc shape. Figure 17In the distribution shown, the scanning cross-section of the laser scanning radar must remain parallel to the installation plane of the linear array camera, and adjacent cameras must have an overlapping field of view of no less than 20%, thereby improving the accuracy of stitching and registration of the laser image scanned by the laser radar and the high-definition image captured by the linear array camera. Furthermore, the linear array camera is equipped with arc-shaped light source sheets arranged in two rows, with the linear array camera located between the two rows of light source sheets to provide a better lighting environment for the linear array camera, improving the shooting illumination and image clarity; the inner diameter of the arc formed by the distribution of light source sheets is larger than the inner diameter of the arc formed by the distribution of linear array cameras, that is, the linear array camera and the light source sheets are arranged at an interval.

[0079] It should be noted that, in this application, the cross-sectional laser scanner can scan 100 circles per second in the direction perpendicular to the platform movement direction, and the cross-sectional point cloud refers to the cross-sectional points of this circle collected by the laser scanner in 0.01 seconds.

[0080] In this application, high-definition images refer to images with a single pixel physical resolution of 0.2mm. This level of resolution can be used to identify cracks with a width of 0.2mm.

[0081] After step S11, step S12 is also included: based on the point cloud data, the cross-section mileage L is obtained. i The cross-section point cloud is expanded according to the design cross-section contour line to calculate the shaft diameter difference △S ij and the central angle θ ij ;

[0082] According to the principle of cross-section contour line expansion projection, the cross-section coordinate system coordinates are calculated to obtain Figure 14 The two-dimensional cross-sectional point cloud dataset D 点云 (L i , P xj , P yj , Ref ij , △S ij ,θ ij );

[0083] Taking a circular tunnel with a cross-section radius of R as an example, .

[0084] In this embodiment, ΔS ij The physical meaning is the difference between the actual distance from the cross-section point to the center of the fitted cross-section and the theoretical distance between the cross-section point at that position and the cross-section center in the theoretical cross-section contour diagram. For example, if the theoretical cross-section is a circle with a radius of R, then θ ij is the angle between the section point and the origin of the section coordinate system; L is the section mileage, i is the section number obtained by the section scanning radar, and j is the section point number in each section.

[0085] After step S12, the process further includes step S13: according to the cross-section contour line expansion projection principle, each cross-section point cloud is converted into the simulated laser image coordinate system to obtain the laser image coordinate system coordinate I 激光坐标 (I x激光 ,I y激光 );

[0086] Furthermore, according to the range mapping method, the laser reflection intensity value Ref ij Convert to 0~255 level image grayscale value Grey 反射强度 , based on I 激光坐标 (I x激光 ,I y激光 ) Obtain the 3D laser point cloud grayscale image expansion map dataset I in the tunnel 激 (I x激光 ,I y激光 , Grey 反射强度 , Ref ij , △S ij ,θ ij ), and then the laser image in step S1 is obtained.

[0087] Among them, I x激光 and I y激光 are the horizontal and vertical coordinates of each pixel on the laser image; I y激光 The absolute value of is equal to the expanded length L of the cross-section contour line from each point in the cross-section point cloud to the zero point of the cross-section j The corresponding number of pixels; the zero point of the section is the horizontal coordinate P of the section coordinate system in the section point cloud x =0 and P y >0, I x激光 =L i / column pixel actual resolution, I y激光 =﹣L j / line pixel actual resolution.

[0088] In this embodiment, the definitions of steps i and j are the same as those in step S12.

[0089] Based on any of the above embodiments, step S2 is further expanded; Figure 3 、 Figures 10 to 17 As shown, step S2 specifically includes step S21 and step S22.

[0090] Specifically, step S21: according to the tunnel design segment size, set the pixel line ln of the single frame image collected and stored by the camera to obtain the original image taken by the line array camera; Figure 4As shown, there are 9 (9a~9i) original color high-definition images of the shield tunnel with ln=5000 lines respectively acquired by the 9 cameras of this application, wherein the original images are rotated 90 degrees clockwise; the image height is the forward direction of the acquisition platform, so the height is ln, and the image width is the nominal resolution of the line array camera. In this application, the example is 8096 pixels acquired by a nominal resolution 8K camera.

[0091] Furthermore, the original images captured by multiple linear array cameras on the same section of the track inspection vehicle are obtained; among them, the Canny edge detection algorithm is used to detect all linear features, and the Hough line detection algorithm is used to detect the vertical joints of the pipe ring perpendicular to the tunnel forward direction and the internal joints of the ring with a specific design angle with the vertical joints in the linear feature results, and the following is obtained: Figure 5 The intersection points of the two types of seams shown in the figure are red for the intersection points of the seams between the rings, and yellow for the intersection points of the blocks within the ring.

[0092] The Canny edge detection algorithm is more suitable for extracting linear features such as tunnel annular joints in the presence of complex background noise. It can detect annular joint edges of varying thicknesses and connect disconnected edges using a dual threshold, making it more suitable for extracting tunnel annular joint features. The Hough line detection algorithm can extract linear features within a specific angle range and is particularly suitable for extracting vertical joints and internal joints in this area.

[0093] In this application, in accordance with the tunnel design segment size, cameras are set according to the tunnel section angle range to be photographed and the field of view angle of each camera, so that the field of view range composed of all cameras can cover the tunnel section angle range to be photographed under the premise that the overlapping field of view angle of adjacent cameras is not less than 5°.

[0094] Furthermore, the feature extractor is used to detect and extract the feature mask of the bolt hole area in the high-definition original image. The detection result is superimposed on the original image, and the area is set to blue (0, 0, 255). The following is obtained: Figure 6 The high-definition image feature diagram I shown 高清特征 (L i ,I x高清 ,I y高清 ,R 特 ,G 特 ,B 特 );

[0095] In the Mask R-CNN algorithm of the feature extractor, ResNet-50 is used as the backbone network to extract multi-level features of the input image. These features are then used in subsequent tasks such as the Region Proposal Network (RPN) and mask generation. The specific process is as follows:

[0096] 1) Image preprocessing, fixed size and batch normalization;

[0097] 2) Unlike ResNet34, ResNet-50 is a deep convolutional neural network that downsamples the input image through convolution and pooling operations to extract semantic information with different resolutions and orders;

[0098] 3) Enhance multi-scale feature extraction through the Feature Pyramid Network (FPN) and perform feature fusion (low-level and high-level feature fusion, adding feature map channels at different stages) to output feature maps;

[0099] 4) Each layer of FPN is input into RPN, and candidate regions (Region Proposals) are generated through classification (predicting candidate region categories, such as bolt holes) and regression branches (adjusting candidate region positions and sizes). Fixed-size features are extracted through bilinear interpolation (RoIAlign) operations for target detection and mask generation.

[0100] It should be noted that the Mask R-CNN algorithm based on ResNet-50 can generate pixel-level masks and accurately locate the boundaries of bolt holes. FPN combined with a deep convolutional network can effectively extract multi-level features. The model can process large-scale tunnel data in batches and is highly robust. The algorithm integrates target detection and segmentation, and has multifunctional unified bolt hole classification and positioning. It can continuously perform data-driven optimization, that is, increase samples, enhance model prediction accuracy, and reduce labor costs.

[0101] After step S21, the process further includes step S22: Figure 7 As shown, according to the high-definition image feature diagram I 高清特征 The actual length W of the bolt hole width, the actual length H of the height and the designed length are used to calculate the scaling ratio Sc of the width and height of the original image of the linear array camera. 宽 and Sc 高 ;

[0102] Sc 宽 = Actual width of bolt hole in HD feature image / designed width;

[0103] Sc 高 = Actual height of bolt hole in HD feature image / designed height;

[0104] Further, such as Figure 8 As shown in the figure, the bolt hole features of multiple high-definition original images of the same section taken by the multi-line array camera after scaling correction are image registered to obtain Figure 8The inter-ring stitching image with a complete tunnel section is shown in (a); then, image stitching in the mileage direction is performed based on the inter-ring stitching image to obtain the high-definition image stitching dataset I of the entire tunnel line array camera. 高清拼接 (I x高清 ,I y高清 ,R ij ,G ij ,B ij ), thus obtaining Figure 8 (b) shows a high-definition visible light image mosaic of multiple rings.

[0105] Based on any of the above embodiments, step S3 is further expanded; step S3 includes step S31 and step S32.

[0106] Specifically, step S31: according to the range mapping method, through the I 激 (I x激光 ,I y激光 ,Grey 反射强度 ,Ref ij ,△S ij ,θ ij ) Obtain the laser point cloud axis diameter difference depth map I 深 (I x激光 ,I y激光 ,Grey 轴径差 ,R D ,G D ,B D );

[0107] Among them, the cross-section diameter difference △S ij Mapped from small to large to 0~255 grayscale Grey 轴径差 , and △S ij Pixels with values ​​> 0.01 are set to blue (0, 0, 255), △S ij Pixels with values ​​< -0.01 are set to yellow (255, 255, 0) to clearly distinguish between facilities that are concave (blue), such as bolt holes, and facilities that are convex (yellow), such as contact network cable racks and equipment boxes, relative to the design cross-section. Laser point cloud axis diameter difference depth map I 深 (I x激光 ,I y激光 ,Grey 轴径差 ,R D ,G D ,B D ), (R D ,G D ,B D ) is the cross-section axis diameter difference expressed in RGB color value, R D The D in the formula represents the depth; Figure 9 shown.

[0108] Furthermore, after step S31, the process further includes step S32: comparing the laser image with the tunnel mileage field L in the high-definition image dataset. i Initial position alignment in registration;

[0109] Through the SIFT image registration algorithm, the laser point cloud axis diameter difference depth map I 深 and I 高清特征 The bolt hole area marked in the image is used as a feature for automatic image registration to obtain the laser image and high-definition image stitching image I 高清拼接 The joint dataset Union (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij );

[0110] Through this joint dataset, we can obtain the linked query display, feature annotation, and dimension measurement calculation of laser images and high-definition images.

[0111] It should be noted that the high-definition image corresponding to the joint dataset is an orthophoto image with measurable characteristics. The measured distance size is the scaled value of the actual distance size on the tunnel surface. This image is not only consistent with the 3D laser scanning point cloud (L i， P xj , P yj ) point by point, and in the tunnel mileage L i The direction corresponds to each scanned 2D point cloud section point by point (P xj , P yj );like Figures 10 to 14 As shown, the high-definition image expansion map, three-dimensional point cloud and two-dimensional cross-sectional map are linked to query and display the precise tunnel mileage and cross-sectional coordinates of each point and calculate the length and area.

[0112] Based on any of the above embodiments, step S4 is further expanded; step S4 includes step S41 and step S42;

[0113] Specifically, step S41: Figure 15 As shown in the figure, according to the relative position relationship between the cross-sectional scanning radar and the linear array camera, the center of each linear array camera (Xc k ,Yc k ,Zc k) and the cross-section coordinate system rotation angle α of the scanning radar center k (k=1,2,…,n is the number of the line scan camera);

[0114] Scan the laser point cloud cross-section coordinates P through the tunnel ij (L i ,P xj ,P yj ), calculate the coordinates P of the cross-section coordinate system of the linear array camera k ij (L i ,P xj ,P yj ), obtain the distance Dc from the tunnel section point to the corresponding camera optical center ij ;

[0115] Where i is the section number obtained by the cross-section scanning radar, and j is the section point number in each section;

[0116] P k xj =P xj cosα k -P yj sinα k +△x k ;

[0117] P k yj =P xj sinα k -P yj cosα k +△y k ;

[0118] .

[0119] Furthermore, according to the camera center distance Dc of each tunnel section point ij , obtain the precise magnification Sc of the corresponding visible light image pixel ij , and a high-definition image precision dataset with point-by-point accurate magnification I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij );

[0120] Among them, the object magnification Sc ij =Dc ij / f, where f is the focal length of the camera.

[0121] Furthermore, after step S41, the method further includes step S42: obtaining the length, width and area of ​​the object in the visible light image through the high-definition image precise imaging data set;

[0122] (1) First calculate the area of ​​the feature based on the total number of pixels of the extracted object feature.

[0123] The area of ​​an object feature = the sum of the areas of all pixels covered by the object feature;

[0124] Single pixel area = camera photosensitive unit pixel size × magnification Sc ij ;

[0125] (2) The representative size of the measured feature, such as the length of a crack feature;

[0126] (3) Calculating another dimension parameter of the object feature, such as width (or length), based on one dimension parameter of the measured planar object feature, such as length (or width); wherein,

[0127] Characteristic length of an object = characteristic area ÷ representative width;

[0128] Object characteristic width = characteristic area ÷ representative length.

[0129] It should be noted that in step S42, when measuring the object in the image, the size corresponding to a single pixel = the camera's photosensitive pixel size * the magnification Sc ij The sum of all pixel sizes measured is the object size. Paste the standard crack width reference card (such as Figure 16 ) can perform correction accuracy inspection on the image after magnification correction, so as to accurately calculate the actual resolution of each pixel point and improve the measurement accuracy of tiny dimensions such as crack width.

[0130] In step S42, the object feature size measurement method prioritizes calculating the pixel area of ​​the feature, then calculates another feature size based on the user-assisted input of a representative feature size. This method controls the measurement of object features through a more precise total area, making it more accurate than traditional methods of directly measuring the number of pixels to calculate length, and more conducive to accurately detecting changes in small features, such as the development of crack disease characteristics.

[0131] Take the most common circular shield subway tunnel with an internal diameter of 5500mm as an example. Figure 17As shown in the figure, the maximum image scaling error caused by the change in the distance from the camera lens to the tunnel wall due to the "replacing the arc with the chord" error of the arc ring on the camera plane is approximately , so the maximum area error is about That is, the processing method of the present application can improve the measurement accuracy of linear features by more than 3% and the measurement accuracy of surface features by more than 6%, which has high practical significance.

[0132] It should be noted that the various embodiments of the present application can be arbitrarily combined into new embodiments if the solutions do not conflict and the technical solutions can coexist.

[0133] The present application has been described in detail above. Specific examples have been used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is intended only to facilitate understanding of the present application and its core concepts. It should be noted that, without departing from the principles of the present application, a number of improvements and modifications may be made to the present application by a person skilled in the art, and such improvements and modifications shall fall within the scope of protection of the claims of the present application.

Claims

1. A data registration and calibration method based on tunnel laser scanning and high-definition imaging, characterized in that: The method comprises: Expand and project the laser point cloud data according to the cross-sectional contour line to obtain the laser image; Obtain a high-definition image feature map through the original visible light image map to obtain a high-definition visible light image mosaic map; The laser image is registered with the high-definition visible light image mosaic to obtain a joint dataset Union (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij ); The actual magnification Sc of the object is obtained based on the joint data set ij , obtain high-definition image accurate data set I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij ) to obtain high-definition visible light images; Among them, L i Point is the cross-section point numbered (i, j) ij Tunnel mileage, (I x激光 , I y激光 ) is the row and column coordinates of the cross-section point in the laser image calculated by expanding the cross-section contour line projection design, (I x高清 , I y高清 ) is the row and column coordinates of the cross-section point in the high-definition visible laser image mosaic, (P xj , P yj ) is the cross-sectional coordinate system coordinate of the laser point cloud of the cross-sectional point, △S ij The physical meaning is the difference between the actual distance between the cross-section point and the center point of the fitted cross-section and the theoretical distance between the cross-section point at that position and the center point of the cross-section in the theoretical cross-section contour diagram; θ ij is the central angle, which is the angle between the line connecting the section point and the origin of the section coordinate system and the x-axis when the section point is in the section coordinate system; ij is the laser reflection intensity value, (R ij , G ij , B ij ) is the RGB value of the high-definition visible light image at the cross-section point, Sc ij is the actual magnification of the image of the cross-section point, i is the cross-section number obtained by the cross-section scanning radar, and j is the cross-section point number in each cross-section; According to the relative position relationship between the cross-sectional scanning radar and the linear array camera, the center of each linear array camera (Xc k ,Yc k ,Zc k ) and the cross-section coordinate system rotation angle α of the scanning radar center k (k=1,2,…,n is the number of the line scan camera); Scan the laser point cloud cross-section coordinates P through the tunnel ij (L i ,P xj ,P yj ), calculate the coordinates P of the cross-section coordinate system of the linear array camera k ij (L i ,P k xj ,P k yj ), obtain the distance Dc from the tunnel section point to the corresponding camera optical center ij ; P k xj =P xj cosα k -P yj sinα k +△x k ; P k yj =P xj sinα k -P yj cosα k +△y k ; ； According to the camera center distance Dc of each tunnel section point ij , obtain the precise magnification Sc of the corresponding visible light image pixel ij , and a high-definition image precision dataset with point-by-point accurate magnification I 精 (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij , Sc ij ); Among them, the object magnification Sc ij =Dc ij / f, f is the focal length of the camera; △x k is the translation of the camera center numbered k relative to the scanning radar center on the x-axis, △y k is the translation of the camera center numbered k relative to the scanning radar center on the y-axis.

2. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 1 is characterized in that: The laser radar is installed on the track inspection vehicle to scan the tunnel section and obtain the tunnel 3D point cloud data (L i , P xj , P yj , Ref ij ).

3. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 2 is characterized in that: Point Cloud Based on the point cloud data, the cross-sectional mileage L is obtained. i The cross-section point cloud is expanded according to the design cross-section contour line to calculate the shaft diameter difference △S ij and the central angle θ ij ; According to the cross-section contour line expansion projection principle, the cross-section coordinate system coordinates are calculated to obtain the cross-section point cloud dataset D 点云 (L i , P xj , P yj , Ref ij , △S ij ,θ ij ); in, , is the actual distance from the section point to the center point of the fitted section, and R is the theoretical distance between the section point at that position and the center of the section in the theoretical section contour diagram.

4. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 3 is characterized in that: According to the principle of cross-section contour line expansion projection, each cross-section point cloud is converted to the laser image coordinate system to obtain the laser image coordinate system coordinate I 激光坐标 (I x激光 , I y激光 ); According to the range mapping method, the laser reflection intensity value Ref ij Convert to 0~255 level image grayscale value Grey 反射强度 , based on I 激光坐标 (I x激光 , I y激光 ) Obtain the 3D laser point cloud grayscale image expansion map dataset I in the tunnel 激 (I x激光 , I y激光 , Grey 反射强度 , Ref ij , △S ij ,θ ij ); Among them, I x激光 and I y激光 are the horizontal and vertical coordinates of each pixel on the laser image; I y激光 The absolute value of is equal to the expanded length L of the cross-section contour line from each point in the cross-section point cloud to the zero point of the cross-section j The corresponding number of pixels; the cross-section zero point is the horizontal coordinate P of the cross-section coordinate system in the cross-section point cloud. x =0 and P y >0, I x激光 =L i / column pixel actual resolution, I y激光 =﹣L j / line pixel actual resolution; Among them, L i Point is the cross-section point numbered (i, j) ij Tunnel mileage, L j The expanded length of the section contour line from each point in the section point cloud to the section zero point.

5. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 4 is characterized in that: The camera is set according to the designed segment size of the tunnel, and the pixel line ln of the single frame image is collected and stored to obtain the original image taken by the line array camera; Obtaining original images captured by multiple linear array cameras on the same section of the track inspection vehicle; Use the feature extractor to detect and extract the bolt hole area feature mask in the high-definition original image, superimpose the detection result on the original image, and set the area to blue to obtain the high-definition image feature map I 高清特征 (L i ,I x高清 ,I y高清 ,R 特 ,G 特 ,B 特 ); The Mask R-CNN algorithm of the feature extractor adopts the pre-trained convolutional neural network ResNet-50.

6. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 5 is characterized in that: According to the high-definition image feature map I 高清特征 The actual length of the bolt hole width, the actual length of the height and the designed length are used to calculate the scaling ratio Sc of the width and height of the original image of the linear array camera. 宽 and Sc 高 ; The design length includes the design width and design height; Sc 宽 = Actual width of bolt hole in HD feature image / designed width; Sc 高 = Actual height of bolt hole in HD feature image / designed height; The bolt hole features of multiple high-definition original images of the same section after scaling correction are registered to obtain an inter-ring mosaic image with a complete tunnel section. The inter-ring stitching images are stitched in the mileage direction to obtain the high-definition image mosaic dataset of the entire tunnel line array camera I. 高清拼接 (I x高清 ,I y高清 ,R ij ,G ij ,B ij ).

7. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 6 is characterized in that: According to the range mapping method, through the I 激 (I x激光 ,I y激光 ,Grey 反射强度 ,Ref ij ,△S ij ,θ ij ) Obtain the laser point cloud axis diameter difference depth map I 深 (I x激光 ,I y激光 ,Grey 轴径差 ,R D ,G D ,B D ); Among them, the cross-section diameter difference △S ij Mapped from small to large to 0~255 grayscale Grey 轴径差 ,(R D ,G D ,B D ) is the cross-section axis diameter difference of the cross-section point expressed in RGB color value: ij Pixels with values ​​> 0.01 are set to blue (0, 0, 255), △S ij Pixels with a value < -0.01 are set to yellow (255, 255, 0).

8. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 7 is characterized in that: The laser image is compared with the tunnel mileage field L in the high-definition image dataset. i Initial position alignment in registration; Through the SIFT image registration algorithm, the laser point cloud axis diameter difference depth map I 深 and I 高清特征 The bolt hole area marked in the image is used as a feature for automatic image registration to obtain the laser image and high-definition image stitching image I 高清拼接 The joint dataset Union (L i , I x激光 , I y激光 , P xj , P yj , Ref ij , △S ij ,θ ij , I x高清 , I y高清 , R ij , G ij , B ij ); Through this joint dataset, we can obtain the linked query display, feature annotation, and dimension measurement calculation of laser images and high-definition images.

9. The data registration and calibration method based on tunnel laser scanning and high-definition imaging according to claim 7 is characterized in that: Obtaining the length, width and area of ​​objects in visible light images through the high-definition image precision imaging data set; Among them, the area of ​​the object feature = the sum of the areas of all pixels covered by the object feature; Single pixel area = camera photosensitive unit pixel size × magnification Sc ij ; Characteristic length of an object = characteristic area ÷ representative width; Object characteristic width = characteristic area ÷ representative length; The representative length is the dimension representing the length of the measurement feature, and the representative width is the dimension representing the width of the measurement feature.

Images