Laser point cloud color assigning method based on linear array image

Abstract

The invention belongs to the technical field of tunnel detection, and particularly relates to a laser point cloud color giving method based on a linear array image, which comprises the following steps of: 1, scanning and preprocessing the linear array image, generating a laser intensity image by using point cloud reflection intensity information of a laser section scanner, storing laser intensity of all points of a section in each column of the image, and obtaining a laser intensity image; the columns are distributed in the mileage direction; 2, image mileage direction alignment is carried out, and the linear array image and the section laser intensity image represent data of one section in each column and are arranged along the mileage direction; 3, longitudinally stretching and correcting the image; 4, performing multi-source image feature matching; and 5, constructing a pixel coordinate transformation relation. According to the method, projection consistency of the laser intensity image and the optical image is achieved through re-projection, then accurate registration of the laser intensity image and the optical image is achieved through feature matching, and finally unification of the laser intensity image and the optical image is achieved through a polynomial model, so that corresponding color information is found for each laser point cloud.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel detection, and in particular to a laser point cloud coloring method based on linear array images. Background Art

[0002] In order to carry out accurate inspection and maintenance, it is particularly important to obtain high-precision three-dimensional coordinates and surface images of the inner wall of the tunnel. Based on mobile laser scanning technology, high-density, high-precision three-dimensional coordinates of the inner wall of the tunnel can be obtained section by section, and laser intensity images of each section can be generated based on the laser echo intensity, becoming an indispensable tool in the fields of engineering measurement and building modeling. On the other hand, tunnel inner wall surface imaging technology based on linear array cameras can obtain optical images of the tunnel surface section by section, providing rich color and texture information, which is easier to understand and analyze in terms of human eye visualization.

[0003] However, laser point cloud data usually lacks real color and texture information. There is usually a big difference between the laser intensity image and the texture actually observed by the human eye. The generated three-dimensional point cloud is quite different from the actual scene, resulting in limitations in visualization and detail identification. Linear array images usually lack three-dimensional structural information and cannot realize the observation of tunnel geometry.

[0004] Traditional tunnel detection technology can only solve and analyze two types of data separately, and has deficiencies in multi-source data fusion. It is difficult to fully understand the scene with only two-dimensional images or only three-dimensional point clouds, and their respective advantages cannot be fully utilized, resulting in limited recognition and understanding capabilities in complex scenes. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] The purpose of the present invention is to solve the problem in the prior art that laser point cloud data usually lacks real color and texture information and line array images usually lack three-dimensional structure information, and to propose a laser point cloud coloring method based on line array images.

[0007] 2. Technical solution

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] The laser point cloud coloring method based on linear array imaging includes the following steps:

[0010] Step 1: Scanning and linear array image preprocessing, using the point cloud reflection intensity information of the laser cross-section scanner to generate a laser intensity image. Each column of the image stores the laser intensity of all points in a cross section and is distributed column by column along the mileage direction;

[0011] Step 2: Align the image in the mileage direction. The linear array image and the cross-sectional laser intensity image represent the data of a cross section in each column and are arranged along the mileage direction. The data composition of the two is consistent, which can simplify the registration process.

[0012] Step 3: Correction of longitudinal stretching of the image. The linear array image was reprojected and stitched during preprocessing, and the geometric structure of the image was not preserved. Therefore, the solution was mainly based on texture information. Taking the shield tunnel as an example, O1 is the center of the scanner, O2 is the center of the tunnel, dy is the y coordinate of the two centers, p is any point on the inner wall of the tunnel, l is the distance from a point on the tunnel to the center of the tunnel, and θ1 and θ2 are the vertical angles of different centers. Since the linear array image has achieved uniform distribution of resolution through reprojection and is symmetrical with respect to the top of the tunnel, the θ2 corresponding to each pixel can be obtained through the image resolution and the y coordinate of the pixel point in the image coordinate system. The corresponding geometric relationship is shown in Formula 1. The linear array image is reprojected to be consistent with the laser intensity image. The θ1 of each row of the laser intensity image is calculated, and the corresponding θ2 is obtained, and the corresponding reprojection rows are obtained row by row. The conversion relationship is shown in Formula 2. The linear array image is reprojected according to the row pair relationship to reduce the difference between the two in the longitudinal direction.

[0013]

[0014]

[0015] Step 4: Multi-source image feature matching. Image registration is based on the feature matching results of the two images. Since the original data is large, the matching is performed in segments in the mileage direction. The laser intensity image is obtained with a fixed number of columns, and the pre-processed linear array image of the corresponding area is obtained. To ensure the matching results, the linear array image needs to be widened by a certain number of columns. The SURF operator is used to calculate the key points and descriptors of the image, and then the BF brute force feature matching method is used to perform feature matching to obtain the rough matching results of the feature matching.

[0016] Step 5: Construct pixel coordinate transformation relationship. Use a polynomial model that avoids the spatial geometric relationship of imaging to construct pixel coordinate transformation relationship. The polynomial model constructs a linear polynomial with multiple parameters, which directly simulates the image deformation itself mathematically. The overall deformation of the image is regarded as the comprehensive result of various basic deformations. See Formula 3. The result of feature matching is brought into the polynomial model, and the least squares method is used for adjustment to solve the parameters of the polynomial model.

[0017]

[0018] Using the transformation relationship model, starting from the output image array of the same size as the laser intensity image, the corresponding position of each pixel in the line array image is calculated in turn, and then the pixel corresponding to the position of the line array image is calculated by resampling, and then the calculated pixel is assigned to the output image pixel. Finally, a color image that completely corresponds to the coordinates of the laser intensity image is obtained, and the image is used to directly color the point cloud. When generating the point cloud file, the color information is written into the standard LAS format file.

[0019] Preferably, in step 1, in order to ensure the resolution effect of the laser intensity image, the intensity value is mapped to the range of 0-255 by a piecewise linear fitting method based on the prior parameters to generate the laser intensity image.

[0020] Preferably, in step 1, in order to ensure uniform resolution of the linear array image in the cross-sectional direction, different cross-sectional types are flexibly adapted according to methods such as piecewise polynomial fitting to generate orthophoto linear array images.

[0021] Preferably, under normal acquisition conditions in step 2, the mileage resolution of the linear array image is higher than that of the laser intensity image. The linear array image is downsampled in the mileage direction according to the mileage direction resolution ratio between the two, so as to achieve preliminary consistency of the mileage direction resolution. The mileage alignment of the two can be measured by an odometer, or a starting mileage can be manually marked and then a preliminary alignment can be performed by the mileage resolution ratio, and precise correction can be performed by the subsequent matching results to achieve accurate alignment in the mileage direction.

[0022] Preferably, in step 3, the geometric relationship of the projection is first used to achieve a preliminary transformation of the linear array image so that its projection mode is consistent with the laser intensity image, and the projection is further refined by feature matching.

[0023] Preferably, in step 3, in order to further improve the accuracy of the longitudinal stretch correction of the image, feature matching can be performed and a polynomial correction model can be obtained based on the matching results, and the corresponding rows of the reprojection are calculated row by row based on the correction model; when the tunnel parameters do not change, it is only necessary to calculate the reprojection parameters once to improve the calculation efficiency, and reprojection is performed to ensure that the feature matching obtains enough correct same-name points. For this purpose, it is only necessary to correct a large longitudinal stretch, and no particularly high projection accuracy is required.

[0024] Preferably, in step 4, the relationship between rows and columns of the corrected images is used to eliminate false matches, the corrected laser intensity image and the linear array image basically satisfy a linear relationship in the row and column directions, linear fitting is performed on the horizontal and vertical coordinate differences respectively, and the interquartile range IQR is used to eliminate false matches.

[0025] 3. Beneficial effects

[0026] Compared with the prior art, the advantages of the present invention are:

[0027] (1) In the present invention, the consistency of the laser intensity image and the optical image projection is achieved by reprojection, and then the laser intensity image and the optical image are accurately aligned by feature matching, and finally the unification of the two is achieved by a polynomial model, so that each laser point cloud can find the corresponding color information.

[0028] (2) In the present invention, the laser point cloud is fused with the optical image to simultaneously obtain the three-dimensional geometric shape and surface texture, thereby improving the visualization and comprehensibility of the point cloud data. In complex engineering and natural environments, the fused data can provide more detailed environmental perception information and can more accurately detect problems such as cracks, deformation and erosion on the tunnel wall. In addition, the fused laser point cloud data has significant advantages in computer vision and machine learning tasks. The point cloud data with color information can significantly improve the accuracy and reliability of tasks such as object recognition, classification and segmentation.

[0029] (3) In the present invention, different cross-section types, including circular, rectangular, horseshoe and other irregular contours, are flexibly adapted through piecewise polynomial fitting and other methods. The traditional laser point cloud coloring method is mainly based on array images, which usually requires high-precision geometric correspondence. The present method takes into account the similarity between laser cross-section data and linear array image data, and realizes the matching of linear array images and laser intensity images based on texture features, without the need for high-precision indoor calibration parameters; the feature matching results are used to accurately align the two in the mileage direction, and only the initial mileage alignment is required, without the need for high-precision manual mileage calibration. The method has a high degree of automation, and after the initial parameters are set, basically no human intervention is required. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The original laser intensity image (after cropping) (left) and the line array image (right) of the same area proposed by the present invention;

[0031] Figure 2 Laser intensity image (top) and line array image (bottom) after mileage correction proposed by the present invention (rotated 90 degrees counterclockwise);

[0032] Figure 3 The projection relationship between the tunnel inner wall linear array image and the laser intensity image proposed by the present invention (taking the shield tunnel as an example);

[0033] Figure 4 Laser intensity image (top) and line array image (bottom) after preliminary reprojection proposed by the present invention (rotated 90 degrees counterclockwise);

[0034] Figure 5The laser intensity image (top) and line array image (bottom) reprojected after feature matching proposed by the present invention (rotated 90 degrees counterclockwise);

[0035] Figure 6 The coordinate differences of the corresponding matching points in the horizontal and vertical directions before the mismatching is eliminated proposed by the present invention;

[0036] Figure 7 The coordinate difference of the corresponding matching points in the horizontal and vertical directions after eliminating the mismatch proposed by the present invention;

[0037] Figure 8 The feature matching result proposed by the present invention (rotated 90 degrees counterclockwise);

[0038] Fig. 9 Laser intensity image (top) and corresponding color image (bottom) proposed by the present invention (rotated 90 degrees counterclockwise);

[0039] Fig.10 The present invention provides a three-dimensional laser point cloud with color. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0041] Embodiment 1:

[0042] The laser point cloud coloring method based on linear array imaging includes the following steps:

[0043] Step 1: Scanning and preprocessing of linear array images. The laser intensity image is generated using the point cloud reflection intensity information of the laser section scanner. Each column of the image stores the laser intensity of all points in a section and is distributed column by column along the mileage direction. To ensure the resolution effect of the laser intensity image, the intensity value is mapped to the range of 0-255 based on the prior parameters through the piecewise linear fitting method to generate the laser intensity image. When collecting linear array images, multiple linear array cameras are often used to synchronously collect images of a single section, and the synchronous stitching of multi-camera images is achieved through matching and fusion methods. In order to ensure the uniform resolution of the linear array image in the section direction, different section types are flexibly adapted based on methods such as piecewise polynomial fitting to generate orthophotod linear array images.

[0044] Step 2: Align the image in the mileage direction. The linear array image and the cross-sectional laser intensity image represent the data of a cross section in each column and are arranged along the mileage direction. The data composition of the two is consistent, which can simplify the registration process. Under the premise that the carrier carrying the instrument runs at a uniform speed, the two are evenly distributed in the mileage direction and have a simple proportional relationship, such as Figure 1As shown. The resolution of the laser intensity image in the mileage direction is determined by the movement speed of the carrier car and the rotation speed of the scanner, and the resolution of the line array image in the mileage direction is determined by the movement speed of the carrier car and the sampling frequency of the line array camera, both of which are known quantities. Under normal acquisition conditions, the mileage resolution of the line array image is higher than that of the laser intensity image. The line array image is downsampled in the mileage direction based on the mileage resolution ratio of the two to achieve preliminary consistency of the mileage resolution. The mileage alignment of the two can be measured by an odometer, or by manually marking a starting mileage, and then performing preliminary alignment through the mileage resolution ratio, and then performing precise corrections through subsequent matching results to achieve accurate alignment in the mileage direction, as shown in Figure 2 shown.

[0045] Step 3: Correction of longitudinal stretching of the image. The linear array images of this system were reprojected and stitched during preprocessing, and the geometric structure of the imaging could not be preserved. Therefore, the solution was mainly based on texture information. The linear array images were evenly projected onto the inner wall of the tunnel during the preprocessing stage. The longitudinal resolution after processing was a known quantity. The projection center was regarded as the tunnel center, while the laser intensity image could be regarded as a projection with the scanning center of the scanner as the projection center. There was a large deviation between the projection methods of the two, and the image was greatly stretched in the longitudinal direction. Therefore, the geometric relationship of the projection was first used to realize the preliminary transformation of the linear array image so that its projection method was consistent with that of the laser intensity image, and the projection was further refined through feature matching.

[0046] like Figure 3 As shown, taking the shield tunnel as an example, O1 is the center of the scanner, O2 is the center of the tunnel, dy is the y coordinate of the two centers, p is any point on the inner wall of the tunnel, l is the distance from a point on the tunnel to the center of the tunnel, and θ1 and θ2 are the vertical angles of different centers. Since the linear array image has achieved uniform distribution of resolution through reprojection, and is symmetrical with respect to the top of the tunnel, the θ2 corresponding to each pixel can be obtained through the image resolution and the y coordinate of the pixel point in the image coordinate system. The corresponding geometric relationship is shown in Formula 1. The linear array image is reprojected to be consistent with the laser intensity image. The θ1 of each row of the image on the laser intensity image is calculated, and the corresponding θ2 is obtained row by row of reprojection. The conversion relationship is shown in Formula 2. The linear array image is reprojected according to the row pair relationship to reduce the difference between the two in the longitudinal direction. The result is as follows. Figure 4 shown.

[0047]

[0048]

[0049] To further improve the accuracy of image longitudinal stretch correction, feature matching can be performed and a polynomial correction model can be obtained based on the matching results. The laser calculates the corresponding lines of the reprojection line by line based on the correction model. The results are as follows: Figure 5 As shown in the figure, when the tunnel parameters do not change, only one reprojection parameter needs to be calculated to improve the calculation efficiency. Reprojection ensures that feature matching can obtain enough correct points of the same name. For this purpose, only large longitudinal stretching needs to be corrected, and no particularly high projection accuracy is required.

[0050] Step 4: Multi-source image feature matching. Image registration is based on the feature matching results of the two images. Since the original data is large, the matching is performed in segments in the mileage direction. The laser intensity image is acquired with a fixed number of columns, and the pre-processed linear array image of the corresponding area is acquired. To ensure the matching result, the linear array image needs to be widened by a certain number of columns. The SURF operator is used to calculate the key points and descriptors of the image, and then the BF (Brute-Force) violent feature matching method is used to perform feature matching to obtain the rough matching results of the feature matching.

[0051] Due to the huge difference in texture features between the laser intensity image and the optical image, there are a lot of mismatches. The traditional mismatch elimination method based on geometric relationships cannot meet the requirements. Therefore, the corrected image row and column relationship is used to eliminate mismatches. The corrected laser intensity image and the linear array image basically meet the linear relationship in the row and column directions, such as Figure 6 As shown, linear fitting is performed on the horizontal and vertical coordinate differences respectively, and the interquartile range (IQR) is used to eliminate false matches. The IQR method is not affected by the maximum or minimum values ​​and is often used to describe the degree of discreteness of non-normally distributed data. The IQR method divides a data set sorted by rank into quartiles, namely Q1, Q2, Q3, and Q4. IQR is defined as Q3-Q1, and the range of (Q1-1.5IQR, Q3+1.5IQR) is regarded as the valid range, and the data outside of it is regarded as outliers, thereby obtaining a set of valid feature matching points, such as Figure 7 and Figure 8 shown.

[0052] Step 5: Construct pixel coordinate transformation relationship, using a polynomial model that avoids the spatial geometric relationship of imaging to construct pixel coordinate transformation relationship. The polynomial model constructs a linear polynomial with multiple parameters, which directly simulates the image deformation itself mathematically, and regards the overall deformation of the image as the result of the combined effect of various basic deformations, see formula 3, bring the result of feature matching into the polynomial model, and use the least squares method for adjustment to solve the various parameters of the polynomial model.

[0053]

[0054] Using the transformation relationship model, starting from the output image (blank image) array of the same size as the laser intensity image, the corresponding position of each pixel in the line array image is calculated in turn, and then the pixel corresponding to the position of the line array image is calculated by resampling, and then the calculated pixel is assigned to the output image pixel, and finally a color image completely corresponding to the coordinates of the laser intensity image is obtained, such as Fig. 9 As shown in the figure, the image is used to directly color the point cloud. When generating the point cloud file, the color information is written into the standard LAS format file, such as Fig.10 shown.

[0055] In the present invention, the laser point cloud is fused with the optical image to obtain the three-dimensional geometric shape and surface texture at the same time, thereby improving the visualization and comprehensibility of the point cloud data. In complex engineering and natural environments, the fused data can provide more detailed environmental perception information and can more accurately detect problems such as cracks, deformation and erosion on the tunnel wall. In addition, the fused laser point cloud data has significant advantages in computer vision and machine learning tasks. The point cloud data with color information can significantly improve the accuracy and reliability of tasks such as object recognition, classification and segmentation.

[0056] In the present invention, different section types, including circular, rectangular, horseshoe and other irregular contours, are flexibly adapted through piecewise polynomial fitting and other methods. The traditional laser point cloud coloring method is mainly based on array images, which usually requires high-precision geometric correspondence. This method takes into account the similarity between laser section data and linear array image data, and realizes the matching of linear array images and laser intensity images based on texture features, without the need for high-precision indoor calibration parameters; the feature matching results are used to accurately align the two in the mileage direction, and only the initial mileage alignment is required, without the need for high-precision manual mileage calibration, with a high degree of automation. After setting the initial parameters, basically no human intervention is required.

[0057] In the present invention, the consistency of the laser intensity image and the optical image projection is achieved through reprojection, and then the laser intensity image and the optical image are accurately aligned using feature matching, and finally the unification of the two is achieved through a polynomial model, so that each laser point cloud can find the corresponding color information.

[0058] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A laser point cloud coloring method based on linear array imaging, characterized in that: The following steps are involved: Step 1: Scanning and linear array image preprocessing, using the point cloud reflection intensity information of the laser cross-section scanner to generate a laser intensity image. Each column of the image stores the laser intensity of all points in a cross section and is distributed column by column along the mileage direction. Step 2: Align the image in the mileage direction. The linear array image and the cross-sectional laser intensity image represent the data of a cross section in each column and are arranged along the mileage direction. The data composition of the two is consistent, which can simplify the registration process. Step 3: Correction of longitudinal stretching of the image. The linear array image was reprojected and stitched during preprocessing, and the geometric structure of the image was not preserved. Therefore, the solution was mainly based on texture information. Taking the shield tunnel as an example, O1 is the center of the scanner, O2 is the center of the tunnel, dy is the y coordinate of the two centers, p is any point on the inner wall of the tunnel, l is the distance from a point on the tunnel to the center of the tunnel, and θ1 and θ2 are the vertical angles of different centers. Since the linear array image has achieved uniform distribution of resolution through reprojection and is symmetrical with respect to the top of the tunnel, the θ2 corresponding to each pixel can be obtained through the image resolution and the y coordinate of the pixel point in the image coordinate system. The corresponding geometric relationship is shown in Formula 1. The linear array image is kept consistent with the laser intensity image through reprojection. The θ1 of each row of the image on the laser intensity image is calculated, and the corresponding θ2 is obtained line by line. The conversion relationship is shown in Formula 2. The linear array image is reprojected according to the row correspondence relationship, thereby reducing the difference between the two in the longitudinal direction. Step 4: Multi-source image feature matching. Image registration is based on the feature matching results of the two images. Since the original data is large, the matching is performed in segments in the mileage direction. The laser intensity image is obtained with a fixed number of columns, and the pre-processed linear array image of the corresponding area is obtained. To ensure the matching results, the linear array image needs to be widened by a certain number of columns. The SURF operator is used to calculate the key points and descriptors of the image, and then the BF brute force feature matching method is used to perform feature matching to obtain the rough matching results of the feature matching. Step 5: Construct pixel coordinate transformation relationship. Use a polynomial model that avoids the spatial geometric relationship of imaging to construct pixel coordinate transformation relationship. The polynomial model constructs a linear polynomial with multiple parameters, which directly simulates the image deformation itself mathematically. The overall deformation of the image is regarded as the comprehensive result of various basic deformations. See Formula 3. The result of feature matching is brought into the polynomial model, and the least squares method is used for adjustment to solve the parameters of the polynomial model. Using the transformation relationship model, starting from the output image array of the same size as the laser intensity image, the corresponding position of each pixel in the line array image is calculated in turn, and then the pixel corresponding to the position of the line array image is calculated by resampling, and then the calculated pixel is assigned to the output image pixel. Finally, a color image that completely corresponds to the coordinates of the laser intensity image is obtained, and the image is used to directly color the point cloud. When generating the point cloud file, the color information is written into the standard LAS format file.

2. The laser point cloud coloring method based on linear array imaging according to claim 1, characterized in that: In step 1, in order to ensure the resolution effect of the laser intensity image, the intensity value is mapped to the range of 0-255 by a piecewise linear fitting method based on the prior parameters to generate the laser intensity image.

3. The laser point cloud coloring method based on linear array imaging according to claim 2 is characterized in that: In step 1, the resolution of the linear array image in the cross-sectional direction is ensured to be uniform, and different cross-sectional types are flexibly adapted according to methods such as piecewise polynomial fitting to generate an orthophoto linear array image.

4. The laser point cloud coloring method based on linear array imaging according to claim 3 is characterized in that: Under normal acquisition conditions in step 2, the mileage resolution of the linear array image is higher than that of the laser intensity image. The linear array image is downsampled in the mileage direction according to the mileage resolution ratio between the two images, so that the initial consistency of the mileage resolution can be achieved. The mileage alignment of the two images can be measured by an odometer, or a starting mileage can be manually marked and then initially aligned by the mileage resolution ratio, and then accurately corrected by the subsequent matching results to achieve accurate alignment in the mileage direction.

5. The laser point cloud coloring method based on linear array imaging according to claim 1, characterized in that: In step 3, the geometric relationship of the projection is first used to achieve a preliminary transformation of the linear array image so that its projection mode is consistent with the laser intensity image, and the projection is further refined through feature matching.

6. The method for coloring laser point cloud based on linear array imaging according to claim 5, characterized in that: In step 3, in order to further improve the accuracy of the longitudinal stretch correction of the image, feature matching can be performed and a polynomial correction model can be obtained based on the matching results, and the corresponding reprojection rows are calculated row by row based on the correction model; when the tunnel parameters do not change, it is only necessary to calculate the reprojection parameters once to improve the calculation efficiency, and reprojection is performed to ensure that the feature matching obtains enough correct same-name points. For this purpose, it is only necessary to correct a large longitudinal stretch, and no particularly high projection accuracy is required.

7. The laser point cloud coloring method based on linear array imaging according to claim 1, characterized in that: In step 4, the relationship between rows and columns of the corrected images is used to eliminate false matches. The corrected laser intensity image and the linear array image basically satisfy a linear relationship in the row and column directions. Linear fitting is performed on the horizontal and vertical coordinate differences, respectively, and the interquartile range IQR is used to eliminate false matches.

Images