Roadway deformation intelligent monitoring and evaluation system

Abstract

This invention relates to an intelligent monitoring and evaluation system for roadway deformation, belonging to the field of roadway deformation monitoring technology. It includes: a data acquisition and processing module, which denoises the original point cloud data and image data to obtain point cloud data; a coarse registration module, which performs coarse registration on the point cloud data to obtain coarsely registered point cloud data; a feature extraction module, which acquires the geometric features of the point cloud for each valid point and the image features of its corresponding valid projection points, and fuses them to obtain the fused features of each valid point; a displacement field acquisition module, which generates the final displacement field of each point in the coarsely registered point cloud data; a slicing module, which performs sliding slicing on the standard point cloud data to obtain multiple continuous cross-sectional point clouds; a label configuration module, which divides the cross-sectional point cloud regions and configures key region labels; a risk index calculation module, which calculates the risk index of each cross-sectional point cloud; and a deformation evaluation module, which determines the roadway deformation evaluation result for each cross-sectional point cloud. This improves the comprehensiveness and accuracy of roadway deformation analysis.

Description

Technical Field

[0001] This invention relates to the field of tunnel deformation monitoring technology, and in particular to an intelligent monitoring and evaluation system for tunnel deformation. Background Technology

[0002] In underground operations such as coal mining, roadways serve as crucial passageways for underground production, transportation, and personnel access. Their structural stability is a core prerequisite for ensuring operational safety and production continuity. With increasing mining depth, the effects of ground stress become more pronounced, and the risk of roadway deformation due to mining activities continues to rise. Therefore, higher demands are placed on the accuracy and reliability of monitoring technologies. Currently, roadway deformation monitoring has gradually shifted towards a technical approach that integrates 3D laser point clouds and optical images. However, in complex underground environments, existing technologies still face many unresolved issues.

[0003] For example, in the multi-source data fusion stage, existing technologies struggle to guarantee the spatiotemporal consistency between 3D laser point clouds and optical images, resulting in insufficient spatial coordinate mapping accuracy. This leads to unreliable foundations for multi-source data association, affecting the accuracy of subsequent roadway deformation analysis. In the deformation registration stage, existing algorithms are limited by the assumption of rigid transformation, failing to adapt to the actual non-rigid deformations in roadways. This results in significant registration errors and even false deformation results, making it difficult to accurately reflect the true deformation state of the roadway. Finally, in the feature extraction stage, feature extraction techniques have limitations; the extracted features cannot fully characterize the complex deformations of the roadway, impacting the comprehensiveness and accuracy of roadway deformation analysis. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an intelligent monitoring and evaluation system for tunnel deformation. The technical solution of this invention is as follows:

[0005] The intelligent monitoring and evaluation system for tunnel deformation includes:

[0006] The data acquisition and processing module is used to acquire raw point cloud data and image data of the tunnel, and to denoise the raw point cloud data by improving the bilateral filtering algorithm to obtain point cloud data.

[0007] The coarse registration module is used to construct the mapping relationship between point cloud data and image data, and to perform coarse registration on the point cloud data based on the reference point cloud data and the mapping relationship to obtain coarsely registered point cloud data.

[0008] The feature extraction module is used to project all points in the coarse registration point cloud data, and determine multiple effective points in the coarse registration point cloud data and multiple effective projection points of the multiple effective points in the image plane where the image data is located based on the projection results. It obtains the point cloud geometric features of each effective point and the image features of each corresponding effective projection point, and fuses the point cloud geometric features of each effective point and the image features of its corresponding effective projection point to obtain the fused features of each effective point.

[0009] The displacement field acquisition module is used to calculate the initial displacement field of each point in the coarse registration point cloud data based on the fusion features of each valid point, and to perform global coarse adjustment and local intermediate adjustment of the initial displacement field of all points based on the reference point cloud data to obtain the final displacement field of each point in the coarse registration point cloud data.

[0010] The slicing module is used to generate standard point cloud data based on the final displacement field of each point in the coarse registration point cloud data, and to perform sliding slicing on the standard point cloud data according to the roadway direction to obtain multiple continuous cross-sectional point clouds, wherein each cross-sectional point cloud is composed of multiple standard points.

[0011] The label configuration module is used to divide the standard points in each cross-sectional point cloud into regions and configure key region labels for each standard point in each cross-sectional point cloud.

[0012] The risk indicator calculation module is used to calculate the risk indicator of each cross-sectional point cloud based on the key area label of each standard point in each cross-sectional point cloud.

[0013] The deformation evaluation module is used to determine the roadway deformation evaluation result for each cross-sectional point cloud based on the risk index of each cross-sectional point cloud.

[0014] Preferably, the data acquisition and processing module includes:

[0015] The data acquisition unit is used to simultaneously acquire raw point cloud data and image data of the tunnel using a high-precision synchronously triggered lidar and camera.

[0016] The denoising unit is used to distinguish between noise and real points in the original point cloud data by improving the bilateral filtering algorithm, and to remove noise from the original point cloud data to obtain the point cloud data.

[0017] Preferably, the coarse registration module includes:

[0018] The mapping relationship construction unit is used to establish a mapping relationship between point cloud data and image data based on camera information and LiDAR information;

[0019] The alignment unit is used to align the point cloud data with the image data according to the mapping relationship, and to eliminate error points in the point cloud data and the image data to obtain the first point cloud data.

[0020] The pose calculation unit is used to extract multiple image key points from the image data, match the multiple image key points with reference key points in the reference image data, and calculate the initial pose transformation matrix of the image data relative to the reference image data based on the matching results.

[0021] The registration unit is used to perform nearest-point registration between the first point cloud data and the reference point cloud data based on the initial pose transformation matrix. During the nearest-point registration process, the initial pose transformation matrix is ​​iterated based on the objective function to obtain the optimal pose transformation matrix. The coarsely registered point cloud data is determined based on the optimal pose transformation matrix and the first point cloud data.

[0022] Preferably, the mapping relationship construction unit establishes any target point in the point cloud data based on camera information and lidar information. Corresponding pixel in image data The mapping relationship between them is achieved through formulas (1) and (2):

[0023] (1);

[0024] (2);

[0025] In formula (1), For the camera's extrinsic parameter matrix, This represents the rotation matrix in the camera's extrinsic parameter matrix. This represents the translation vector in the camera's extrinsic parameter matrix; This represents the intrinsic parameter matrix of the camera. This represents the depth value of the target point in the camera coordinate system;

[0026] In formula (2), This represents the rotation matrix between the camera and the lidar. This represents the translation vector between the camera and the lidar. This represents the coordinates of the target point in the camera coordinate system after coordinate transformation.

[0027] Preferably, the feature extraction module includes:

[0028] The projection unit is used to project all points in the coarse registration point cloud data using camera information to obtain multiple projection points located on the image plane where the image data is located.

[0029] The filtering unit is used to filter multiple valid projection points from multiple projection points according to the mask, and to determine the valid point corresponding to each valid projection point in the coarse registration point cloud data according to the projection relationship.

[0030] The image feature extraction unit is used to extract image features for each effective projection point through a feature extraction convolutional neural network.

[0031] The point cloud geometric feature extraction unit is used to filter multiple sampling points from multiple valid points through sampling processing, extract the point cloud geometric features of each sampling point through a point cloud simplification processing algorithm, and finally expand the point cloud geometric features with the same number of sampling points to the same number of valid points through interpolation to obtain the point cloud geometric features of each valid point.

[0032] The feature fusion unit concatenates the point cloud geometric features of each valid point with the image features of each valid projection point in the channel dimension to obtain the initial fused features, and then performs dimensionality reduction on each initial fused feature based on the channel attention mechanism to obtain the fused features of each valid point.

[0033] Preferably, the displacement field acquisition module includes:

[0034] The initial displacement field acquisition unit is used to map the fusion features of each valid point through a multilayer perceptron to obtain the effective displacement field of each valid point. It performs interpolation processing on all other points in the coarse registration point cloud data except for the valid points to generate other displacement fields for each other point. The effective displacement fields of all valid points and the other displacement fields of all other points are used as the initial displacement field of each point in the coarse registration point cloud data.

[0035] The global coarse adjustment unit is used to calculate the optimal pose information between the reference point cloud data and the coarse registration point cloud data through the iterative nearest point algorithm, and to superimpose the optimal pose information onto the initial displacement field of each point to obtain the coarse adjustment displacement field of each point in the coarse registration point cloud data.

[0036] The local adjustment unit is used to calculate the residual between the coarse adjustment displacement field of each point in the coarse registration point cloud data and the average coarse adjustment displacement field of all its neighboring points. If the absolute value of the residual between the coarse adjustment displacement field of any point and the average coarse adjustment displacement field of all its neighboring points is greater than a preset threshold, the coarse adjustment displacement field of that point is superimposed with a preset adjustment amount and the iteration continues until a preset iteration termination condition is met to obtain the final displacement field of that point. The preset adjustment amount is the product of the residual of each point and the preset residual weight. Otherwise, the coarse adjustment displacement field of that point is taken as its final displacement field.

[0037] Preferably, the slicing module includes:

[0038] The standard point cloud data generation unit is used to superimpose the coordinates of each point in the coarse registration point cloud data with the corresponding final displacement field to obtain each standard point, and combine all standard points to obtain standard point cloud data.

[0039] The slicing unit is used to slide slice the standard point cloud data along the roadway direction according to the preset slice length and sliding step size to obtain multiple continuous cross-sectional point clouds.

[0040] Preferably, the label configuration module includes:

[0041] The center determination unit is used to calculate the center coordinates of the center of each cross-section point cloud in the lidar coordinate system according to the direction of the roadway, and to calculate the polar angle of the center coordinates of each cross-section point cloud and each standard point, so as to obtain the polar angle of each standard point in each cross-section point cloud.

[0042] The label configuration unit is used to configure key region labels for each standard point according to the polar angle of each standard point in each cross-sectional point cloud. The types of key region labels include top plate, bottom plate, left side and right side.

[0043] Preferably, the center determination unit calculates the center coordinates of the center of any cross-sectional point cloud in the lidar coordinate system. With any of its standard points polar angle This can be achieved through formulas (3) and (4):

[0044] (3);

[0045] (4);

[0046] In formula (3), Indicates the center of the point cloud of this cross section. In the X-axis coordinate, Indicates the center of the point cloud of this cross section. In the Z-axis coordinate, This indicates the number of standard points within the point cloud of that cross section. This represents the sum of the X-axis coordinates of all standard points within the point cloud of this cross-section. This represents the sum of the coordinates of all standard points within the point cloud of this cross section on the Z-axis. The origin of the lidar coordinate system is the lidar's transmission center, the X-axis represents the area directly above the tunnel, the Y-axis represents the tunnel's direction, and the Z-axis represents the direction perpendicular to the X and Y axes.

[0047] In formula (4), Representing standard points In the X-axis coordinate, Representing standard points In the Z-axis coordinate, This represents the two-parameter arctangent function.

[0048] Preferably, the risk indicators include convergence, roof subsidence, and torsion angle, and the risk indicator calculation module includes:

[0049] The convergence calculation unit is used to obtain the standard points labeled as left and right sides of the key regions in each cross-sectional point cloud. The coordinates of the leftmost standard point labeled as left side of the key region are taken as the left side boundary of each cross-sectional point cloud, and the coordinates of the rightmost standard point labeled as right side of the key region are taken as the right side boundary of each cross-sectional point cloud. The absolute value of the difference between the left side boundary and the right side boundary of each cross-sectional point cloud is taken as the cross-sectional width of each cross-sectional point cloud. The absolute value of the difference between the cross-sectional width of each cross-sectional point cloud and the cross-sectional width of the corresponding cross-sectional point cloud in the reference point cloud data is taken as the convergence amount of each cross-sectional point cloud.

[0050] The top plate subsidence calculation unit is used to take the absolute value of the difference between the average coordinate of all standard points labeled as top plate in the key area of ​​each cross-section point cloud and the preset top plate height in the corresponding cross-section point cloud data as the top plate subsidence of each cross-section point cloud.

[0051] The torsion angle calculation unit is used to perform plane fitting on all standard points in each cross-sectional point cloud to obtain the plane fitting equation of each cross-sectional point cloud, and obtain the normal vector of the plane fitting equation. The angle between the normal vector and the horizontal plane is taken as the torsion angle of each cross-sectional point cloud.

[0052] All of the above-mentioned optional technical solutions can be combined arbitrarily, and the present invention will not provide a detailed description of the structure after each combination.

[0053] By means of the above solution, the beneficial effects of the present invention are as follows:

[0054] The data acquisition and processing module synchronously acquires raw point cloud data and image data of the tunnel. After denoising the raw point cloud data, the point cloud data is obtained. The mapping relationship between the point cloud data and the image data is constructed by the coarse registration module. Then, the point cloud data is coarsely registered by combining the reference point cloud data to obtain coarsely registered point cloud data. By combining the advantages of point cloud data and image data through synchronous acquisition, the problem of spatiotemporal consistency between 3D laser point cloud and optical image is solved. The quality of tunnel point cloud data is effectively improved by denoising and coarse registration, providing an accurate data foundation for subsequent tunnel deformation analysis.

[0055] The feature extraction module filters multiple valid points from the coarsely registered point cloud data and their corresponding multiple valid projection points in the image plane of the image data. After obtaining the point cloud geometric features of each valid point and the image features of each corresponding valid projection point, the fusion is performed to obtain the fusion features of each valid point. This realizes the combination of point cloud geometric features and image features, and provides accurate and multi-source fusion features for subsequent tunnel deformation analysis.

[0056] The displacement field acquisition module calculates the initial displacement field based on the fusion characteristics of each effective point, and performs global coarse adjustment and local intermediate adjustment on all initial displacement fields to obtain the final displacement field. This can significantly improve the registration accuracy between coarse registration point cloud data and reference point cloud data, and provide a high-quality registration basis for subsequent roadway deformation analysis.

[0057] The slicing module generates standard point cloud data based on the final displacement field of each point in the coarse registration point cloud data. Then, it performs sliding slices on the standard point cloud data according to the roadway orientation to obtain multiple continuous cross-sectional point clouds. The label configuration module assigns key region labels to each standard point in each cross-sectional point cloud, and the risk index calculation module calculates the risk index for each cross-sectional point cloud based on the key region labels of each standard point. Since the sliced ​​cross-sectional point clouds contain both rigid and non-rigid deformation, this invention can accurately analyze the non-rigid deformation of the roadway, solving the problem of large registration errors caused by non-rigid deformation in existing technologies and improving the comprehensiveness and accuracy of roadway deformation analysis.

[0058] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the intelligent monitoring and evaluation system for roadway deformation provided in an embodiment of the present invention. Detailed Implementation

[0060] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0061] like Figure 1 As shown in the figure, this embodiment of the invention provides an intelligent monitoring and evaluation system for roadway deformation, including the following modules:

[0062] The data acquisition and processing module is used to acquire raw point cloud data and image data of the tunnel, and to denoise the raw point cloud data by improving the bilateral filtering algorithm to obtain point cloud data.

[0063] The coarse registration module is used to construct the mapping relationship between point cloud data and image data, and to perform coarse registration on the point cloud data based on the reference point cloud data and the mapping relationship to obtain coarsely registered point cloud data.

[0064] The feature extraction module is used to project all points in the coarse registration point cloud data, and determine multiple effective points in the coarse registration point cloud data and multiple effective projection points of the multiple effective points in the image plane where the image data is located based on the projection results. It obtains the point cloud geometric features of each effective point and the image features of each corresponding effective projection point, and fuses the point cloud geometric features of each effective point and the image features of its corresponding effective projection point to obtain the fused features of each effective point.

[0065] The displacement field acquisition module is used to calculate the initial displacement field of each point in the coarse registration point cloud data based on the fusion features of each valid point, and to perform global coarse adjustment and local intermediate adjustment of the initial displacement field of all points based on the reference point cloud data to obtain the final displacement field of each point in the coarse registration point cloud data.

[0066] The slicing module is used to generate standard point cloud data based on the final displacement field of each point in the coarse registration point cloud data, and to perform sliding slicing on the standard point cloud data according to the roadway direction to obtain multiple continuous cross-sectional point clouds, wherein each cross-sectional point cloud is composed of multiple standard points.

[0067] The label configuration module is used to divide the standard points in each cross-sectional point cloud into regions and configure key region labels for each standard point in each cross-sectional point cloud.

[0068] The risk indicator calculation module is used to calculate the risk indicator of each cross-sectional point cloud based on the key area label of each standard point in each cross-sectional point cloud.

[0069] The deformation evaluation module is used to determine the roadway deformation evaluation result for each cross-sectional point cloud based on the risk index of each cross-sectional point cloud.

[0070] Specifically, in the data acquisition and processing module, a precise mechanical structure is used to fix the lidar and camera to synchronously acquire raw point cloud data and image data of the tunnel. When acquiring raw point cloud data and image data, a high-precision synchronous triggering device is used to ensure the consistency of the two types of data in time.

[0071] In the coarse registration module, the mapping relationship refers to the coordinate transformation relationship between point cloud data and image data. Coarse registration aims to eliminate pose errors between point cloud data and reference point cloud data, facilitating subsequent roadway deformation analysis based on deformation differences in point cloud data at different times. Reference point cloud data refers to the point cloud data collected by the lidar before roadway mining, obtained after coarse registration. For example, if the lidar collects initial point cloud data on the first day of roadway mining and second initial point cloud data on the second day before mining, then the first and second initial point cloud data are coarsely registered, and the coarsely registered first point cloud data is used as the reference point cloud data. By obtaining the reference point cloud data through coarse registration of the first and second initial point cloud data, errors caused by lidar angle changes can be eliminated.

[0072] In the feature extraction module, valid points refer to points in the coarse-registered point cloud data that, after screening, have a good projection relationship with the image data. Valid projection points are the points on the image plane where the image data is located, projected onto the valid points according to the projection relationship. The projection relationship refers to a geometric correspondence established between the point cloud data and the image data, which is determined by camera information in this embodiment. Image features describe the visual characteristics of each valid projection point in the image data, including color, texture, shape, edges, and corners. In this embodiment, the image features are 256-dimensional. Point cloud geometric features are numerical features describing the spatial location, shape, and structural characteristics of each valid point in the coarse-registered point cloud data, including coordinates, normal vectors, curvature, and density. In this embodiment, the point cloud geometric features are 128-dimensional. The fusion feature combines the point cloud geometric features of each valid point in the coarse-registered point cloud data with the image features of its corresponding valid projection point to form a multimodal comprehensive feature representation of each valid point. In this embodiment, the fusion feature is 128-dimensional.

[0073] In the displacement field acquisition module, the initial displacement field refers to the initial displacement estimation vector of the coarse-registered point cloud data relative to the reference point cloud data, calculated based on the fusion features of each valid point. Global coarse adjustment involves globally adjusting the coarse-registered point cloud data against the reference point cloud data to eliminate registration errors in the initial displacement field across the entire global scope of the coarse-registered point cloud data. Local mid-adjustment involves adjusting the initial displacement field of each point in the globally coarse-registered point cloud data and its neighboring points to obtain the final displacement field for each point. The final displacement field is the displacement vector of the coarse-registered point data relative to the reference point cloud data. By adjusting the coordinates of each point in the coarse-registered point cloud data according to the final displacement field, standard point cloud data is obtained.

[0074] In the slicing module, sliding slicing refers to generating multiple continuous two-dimensional slices (i.e., cross-sectional point clouds) in standard point cloud data by sliding a window according to a certain step size and direction. The step size is generally 1m, and the direction is the tunnel direction.

[0075] In the label configuration module, the types of labels for key areas include top plate, bottom plate, left side and right side.

[0076] In the risk indicator calculation module, the risk indicator is an indicator that quantifies the degree of deformation of the roadway structure, including convergence, roof subsidence, and torsion angle.

[0077] In the deformation evaluation module, the roadway deformation evaluation result is the result obtained after comprehensive analysis and evaluation of the roadway deformation. For example, the risk level of the point cloud at the 10th cross section is Level II, the evaluation result is increased deformation, the observation frequency is increased to once a day, and the focus is on checking the integrity of the support.

[0078] In one specific embodiment, the data acquisition and processing module includes:

[0079] The data acquisition unit is used to simultaneously acquire raw point cloud data and image data of the tunnel using a high-precision synchronously triggered lidar and camera.

[0080] The denoising unit is used to distinguish between noise and real points in the original point cloud data by improving the bilateral filtering algorithm, and to remove noise from the original point cloud data to obtain the point cloud data.

[0081] Specifically, in the data acquisition unit, the mechanical structure that fixes the lidar and camera is triggered synchronously typically once a day. Spatially, the lidar and camera are fixed by a precise mechanical structure to ensure their relative positions are stable, providing a good foundation for subsequent spatiotemporal calibration.

[0082] When processing the original point cloud data using an improved bilateral filtering algorithm, the denoising unit first calculates the weight value of the difference in reflection intensity between each point in the original point cloud data and all points in its neighborhood. Specifically, in calculating the weight value of the first point in the original point cloud data... i The weighted value of the difference in reflection intensity between a point and all points in its neighborhood is achieved by formula (5):

[0083] (5);

[0084] In formula (5), This represents the first point in the original point cloud data. i The weighted value of the difference in reflection intensity between a point and all points in its neighborhood; This represents the first point in the original point cloud data. i One point, This indicates the first point in the original point cloud data.i Within the neighborhood of point 1 j One point; Indicates the first i The point and its neighborhood of the i-th point j Spatial distance between points; Let represent the distance weight kernel, and satisfy . ;in, This represents the distance coefficient, which is adaptively adjusted based on the density of the original point cloud data. When the density of the original point cloud data is high... Take the smaller value, and vice versa. Take the larger value. Generally within the range of 0.01-0.1; The intensity weight kernel represents the weight values ​​assigned to all points in the neighborhood of each point in the original point cloud data based on the difference in reflection intensity. Using the Gaussian function, the formula is: ,in, The first point cloud data i The reflection intensity at each point The first point cloud data i Within the neighborhood of point 1 j The reflection intensity of each point is a parameter that the lidar directly measures and records when acquiring raw point cloud data. It does not require additional calculation and is one of the inherent attributes of the raw point cloud data. No. i Preset smoothing coefficient for each point; normalization factor The calculation strictly follows This process ensures that the filtered original point cloud data does not experience an overall shift. Indicates the first i The set consisting of all points within the domain of a given point; express In formula (5), the smaller the difference in reflection intensity between a point and all points in its neighborhood, the larger the weight value of the difference in reflection intensity assigned to that point. This allows for better retention of real points with smaller weight values ​​of the difference in reflection intensity between the point and its neighbors during filtering, while suppressing noise points with larger weight values ​​of the difference in reflection intensity between the point and all points in its neighborhood.

[0085] Furthermore, the denoising unit distinguishes all points in the original point cloud data into real points and noise points according to a preset filtering weight threshold. Specifically, points in the original point cloud data whose weight value of the difference in reflection intensity is greater than the preset filtering weight threshold are classified as noise and removed. The preset filtering weight threshold is a historical empirical value for distinguishing between real points and noise points, determined based on historical point cloud data.

[0086] In one specific embodiment, the coarse registration module includes:

[0087] The mapping relationship construction unit is used to establish a mapping relationship between point cloud data and image data based on camera information and LiDAR information;

[0088] The alignment unit is used to align the point cloud data with the image data according to the mapping relationship, and to eliminate error points in the point cloud data and the image data to obtain the first point cloud data.

[0089] The pose calculation unit is used to extract multiple image key points from the image data, match the multiple image key points with reference key points in the reference image data, and calculate the initial pose transformation matrix of the image data relative to the reference image data based on the matching results.

[0090] The registration unit is used to perform nearest-point registration between the first point cloud data and the reference point cloud data based on the initial pose transformation matrix. During the nearest-point registration process, the initial pose transformation matrix is ​​iterated based on the objective function to obtain the optimal pose transformation matrix. The coarsely registered point cloud data is determined based on the optimal pose transformation matrix and the first point cloud data.

[0091] Specifically, in the mapping relationship construction unit, camera information includes the camera's extrinsic parameter matrix and intrinsic parameter matrix. LiDAR information includes the pose information between the camera and the LiDAR.

[0092] When aligning point cloud data with image data according to the mapping relationship, the alignment unit first maps each point in the point cloud data to the image data according to the mapping relationship, obtaining the mapping point of each point in the point cloud data in the image data. Each mapping point is paired with the nearest point in the image data to form a point pair. Then, the pixel error of each point pair is calculated using the least squares method. The corresponding points in the point cloud data of the mapping points with errors exceeding 0.5 pixels are regarded as error points and deleted.

[0093] In the pose calculation unit, image keypoints refer to points that are salient and easily identifiable in the image data. Image keypoints are found in the image data using the SIFT algorithm. The SIFT algorithm for finding image keypoints employs a multi-scale spatial extremum detection method. Specifically, a Gaussian pyramid is constructed at different scales, with each octave (a unit of Gaussian pyramid size) containing images at five scales. A difference Gaussian pyramid is obtained by performing a difference operation on the reference image data and the image data. Local extrema are detected on the difference Gaussian pyramid as initial image keypoints. After steps such as localization and orientation assignment, the final image keypoints with scale and rotation invariance in the image data are obtained. The reference image data consists of images of the tunnel before mining, captured by a camera and acquired at the same time as the reference point cloud data. Any image keypoint can be represented as... ,in, The scale at which the key points of the image are located; The main direction of the key points in the image (reflecting the rotation angle of the features). and These represent the x-coordinate and y-coordinate of the key points in the image in the camera coordinate system, respectively.

[0094] When calculating the initial pose transformation matrix, the pose calculation unit obtains the transformation matrix between any pair of keypoints and uses this transformation matrix as the initial pose transformation matrix of the image data relative to the reference image data. Specifically, a pair of keypoints is defined as those that are present in both the reference and reference image data and have the highest similarity to their pixel values. The initial pose transformation matrix includes the rotation matrix and translation vector of the image data relative to the reference image data.

[0095] In the registration unit, nearest-point registration is achieved by iteratively minimizing the objective function, which can be expressed as: ;in, This represents the rotation matrix in the initial pose transformation matrix. This represents the translation vector in the initial pose transformation matrix. This represents any resource point in the first cloud data. This represents the reference point corresponding to the resource point in the reference point cloud data. In each iteration, the resource point closest to the resource point is selected to continue the iteration. The iteration process calculates the optimal rotation matrix and translation matrix by minimizing the objective function, and then continues to search for the reference point corresponding to the next resource point in the reference point cloud data until the number of iterations is completed and the optimal pose transformation matrix is ​​obtained. Generally, the number of iterations is set to 20-50 times to lay a good foundation for subsequent fine registration.

[0096] It should be noted that in this embodiment of the invention, the mapping between image data and point cloud data only addresses the correspondence between point cloud data and image data at the same time. That is, it binds point cloud data and image data at the same time. For example, for a pixel in the image data, its corresponding 3D coordinates in the point cloud data can be found. However, this only establishes the association rules between the two types of data and does not involve comparing data from different times. This is the basis for data fusion. Coarse registration, on the other hand, aligns and compares point cloud data from different times, such as aligning and comparing point cloud data from the first day and the second day. The purpose is to calculate the deformation difference between the two times, which is a prerequisite for deformation analysis.

[0097] In one specific embodiment, the mapping relationship construction unit establishes any target point in the point cloud data based on camera information and LiDAR information. Corresponding pixel in image data The mapping relationship between them is achieved through formulas (1) and (2):

[0098] (1);

[0099] (2);

[0100] In formula (1), For the camera's extrinsic parameter matrix, This represents the rotation matrix in the camera's extrinsic parameter matrix. This represents the translation vector in the camera's extrinsic parameter matrix; This represents the intrinsic parameter matrix of the camera. This represents the depth value of the target point in the camera coordinate system;

[0101] In formula (2), This represents the rotation matrix between the camera and the lidar. This represents the translation vector between the camera and the lidar. This represents the coordinates of the target point in the camera coordinate system after coordinate transformation.

[0102] It should be noted that the camera's intrinsic parameter matrix needs to be obtained through a rigorous camera calibration process. The camera's intrinsic parameter matrix consists of intrinsic parameters such as focal length and principal point coordinates. A high-precision calibration board is used during the calibration of the intrinsic parameters, and the calibration is repeated more than 10 times. The average value of the intrinsic parameters from all calibrations is then taken to reduce errors.

[0103] In one specific embodiment, the feature extraction module includes:

[0104] The projection unit is used to project all points in the coarse registration point cloud data using camera information to obtain multiple projection points located on the image plane where the image data is located.

[0105] The filtering unit is used to filter multiple valid projection points from multiple projection points according to the mask, and to determine the valid point corresponding to each valid projection point in the coarse registration point cloud data according to the projection relationship.

[0106] The image feature extraction unit is used to extract image features for each effective projection point through a feature extraction convolutional neural network.

[0107] The point cloud geometric feature extraction unit is used to filter multiple sampling points from multiple valid points through sampling processing, extract the point cloud geometric features of each sampling point through a point cloud simplification processing algorithm, and finally expand the point cloud geometric features with the same number of sampling points to the same number of valid points through interpolation to obtain the point cloud geometric features of each valid point.

[0108] The feature fusion unit concatenates the point cloud geometric features of each valid point with the image features of each valid projection point in the channel dimension to obtain the initial fused features, and then performs dimensionality reduction on each initial fused feature based on the channel attention mechanism to obtain the fused features of each valid point.

[0109] Specifically, in the projection unit, for any point in the coarsely aligned point cloud data... After combining the camera information and projecting this point onto the image plane containing the image data, the pixel coordinates of the corresponding projected point are: ,in: , ;in, and This refers to the camera's focal length (in pixels) on the horizontal and vertical axes, as shown in the camera information. and The principal point coordinates of the image data are inherent parameters in the camera information; this formula represents the projection relationship.

[0110] Projection points include valid and invalid projection points. The filtering unit filters invalid projection points from all projection points based on the mask. Specifically, the filtering unit judges any point... The formula for determining whether a point is an invalid projection point is: ;in, and These are the width and height of the image data, respectively. This condition is to exclude points that are close together. This represents the filtering result obtained by masking. The mask is a binary image of the same size as the image data, which filters all points projected from the coarse registration point cloud data onto the image plane containing the image data using two pixel values. After masking, the number of valid projected points retained is N' (N'≤N, where N is the number of points in the coarse registration point cloud data), where the valid projected points are important structural points of the roadway. The number of valid projected points is the same as the number of valid points. For example, a LiDAR can scan points in image blind spots that a camera cannot capture. The mask can filter and remove these points, leaving important structural points of the roadway and avoiding errors in subsequent cross-modal feature fusion. Suppose that the coarse registration point cloud data of a certain roadway includes 8000 points, but 500 of them correspond to image blind spots. After masking, these 500 points are further removed, and finally 7500 points are taken as valid points.

[0111] In the image feature extraction unit, the feature extraction convolutional neural network uses ResNet34 to extract features from effective projection points. The input of the feature extraction convolutional neural network is image data, and the output is a feature map. The ResNet34 input... Image data (3 for RGB channels) is compressed in stages using residual blocks. The four stages are as follows: Stage 1: The convolutional kernel has a stride of 2, and the output size is... The stride of the convolutional kernel in stage 2 is 2, and the output size is [missing value]. The stride of the stage 3 convolutional kernel is 2, and the output size is... The stride of the convolutional kernel in stage 4 is 2, and the output size is... After four downsampling steps with a step size of 2, the final output size is The feature map. Here and This refers to the size of the image data after downsampling. This downsampling process reduces the amount of data while preserving the key structural visual features of the image data. Furthermore, since the size of the feature map is 1 / 16 of the image data, any effective projection point needs to be scaled to the size of the feature map. Specifically, this involves scaling the pixel coordinates of the effective projection point (...). U , V ) Scale proportionally to the size of the feature map, i.e. , .because and The value may be non-integer, so bilinear interpolation is needed to sample the features at the corresponding location from the feature map to obtain the image features of the effective projection point with a dimension of 256. Each valid projection point corresponds to an image feature.

[0112] In the point cloud geometric feature extraction unit, one valid point is randomly selected from N' valid points as the initial sampling point. Then, the valid point farthest from the initial sampling point is selected as the second sampling point. Next, the valid point farthest from the second sampling point is selected as the next sampling point, until the number of sampling points reaches M (M=N' / 32, e.g., 100 sampling points are taken when N'=3200, 32 is an adjustable parameter), ensuring that all sampling points uniformly cover the entire range of the coarse registration point cloud data. Next, Ball Query grouping is performed, specifically: a spherical neighborhood with a radius of 0.2m is defined with each sampling point as the center (the setting is based on the density of the coarse registration point cloud data, ensuring that each sphere contains 16-32 points), and the valid points of the coarse registration point cloud data within the same spherical neighborhood are grouped together. Finally, MLP (Multilayer Perceptron) aggregation is used. For points within each group, features are extracted and aggregated through a 3-layer MLP: the first layer maps the point cloud coordinates of valid points to 64-dimensional features; the second layer increases the dimensionality to 128 dimensions; and the third layer uses max pooling to maximize the 128-dimensional features within the group, obtaining the global features of the sampled points. Since there are only M sampled points, KNN interpolation (taking the 3 nearest sampled points) is used to expand the M features to N' features, ultimately outputting the 128-dimensional point cloud geometric features for each valid point. .

[0113] In the feature fusion unit, the point cloud geometric features of any valid point are concatenated with the image features of the corresponding valid projection point along the channel dimension to enhance the consistency of the local structure. Specifically, during fusion, the point cloud geometric features of the valid point are first... (Dimension 128) and image features (256-dimensional) concatenated into a feature vector of dimension 384. Then, a one-dimensional convolutional neural network is used to compress the number of channels to 128 dimensions, resulting in an initial fused feature vector of dimension 128. Next, the initial fused features are mapped to query vectors through a linear layer. (Dimension 64). Next, the feature map output by ResNet34 is flattened into a dimension of... The features are then mapped to bonds through a linear layer. (dimension is) ) and value (dimension is) Then, a mask is introduced (consistent with the mask in the above embodiment, used to filter invalid image regions), and the association weight of each dimension feature is calculated, as follows: ;in, Indicates attention weights, It is an activation function used to normalize the weights to a range of 0 to 1. It is a scaling factor, designed to prevent gradient problems caused by excessively large weight values. As a mask. Values ​​are adjusted using attention weights. Weighted summation yields a local fusion feature with dimension 64. The local fusion features can reflect the position of the projection point in the overall space of the tunnel. Next, the initial fusion features... With local fusion features The features are concatenated into a 192-dimensional feature set, and then gating weights of dimension 128 are generated using an MLP (192→128) and a sigmoid activation function (the weight values ​​of the gating weights are between 0 and 1). Finally, the initial fused features for this valid point are... With local fusion features The weighted fusion formula is as follows: ,in, This indicates the preset fusion weight coefficients, and the final output is the fusion feature with a dimension of 128 for this valid point. The fusion characteristics of other effective points can be obtained similarly.

[0114] In one specific embodiment, the displacement field acquisition module includes:

[0115] The initial displacement field acquisition unit is used to map the fusion features of each valid point through a multilayer perceptron to obtain the effective displacement field of each valid point. It performs interpolation processing on all other points in the coarse registration point cloud data except for the valid points to generate other displacement fields for each other point. The effective displacement fields of all valid points and the other displacement fields of all other points are used as the initial displacement field of each point in the coarse registration point cloud data.

[0116] The global coarse adjustment unit is used to calculate the optimal pose information between the reference point cloud data and the coarse registration point cloud data through the iterative nearest point algorithm, and to superimpose the optimal pose information onto the initial displacement field of each point to obtain the coarse adjustment displacement field of each point in the coarse registration point cloud data.

[0117] The local adjustment unit is used to calculate the residual between the coarse adjustment displacement field of each point in the coarse registration point cloud data and the average coarse adjustment displacement field of all its neighboring points. If the absolute value of the residual between the coarse adjustment displacement field of any point and the average coarse adjustment displacement field of all its neighboring points is greater than a preset threshold, the coarse adjustment displacement field of that point is superimposed with a preset adjustment amount and the iteration continues until a preset iteration termination condition is met to obtain the final displacement field of that point. The preset adjustment amount is the product of the residual of each point and the preset residual weight. Otherwise, the coarse adjustment displacement field of that point is taken as its final displacement field.

[0118] Specifically, in the initial displacement field acquisition unit, the effective displacement field of each effective point is calculated based on the fusion features. Specifically, a bottleneck-type MLP (128→64→32→3) is used to map the fusion features of any effective point, and the bottleneck-type MLP outputs the displacement vector of that effective point. , here This is the initial displacement field of the valid point. For other points in the coarsely registered point cloud data that did not pass the mask screening, the KNN-32 algorithm (i.e., taking the 32 nearest valid points) is used to interpolate each other point, and the other displacement fields of each other point are used as the initial displacement fields.

[0119] When the global coarse adjustment unit calculates the optimal pose information between the reference point cloud data and the coarse registration point cloud data using the iterative nearest point algorithm, it performs ICP calculation for each point pair between the reference point cloud data and the coarse registration point cloud data (any point in the coarse registration point cloud data and its corresponding reference point in the reference point cloud data form a point pair). The formula is as follows: ;in, This represents the Iterative Closest Point Algorithm, used to calculate the optimal rotation matrix and optimal translation vector (optimal pose information) for any point pair between the reference point cloud data and the coarsely registered point cloud data. This represents any point in the coarsely registered point cloud data. This indicates the reference point corresponding to this point in the reference point cloud data. and Let represent the optimal rotation matrix and the optimal translation vector, respectively. Superimposing the optimal rotation matrix and the optimal translation vector onto the initial displacement field at that point yields the coarse-tuned displacement field at that point, which can be expressed as: ,in, This represents the coarse-tuned displacement field at that point. This represents the initial displacement field at that point. It should be noted that since the optimal rotation matrix and optimal translation vector are determined based on the reference point cloud data, the coarse-adjusted displacement field of each point refers to the coarse-adjusted displacement field of each point relative to each corresponding reference point in the reference point cloud data.

[0120] In the local mid-level adjustment unit, for a point in the coarse registration point cloud data, the residual of the average coarse adjustment displacement between that point and its 32 neighboring points is calculated using the following formula: ,in, This represents the average coarse-adjustment displacement field of the 32 neighboring points of this point (i.e., the average value of the coarse-adjustment displacement field of the 32 neighboring points). This represents the coarse-tuned displacement field at that point. This represents the residual. If... If a significant bias is considered to exist, then the residual weights are preset. ( The value of (between 0 and 1) is used to adjust the coarse displacement field at this point. The adjustment formula is: ,in, This represents the coarse-adjusted displacement field at that point. If the absolute value of the residual between the coarse-adjusted displacement field at that point and the average coarse-adjusted displacement field of all its neighboring points is greater than a preset threshold, the adjustment continues using the adjustment formula until the preset iteration termination condition is met. Generally, three iterations are repeated to gradually correct the deviation. The first iteration mainly corrects deviations greater than 5 mm, while the third iteration corrects small deviations of 1-2 mm. Ultimately, the registration error is controlled within 1-3 mm, while retaining true inelastic local mutations.

[0121] In one specific embodiment, the slicing module includes:

[0122] The standard point cloud data generation unit is used to superimpose the coordinates of each point in the coarse registration point cloud data with the corresponding final displacement field to obtain each standard point, and combine all standard points to obtain standard point cloud data.

[0123] The slicing unit is used to slide slice the standard point cloud data along the roadway direction according to the preset slice length and sliding step size to obtain multiple continuous cross-sectional point clouds.

[0124] Specifically, in the slicing unit, the preset slice length is 1m, and the sliding step size is 0.5m. An overlap strategy (overlap rate of 50%) ensures the data continuity between adjacent cross-sectional slices, which is beneficial for subsequent statistical smoothing processing. Each cross-sectional point cloud contains all standard points, the final displacement field of all standard points, and the identifier of that cross-sectional point cloud. The identifier is generally the slice number 0, 1, 2, ...

[0125] In one specific embodiment, the label configuration module includes:

[0126] The center determination unit is used to calculate the center coordinates of the center of each cross-section point cloud in the lidar coordinate system according to the direction of the roadway, and to calculate the polar angle of the center coordinates of each cross-section point cloud and each standard point, so as to obtain the polar angle of each standard point in each cross-section point cloud.

[0127] The label configuration unit is used to configure key region labels for each standard point according to the polar angle of each standard point in each cross-sectional point cloud. The types of key region labels include top plate, bottom plate, left side and right side.

[0128] Specifically, in the center determination unit, the center coordinates of a certain cross-sectional point cloud refer to the coordinates of the geometric center of the cross-sectional point cloud, and the polar angle is the angle of a certain standard point in the cross-sectional point cloud relative to the center coordinates of the cross-sectional point cloud.

[0129] In the tag configuration unit, based on polar angle θ When configuring four key region labels for each standard point, when the polar angle of a certain standard point... When the critical area label for a standard point is set to the top plate, the polar angle of a certain standard point is... When the critical area label configured for this standard point is the base plate; when the polar angle of a certain standard point is... When the critical area label for a standard point is set to left, the polar angle of a standard point is set to left. The key area label configured for this standard point is the right side.

[0130] In one specific embodiment, the center determination unit calculates the center coordinates of the center of any cross-sectional point cloud in the lidar coordinate system. With any of its standard points polar angle This can be achieved through formulas (3) and (4):

[0131] (3);

[0132] (4);

[0133] In formula (3), Indicates the center of the point cloud of this cross section. In the X-axis coordinate, Indicates the center of the point cloud of this cross section. In the Z-axis coordinate, This indicates the number of standard points within the point cloud of that cross section. This represents the sum of the X-axis coordinates of all standard points within the point cloud of this cross-section. This represents the sum of the coordinates of all standard points within the point cloud of this cross section on the Z-axis. The origin of the lidar coordinate system is the lidar's transmission center, the X-axis represents the area directly above the tunnel, the Y-axis represents the tunnel's direction, and the Z-axis represents the direction perpendicular to the X and Y axes.

[0134] In formula (4), Representing standard points In the X-axis coordinate, Representing standard points In the Z-axis coordinate, This represents a two-parameter arctangent function, where the X-axis represents the area directly above the tunnel, the Y-axis represents the direction of the tunnel, and the Z-axis represents the direction perpendicular to the X and Y axes.

[0135] In one specific embodiment, the risk indicators include convergence amount, roof subsidence amount, and torsion angle, and the risk indicator calculation module includes:

[0136] The convergence calculation unit is used to obtain the standard points labeled as left and right sides of the key regions in each cross-sectional point cloud. The coordinates of the leftmost standard point labeled as left side of the key region are taken as the left side boundary of each cross-sectional point cloud, and the coordinates of the rightmost standard point labeled as right side of the key region are taken as the right side boundary of each cross-sectional point cloud. The absolute value of the difference between the left side boundary and the right side boundary of each cross-sectional point cloud is taken as the cross-sectional width of each cross-sectional point cloud. The absolute value of the difference between the cross-sectional width of each cross-sectional point cloud and the cross-sectional width of the corresponding cross-sectional point cloud in the reference point cloud data is taken as the convergence amount of each cross-sectional point cloud.

[0137] The top plate subsidence calculation unit is used to take the absolute value of the difference between the average coordinate of all standard points labeled as top plate in the key area of ​​each cross-section point cloud and the preset top plate height in the corresponding cross-section point cloud data as the top plate subsidence of each cross-section point cloud.

[0138] The torsion angle calculation unit is used to perform plane fitting on all standard points in each cross-sectional point cloud to obtain the plane fitting equation of each cross-sectional point cloud, and obtain the normal vector of the plane fitting equation. The angle between the normal vector and the horizontal plane is taken as the torsion angle of each cross-sectional point cloud.

[0139] Specifically, in the convergence calculation unit, the convergence reflects the relative compression deformation of the left and right sides of the roadway. When calculating the convergence of any cross-sectional point cloud, the extreme values ​​of the X-axis coordinates of the standard points labeled as the left and right sides in the key regions of the cross-sectional point cloud are first obtained. Specifically, the left side boundary... ;in, This indicates the X-axis coordinates of the d-th standard point on the left slab in the point cloud of the cross-section; the right slab boundary. ;

[0140] in: This indicates the X-axis coordinates of the k-th standard point labeled as the right flank in the key region point cloud of this cross-section. Then, the cross-sectional width of this point cloud is calculated. The calculation formula is: Finally, the convergence of the point cloud along this cross section is calculated. The formula is: ;in, The cross-sectional width of the corresponding cross-sectional point cloud in the reference point cloud data.

[0141] In the roof subsidence calculation unit, the roof subsidence reflects the vertical deformation of the roadway roof. When calculating the roof subsidence of a certain cross-sectional point cloud, the preset roof height of the corresponding cross-sectional point cloud in the reference point cloud data is first calculated. The calculation formula is: ,in, This indicates that the key region label in the point cloud of this section is the sum of the Z-axis coordinates of all reference points in the reference point cloud data corresponding to all standard points on the top plate. This represents the Z-axis coordinate of the reference point corresponding to the m-th standard point in the cross-sectional point cloud data. This indicates the number of reference points included in the corresponding cross-sectional point cloud data; then, the mean Z-axis coordinates of all standard points labeled "top plate" in the key region of this cross-sectional point cloud are calculated. The calculation formula is: ,in, This indicates the number of standard points in the point cloud of this cross-section whose key region is labeled as the top plate. The Z-axis coordinate of the m-th standard point in the cross-sectional point cloud is given; finally, the top plate subsidence of the cross-sectional point cloud is calculated. The calculation formula is: The larger the value of the top plate subsidence, the more severe the vertical deformation of the top plate in the cross-sectional point cloud.

[0142] In the twist angle calculation unit, the plane fitting equation for the point cloud of any cross section can be expressed as: The normal vector of the plane fitting equation is Then, the angle between the normal vector of the cross-sectional point cloud and the horizontal plane (0,0,0) is calculated as the twist angle of the cross-sectional point cloud.

[0143] In one specific embodiment, the deformation evaluation module determines the roadway deformation evaluation result of each cross-sectional point cloud based on the risk index of each cross-sectional point cloud and a preset risk classification threshold table.

[0144] Specifically, the preset risk grading threshold table is a table that stores the relationship between risk indicators and predetermined risk levels, as well as the correlation between evaluation results. Risk levels are generally divided into four levels. The deformation evaluation module compares the risk indicators of each cross-sectional point cloud with the preset risk grading threshold table and determines the roadway deformation evaluation result based on the comparison results. The specific content of the preset risk grading threshold table can be found in Table 1. The risk indicators, risk levels, cross-sectional point cloud identifiers, and evaluation results are integrated and output to obtain the roadway deformation evaluation result for each cross-sectional point cloud.

[0145]

[0146] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A roadway deformation intelligent monitoring and evaluation system, characterized in that, include: The data acquisition and processing module is used to acquire raw point cloud data and image data of the tunnel, and to denoise the raw point cloud data by improving the bilateral filtering algorithm to obtain point cloud data. The coarse registration module is used to construct the mapping relationship between point cloud data and image data, and to perform coarse registration on the point cloud data based on the reference point cloud data and the mapping relationship to obtain coarsely registered point cloud data. The feature extraction module is used to project all points in the coarse registration point cloud data, and determine multiple effective points in the coarse registration point cloud data and multiple effective projection points of the multiple effective points in the image plane where the image data is located based on the projection results. It obtains the point cloud geometric features of each effective point and the image features of each corresponding effective projection point, and fuses the point cloud geometric features of each effective point and the image features of its corresponding effective projection point to obtain the fused features of each effective point. The displacement field acquisition module is used to calculate the initial displacement field of each point in the coarse registration point cloud data based on the fusion features of each valid point, and to perform global coarse adjustment and local intermediate adjustment of the initial displacement field of all points based on the reference point cloud data to obtain the final displacement field of each point in the coarse registration point cloud data. The slicing module is used to generate standard point cloud data based on the final displacement field of each point in the coarse registration point cloud data, and to perform sliding slicing on the standard point cloud data according to the roadway direction to obtain multiple continuous cross-sectional point clouds, wherein each cross-sectional point cloud is composed of multiple standard points. The label configuration module is used to divide the standard points in each cross-sectional point cloud into regions and configure key region labels for each standard point in each cross-sectional point cloud. The risk indicator calculation module is used to calculate the risk indicator of each cross-sectional point cloud based on the key area label of each standard point in each cross-sectional point cloud. The deformation evaluation module is used to determine the roadway deformation evaluation result for each cross-sectional point cloud based on the risk index of each cross-sectional point cloud.

2. The intelligent monitoring and evaluation system for roadway deformation according to claim 1, characterized in that, The data acquisition and processing module includes: The data acquisition unit is used to simultaneously acquire raw point cloud data and image data of the tunnel using a high-precision synchronously triggered lidar and camera. The denoising unit is used to distinguish between noise and real points in the original point cloud data by improving the bilateral filtering algorithm, and to remove noise from the original point cloud data to obtain the point cloud data.

3. The intelligent monitoring and evaluation system for roadway deformation according to claim 1 or 2, characterized in that, The coarse registration module includes: The mapping relationship construction unit is used to establish a mapping relationship between point cloud data and image data based on camera information and LiDAR information; The alignment unit is used to align the point cloud data with the image data according to the mapping relationship, and to eliminate error points in the point cloud data and the image data to obtain the first point cloud data. The pose calculation unit is used to extract multiple image key points from the image data, match the multiple image key points with reference key points in the reference image data, and calculate the initial pose transformation matrix of the image data relative to the reference image data based on the matching results. The registration unit is used to perform nearest-point registration between the first point cloud data and the reference point cloud data based on the initial pose transformation matrix. During the nearest-point registration process, the initial pose transformation matrix is ​​iterated based on the objective function to obtain the optimal pose transformation matrix. The coarsely registered point cloud data is determined based on the optimal pose transformation matrix and the first point cloud data.

4. The intelligent monitoring and evaluation system for roadway deformation according to claim 3, characterized in that, The mapping relationship construction unit establishes any target point in the point cloud data based on camera information and LiDAR information. Corresponding pixel in image data The mapping relationship between them is achieved through formulas (1) and (2): （1）； （2）； In formula (1), For the camera's extrinsic parameter matrix, This represents the rotation matrix in the camera's extrinsic parameter matrix. This represents the translation vector in the camera's extrinsic parameter matrix; This represents the intrinsic parameter matrix of the camera. This represents the depth value of the target point in the camera coordinate system; In formula (2), This represents the rotation matrix between the camera and the lidar. This represents the translation vector between the camera and the lidar. This represents the coordinates of the target point in the camera coordinate system after coordinate transformation.

5. The intelligent monitoring and evaluation system for roadway deformation according to claim 1, characterized in that, The feature extraction module includes: The projection unit is used to project all points in the coarse registration point cloud data using camera information to obtain multiple projection points located on the image plane where the image data is located. The filtering unit is used to filter multiple valid projection points from multiple projection points according to the mask, and to determine the valid point corresponding to each valid projection point in the coarse registration point cloud data according to the projection relationship. The image feature extraction unit is used to extract image features for each effective projection point through a feature extraction convolutional neural network. The point cloud geometric feature extraction unit is used to filter multiple sampling points from multiple valid points through sampling processing, extract the point cloud geometric features of each sampling point through a point cloud simplification processing algorithm, and finally expand the point cloud geometric features with the same number of sampling points to the same number of valid points through interpolation to obtain the point cloud geometric features of each valid point. The feature fusion unit concatenates the point cloud geometric features of each valid point with the image features of each valid projection point in the channel dimension to obtain the initial fused features, and then performs dimensionality reduction on each initial fused feature based on the channel attention mechanism to obtain the fused features of each valid point.

6. The intelligent monitoring and evaluation system for roadway deformation according to claim 1, characterized in that, The displacement field acquisition module includes: The initial displacement field acquisition unit is used to map the fusion features of each valid point through a multilayer perceptron to obtain the effective displacement field of each valid point. It performs interpolation processing on all other points in the coarse registration point cloud data except for the valid points to generate other displacement fields for each other point. The effective displacement fields of all valid points and the other displacement fields of all other points are used as the initial displacement field of each point in the coarse registration point cloud data. The global coarse adjustment unit is used to calculate the optimal pose information between the reference point cloud data and the coarse registration point cloud data through the iterative nearest point algorithm, and to superimpose the optimal pose information onto the initial displacement field of each point to obtain the coarse adjustment displacement field of each point in the coarse registration point cloud data. The local adjustment unit is used to calculate the residual between the coarse adjustment displacement field of each point in the coarse registration point cloud data and the average coarse adjustment displacement field of all its neighboring points. If the absolute value of the residual between the coarse adjustment displacement field of any point and the average coarse adjustment displacement field of all its neighboring points is greater than a preset threshold, the coarse adjustment displacement field of that point is superimposed with a preset adjustment amount and the iteration continues until a preset iteration termination condition is met to obtain the final displacement field of that point. The preset adjustment amount is the product of the residual of each point and the preset residual weight. Otherwise, the coarse adjustment displacement field of that point is taken as its final displacement field.

7. The intelligent monitoring and evaluation system for roadway deformation according to claim 1, characterized in that, The slicing module includes: The standard point cloud data generation unit is used to superimpose the coordinates of each point in the coarse registration point cloud data with the corresponding final displacement field to obtain each standard point, and combine all standard points to obtain standard point cloud data. The slicing unit is used to slide slice the standard point cloud data along the roadway direction according to the preset slice length and sliding step size to obtain multiple continuous cross-sectional point clouds.

8. The intelligent monitoring and evaluation system for roadway deformation according to claim 1, characterized in that, The label configuration module includes: The center determination unit is used to calculate the center coordinates of the center of each cross-section point cloud in the lidar coordinate system according to the direction of the roadway, and to calculate the polar angle of the center coordinates of each cross-section point cloud and each standard point, so as to obtain the polar angle of each standard point in each cross-section point cloud. The label configuration unit is used to configure key region labels for each standard point according to the polar angle of each standard point in each cross-sectional point cloud. The types of key region labels include top plate, bottom plate, left side and right side.

9. The intelligent monitoring and evaluation system for roadway deformation according to claim 8, characterized in that, The center determination unit calculates the center coordinates of the center of any cross-sectional point cloud in the lidar coordinate system. With any of its standard points polar angle This can be achieved through formulas (3) and (4): （3）； （4）； In formula (3), Indicates the center of the point cloud of this cross section. In the X-axis coordinate, Indicates the center of the point cloud of this cross section. In the Z-axis coordinate, This indicates the number of standard points within the point cloud of that cross section. This represents the sum of the X-axis coordinates of all standard points within the point cloud of this cross-section. This represents the sum of the coordinates of all standard points within the point cloud of this cross section on the Z-axis. The origin of the lidar coordinate system is the lidar's transmission center, the X-axis represents the area directly above the tunnel, the Y-axis represents the tunnel's direction, and the Z-axis represents the direction perpendicular to the X and Y axes. In formula (4), Representing standard points In the X-axis coordinate, Representing standard points In the Z-axis coordinate, This represents the two-parameter arctangent function.

10. The intelligent monitoring and evaluation system for roadway deformation according to claim 9, characterized in that, The risk indicators include convergence, roof subsidence, and torsion angle. The risk indicator calculation module includes: The convergence calculation unit is used to obtain the standard points labeled as left and right sides of the key regions in each cross-sectional point cloud. The coordinates of the leftmost standard point labeled as left side of the key region are taken as the left side boundary of each cross-sectional point cloud, and the coordinates of the rightmost standard point labeled as right side of the key region are taken as the right side boundary of each cross-sectional point cloud. The absolute value of the difference between the left side boundary and the right side boundary of each cross-sectional point cloud is taken as the cross-sectional width of each cross-sectional point cloud. The absolute value of the difference between the cross-sectional width of each cross-sectional point cloud and the cross-sectional width of the corresponding cross-sectional point cloud in the reference point cloud data is taken as the convergence amount of each cross-sectional point cloud. The top plate subsidence calculation unit is used to take the absolute value of the difference between the average coordinate of all standard points labeled as top plate in the key area of ​​each cross-section point cloud and the preset top plate height in the corresponding cross-section point cloud data as the top plate subsidence of each cross-section point cloud. The torsion angle calculation unit is used to perform plane fitting on all standard points in each cross-sectional point cloud to obtain the plane fitting equation of each cross-sectional point cloud, and obtain the normal vector of the plane fitting equation. The angle between the normal vector and the horizontal plane is taken as the torsion angle of each cross-sectional point cloud.

Images