Tunnel surrounding rock modeling and all types of rock mass structural surface identification method and device

Abstract

The present invention discloses a method and device for modeling tunnel surrounding rock and identifying all types of rock structure surfaces, the method comprising: using depth cameras arranged at multiple angles to obtain image data of tunnel surrounding rock, the image frames obtained after processing include point cloud models with color data; based on color consistency and depth consistency, post-registering the point cloud model to obtain a complete tunnel point cloud model; obtaining the geometric posture of the tunnel point cloud model, converting it into color data, and mapping it to a two-dimensional image to obtain a geometric posture image; identifying the degree of association between pixels and neighborhoods in the geometric posture image, determining the neighborhood to which the pixels belong based on the degree of association, extracting surrounding rock surface semantic areas reflecting similar geological features; merging surrounding rock surface semantic areas to obtain rock structure surfaces. The above technical solution takes into account the diversity of geometric features and texture features, overcomes the differences in structural surface morphology and the influence of lighting, and realizes accurate and comprehensive extraction of various types of structural surfaces.

Description

Technical Field

[0001] The present invention relates to the technical field of photogrammetry and geological analysis, and in particular to a method and a device for modeling tunnel surrounding rock and identifying all types of rock mass structural surfaces. Background Art

[0002] Existing methods for extracting rock mass structure information mainly include the following two types: (1) contact extraction, such as manual measurement. This type of method requires manual field survey. The advantage is that it can ensure high accuracy, but the time cost is high. In addition, due to the structural characteristics of the tunnel itself, there will be safety risks if personnel stay in the tunnel for too long. (2) non-contact extraction, which is based on the three-dimensional model obtained by photogrammetry. Photogrammetry can quickly obtain a three-dimensional model of the rock mass, meeting the requirements of speed, accuracy and safety.

[0003] The existing non-contact extraction methods have certain shortcomings in practical applications. Due to the complex internal environment and structure of the tunnel and the changeable lighting conditions, the brightness of the inner surface of the tunnel is uneven, and the color is different from the actual color. This has a great negative impact on the recognition method based on image data, resulting in low segmentation accuracy and large errors in the structural surface. Summary of the invention

[0004] Purpose of the invention: The present invention provides a method and device for modeling tunnel surrounding rock and identifying all types of rock structure surfaces, aiming to solve the technical problems existing in the prior art of low accuracy of structure surface segmentation due to the complex and large differences in the morphology of rock structure surfaces and poor lighting conditions.

[0005] Technical solution: The present invention provides a method for modeling tunnel surrounding rock and identifying all types of rock structure surfaces, including: using depth cameras arranged at multiple angles to acquire image data of tunnel surrounding rock, constructing a point cloud model based on the depth data of each image frame in the image data, attaching the corresponding color data in the image data to the point cloud model, and obtaining an image frame of the image data including a point cloud model with color data; performing a preliminary registration on the point cloud model of the image frame, and performing a post-registration on the point cloud model based on the color consistency and depth consistency between the point cloud models to obtain a complete tunnel point cloud model; acquiring the geometric posture of the tunnel point cloud model, and converting the geometric posture into The color data is obtained and mapped to a two-dimensional image to obtain a geometric posture image; the degree of correlation between the geometric posture features of the pixels in the geometric posture image and the neighborhood is identified, and the neighborhood to which the pixel belongs is determined according to the degree of correlation, thereby extracting the surrounding rock surface semantic areas reflecting similar geological characteristics on the geometric posture image; based on the similarity between the surrounding rock surface semantic areas, the surrounding rock surface semantic areas are merged to obtain the rock structure surface, and the tunnel depth map and RGB image are transferred and learned based on the stacked ResNet model to identify the linear exposed cracks in the tunnel, and the surrounding rock surface structural surface and the linear exposed cracks are merged to obtain a comprehensive rock structure surface recognition result.

[0006] Specifically, after sharpening and bilateral filtering the image frame, the three-dimensional coordinates of the point cloud model of the image frame are preliminarily determined according to the parameters of the depth camera, and the initial alignment of the point cloud models of image frames at different angles is completed.

[0007] Specifically, the color of the point cloud model is converted into grayscale, and a first point cloud model and a second point cloud model whose original positions are known are selected in the point cloud model; the transformation matrix is ​​used to transform the first point cloud model to obtain a second point cloud model; after the transformation of the transformation matrix, the grayscale consistency and depth consistency between the points in the first point cloud model and the corresponding points in the second point cloud model are calculated, and the consistency and value of the grayscale consistency and the depth consistency are calculated; based on the consistency and value, the transformation matrix is ​​adjusted to obtain the optimal transformation matrix, and the point cloud models of image frames at different angles are post-registered based on the optimal transformation matrix.

[0008] Specifically, the grayscale of the point cloud model is calculated using the following formula:

[0009] P C (m c )=P(c)+G c T m,

[0010] Among them, P C (m c ) represents the grayscale of point c in point cloud model C, m c represents the moving direction of point c on the corresponding virtual section, P(c) represents the grayscale of point c on the point cloud model C, Gc Indicates P C (m c ), the superscript T indicates transposition;

[0011] The depth of the point cloud model is calculated using the following formula:

[0012] D C (m c )=(O C -c) T n c ,

[0013] Among them, D C (m c ) represents the depth of point c in the point cloud model C, O C Represents the origin of the depth camera corresponding to the point cloud model C, n c Represents the normal vector of the virtual section corresponding to point c;

[0014] Grayscale consistency is calculated using the following formula;

[0015] Δ z (S z )=(P A (m b’ )-(P B (m b )) 2 ,

[0016] Among them, Δ z (S z ) represents the transformation matrix S z The grayscale difference between the transformed point b' of the first point cloud model A and the corresponding point b of the second point cloud model under the transformation of ;

[0017] The depth consistency is calculated using the following formula;

[0018] Δ v (S v )=(D A (m b’ )-(D B (m b )) 2 ,

[0019] Among them, Δ v (S v ) represents the transformation matrix S v The depth difference between the transformed point b' of the first point cloud model A and the corresponding point b of the second point cloud model under the transformation of ;

[0020] The consistency and value are calculated using the following formula:

[0021] Δ(S)=μΔz (S z )+σΔ v (S v ),

[0022] Among them, Δ(S) represents the sum of the grayscale difference and the depth difference under the transformation matrix S, and μ and σ represent the weights of the grayscale difference and the depth difference, respectively.

[0023] Specifically, the inclination, inclination and direction of the points in the tunnel point cloud model are obtained and converted into corresponding colors respectively; the mapping relationship between the points in the point cloud model and the two-dimensional image is determined according to the parameters of the depth camera, and the geometric posture image is mapped.

[0024] Specifically, convolution and pooling are performed on the geometric posture image to generate an inclination feature map, an inclination feature map, and a strike feature map; the correlation degree matrices between pixels in the geometric posture image and adjacent superpixels in the inclination feature map, adjacent superpixels in the inclination feature map, and adjacent superpixels in the strike feature map are calculated respectively to obtain an inclination correlation matrix, an inclination correlation matrix, and a strike feature correlation matrix; the correlation matrices are summed up to obtain a fusion matrix, and the adjacent superpixels with the highest correlation with the pixels in the geometric posture image are determined according to the fusion matrix, thereby determining the neighborhood to which the pixels in the geometric posture image belong.

[0025] Specifically, the attribute similarity and spatial similarity between the surrounding rock surface semantic areas are calculated, and the similarity between the surrounding rock surface semantic areas is calculated by weighted summation. If the similarity is higher than the standard similarity, the corresponding surrounding rock surface semantic areas are merged; the attribute similarity is calculated by the geometric posture similarity between the surrounding rock surface semantic areas; the spatial similarity is calculated by the distance between the centroids of the point cloud models corresponding to the surrounding rock surface semantic areas.

[0026] Specifically, the undulation of each point in the surrounding rock surface semantic area is calculated, and the part with undulation higher than the standard undulation is excluded.

[0027] Specifically, the linear structure recognition model is used to identify the depth data and color data, and the identified linear exposed cracks are merged into the rock mass structural surface.

[0028] The present invention also provides a tunnel surrounding rock modeling and all-type rock structure surface recognition device, including: an image acquisition unit, a registration unit, a mapping unit, a surrounding rock surface extraction unit and a structure surface acquisition unit, wherein: the image acquisition unit is used to acquire image data of the tunnel surrounding rock using depth cameras arranged at multiple angles, build a point cloud model based on the depth data of each image frame in the image data, attach the corresponding color data in the image data to the point cloud model, and the image frame of the obtained image data includes a point cloud model with color data; the registration unit is used to perform initial registration on the point cloud model of the image frame, based on the color consistency and depth consistency between the point cloud models, The point cloud model is post-registered to obtain a complete tunnel point cloud model; the mapping unit is used to obtain the geometric posture of the tunnel point cloud model, convert the geometric posture into color data, and map it to a two-dimensional image to obtain a geometric posture image; the surrounding rock surface extraction unit is used to identify the degree of association between the geometric posture features of the pixels and the neighborhood in the geometric posture image, determine the neighborhood to which the pixels belong according to the degree of association, and thus extract the surrounding rock surface semantic areas reflecting similar geological features on the geometric posture image; the structural surface acquisition unit is used to merge the surrounding rock surface semantic areas based on the similarity between the surrounding rock surface semantic areas to obtain the rock mass structural surface.

[0029] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: based on the registration of point cloud models by color and depth, similar surrounding rock surfaces are merged based on geometric posture geological characteristics to obtain rock mass structural surfaces, taking into account the diversity of geometric characteristics and texture characteristics, overcoming the differences in structural surface morphology and the influence of lighting, and realizing accurate and comprehensive extraction of various types of structural surfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of the flow of the tunnel surrounding rock modeling and all types of rock mass structural surface identification method provided by the present invention;

[0031] Figure 2 A schematic diagram of a depth camera acquisition method provided by the present invention;

[0032] Figure 3 A schematic diagram of the process of integrated learning of the fused stacked AINet model provided by the present invention. DETAILED DESCRIPTION

[0033] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0034] See also Figure 1 , which is a flow chart of the tunnel surrounding rock modeling and all types of rock mass structural surface identification method provided by the present invention.

[0035] In an embodiment of the present invention, image data of the tunnel surrounding rock is acquired using depth cameras arranged at multiple angles, a point cloud model is constructed based on the depth data of each image frame in the image data, and the corresponding color data in the image data is attached to the point cloud model. The image frames (RGB-D frames) of the obtained image data include a point cloud model with color data.

[0036] See also Figure 2 , which is a schematic diagram of the depth camera acquisition method provided by the present invention.

[0037] In a specific implementation, the tunnel can be divided into multiple sections, and each section is used to obtain its image data with a set of depth camera sequences arranged at multiple angles. The multi-angle depth camera sequence advances forward to obtain complete tunnel surrounding rock image data. The image data includes depth data and color data. Each depth camera shoots an angle range, and the angle ranges of adjacent depth cameras can overlap.

[0038] In the specific implementation, the point cloud model is generated by the depth map data obtained by the depth camera. The point cloud model has no color texture information. According to the built-in calibration parameters of the depth camera, the corresponding RGB texture information can be obtained for each depth pixel. By fusing the synchronized depth data with the RGB data, a depth point cloud model with texture can be generated. Regarding the RGB-D (D stands for Depth) frame, the original image data collected by the depth camera is encapsulated video data. The video is extracted into multiple frames of RGB color data and depth image data according to the frame rate at the time of acquisition. Each frame of data corresponds to a point cloud model segment. The RGB-D frame is a set of RGB and depth data in the acquired data. The pose of the image frame corresponds to the pose of the point cloud model segment. After obtaining the pose of the point cloud model segment, the point cloud model can be registered. The point cloud model segments of the image frames of each depth camera at multiple angles are fused to obtain a complete tunnel point cloud model.

[0039] In the embodiment of the present invention, the point cloud model of the image frame is initially registered, and based on the color consistency and depth consistency between the point cloud models, the point cloud model is post-registered to obtain a complete tunnel point cloud model.

[0040] In the embodiment of the present invention, after the image frame is sharpened and bilaterally filtered, the three-dimensional coordinates of the point cloud model of the image frame are preliminarily determined according to the parameters of the depth camera, and the initial alignment of the point cloud models of image frames at different angles is completed.

[0041] In the specific implementation, the acquired tunnel image is sharpened to highlight the edge and detail information in the image, and the Laplace operator can be used for sharpening. The Laplace operator can enhance the clarity of the edge of the rock structure surface and improve the accuracy of structure surface extraction.

[0042] In the specific implementation, the sharpened image is then subjected to bilateral filtering to reduce the noise introduced by sharpening and further improve the sharpening effect. After bilateral filtering, the photometric optimization work before feature matching can be completed. The processed image is subjected to wide baseline feature matching through the ORB algorithm to obtain the approximate posture of the depth camera in the RGB-D frame. Based on the principle of the depth camera, according to the relative position relationship inside the depth camera, the posture of the depth camera is converted into the posture of the point cloud, that is, the three-dimensional coordinates of the point cloud model of the image frame are preliminarily determined, and the initial registration of the point cloud model can be completed. After completing the initial rough registration, the RANSAC algorithm is used for filtering to remove the noise of the point cloud model, and then post-registration is performed.

[0043] In the specific implementation, the image enhancement method of image sharpening and image filtering is used, and the tunnel is reasonably divided into multiple sections and regions for data acquisition to adapt to the point cloud model reconstruction of the tunnel scene, overcoming the problem that the existing RGB-D frame reconstruction algorithm is greatly affected by lighting.

[0044] In an embodiment of the present invention, the color of the point cloud model is converted into grayscale, and a first point cloud model and a second point cloud model whose original positions are known are selected in the point cloud model; the first point cloud model and the second point cloud model usually come from image frames of depth cameras arranged at different angles; the transformation matrix S is used to transform the first point cloud model to obtain the second point cloud model; after the transformation of the transformation matrix, the grayscale consistency and depth consistency between the points in the first point cloud model and the corresponding points in the second point cloud model are calculated, and the consistency and value of the grayscale consistency and the depth consistency are calculated; based on the consistency and value, the transformation matrix is ​​adjusted to obtain the optimal transformation matrix, and the point cloud models of image frames at different angles are post-registered based on the optimal transformation matrix.

[0045] In the specific implementation, in RGB-D point cloud processing, grayscale intensity is a scalar representation extracted from the color information (RGB value) of the three-dimensional point cloud in order to simplify the calculation. Grayscale intensity is regarded as a continuous function P(c) in actual processing, where c represents a point in the point cloud. Due to the discrete nature of the point cloud, directly calculating the grayscale at each point will lead to discontinuity and computational complexity, so it is necessary to approximate the grayscale function so that it changes continuously in space. In the point cloud, the grayscale intensity is a discrete distribution, and the virtual section can be used to approximate the continuous change of the grayscale intensity in the local area. By projecting the points in the point cloud onto the virtual section, the local discrete error can be basically eliminated, and more accurate calculations can be performed on this basis. The virtual section is determined based on the position of the point in the point cloud model and the virtual section normal vector. Define the point cloud model with color as C, the point in the point cloud model as c, and the point on the point cloud model as discrete points. Since discrete points are difficult to calculate with functions, the grayscale function P(c) corresponding to the image frame of the two-dimensional plane is approximated as a continuous function, and the grayscale function P(c) is extended from the two-dimensional plane image frame to the three-dimensional space. After being extended to the three-dimensional space, it is equivalent to introducing a virtual section for each point, converting the grayscale of the point into the grayscale information of the section, which is convenient for calculation. The virtual section is determined based on the position of the corresponding point and the normal vector of the virtual section. m represents the moving direction of point c on the virtual section, that is, the direction of the projection point of point c to other points in its neighborhood on the section plane, and n represents the normal vector of the virtual section corresponding to point c, which is obtained by local fitting of the point cloud. The normal vector of the section determines the direction of the section, and also indirectly determines whether the section is the best fitting plane for the local area of ​​point c.

[0046] In the embodiment of the present invention, the grayscale of the point cloud model C is calculated using the following formula:

[0047] P C (m c )=P(c)+G c T m,

[0048] Among them, P C (m c ) represents the grayscale of point c in the point cloud model C approximated by the virtual section, m c represents the moving direction of point c on the virtual section, that is, the direction from point c to the projection point of other points in its neighborhood on the virtual section. P(c) represents the grayscale of point c on the point cloud model C. G c Indicates P C (m c ) reflects the rate of change of grayscale at point c, and the superscript T indicates transposition;

[0049] In the specific implementation, in order to calculate the gradient G c, through the local neighborhood F of point c c , based on the least squares method, the gradient estimate is obtained. The specific method is as follows:

[0050] Let Y(x) be the function that projects a three-dimensional point x to the virtual tangent plane of point c. Y(x) is expressed as:

[0051] Y(x)=xn(xc) T n,

[0052] The grayscale gradient fitting problem is expressed as minimizing the error term W(G c ),

[0053] .

[0054] In the embodiment of the present invention, the depth of the point cloud model C is calculated using the following formula:

[0055] D C (m c )=(O C -c) T n c ,

[0056] Among them, D C (m c ) represents the depth of point c in the point cloud model C, O C Represents the origin of the depth camera corresponding to the point cloud model C, n c represents the normal vector of the virtual section corresponding to point c; since the gradient of the depth data at the origin of the depth camera is 0, c' represents a point in the neighborhood of point c, so D C (m c ) can be expressed as a linear function.

[0057] In the embodiment of the present invention, the grayscale consistency is calculated using the following formula:

[0058] Δ z (S z )=(P A (m b’ )-(P B (m b )) 2 ,

[0059] Among them, Δ z (S z ) represents the transformation matrix S z The grayscale difference between the transformed point b' of the first point cloud model A and the corresponding point b of the second point cloud model under the transformation matrix S. Point b' is the grayscale difference between the transformed point b' of the first point cloud model A and the corresponding point b of the second point cloud model. z After transformation, the point corresponding to point b in the second point cloud model B. Grayscale difference Δ z(S z ) is smaller, indicating higher consistency. A (m b’ ) and P B (m b ) refers to the above grayscale calculation formula.

[0060] In the embodiment of the present invention, the depth consistency is calculated using the following formula:

[0061] Δ v (S v )=(D A (m b’ )-(D B (m b )) 2 ,

[0062] Among them, Δ v (S v ) represents the transformation matrix S v The depth difference between the transformed point b' of the first point cloud model A and the corresponding point b of the second point cloud model under the transformation of . Depth difference Δ v (S v ) is smaller, indicating higher consistency. A (m b’ ) and D B (m b ) refers to the above depth calculation formula.

[0063] In the embodiment of the present invention, the following formula is used to calculate the consistency and value:

[0064] Δ(S)=μΔ z (S z )+σΔ v (S v ),

[0065] Among them, Δ(S) represents the sum of grayscale difference and depth difference under the transformation matrix S, μ and σ represent the weights of grayscale difference and depth difference respectively, satisfying μ+σ=1. Δ(S) can be minimized by optimization calculation, and the corresponding transformation matrix S is used as the optimal transformation matrix.

[0066] In a specific implementation, the transformation matrix represents the three-dimensional positional relationship between the first point cloud model and the second point cloud model, thereby the point cloud models can be registered and fused to obtain a complete tunnel point cloud model.

[0067] In the specific implementation, similarly, grayscale features and depth features are used to complete the fusion and establishment of the point cloud model, taking into account the diversity of geometric features and texture features, which can overcome the influence of complex structural surfaces and lighting changes.

[0068] In the embodiment of the present invention, the geometric posture of the tunnel point cloud model is obtained, the geometric posture is converted into color data, and mapped to a two-dimensional image to obtain a geometric posture image.

[0069] In an embodiment of the present invention, the geometric posture (inclination, tilt and direction) of the points in the tunnel point cloud model is obtained and converted into corresponding colors respectively; the mapping relationship between the points in the point cloud model and the two-dimensional image is determined according to the parameters of the depth camera, and the geometric posture image is mapped.

[0070] In a specific implementation, the direction represents the changing direction of points in the point cloud model.

[0071] In the specific implementation, the dip range is (-90°~90°), the inclination range is (0°~360°), and the strike range is (0°~360°). Each point in the point cloud model has dip, inclination and strike information. After calculating the dip, inclination and strike of the point, the obtained dip, inclination and strike values ​​are normalized into color ranges, and colors are used to display the differences in dip, inclination and strike, and to distinguish different areas.

[0072] In the specific implementation, the conversion relationship between the point cloud model and the image space is determined according to the internal and external parameters of the depth camera, and the projection method is determined. The semantic information of geological features represented by colors is projected into the image space using the conversion relationship to obtain the semantic information of geological features that corresponds one to one at the pixel level. Each pixel includes semantic information represented by three colors (inclination, dip and strike).

[0073] In the specific implementation, subsequent structural surface recognition, extraction and merging based on geometric posture can also overcome the negative impact of structural surface morphology differences and poor lighting conditions on structural surface recognition. Specifically, structural surface morphology differences, uneven brightness caused by lighting changes, and color differences from actual conditions will have a negative impact on structural surface recognition.

[0074] See also Figure 3 , which is a schematic diagram of the process of integrated learning of the fused stacked AINet (convolutional neural network) model provided by the present invention.

[0075] In an embodiment of the present invention, the degree of association between the geometric posture features of pixels and neighborhoods in a geometric posture image is identified, and the neighborhood to which the pixel belongs is determined based on the degree of association, thereby extracting surrounding rock surface semantic areas reflecting similar geological features on the geometric posture image.

[0076] In an embodiment of the present invention, convolution and pooling are performed on a geometric posture image to generate an inclination feature map, an inclination feature map, and a trend feature map; correlation degree matrices between pixels in the geometric posture image and adjacent superpixels in the inclination feature map, adjacent superpixels in the inclination feature map, and adjacent superpixels in the trend feature map are calculated respectively to obtain an inclination correlation matrix, an inclination correlation matrix, and a trend feature correlation matrix; the correlation matrices are summed to obtain a fusion matrix, and the adjacent superpixels with the highest correlation with the pixels in the geometric posture image are determined based on the fusion matrix, thereby determining the neighborhood to which the pixels in the geometric posture image belong.

[0077] In the specific implementation, the sampling interval is set to n, and the input geometric posture image is subjected to multiple convolutions and maximum pooling compression to generate three feature maps of the grid cells containing geological feature semantics, namely, the dip feature map, the inclination feature map, and the strike feature map.

[0078] In a specific implementation, the fused stacked AINet model is a semantic segmentation model that is more suitable for tunnels after the AINet model is modified by the present invention. The original AINet model has only one input channel. On this basis, multiple input channels are added to comprehensively consider different types of data, so that the superpixel segmentation effect of tunnel data is better.

[0079] In specific implementations, superpixels refer to segmenting an image into coherent regions that are larger than pixels but smaller than the entire image. Each superpixel typically contains a group of pixels with similar color, texture, or brightness characteristics, and is as compact and regular in shape as possible.

[0080] In a specific implementation, for any of the inclination feature map, the tilt feature map, and the strike feature map, for example, the inclination feature map, the fused stacked AINet model combines each pixel in the geometric posture image with its surrounding 9 superpixels from the inclination feature map to form a 3×3×D matrix SP:

[0081] ,

[0082] Among them, e p represents the pixels in the geometric pose image, represents a superpixel, and its subscript represents the relative pixel e p position.

[0083] The matrix SP can be converted into a 1×1×9 matrix by a 3×3 convolution, and the 1×1×9 matrix is ​​used as the pixel e p The dip angle correlation matrix of the surrounding adjacent superpixels is calculated in the same way as above.

[0084] In the specific implementation, the weighted sum of the dip angle correlation matrix, the inclination correlation matrix and the strike feature correlation matrix is ​​calculated to obtain a fusion matrix, which represents which neighboring superpixel has the highest similarity between the pixel and the surrounding pixel, and can determine the neighboring superpixel with the highest similarity and divide it into the corresponding area. The above steps are performed one by one on the pixels in the geometric posture image to determine the area to which each pixel belongs, thereby obtaining the surrounding rock surface semantic area.

[0085] In the embodiment of the present invention, based on the similarity between the surrounding rock surface semantic regions, the surrounding rock surface semantic regions are merged to obtain the rock mass structural surface.

[0086] In an embodiment of the present invention, the attribute similarity and spatial similarity between the surrounding rock surface semantic regions are calculated, and the similarity between the surrounding rock surface semantic regions is calculated by weighted summation. If the similarity is higher than the standard similarity, the corresponding surrounding rock surface semantic regions are merged; the attribute similarity is calculated by the geometric posture similarity between the surrounding rock surface semantic regions; the spatial similarity is calculated by the distance between the centroids of the point cloud models corresponding to the surrounding rock surface semantic regions.

[0087] In a specific implementation, the geometric posture similarity between surrounding rock surface semantic areas can be calculated in a variety of ways, such as calculating the mean of the geometric posture (weighted sum of dip, inclination and strike, or calculating them separately) of the surrounding rock surface semantic areas, and determining the similarity based on the degree of deviation of the mean.

[0088] In the embodiment of the present invention, the undulation of each point in the surrounding rock surface semantic area is calculated, and the portion with a undulation higher than the standard undulation is excluded.

[0089] In the specific implementation, according to the tunnel point cloud model, the point cloud shows the degree of fluctuation calculated by the degree of change of the point cloud normal vector, and then normalized to facilitate the judgment of the structural surface. Set the corresponding threshold, and judge whether it is a structural surface according to the semantics of the fluctuation of the merged surrounding rock surface structural surface. If the fluctuation is higher than the threshold, it means that the surface of the surrounding rock surface structural surface is too undulating and it is a non-structural surface area, or it is a non-rock area and should be excluded.

[0090] In the embodiment of the present invention, a linear structure recognition model is used to recognize depth data and color data, and the recognized linear exposed cracks are merged into the rock mass structural surface.

[0091] In the specific implementation, a linear structural surface sample is produced, and the already trained recognition model (such as a stacked ResNet model, including multiple input channels, capable of simultaneously inputting depth data and color data) is migrated to obtain a linear structure recognition model, which is used to identify the linear exposed cracks in the tunnel in the depth data and color data of the image data, and project the linear exposed crack information into the point cloud model space. The surrounding rock surface structural surface and the linear exposed crack are merged to obtain a comprehensive rock mass structural surface recognition result.

[0092] In specific implementation, linear exposed cracks are a type of structural surface, which is a fracture or opening surface in the rock mass. The geometric features are usually linear, and the cracks are mostly small-scale, local features of the rock mass. This linear structural surface is relatively complex, and some parts have both two-dimensional and three-dimensional morphological features. Using the recognition model obtained by migration to simultaneously recognize depth data and color data can ensure the accuracy of linear structural surface exposure recognition.

[0093] The present invention also provides a tunnel surrounding rock modeling and all-type rock structure surface recognition device, including: an image acquisition unit, a registration unit, a mapping unit, a surrounding rock surface extraction unit and a structure surface acquisition unit, wherein: the image acquisition unit is used to acquire image data of the tunnel surrounding rock using depth cameras arranged at multiple angles, build a point cloud model based on the depth data of each image frame in the image data, attach the corresponding color data in the image data to the point cloud model, and the image frame of the obtained image data includes a point cloud model with color data; the registration unit is used to perform initial registration on the point cloud model of the image frame, based on the color consistency and depth consistency between the point cloud models, The point cloud model is post-registered to obtain a complete tunnel point cloud model; the mapping unit is used to obtain the geometric posture of the tunnel point cloud model, convert the geometric posture into color data, and map it to a two-dimensional image to obtain a geometric posture image; the surrounding rock surface extraction unit is used to identify the degree of association between the geometric posture features of the pixels and the neighborhood in the geometric posture image, determine the neighborhood to which the pixels belong according to the degree of association, and thus extract the surrounding rock surface semantic areas reflecting similar geological features on the geometric posture image; the structural surface acquisition unit is used to merge the surrounding rock surface semantic areas based on the similarity between the surrounding rock surface semantic areas to obtain the rock mass structural surface.

[0094] In a specific implementation, the tunnel surrounding rock modeling and all-type rock structure surface identification device provided by the present invention, wherein the execution unit for executing methods, steps or functions, the methods, steps or functions executed by it can refer to the tunnel surrounding rock modeling and all-type rock structure surface identification method provided by the present invention.

Claims

1. A method for modeling surrounding rock of a tunnel and identifying all types of rock mass structural surfaces, characterized in that: include: The image data of the tunnel surrounding rock is acquired by using depth cameras arranged at multiple angles, a point cloud model is constructed based on the depth data of each image frame in the image data, and the corresponding color data in the image data is attached to the point cloud model. The image frame of the acquired image data includes a point cloud model with color data. Performing preliminary registration on the point cloud model of the image frame; converting the color of the point cloud model into grayscale, selecting a first point cloud model and a second point cloud model whose original positions are known in the point cloud model; using a transformation matrix to transform the first point cloud model to obtain a second point cloud model; calculating the grayscale consistency and depth consistency between the points in the first point cloud model and the corresponding points in the second point cloud model after the transformation of the transformation matrix, and calculating the consistency and value of the grayscale consistency and the depth consistency; The transformation matrix is ​​adjusted based on the consistency and value to obtain the optimal transformation matrix, and the point cloud models of image frames at different angles are post-registered based on the optimal transformation matrix to obtain a complete tunnel point cloud model. Acquire the geometric posture of the tunnel point cloud model, convert the geometric posture into color data, and map it to a two-dimensional image to obtain a geometric posture image; Identify the degree of correlation between the geometric posture features of pixels and neighborhoods in the geometric posture image, determine the neighborhood to which the pixel belongs based on the degree of correlation, and thereby extract the surrounding rock surface semantic area reflecting similar geological features on the geometric posture image; Based on the similarity between the surrounding rock surface semantic regions, the surrounding rock surface semantic regions are merged to obtain the rock mass structural surface.

2. The method for tunnel surrounding rock modeling and all-type rock mass structural surface identification according to claim 1 is characterized in that: The initial registration of the point cloud model of the image frame includes: After sharpening and bilateral filtering the image frame, the three-dimensional coordinates of the point cloud model of the image frame are preliminarily determined according to the parameters of the depth camera, and the initial registration of the point cloud models of image frames at different angles is completed.

3. The method for tunnel surrounding rock modeling and all-type rock mass structural surface identification according to claim 1 is characterized in that: The grayscale of the point cloud model is calculated using the following formula: P C (m c )=P(c)+G c T m, Among them, P C (m c ) represents the grayscale of point c in point cloud model C, m c represents the moving direction of point c on the corresponding virtual section, m represents the moving direction of the point on the virtual section, P(c) represents the grayscale of point c on the point cloud model C, G c Pc(m c ), the superscript T indicates transposition; The depth of the point cloud model is calculated using the following formula: D C (m c )=(O C -c) T n c , Among them, D C (m c ) represents the depth of point c in the point cloud model C, O C Represents the origin of the depth camera corresponding to the point cloud model C, n c Represents the normal vector of the virtual section corresponding to point c; Grayscale consistency is calculated using the following formula; Δ z (S z )=(P A (m b’ )-(P B (m b )) 2 , Among them, Δ z (S z ) represents the transformation matrix S z The grayscale difference between the transformed point b' of the first point cloud model A and the corresponding point b of the second point cloud model under the transformation of ; The depth consistency is calculated using the following formula; Δ v (S v )=(D A (m b’ )-(D B (m b )) 2 , Among them, Δ v (S v ) represents the transformation matrix S v The depth difference between the transformed point b' of the first point cloud model A and the corresponding point b of the second point cloud model under the transformation of ; The consistency and value are calculated using the following formula: Δ(S)=μΔ z (S z )+σΔ v (S v ), Among them, Δ(S) represents the sum of the grayscale difference and the depth difference under the transformation matrix S, and μ and σ represent the weights of the grayscale difference and the depth difference, respectively.

4. The method for tunnel surrounding rock modeling and all-type rock mass structural surface identification according to claim 1 is characterized in that: The step of obtaining the geometric posture of the tunnel point cloud model, converting the geometric posture into color data, and mapping the data to a two-dimensional image to obtain a geometric posture image includes: Obtain the inclination, dip and direction of the points in the tunnel point cloud model and convert them into corresponding colors respectively; The mapping relationship between the points in the point cloud model and the two-dimensional image is determined according to the parameters of the depth camera, and the geometric posture image is mapped.

5. The method for tunnel surrounding rock modeling and all-type rock mass structural surface identification according to claim 1 is characterized in that: The step of identifying the degree of association between geometric posture features of pixels in the geometric posture image and the neighborhood, and determining the neighborhood to which the pixel belongs according to the degree of association, comprises: Perform convolution and pooling on the geometric posture image to generate an inclination feature map, a tilt feature map, and a direction feature map; Respectively calculate the correlation degree matrix between the pixels in the geometric posture image and the adjacent superpixels in the inclination feature map, the adjacent superpixels in the tendency feature map, and the adjacent superpixels in the trend feature map to obtain the inclination correlation matrix, the tendency correlation matrix, and the trend feature correlation matrix; The association matrix is ​​summed up to obtain a fusion matrix, and the adjacent superpixels with the highest association with the pixels in the geometric posture image are determined according to the fusion matrix, thereby determining the neighborhood to which the pixels in the geometric posture image belong.

6. The method for tunnel surrounding rock modeling and all-type rock mass structural surface identification according to claim 1 is characterized in that: The merging of surrounding rock surface semantic regions based on the similarity between the surrounding rock surface semantic regions includes: The attribute similarity and spatial similarity between the surrounding rock surface semantic areas are calculated, and the similarity between the surrounding rock surface semantic areas is calculated by summing them up. If the similarity is higher than the standard similarity, the corresponding surrounding rock surface semantic areas are merged; the attribute similarity is calculated by the geometric posture similarity between the surrounding rock surface semantic areas; the spatial similarity is calculated by the distance between the centroids of the point cloud models corresponding to the surrounding rock surface semantic areas.

7. The method for tunnel surrounding rock modeling and all-type rock mass structural surface identification according to claim 1 is characterized in that: The merging of surrounding rock surface semantic areas then includes: Calculate the undulation of each point in the surrounding rock surface semantic area and exclude the part with undulation higher than the standard undulation.

8. The method for tunnel surrounding rock modeling and all-type rock mass structural surface identification according to claim 1 is characterized in that: The merging of surrounding rock surface semantic areas then includes: The linear structure recognition model is used to identify the depth data and color data, and the identified linear exposed cracks are merged into the rock mass structural surface.

9. A tunnel surrounding rock modeling and all types of rock mass structural surface identification device, characterized in that: include: Image acquisition unit, registration unit, mapping unit, surrounding rock surface extraction unit and structural surface acquisition unit, wherein: The image acquisition unit is used to acquire image data of the tunnel surrounding rock using depth cameras arranged at multiple angles, construct a point cloud model based on the depth data of each image frame in the image data, attach the corresponding color data in the image data to the point cloud model, and the image frame of the acquired image data includes a point cloud model with color data; The registration unit is used to perform preliminary registration on the point cloud model of the image frame; to convert the color of the point cloud model into grayscale, and to select the first point cloud model and the second point cloud model whose original positions are known in the point cloud model; the transformation matrix is ​​used to transform the first point cloud model into the second point cloud model; after the transformation of the transformation matrix, the grayscale consistency and depth consistency between the points in the first point cloud model and the corresponding points in the second point cloud model are calculated, and the consistency and value of the grayscale consistency and the depth consistency are calculated; based on the consistency and value, the transformation matrix is ​​adjusted to obtain the optimal transformation matrix, and the point cloud models of the image frames at different angles are post-registered based on the optimal transformation matrix to obtain a complete tunnel point cloud model; The mapping unit is used to obtain the geometric posture of the tunnel point cloud model, convert the geometric posture into color data, and map it to a two-dimensional image to obtain a geometric posture image; The surrounding rock surface extraction unit is used to identify the degree of association between the geometric posture features of the pixels and the neighborhood in the geometric posture image, determine the neighborhood to which the pixel belongs according to the degree of association, and thereby extract the surrounding rock surface semantic area reflecting similar geological features on the geometric posture image; The structural surface acquisition unit is used to merge the surrounding rock surface semantic areas based on the similarity between the surrounding rock surface semantic areas to obtain the rock mass structural surface.

Images