Surrounding rock fracture monitoring method and device based on borehole peeping

Abstract

The invention provides a surrounding rock fracture monitoring method and device based on borehole peeping. The method comprises the following steps: acquiring depth image data of a rock stratum in a peephole at a loose circle distribution position in a roadway surrounding rock through a borehole peeping instrument; the drilling peeping instrument is integrated by a depth camera; according to the depth image data, obtaining visual point cloud data of a rock stratum in the peephole; based on the visual point cloud data, a trained target recognition model is adopted to recognize fractures of the rock stratum in the peephole, so that fracture point cloud information of the fractures in the rock stratum in the peephole is obtained; and analyzing the fracture point cloud information of the fracture to obtain target monitoring information of the fracture in the rock stratum in the peephole. According to the method, the drilling observation instrument is combined with the depth camera and the target identification model, automatic and rapid identification and analysis of the rock stratum fractures at the distribution positions of the loose circles in the surrounding rock of the roadway are achieved, monitoring of the rock stratum fractures is achieved, limitation of traditional human eye observation is eliminated, and powerful technical support is provided for safe production and efficient mining of a coal mine.

Description

Technical Field

[0001] The present application relates to the technical field of borehole peeping, and in particular to a method and device for monitoring surrounding rock fissures based on borehole peeping. Background Art

[0002] In order to evaluate the stability of the tunnel surrounding rock, the effectiveness of support measures, and explore geological characteristics, the borehole peep method is often used. A peep hole is drilled in the coal rock mass, and a borehole peep instrument is used to observe and test the structural surface distribution of the rock strata in the peep hole to monitor the development characteristics of the surrounding rock loosening zone.

[0003] However, in the related technologies, borehole voyeur imaging in coal mines often relies on the naked eye observation of relevant staff, and is limited by the special low illumination and unclear crack characteristics of coal rock, which makes it difficult to identify the cracks and drill marks of the borehole. Furthermore, it makes it impossible to accurately measure the depth, width, length, etc. of the cracks in a timely manner, and it is impossible to timely and effectively count and classify the information of cracks of different widths in the borehole at the distribution position of the loose circle of the surrounding rock, which hinders the relevant staff from identifying the current bearing capacity of the surrounding rock through the cracks, and brings challenges to the surrounding rock stability assessment and the optimization of the tunnel support scheme. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the first purpose of the present application is to propose a surrounding rock crack monitoring method based on borehole peeping.

[0006] The second purpose of the present application is to provide a surrounding rock crack monitoring device based on borehole peeping.

[0007] The third objective of the present application is to provide an electronic device.

[0008] A fourth objective of the present application is to provide a computer-readable storage medium.

[0009] A fifth object of the present application is to provide a computer program product.

[0010] To achieve the above-mentioned purpose, the first aspect of the embodiment of the present application proposes a surrounding rock crack monitoring method based on borehole peeping, including: obtaining depth image data of the rock layer in the peephole at the distribution position of the loose circle in the surrounding rock of the tunnel through a borehole peeper; wherein the borehole peeper is integrated with a depth camera; according to the depth image data, obtaining visual point cloud data of the rock layer in the peephole; based on the visual point cloud data, using a trained target recognition model to identify the cracks in the rock layer in the peephole to obtain crack point cloud information of the cracks in the rock layer in the peephole; analyzing the crack point cloud information of the cracks to obtain target monitoring information of the cracks in the rock layer in the peephole.

[0011] To achieve the above-mentioned purpose, the second aspect of the embodiment of the present application proposes a surrounding rock crack monitoring device based on borehole peeping, including: a first acquisition module, used to obtain depth image data of the rock layer in the peephole at the distribution position of the loose circle in the tunnel surrounding rock through a borehole peeper; wherein the borehole peeper is integrated with a depth camera; a second acquisition module, used to obtain visual point cloud data of the rock layer in the peephole according to the depth image data; an identification module, used to identify the cracks of the rock layer in the peephole based on the visual point cloud data and using a trained target recognition model to obtain crack point cloud information of the cracks in the rock layer in the peephole; and an analysis module, used to analyze the crack point cloud information of the cracks to obtain target monitoring information of the cracks in the rock layer in the peephole.

[0012] To achieve the above-mentioned purpose, the third aspect embodiment of the present application proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method described in the first aspect embodiment above.

[0013] To achieve the above-mentioned purpose, the fourth aspect embodiment of the present application proposes a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the method described in the first aspect embodiment above.

[0014] To achieve the above-mentioned purpose, the fifth aspect of the present application proposes a computer program product, including a computer program, which, when executed by a processor, implements the method described in the first aspect of the present application.

[0015] The present application provides a method and device for monitoring rock cracks based on borehole peephole, which obtains the depth image data of the rock layer in the peephole at the distribution position of the loose circle in the surrounding rock of the tunnel through the borehole peephole; wherein the borehole peephole is integrated with a depth camera; according to the depth image data, the visual point cloud data of the rock layer in the peephole is obtained; based on the visual point cloud data, the cracks in the rock layer in the peephole are identified by using a trained target recognition model to obtain the crack point cloud information of the cracks in the rock layer in the peephole; the crack point cloud information of the cracks is analyzed to obtain the target monitoring information of the cracks in the rock layer in the peephole. Therefore, the borehole peephole can be used in combination with a depth camera and a target recognition model to realize the automatic and rapid identification and analysis of the rock layer cracks at the distribution position of the loose circle in the surrounding rock of the tunnel, so as to realize the monitoring of the rock layer cracks, get rid of the limitations of traditional human eye observation, improve the accuracy and efficiency of identification, reduce manual intervention, reduce the difficulty of operation, and provide strong technical support for the safe production and efficient mining of coal mines.

[0016] The method and device for monitoring surrounding rock cracks based on borehole peeping provided in this application can not only be used to evaluate the stability of tunnel surrounding rock to guide the design and optimization of tunnel support schemes, but also can be used to monitor geological disaster risks to take preventive measures in advance.

[0017] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A schematic diagram of a flow chart of a surrounding rock fracture monitoring method based on borehole peek provided in an embodiment of the present application;

[0020] Figure 2 A schematic flow chart of a surrounding rock fracture monitoring method based on borehole peek provided in another embodiment of the present application;

[0021] Figure 3 A schematic flow chart of a surrounding rock fracture monitoring method based on borehole peek provided in another embodiment of the present application;

[0022] Figure 4 A schematic structural diagram of a surrounding rock fissure monitoring device based on borehole peeping provided in another embodiment of the present application. DETAILED DESCRIPTION

[0023] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0024] The following describes a method and device for monitoring surrounding rock fractures based on borehole peeping according to an embodiment of the present application with reference to the accompanying drawings.

[0025] Figure 1 A schematic flow chart of a method for monitoring surrounding rock fractures based on borehole observation provided in one embodiment of the present application.

[0026] The embodiment of the present application takes the method for monitoring surrounding rock fractures based on borehole peeping as an example, in which the method is configured in a surrounding rock fracture monitoring device based on borehole peeping. The surrounding rock fracture monitoring device based on borehole peeping can be applied to any electronic device so that the electronic device can perform the surrounding rock fracture monitoring function based on borehole peeping.

[0027] Among them, the electronic device can be any device with computing capabilities, such as a personal computer (PC), an industrial computer, a host computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0028] like Figure 1 As shown, the method comprises the following steps:

[0029] Step S101, obtaining depth image data of rock layers in a peephole at a distribution position of loose circles in surrounding rocks of a tunnel by means of a borehole peep instrument.

[0030] The borehole peephole may be integrated with a depth camera, wherein the depth camera may be, for example, a ToF (Time of Flight) camera.

[0031] It should be noted that peepholes can be drilled at the locations where loose circles are distributed in the surrounding rock of the tunnel, and since rock dust, rock chips, gravel, etc. will be generated during the drilling process and interfere with the monitoring of the surrounding rock, the peepholes can be cleaned. In addition, the accumulated water and water film in the rock stratum fissures will reflect and scatter the structured light emitted by the depth camera, causing the camera to miscalculate the depth information of the pixels, increase more distortion and invalid pixels, and affect the actual observation of the rock stratum fissures. Therefore, after cleaning the peephole, the borehole can be dried.

[0032] In an embodiment of the present application, a borehole peephole can be used to photograph the rock strata in the peephole at the distribution position of the loose circle in the surrounding rock of the tunnel, so as to obtain depth image data of the rock strata in the peephole at the distribution position of the loose circle in the surrounding rock of the tunnel.

[0033] As a possible implementation manner, the depth image data may include multiple frames of continuous depth images, wherein the value of each pixel in the depth image represents the depth (or distance) of the corresponding pixel from the depth camera in the camera coordinate system.

[0034] It can be understood that before obtaining the depth image data of the rock layer in the peephole at the distribution position of the loose circle in the surrounding rock of the tunnel through the borehole peephole, the depth camera in the borehole peephole for borehole peephole can be calibrated and aligned to obtain the internal and external parameters of the depth camera.

[0035] It should be noted that in the process of acquiring depth image data by the depth camera, due to the complex reflection characteristics of light in cracks or drill marks, a multipath effect will be generated. This effect will cause certain interference to the subsequent image processing. In order to improve the accuracy and effectiveness of the subsequent image processing process, a deep learning network can be used to identify and correct the multipath interference image area in the depth image data.

[0036] Step S102, obtaining visual point cloud data of the rock layer in the peephole according to the depth image data.

[0037] Among them, the visual point cloud data may include the spatial coordinates of multiple spatial points, the spatial points are coordinate points in three-dimensional space, and the coordinate system corresponding to the spatial points (referred to as the target coordinate system in this application) is the xyz rectangular coordinate system, where the z-axis represents the depth direction.

[0038] Specifically, the pixel coordinates of each pixel in the depth image data can be uniformly converted to obtain visual point cloud data of the rock layer in the peephole, that is, to obtain the spatial coordinates of multiple spatial points. It can be understood that when the pixel coordinates of the two-dimensional space are converted into the spatial coordinates of the three-dimensional space, the z-axis coordinate in the spatial coordinates of the three-dimensional space represents the depth of the corresponding pixel point.

[0039] In a possible implementation of the embodiment of the present application, the visual point cloud data may be voxelized, thereby effectively reducing the amount of data, facilitating subsequent data processing, and helping to increase the data processing speed.

[0040] In another possible implementation of the embodiment of the present application, the rock layer in the peephole can be reconstructed based on the visual point cloud data to obtain a three-dimensional surface model of the rock layer in the peephole. For example, the rock layer in the peephole can be reconstructed based on the visual point cloud data using a Poisson surface reconstruction algorithm to obtain a three-dimensional surface model of the rock layer in the peephole. In this way, an intuitive view of the inside of the peephole can be provided to relevant staff, so that relevant staff can more easily understand and analyze the structure and characteristics of the rock layer in the peephole.

[0041] Step S103, based on the visual point cloud data, using a trained target recognition model to identify the cracks in the rock formation in the peephole, so as to obtain crack point cloud information of the cracks in the rock formation in the peephole.

[0042] The crack point cloud information may include the spatial coordinates of multiple spatial points.

[0043] In an embodiment of the present application, the target recognition model can be used to identify cracks in the rock formation in the peephole. Specifically, the visual point cloud data can be input into a trained target recognition model, and the output of the trained target recognition model can be determined as the crack point cloud information of the cracks in the rock formation in the peephole obtained by identifying the cracks in the rock formation in the peephole.

[0044] It should be noted that the number of cracks in the rock formation in the peephole can be but is not limited to one, and this application does not impose any restrictions on this.

[0045] As a possible implementation method, the following steps can be used to obtain a trained target recognition model:

[0046] Step S1031, obtaining multiple groups of sample depth image data of the rock layer in the sample peephole.

[0047] It should be noted that this application does not limit the number of sample peepholes.

[0048] It should also be noted that any set of sample depth image data may include but is not limited to being a sample depth image.

[0049] In an embodiment of the present application, multiple sets of sample depth image data of the rock formation in the sample peephole can be obtained. As an example, a sample borehole peephole with an integrated depth camera can be used to photograph the rock formation in the sample peephole, thereby obtaining multiple sets of sample depth image data.

[0050] Step S1032 : for any group of sample depth image data, obtain sample point cloud data corresponding to the group of sample depth image data according to the group of sample depth image data, wherein the sample point cloud data includes a plurality of sample space points.

[0051] It should be noted that the method for acquiring the sample point cloud data corresponding to any set of sample depth image data is similar to the method for acquiring the visual point cloud data in step S102, and will not be elaborated herein.

[0052] Step S1033 , obtaining the annotation information corresponding to each sample space point in the sample point cloud data corresponding to each group of sample depth image data, wherein the annotation information is used to indicate the attribute information of the sample space point.

[0053] The attribute information may include first-level categories and / or second-level categories under the first-level categories, wherein the first-level categories may include cracks, drill marks, and non-cracks and non-drill marks; when the first-level category is cracks, the second-level category is used to indicate the category to which the corresponding cracks belong, and when the first-level type is drill marks, the second-level category can be used to indicate the category to which the corresponding drill marks belong.

[0054] As an example, assume that there is a set of sample point cloud data corresponding to sample depth image data, and the sample point cloud data includes sample space points belonging to crack 1, sample space points belonging to crack 2, sample space points belonging to crack 3, sample space points belonging to drill mark 1, sample space points belonging to drill mark 2, and sample space points belonging to non-crack and non-drill mark, wherein the first-level categories include cracks, drill marks, and non-cracks and non-drill marks. When the first-level category is cracks, the second-level category is used to indicate the category to which the corresponding cracks belong, i.e., crack 1, crack 2, and crack 3. When the first-level type is drill marks, the second-level category can be used to indicate the category to which the corresponding drill marks belong, i.e., drill mark 1 and drill mark 2.

[0055] In the embodiment of the present application, annotation information corresponding to each sample space point in the sample point cloud data corresponding to each group of sample depth image data may be obtained.

[0056] As an example, manual annotation may be used to annotate each sample space point in the sample point cloud data corresponding to any set of sample depth image data to obtain annotation information corresponding to each sample space point.

[0057] As another example, each sample space point in the sample point cloud data corresponding to any set of sample depth image data can be annotated by machine annotation. That is, a trained annotation model can be used to annotate each sample space point in the sample point cloud data corresponding to any set of sample depth image data. At the same time, in order to improve the accuracy and reliability of the annotation results, the machine annotation results can also be manually reviewed to correct the sample space points that are incorrectly annotated by the machine.

[0058] Step S1034 , using each sample space point in the sample point cloud data corresponding to each group of labeled sample depth image data, to train the initial recognition model, to obtain a trained target recognition model.

[0059] In the embodiment of the present application, each sample space point in the sample point cloud data corresponding to each group of labeled sample depth image data can be used to train the initial recognition model to obtain a trained target recognition model. It can be understood that by training the initial recognition model, the trained target recognition model can learn the corresponding relationship between each sample space point and attribute information in the sample point cloud data corresponding to any group of sample depth image data, so that after obtaining each sample space point in the sample point cloud data corresponding to a new group of sample depth image data, each sample space point can be automatically classified.

[0060] Step S104, analyzing the fracture point cloud information of the fracture to obtain target monitoring information of the fracture in the rock formation in the peephole.

[0061] The target monitoring information may include, but is not limited to, the crack depth, size information, distribution location, etc. of the corresponding crack, and the present application does not limit this. The size information may include the crack width and crack length, and the distribution location may include the starting point and end point of the corresponding crack.

[0062] In the disclosed embodiment, the fracture point cloud information of the fracture may be analyzed to determine the target monitoring information of the fracture in the rock formation in the peephole.

[0063] The surrounding rock fissure monitoring method based on borehole peep in the embodiment of the present application obtains the depth image data of the rock layer in the peephole at the distribution position of the loose circle in the surrounding rock of the tunnel through the borehole peep instrument; wherein the borehole peep instrument is integrated with a depth camera; according to the depth image data, the visual point cloud data of the rock layer in the peep hole is obtained; based on the visual point cloud data, the cracks of the rock layer in the peep hole are identified by using a trained target recognition model to obtain the crack point cloud information of the cracks in the rock layer in the peep hole; the crack point cloud information of the cracks is analyzed to obtain the target monitoring information of the cracks in the rock layer in the peep hole. Therefore, the borehole peep instrument can be used in combination with the depth camera and the target recognition model to realize the automatic and rapid identification and analysis of the rock layer cracks at the distribution position of the loose circle in the surrounding rock of the tunnel, so as to realize the monitoring of the rock layer cracks, get rid of the limitations of traditional human eye observation, improve the accuracy and efficiency of identification, reduce manual intervention, reduce the difficulty of operation, and provide strong technical support for the safe production and efficient mining of coal mines.

[0064] When there is at least one crack in the rock formation in the peephole, the target monitoring information includes size information of the corresponding cracks, and the depth image data includes multiple frames of continuous depth images, in order to clearly explain how the crack point cloud information of the cracks is analyzed in the above embodiments of the present application to obtain target monitoring information of the cracks in the rock formation in the peephole, the present application also proposes a surrounding rock crack monitoring method based on borehole peeping.

[0065] Figure 2 A schematic flow chart of a surrounding rock fracture monitoring method based on borehole peeping provided in another embodiment of the present application.

[0066] like Figure 2 As shown, the method comprises the following steps:

[0067] Step S201, obtaining depth image data of rock layers in a peephole at a distribution position of loose circles in surrounding rocks of a tunnel by means of a borehole peep instrument.

[0068] Step S202: acquiring visual point cloud data of the rock layer in the peephole according to the depth image data.

[0069] Step S203, based on the visual point cloud data, using a trained target recognition model to identify the cracks in the rock formation in the peephole, so as to obtain crack point cloud information of the cracks in the rock formation in the peephole.

[0070] It should be noted that the execution process of steps S201 to S203 can refer to the execution process of any embodiment of the present application and will not be repeated here.

[0071] Step S204: for any crack in the rock formation in the peephole, based on the crack point cloud information of the crack, determine a target depth image where the crack exists from multiple frames of continuous depth images.

[0072] It should be noted that the coordinate system corresponding to the spatial point (referred to as the target coordinate system in this application) is an xyz rectangular coordinate system, in which the z-axis represents the depth direction, and the z-axis coordinate of the spatial point in the crack point cloud information represents the depth of the corresponding pixel point. Therefore, as a possible implementation method, for any frame depth image, the z-axis coordinate value in the spatial coordinates of each spatial point in the crack point cloud information of the crack is matched with the depth value corresponding to each pixel coordinate point (also called pixel point) of the depth image to determine the third matching degree between the crack point cloud information of the crack and the depth image; when the third matching degree between the crack point cloud information of the crack and the depth image is greater than the third set threshold, the depth image is determined as the target depth image.

[0073] The third set threshold may be pre-set, such as 90%, 85%, 99%, etc., and this application does not impose any limitation on this.

[0074] It should be noted that the target depth image can be one frame or multiple frames, and this application does not impose any limitation on this.

[0075] In an embodiment of the present application, for any frame depth image, the z-axis coordinate value in the spatial coordinates of each spatial point in the crack point cloud information of the crack can be matched with the depth value corresponding to each pixel coordinate point of the depth image to determine the third matching degree between the crack point cloud information of the crack and the depth image.

[0076] As an example, for any depth image, for any spatial point in the crack point cloud information of the crack, the z-axis coordinate value in the spatial coordinates of the spatial point can be matched with the depth corresponding to each pixel coordinate point of the depth image to determine whether there is a first pixel point in each pixel coordinate point of the depth image, wherein the depth value corresponding to the first pixel point matches (or is equal to) the z-axis coordinate value in the spatial coordinate axis of the spatial point. When the first pixel point exists, the spatial point is determined as the first spatial point; the number of first spatial points in the crack point cloud information of the crack is counted to obtain a first number, and the number of spatial points in the crack point cloud information of the crack is counted to obtain a second number; the ratio of the first number to the second number is determined as the third matching degree between the crack point cloud information of the crack and the depth image.

[0077] Therefore, with the help of the value of the coordinate axis representing the depth direction in the crack point cloud information, the crack point cloud information of the crack is matched with multiple frames of depth images, so as to achieve the matching degree between the crack point cloud information and the depth image based on the crack, and effectively and accurately screen out the target depth image with cracks from multiple frames of continuous depth images.

[0078] Step S205: acquiring a target region image corresponding to the crack according to the target depth image.

[0079] As a possible implementation method, first, target detection can be performed on the target depth image to obtain multiple first detection frames; secondly, the target depth image can be segmented based on the multiple first detection frames to obtain multiple first area images; thirdly, for any first area image, the crack point cloud information of the crack is matched with the first area image to determine a first matching degree between the crack point cloud information of the crack and the first area image; finally, in response to the first matching degree being greater than a first set threshold, the first area image is determined as the target area image corresponding to the crack.

[0080] The first detection frame has a corresponding object, and the object may be a crack, a drill mark, or the like.

[0081] The first set threshold may be pre-set, such as 95%, 99%, etc., and this application does not impose any limitation on this.

[0082] In an embodiment of the present application, a target detection algorithm (such as a YOLO (You Only Look Once) algorithm) may be used to perform target detection on a target depth image, thereby obtaining a first detection frame corresponding to each target based on the detected target (or object).

[0083] In an embodiment of the present application, for any first region image, the crack point cloud information of the crack can be matched with the first region image to determine a first matching degree between the crack point cloud information of the crack and the first region image. As an example, for any first region image, the z-axis coordinate value in the spatial coordinates of each spatial point in the crack point cloud information of the crack can be matched with the depth value corresponding to each pixel coordinate point of the first region image to determine a first matching degree between the crack point cloud information of the crack and the first region image. For example, for any first area image, for any spatial point in the crack point cloud information of the crack, the z-axis coordinate value in the spatial coordinates of the spatial point can be matched with the depth value corresponding to each pixel coordinate point of the first area image to determine whether there is a second pixel point in each pixel coordinate point of the first area image, wherein the depth value corresponding to the second pixel point matches (or is equal to) the z-axis coordinate value in the spatial coordinate axis of the spatial point; when the second pixel point exists, the spatial point is determined as the second spatial point; the number of second spatial points in the crack point cloud information of the crack is counted to obtain a third number, and the number of spatial points in the crack point cloud information of the crack is counted to obtain a fourth number; the ratio of the third number to the fourth number is determined as the first matching degree between the crack point cloud information of the crack and the first area image.

[0084] Therefore, with the help of the value of the coordinate axis representing the depth direction in the crack point cloud information, the crack point cloud information of the crack is matched with multiple first area images, so as to achieve the matching degree between the crack point cloud information of the crack and the first area image, and effectively and accurately determine the target area image corresponding to the crack.

[0085] It is understandable that there may be a situation where the target depth image is multiple frames, that is, there are multiple frames of target depth images, and each frame of the target depth image contains all or part of the corresponding crack. In the case where the target depth image is multiple frames, in order to accurately obtain the target area image corresponding to the crack, in another possible implementation method of the embodiment of the present application, when the number of target depth images is multiple frames, the multiple frames of target depth images can be spliced ​​to obtain a spliced ​​image; target detection is performed on the spliced ​​image to obtain multiple second detection frames; based on the multiple second detection frames, the spliced ​​image is segmented to obtain multiple second area images; for any second area image, the crack point cloud information of the crack is matched with the second area image to obtain the second matching degree between the crack point cloud information of the crack and the second area image; in response to the second matching degree between the crack point cloud information of the crack and the second area image being greater than the second set threshold, the second area image is determined as the target area image corresponding to the crack.

[0086] Among them, the second set threshold can be pre-set, and this application does not limit the value of the second set threshold.

[0087] It should also be noted that the second set threshold may be the same as the first set threshold, or may be different, and this application does not impose any restrictions on this.

[0088] It should be noted that the method for obtaining the second matching degree is similar to the method for obtaining the first matching degree, which will not be described in detail here.

[0089] Therefore, when the target depth image is a multi-frame image, the multi-frame target depth image can be stitched together to obtain a stitched image, so that based on the stitched image, the target area image corresponding to the crack can be accurately acquired.

[0090] Step S206, determining the size information of the crack according to the target area image corresponding to the crack.

[0091] As an example, the size information of the target area image may be determined as the size information of the crack.

[0092] In a possible implementation method of an embodiment of the present application, when the target monitoring information may also include the distribution position and crack depth of the corresponding crack, the distribution position and crack depth of the crack may be determined based on the spatial coordinates of multiple spatial points in the crack point cloud information of the crack; wherein the distribution position includes the starting point and end point of the corresponding crack.

[0093] As an example, the coordinate system corresponding to the spatial point in the crack point cloud information (referred to as the target coordinate system in this application) is an xyz rectangular coordinate system, where the z-axis represents the depth direction. The spatial point with the largest z-axis coordinate value and the spatial point with the smallest z-axis coordinate value are determined from multiple spatial points in the crack point cloud information of the crack; the difference in z-axis coordinate value between the spatial point with the largest z-axis coordinate value and the spatial point with the smallest z-axis coordinate value is determined as the crack depth of the corresponding crack; the spatial point with the smallest z-axis coordinate value is used as the starting point of the corresponding crack, and the spatial point with the largest z-axis coordinate value is determined as the end point of the corresponding crack.

[0094] In this way, the distribution position and depth of cracks can be effectively determined.

[0095] In a possible implementation of the embodiment of the present application, when the size information includes the crack width and the crack length, a first crack statistical report can be generated according to the target detection information of each crack; wherein the first crack statistical report can be used to indicate at least one of the following:

[0096] The number of fractures in each depth statistical range;

[0097] The average fracture depth of each depth statistical range;

[0098] Maximum crack depth in each depth statistical range;

[0099] Maximum crack length in each depth statistical range;

[0100] Average crack length in each depth statistical range;

[0101] The average crack width in each depth statistical range;

[0102] The maximum crack width in each depth statistical range.

[0103] It should be noted that the depth statistical range can be pre-set, for example, the depth statistical range can be 0-1m, 1-2m, 2-3m, 3-4m, 4-5m, 5-6m, 6-7m, 7-8m, 8-9m, etc. This application does not limit the value range of the depth statistical range.

[0104] The surrounding rock crack monitoring method based on borehole peep in the embodiment of the present application determines the target depth image where the crack exists from multiple frames of continuous depth images based on the crack point cloud information of any crack in the rock formation in the peep hole; obtains the target area image corresponding to the crack according to the target depth image; and determines the size information of the crack according to the target area image corresponding to the crack. Thus, the size information of the crack is effectively determined by combining the depth image and the crack point cloud information.

[0105] When the depth image data includes multiple frames of continuous depth images and the visual point cloud data includes the spatial coordinates of multiple spatial points, in order to clearly explain how the visual point cloud data of the rock formation in the peephole is obtained based on the depth image data in any of the above embodiments of the present application, the present application also proposes a surrounding rock crack monitoring method based on borehole peeping.

[0106] Figure 3 A schematic flow chart of a surrounding rock fracture monitoring method based on borehole peeping provided in another embodiment of the present application.

[0107] like Figure 3 As shown, the method comprises the following steps:

[0108] Step S301, obtaining depth image data of rock layers in a peephole at a distribution position of loose circles in surrounding rocks of a tunnel by means of a borehole peep instrument.

[0109] It should be noted that the execution process of step S301 can refer to the execution process of any embodiment of the present application and will not be repeated here.

[0110] Step S302: performing image registration on multiple frames of continuous depth images to determine the camera pose when the depth camera captures each frame of the depth image.

[0111] As an example, for any frame of depth image, SIFT (Scale-Invariant Feature Transform) algorithm, SURF (Speeded-Up Robust Features) algorithm, ORB (Oriented FAST and Rotated BRIEF) algorithm, etc. are used to extract feature points of the depth image to obtain feature points of the depth image; FLANN (Fast Library for Approximate Nearest Neighbors), BFMatcher (Brute-Force Matcher) and other matching algorithms are used to match feature points of any two adjacent frames of depth images to obtain feature point pairs matched between any two adjacent frames of depth images; based on the feature point pairs matched between any two adjacent frames of depth images, a geometric method (such as ICP (Iterative Closest Point) algorithm, PnP (Perspective-n-Point) algorithm, etc.) is used to estimate the camera pose when the depth camera takes each frame of depth image to obtain the corresponding camera pose.

[0112] It should be noted that, for any frame depth image, before feature point extraction is performed on the depth image, in order to improve the accuracy and reliability of feature point extraction, the depth image can be preprocessed, wherein the preprocessing may include denoising, filtering, etc., and this application does not impose any restrictions on this.

[0113] It should also be noted that in order to improve the accuracy of the obtained camera pose, a bundle adjustment (BA) algorithm or a filtering method (such as EKF (Extended Kalman Filter), UKF (Unscented Kalman Filter), etc.) can be used to optimize the camera pose.

[0114] Step S303 , for any frame of depth image, according to the camera posture when the depth camera takes the depth image, the pixel coordinates of the pixel points of the depth image are converted into the spatial coordinates of the corresponding spatial points in the target coordinate system.

[0115] The target coordinate system may be a world coordinate system.

[0116] In an embodiment of the present application, for any frame depth image, the correspondence between the camera coordinate system when the depth camera takes the depth image and the target coordinate system can be determined according to the camera posture when the depth camera takes the depth image, and then based on the value of the pixel point of the depth image (i.e., the depth value) and the internal parameters of the depth camera, the pixel coordinates of the pixel point of the depth image are converted into the three-dimensional coordinates in the corresponding camera coordinate system, and then, based on the corresponding relationship, the three-dimensional coordinates in the camera coordinate system are converted to the spatial coordinates of the corresponding spatial point in the target coordinate system.

[0117] Step S304: generating visual point cloud data according to the spatial coordinates of the spatial points corresponding to the depth images of each frame.

[0118] It can be understood that the spatial coordinates of each spatial point are in the same coordinate system (i.e., the target coordinate system). Therefore, as a possible implementation method, for any frame depth image, the initial point cloud corresponding to the depth image can be generated based on the spatial coordinates of the spatial points corresponding to the depth image; the initial point clouds corresponding to each depth image are fused to obtain visual point cloud data.

[0119] After the initial point clouds corresponding to the depth images are fused, the fused initial point cloud data may be deduplicated to obtain visual point cloud data.

[0120] Step S305 , based on the visual point cloud data, a trained target recognition model is used to identify the cracks in the rock formation in the peephole to obtain crack point cloud information of the cracks in the rock formation in the peephole.

[0121] Step S306, analyzing the fracture point cloud information of the fracture to obtain target monitoring information of the fracture in the rock formation in the peephole.

[0122] It should be noted that the execution process of step S305 to step S306 can refer to the execution process of any embodiment of the present application and will not be repeated here.

[0123] In a possible implementation of the embodiment of the present application, a second crack statistical report may be generated according to target monitoring information of each crack; wherein the second crack statistical report is used to indicate at least one of the following:

[0124] the number of cracks in the rock formation within the peephole;

[0125] Target detection information for each crack.

[0126] The surrounding rock crack monitoring method based on borehole peeping of the embodiment of the present application performs image registration on multiple frames of continuous depth images to determine the camera pose when the depth camera takes each frame of depth image; for any frame of depth image, according to the camera pose when the depth camera takes the depth image, the pixel coordinates of the pixel points of the depth image are converted into the spatial coordinates of the corresponding spatial points in the target coordinate system; according to the spatial coordinates of the spatial points corresponding to each frame of depth image, visual point cloud data is generated. Thus, the pose of the camera is effectively determined through image registration, and then the pixel coordinates of the depth image are converted into spatial coordinates to generate visual point cloud data, which provides a basis for subsequent crack identification and analysis.

[0127] In order to implement the above-mentioned embodiments, the present application also proposes a surrounding rock crack monitoring device based on borehole peeping.

[0128] Figure 4 A schematic structural diagram of a surrounding rock fissure monitoring device based on borehole peeping provided in another embodiment of the present application.

[0129] like Figure 4 As shown, the surrounding rock fracture monitoring device 400 based on borehole peeping includes: a first acquisition module 410, a second acquisition module 420, an identification module 430 and an analysis module 440.

[0130] The first acquisition module 410 is used to acquire the depth image data of the rock layer in the peephole at the distribution position of the loose circle in the surrounding rock of the tunnel through a borehole peephole; wherein the borehole peephole is integrated with a depth camera.

[0131] The second acquisition module 420 is used to acquire visual point cloud data of the rock layer in the peephole according to the depth image data.

[0132] The recognition module 430 is used to identify the cracks in the rock formation in the peephole based on the visual point cloud data by using a trained target recognition model to obtain crack point cloud information of the cracks in the rock formation in the peephole.

[0133] The analysis module 440 is used to analyze the fracture point cloud information of the fracture to obtain the target monitoring information of the fracture in the rock formation in the peephole.

[0134] Furthermore, there is at least one crack in the rock formation in the peephole, the target monitoring information includes size information of the corresponding crack, and the depth image data includes multiple frames of continuous depth images; the analysis module 440 is used to: for any crack in the rock formation in the peephole, based on the crack point cloud information of the crack, determine the target depth image where the crack exists from the multiple frames of continuous depth images; according to the target depth image, obtain the target area image corresponding to the crack; according to the target area image corresponding to the crack, determine the size information of the crack.

[0135] Furthermore, the analysis module 440 is used to: perform target detection on the target depth image to obtain multiple first detection frames; based on the multiple first detection frames, segment the target depth image to obtain multiple first area images; for any of the first area images, match the crack point cloud information of the crack with the first area image to determine a first matching degree between the crack point cloud information of the crack and the first area image; in response to the first matching degree being greater than a first set threshold, determine the first area image as the target area image corresponding to the crack.

[0136] Furthermore, the target depth image is a plurality of frames; the analysis module 440 is used to: stitch the plurality of frames of the target depth image to obtain a stitched image; perform target detection on the stitched image to obtain a plurality of second detection frames; segment the stitched image based on the plurality of second detection frames to obtain a plurality of second area images; for any of the second area images, match the crack point cloud information of the crack with the second area image to obtain a second matching degree between the crack point cloud information of the crack and the second area image; in response to the second matching degree between the crack point cloud information of the crack and the second area image being greater than a second set threshold, determine the second area image as the target area image corresponding to the crack.

[0137] Furthermore, the target monitoring information also includes the distribution position and crack depth of the corresponding cracks, and the crack point cloud information includes the spatial coordinates of multiple spatial points; the analysis module 440 is used to:

[0138] Based on the spatial coordinates of the plurality of spatial points in the crack point cloud information of the crack, the distribution position and the crack depth of the crack are determined; wherein the distribution position includes the starting point and the end point of the corresponding crack.

[0139] Furthermore, the size information includes crack width and crack length, and the device 400 may further include:

[0140] A first generating module, used for generating a first crack statistical report according to target monitoring information of each crack;

[0141] The first crack statistical report is used to indicate at least one of the following:

[0142] The number of fractures in each depth statistical range;

[0143] The average fracture depth of each depth statistical range;

[0144] Maximum crack depth in each depth statistical range;

[0145] Maximum crack length in each depth statistical range;

[0146] Average crack length in each depth statistical range;

[0147] The average crack width in each depth statistical range;

[0148] The maximum crack width in each depth statistical range.

[0149] Furthermore, the depth image data includes multiple frames of continuous depth images, and the visual point cloud data includes spatial coordinates of multiple spatial points; the second acquisition module 420 is used to: perform image registration on the multiple frames of continuous depth images to determine the camera pose when the depth camera takes each frame of the depth image; for any frame of the depth image, according to the camera pose when the depth camera takes the depth image, the pixel coordinates of the pixel points of the depth image are converted into spatial coordinates of corresponding spatial points in the target coordinate system; according to the spatial coordinates of the spatial points corresponding to the depth images of each frame, the visual point cloud data is generated.

[0150] Furthermore, the device 400 may also include:

[0151] A processing module is used to voxelize the visual point cloud data.

[0152] Furthermore, the device 400 may also include:

[0153] A second generating module, used for generating a second crack statistical report according to the target monitoring information of each crack;

[0154] The second crack statistical report is used to indicate at least one of the following:

[0155] the number of fractures in the rock formation within the peephole;

[0156] Target detection information of each of the cracks.

[0157] It should be noted that the aforementioned explanation of the embodiment of the surrounding rock fracture monitoring method based on borehole peeping is also applicable to the surrounding rock fracture monitoring device based on borehole peeping of this embodiment, and will not be repeated here.

[0158] In summary, the surrounding rock fissure monitoring device based on borehole peephole in the embodiment of the present application obtains the depth image data of the rock layer in the peephole at the distribution position of the loose circle in the surrounding rock of the tunnel through the borehole peephole; wherein the borehole peephole is integrated with a depth camera; according to the depth image data, the visual point cloud data of the rock layer in the peephole is obtained; based on the visual point cloud data, the cracks in the rock layer in the peephole are identified by using a trained target recognition model to obtain the crack point cloud information of the cracks in the rock layer in the peephole; the crack point cloud information of the cracks is analyzed to obtain the target monitoring information of the cracks in the rock layer in the peephole. Therefore, the borehole peephole can be used in combination with a depth camera and a target recognition model to realize the automatic and rapid identification and analysis of the rock layer cracks at the distribution position of the loose circle in the surrounding rock of the tunnel, so as to realize the monitoring of the rock layer cracks, get rid of the limitations of traditional human eye observation, improve the accuracy and efficiency of identification, reduce manual intervention, reduce the difficulty of operation, and provide strong technical support for the safe production and efficient mining of coal mines.

[0159] In order to implement the above-mentioned embodiments, the present application also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the surrounding rock fracture monitoring method based on borehole peeping provided in the above-mentioned embodiments.

[0160] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, which stores computer execution instructions. When the computer execution instructions are executed by a processor, they are used to implement the surrounding rock fracture monitoring method based on borehole peeping provided in the above embodiments.

[0161] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which, when executed by a processor, implements the surrounding rock fracture monitoring method based on borehole peeping provided in the above embodiments.

[0162] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0163] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.

[0164] The present application is expected to provide an implementation scheme for users to selectively block the use or access of personal information data. That is, the present disclosure is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.

[0165] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0166] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood to control or imply relative importance or implicitly indicate the number of technical features controlled. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0167] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0168] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0169] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0170] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0171] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0172] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for monitoring surrounding rock cracks based on borehole peek, characterized in that: The method comprises: Acquire the depth image data of the rock layer in the peephole at the distribution position of the loose circle in the surrounding rock of the tunnel through the borehole peephole; wherein the borehole peephole is integrated with a depth camera; Acquire visual point cloud data of the rock formation in the peephole according to the depth image data; Based on the visual point cloud data, a trained target recognition model is used to identify the cracks in the rock formation in the peephole to obtain crack point cloud information of the cracks in the rock formation in the peephole; The fracture point cloud information of the fracture is analyzed to obtain target monitoring information of the fracture in the rock formation in the peephole.

2. The method according to claim 1, characterized in that The number of cracks in the rock formation in the peephole is at least one, the target monitoring information includes size information of the corresponding cracks, and the depth image data includes multiple frames of continuous depth images; The analyzing the crack point cloud information of the crack to obtain target monitoring information of the crack in the rock formation in the peephole includes: For any of the cracks in the rock formation in the peephole, based on the crack point cloud information of the crack, determine a target depth image where the crack exists from the multiple frames of continuous depth images; According to the target depth image, acquiring a target area image corresponding to the fissure; The size information of the crack is determined according to the target area image corresponding to the crack.

3. The method according to claim 1, characterized in that The step of acquiring a target area image corresponding to the fissure according to the target depth image includes: Performing target detection on the target depth image to obtain a plurality of first detection frames; Based on the multiple first detection frames, segment the target depth image to obtain multiple first region images; For any of the first regional images, matching the crack point cloud information of the crack with the first regional image to determine a first matching degree between the crack point cloud information of the crack and the first regional image; In response to the first matching degree being greater than a first set threshold, the first region image is determined as a target region image corresponding to the crack.

4. The method according to claim 2, characterized in that: The target depth image is a plurality of frames; The step of acquiring a target area image corresponding to the fissure according to the target depth image includes: Splicing multiple frames of the target depth image to obtain a spliced ​​image; Performing target detection on the stitched image to obtain a plurality of second detection frames; Based on the multiple second detection frames, segment the stitched image to obtain multiple second region images; For any second region image, matching the crack point cloud information of the crack with the second region image to obtain a second matching degree between the crack point cloud information of the crack and the second region image; In response to a second matching degree between the crack point cloud information of the crack and the second region image being greater than a second set threshold, the second region image is determined as a target region image corresponding to the crack.

5. The method according to claim 2, characterized in that: The target monitoring information also includes the distribution position and crack depth of the corresponding cracks, and the crack point cloud information includes the spatial coordinates of multiple spatial points; The analyzing the crack point cloud information of the crack to obtain target monitoring information of the crack in the rock formation in the peephole includes: Based on the spatial coordinates of the plurality of spatial points in the crack point cloud information of the crack, the distribution position and the crack depth of the crack are determined; wherein the distribution position includes the starting point and the end point of the corresponding crack.

6. The method according to claim 5, characterized in that The size information includes crack width and crack length, and the method further includes: Generate a first crack statistical report based on the target monitoring information of each crack; The first crack statistical report is used to indicate at least one of the following: The number of fractures in each depth statistical range; The average fracture depth of each depth statistical range; Maximum crack depth in each depth statistical range; Maximum crack length in each depth statistical range; Average crack length in each depth statistical range; The average crack width in each depth statistical range; The maximum crack width in each depth statistical range.

7. The method according to claim 1, characterized in that The depth image data includes multiple frames of continuous depth images, and the visual point cloud data includes spatial coordinates of multiple spatial points; The step of acquiring visual point cloud data of the rock formation in the peephole according to the depth image data comprises: Performing image registration on the multiple frames of continuous depth images to determine the camera pose when the depth camera captures each frame of the depth image; For any frame of the depth image, converting the pixel coordinates of the pixel points of the depth image into the spatial coordinates of the corresponding spatial points in the target coordinate system according to the camera pose when the depth camera takes the depth image; The visual point cloud data is generated according to the spatial coordinates of the spatial points corresponding to the depth image of each frame.

8. The method according to any one of claims 1 to 7, characterized in that: After acquiring visual point cloud data of the rock formation in the peephole according to the depth image data, the method further includes: The visual point cloud data is voxelized.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Generating a second crack statistical report according to the target monitoring information of each crack; The second crack statistical report is used to indicate at least one of the following: the number of fractures in the rock formation within the peephole; Target detection information of each of the cracks.

10. A surrounding rock crack monitoring device based on drilling peek, characterized in that: The device comprises: A first acquisition module is used to acquire depth image data of rock layers in a peephole at a distribution position of loose circles in surrounding rocks of a tunnel through a borehole peephole; wherein the borehole peephole is integrated with a depth camera; A second acquisition module is used to acquire visual point cloud data of the rock layer in the peephole according to the depth image data; A recognition module, configured to recognize the cracks in the rock formation in the peephole by using a trained target recognition model based on the visual point cloud data, so as to obtain crack point cloud information of the cracks in the rock formation in the peephole; The analysis module is used to analyze the fracture point cloud information of the fracture to obtain the target monitoring information of the fracture in the rock formation in the peephole.

Images