A rock mass fracture key point identification and skeleton reconstruction method based on multi-task network

Abstract

The present invention provides a method for identifying key points and reconstructing the skeleton of rock fractures based on a multi-task network, comprising: acquiring rock fracture image data and constructing a rock fracture sample dataset; extracting fracture feature information; obtaining fracture skeleton line prediction results and simultaneously generating a fracture key point thermal map; obtaining rock fracture skeleton reconstruction results; determining long fractures on the skeleton reconstruction results based on fracture bifurcation angles and merging them to obtain skeleton line segmentation results; and calculating rock fracture length, inclination, distribution density, and other occurrence parameters. The present invention achieves more efficient end-to-end output for rock fracture key point identification and skeleton line extraction based on a multi-task network, fully considering the relevance of the two tasks. By calculating a joint loss, mutual constraints are implemented between point detection and line extraction, achieving higher recognition and detection accuracy and more reliable calculation results, providing guidance for rock stability evaluation or engineering construction risk assessment.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and computer vision, and in particular to a rock mass fissure key point recognition and skeleton reconstruction method based on a multi-task network. Background Art

[0002] Rock masses are complex geological bodies formed during lengthy geological tectonic movements. Fractures, a major form of rock mass structural surface, are discontinuous and heterogeneous, significantly impacting the mechanical mechanisms and properties of the rock mass. The design of geotechnical engineering construction methods is largely based on the results of engineering geological background surveys, which are in turn closely linked to geological surveys of rock mass fracture development. Non-contact close-range photography is used to capture rock mass images, and computer algorithms are used to accurately extract rock mass fracture areas from these images. This allows for the identification and skeleton reconstruction of key fracture points. Combining the results of rock mass fracture identification and skeletonization with the calculated rock mass fracture characterization parameters can be used for tasks such as rock mass stability evaluation and engineering construction risk assessment.

[0003] Generally speaking, the requirements for crack skeleton extraction and reconstruction mainly include: ensuring the connectivity of the refined curve, preserving the detailed features of the image, and preserving the curve breakpoints. Based on these requirements, early skeletonization algorithms were designed based on rules. For example, a fast parallel refinement algorithm gradually deleted edge pixels in the image through iteration until the refinement goal was achieved, or a preprocessing stage was introduced before applying the standard refinement algorithm to improve the refinement accuracy. This type of refinement algorithm can effectively extract crack skeleton results, but is very sensitive to noise in the image. In addition, it can only be extracted from binary images that have completed crack identification, and is prone to interruptions or redundancy during the refinement process. In recent years, with the development of deep learning, convolutional neural networks have shown great potential in skeletonization tasks. CNNs have been introduced in this field to solve the skeleton interruption and redundancy problems existing in traditional rule-driven methods.

[0004] In the existing technology, the application of data-driven deep learning methods has made significant breakthroughs in improving the accuracy of crack skeletonization. However, the results obtained after skeletonization are not sufficient to be directly used in the calculation process of multiple crack characterization parameters. The current method uses a two-step processing method to apply image morphological processing algorithms to the skeletonization results, manually set key point templates, and then detect the coordinates of the crack skeleton key points to achieve the crack reconstruction process. This method requires two independent serial processing steps and manual template point setting. The processing efficiency is low, and the key point results obtained in complex crack scenes contain many redundant points. In addition, this two-stage processing method does not consider the correlation between the skeleton line extraction task and the key point detection task. The deviation of the skeleton line extraction result in the previous step will inevitably lead to inaccurate key point extraction results in the subsequent step. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a method for rock fracture key point identification and skeleton reconstruction based on a multi-task network to solve the problems raised in the above background technology. The present invention constructs a multi-task network and utilizes two parallel skeleton line extraction branches and key point identification branches to realize an end-to-end process. Moreover, through the mutual constraints of point and line extraction tasks, the efficiency is improved while effectively improving the accuracy of key point identification and rock fracture reconstruction.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solution: a rock mass fracture key point identification and skeleton reconstruction method based on a multi-task network, comprising the following steps:

[0007] S1. Obtain rock fracture image data, perform preprocessing and random enhancement operations on the image, and use annotation tools to annotate the key points and skeleton line true values ​​of the fracture data. The data is divided into three parts: training set, validation set, and test set to complete the construction of the rock fracture sample dataset;

[0008] S2. Using the constructed rock fracture sample dataset as input, a series of convolution, pooling, activation, downsampling and other operations are performed on the fracture dataset through the convolutional neural network encoder structure to extract the fracture feature information in the data;

[0009] S3. A multi-task branch structure is used to decode the feature information extracted by the encoder. The multi-task structure includes two branches: skeleton line detection and key point detection. Through the decoding process of the two branches, the crack skeleton line prediction result is obtained, and a crack key point heat map is generated at the same time.

[0010] S4. Screen the candidate points in the key point heat map to obtain three types of key point results, including endpoints, inflection points, and intersection points. Use the skeleton line prediction results to guide the connection process of the key points and obtain the rock mass fracture skeleton reconstruction results;

[0011] S5. Determine long cracks based on bifurcation angles formed by different line segments in the skeleton reconstruction result, determine line segments with large bifurcation angles as long cracks, and merge them to obtain a skeleton line segmentation result;

[0012] S6. Based on the identification of key fracture points, skeleton line reconstruction and segmentation results, the occurrence parameters such as the length, inclination, and distribution density of the rock mass fractures are calculated.

[0013] Furthermore, in the process of constructing the rock fracture sample dataset, the rock fracture data obtained by field photography with drones is used as the main data source, and the open source fracture image data collected by crawling is used as a supplement, and the preprocessing and random image enhancement operations are superimposed.

[0014] Furthermore, a sliding window of a certain size is used to perform a cropping operation on the image to obtain slice data, and the slice data is divided into a training set, a test set, and a validation set in a certain ratio for subsequent model training and prediction process.

[0015] Furthermore, the multi-task structure performs a decoding operation on the feature information extracted by the encoder, including two parallel branches of key point detection and skeleton line detection.

[0016] Furthermore, the key point detection branch decodes the crack feature information obtained by the encoder. The target key point types are defined as three categories: endpoints, inflection points, and intersections. Intersections include X-shaped intersections and Y-shaped intersections. The decoded feature map contains three channels, each of which predicts a type of key point. A threshold is set to filter the key point clusters on each heat map to obtain the final candidate points. Coordinate offset correction is performed on all candidate points on each feature map, and the coordinate points are mapped to the original image position to obtain the final key point prediction result.

[0017] Furthermore, the skeleton line detection branch also performs a decoding operation on the crack feature information obtained by the encoder, and obtains the crack skeleton line prediction result by regression.

[0018] Furthermore, the loss between the network's predicted key points and the true key points, as well as the loss between the predicted skeleton lines and the true skeleton lines, is calculated. The two branches are combined and the joint loss is calculated and fed back. The model converges over multiple training cycles to achieve mutual constraints between the two branch tasks.

[0019] Furthermore, the long crack determination process in S5 includes the following steps:

[0020] S5.1. Use the fracture line segments obtained by network reconstruction as input, find fracture segments with the same points, and adjust the direction of the fracture vector to ensure the angle is consistent.

[0021] S5.2. Calculate the angles of the crack segments with the same points, merge the crack segments with large vector angles into three-point long cracks, and output the crack segments with small angles directly without processing.

[0022] Furthermore, the process of calculating the occurrence parameters in S6 includes the following steps:

[0023] S6.1. Based on the skeleton line results obtained by network extraction, the coordinate position of the key point is used as the boundary, and the length of a single crack is calculated using the differential method. The length of each micro-segmented crack unit is accumulated to obtain the total length of the single crack.

[0024] S6.2. Based on the skeleton line reconstruction and segmentation results, calculate the apparent inclination angle of a single crack using the linear regression algorithm combined with the loop function.

[0025] S6.3. Based on the results of the fracture skeleton reconstruction, the fracture distribution density parameter is obtained by calculating the number of fractures per unit area using a statistical method.

[0026] Beneficial effects of the present invention:

[0027] 1. The present invention can achieve multi-task end-to-end output with high processing efficiency: For the rock fracture skeletonization results, by constructing a multi-task network, skeleton line regression and key point heat map generation operations are directly performed based on the fracture features learned in the encoder, and the results are extracted in an end-to-end manner, effectively improving the processing efficiency.

[0028] 2. This invention takes into account the relevance of the two tasks, achieving higher accuracy in key point detection and skeleton reconstruction. By integrating the two branches into a multi-task network for end-to-end output and then calculating the joint loss of the two tasks for feedback, the model reaches convergence after multiple training sessions. Point detection and line extraction are mutually constrained, and line generation can guide the point connection process, resulting in higher accuracy in key point detection and skeleton line reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the process of a rock mass fracture key point identification and skeleton reconstruction method based on a multi-task network of the present invention;

[0030] Figure 2 A schematic diagram of a multi-task network structure provided by an embodiment of the present invention;

[0031] Figure 3A schematic diagram of the process of determining long cracks provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.

[0033] See also Figures 1 to 3 The present invention provides the following technical solution: a method for identifying key points of rock fractures and reconstructing their skeleton based on a multi-task network, comprising the following steps:

[0034] S1. Data preparation: Obtain rock fracture image data. Preprocess and randomly enhance the image before inputting it into the network. Use annotation tools to annotate the fracture data with key points and true skeleton lines. Divide the data into three parts: a training set, a validation set, and a test set, to complete the construction of the rock fracture sample set.

[0035] In this embodiment, rock fracture data obtained by field photography using drones is used as the primary data source, and open source fracture data collected by crawling is used as a supplement, and preprocessing and random image enhancement operations are performed on them. In order to construct a sample set suitable for key point and skeleton line detection, the open source annotation tool Labelme is used to perform true value annotation of key points and skeleton lines in the image data. The original image is cropped using a uniform size method, and the crop size can be set to 512×512. The sample set can be divided into a training set and a test set according to a certain ratio. The ratio of training set, validation set, and test set can be set to 6:3:1, with 60% of the data used for training, 30% of the data used for model validation, and 10% of the data used to test the performance of the model.

[0036] S2. Using the constructed rock fracture sample dataset as input, the encoder of the multi-task network performs a series of convolution, pooling, activation, downsampling and other operations on the rock fracture sample dataset to extract the feature information of the rock fracture skeleton lines and key points in the dataset.

[0037] In this embodiment, the encoder is formed by stacking a series of multi-layer feature extraction structures. Each layer of the structure consists of three parts: continuous convolution, normalization, and activation. Through a series of convolution, pooling, activation, downsampling and other operations, the key points and skeleton line feature information of rock fractures in the data set are extracted.

[0038] S3. A multi-task branch structure is used to decode the feature information extracted by the encoder. The multi-task structure includes two branches: skeleton line detection and key point detection. Through the decoding process of the two branches, the crack skeleton line prediction result is obtained, and the crack key point heat map is generated at the same time.

[0039] In this embodiment, the keypoint detection branch decodes the crack feature information obtained by the encoder. The target keypoint types are defined as endpoints, inflection points, and intersections. Intersections include X-shaped and Y-shaped intersections. The decoded feature map contains three channels, each for predicting a different keypoint type.

[0040] In this embodiment, the input image I∈R is of height H and width W. H×W×3 , for each image, there is a corresponding true value heat map, so that Y∈[0,1] H×W The purpose of this process is to generate a corresponding candidate key point heat map Y′∈[0,1] H×W , which represents the vertex of each instance, the prediction Y′=1 corresponds to the key point position detected, and the prediction Y′=0 corresponds to the background detected. For each true value key point p∈R 2 , the real key point mapping is guided by the Gaussian kernel function, and the corresponding kernel function formula is where σ p 2 Is the object size adaptive standard deviation. During the training process, the predicted key point coordinates are constrained to be within the radius of the true value position. Therefore, the training object is set to the loss L with the corrected focus. keypoint , to ensure the balance between the positive and negative positions, the formula definition is set as follows in this embodiment:

[0041]

[0042] Among them, α and β are hyperparameters in the focal loss, N is the number of key points, and in this embodiment, α is set to 2 and β is set to 4. Set a threshold to screen the key point clusters on each heat map to obtain the final candidate points. Perform coordinate offset correction on all candidate points on each feature map and map the coordinate points to the original image position. The resolution of the heat map output result of the key point detection branch in the network is 1 / 4 of the original image. It is necessary to find the maximum position of the predicted information cluster for each key point as the position of the predicted key point. Mapping it to the original image can obtain the coordinates of the key point on the original image.

[0043] The skeleton line detection branch also decodes the crack feature information obtained by the encoder and obtains the crack skeleton line prediction result through regression. The loss value between the network predicted key points and the real key points and the loss value between the predicted skeleton line and the real skeleton line are calculated. The two branches are combined and the joint loss L is calculated and fed back, where L = αL keypoint +βL skeleton , the model is converged during multiple training processes to achieve mutual constraints between the two branch tasks.

[0044] S4. Screen the candidate points in the key point heat map to obtain three types of key point results, including endpoints, inflection points, and intersection points. Use the skeleton line prediction results to guide the connection process of the key points and obtain the rock mass fracture skeleton reconstruction results.

[0045] In this embodiment, the process of skeleton line guiding key point connection specifically includes the following steps: (1) all crack key points are removed, and the crack skeleton line forms multiple disconnected line segments; (2) two removed crack key points are randomly selected for backfilling; (3) whether the two backfilled key points are connected, if they are connected, it can be proved that the two key points are adjacent and can be included in the crack reconstruction result; (4) the two key points previously backfilled are removed, and the three steps (2)(3)(4) are repeated until all key points are backfilled in pairs; (5) the reconstruction result is output.

[0046] S5. Determine long cracks based on the bifurcation angles formed by different line segments in the skeleton reconstruction result. Determine line segments with large bifurcation angles as long cracks and merge them to obtain skeleton line segmentation results.

[0047] In this embodiment, the process of determining long cracks is divided into the following steps: taking the crack line segments obtained by network reconstruction as input, finding crack segments with the same points, and adjusting the direction of the crack vector to ensure consistent angles, calculating the angles of the crack segments with the same points, and merging the cracks with large vector angles into three-point long cracks. The angle threshold here is set to 150 degrees in this example, and the crack segments with angles less than the threshold angle are directly output without processing.

[0048] S6. Based on the identification of key fracture points, skeleton line reconstruction and segmentation results, the occurrence parameters such as the length, inclination, and distribution density of the rock mass fractures are calculated.

[0049] In this embodiment, the crack length is obtained by the main skeleton line, and the true length is gradually approximated by differential accumulation. First, the distance from point m to point m+1 is calculated, and then the distance between each two points is integrated to gradually approximate the true length of the crack. The specific formula is:

[0050]

[0051] Among them, D m is the distance between point m and point m+1; i m 、i m+1 is the horizontal vector of points m and m+1; q m ,q m+1 is the vertical vector of points m and m+1.

[0052] In this embodiment, the crack width can be comprehensively calculated by the crack skeleton line extraction results and the crack identification results, and the average width is used as the crack width parameter for characterization. With the center of the skeleton point and 15 degrees as the interval, the crack contour points are connected with rays to form 10 line segments. The lengths of the 10 line segments are counted, and the minimum length is recorded as the crack width at the skeleton point. The specific formula is:

[0053] ω i =min(ω i -30° ,ω i -15° ,ω i -0° ,ω i 15° ,ω i 30° )

[0054]

[0055] Among them, ω i is the minimum length of the line segment at point i of the crack, and ω′ is the average value of the width of the crack at multiple locations.

[0056] The basic principles, main features and advantages of the present invention are shown and described above. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention.

[0057] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A rock mass fracture key point identification and skeleton reconstruction method based on a multi-task network, characterized in that: The following steps are involved: S1. Obtain rock fracture image data, perform preprocessing and random enhancement operations on the image, and use annotation tools to annotate the key points and skeleton line true values ​​of the fracture image data. The data is divided into three parts: training set, validation set, and test set to complete the construction of the rock fracture sample dataset; S2. Using the constructed rock fracture sample dataset as input, a series of convolution, pooling, activation, and downsampling operations are performed on the fracture dataset through the convolutional neural network encoder structure to extract the fracture feature information in the data; S3. A multi-task branch structure is used to decode the feature information extracted by the encoder. The multi-task structure includes two branches: skeleton line detection and key point detection. Through the decoding process of the two branches, a crack skeleton line prediction result is obtained, and a crack key point heat map is generated at the same time. The multi-task structure decodes the feature information extracted by the encoder, including two parallel branches: key point detection and skeleton line detection. The key point detection branch decodes the crack feature information obtained by the encoder. The target key point type is defined as three categories: endpoint, inflection point, and intersection point, where the intersection point includes X-type intersection point and Y-type intersection point. The decoded feature map contains three channels, each channel is for the prediction of a type of key point, and a threshold is set to screen the key point clusters on each heat map to obtain the final candidate points. The coordinate offset correction is performed on all candidate points on each feature map, and the coordinate points are mapped to the original image position to obtain the final key point prediction result. The skeleton line detection branch also decodes the crack feature information obtained by the encoder and obtains the crack skeleton line prediction result by regression. S4. Screen the candidate points in the key point heat map to obtain three types of key point results, including endpoints, inflection points, and intersections. Use the skeleton line prediction results to guide the connection process of the key points and obtain the rock mass fracture skeleton reconstruction results; calculate the loss value between the network predicted key points and the true key points, as well as the loss value between the predicted skeleton line and the true skeleton line. Combine the two branches and calculate the joint loss for feedback. After multiple training processes, the model reaches convergence to achieve the mutual constraint between the two branch tasks; S5. Determine long cracks based on bifurcation angles formed by different line segments in the skeleton reconstruction result, determine line segments with large bifurcation angles as long cracks, and merge them to obtain a skeleton line segmentation result; S6. Based on the identification of key fracture points, skeleton line reconstruction and segmentation results, calculate the length, dip, distribution density and occurrence parameters of rock mass fractures.

2. The method for identifying key points and reconstructing the skeleton of rock mass fractures based on a multi-task network according to claim 1, characterized in that: In the process of constructing the rock fracture sample dataset, the rock fracture data obtained by field photography with drones is used as the main data source, and the open source fracture image data collected by crawling is used as a supplement, and the preprocessing and random image enhancement operations are performed on them.

3. The method for identifying key points and reconstructing the skeleton of rock mass fractures based on a multi-task network according to claim 2, characterized in that: The image is cropped using a sliding window of a certain size to obtain slice data, which is then divided into a training set, a test set, and a validation set in a certain proportion for subsequent model training and prediction processes.

4. The method for identifying key points and reconstructing the skeleton of rock mass fractures based on a multi-task network according to claim 1, characterized in that: The long crack determination process in S5 includes the following steps: S5.

1. Use the fracture line segments obtained by network reconstruction as input, find fracture segments with the same points, and adjust the direction of the fracture vectors to ensure the angle is consistent; S5.

2. Calculate the angles of the crack segments with the same points, merge the crack segments with large vector angles into three-point long cracks, and output the crack segments with small angles directly without processing.

5. The method for identifying key points and reconstructing the skeleton of rock mass fractures based on a multi-task network according to claim 1, characterized in that: The process of calculating the occurrence parameters in S6 includes the following steps: S6.

1. Based on the skeleton line results obtained from network extraction, use the coordinate position of the key point as the boundary to calculate the length of a single fracture using the differential method. The length of each micro-segmented fracture unit is accumulated to obtain the total length of the single fracture. S6.

2. Based on the skeleton line reconstruction and segmentation results, calculate the apparent inclination angle of a single fracture using a linear regression algorithm combined with a circular function; S6.

3. Based on the results of the fracture skeleton reconstruction, the fracture distribution density parameter is obtained by calculating the number of fractures per unit area using a statistical method.

Images