Rock mass fracture key point identification and skeleton reconstruction method based on multi-task network

Abstract

The invention provides a rock mass fracture key point identification and skeleton reconstruction method based on a multi-task network, and the method comprises the steps: obtaining rock mass fracture image data, and constructing a rock mass fracture sample data set; extracting fracture feature information; acquiring a fracture skeleton line prediction result, and generating a fracture key point thermodynamic diagram; obtaining a rock mass fracture skeleton reconstruction result; performing long fracture judgment on the skeleton reconstruction result according to a fracture bifurcation angle, and performing merging processing to obtain a skeleton line segmentation result; and calculating to obtain occurrence parameters such as rock mass fracture length, dip angle, distribution density and the like. According to the method, more efficient end-to-end output is achieved for rock mass fracture key point recognition and skeleton line extraction based on the multi-task network, the relevance of two tasks is fully considered, mutual constraint between point detection and line extraction is achieved through calculation of joint loss, higher recognition and detection precision is achieved, the calculation result is more reliable, and the method is suitable for large-scale popularization and application. And guidance is provided for rock mass stability evaluation or engineering construction risk assessment.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and computer vision, and in particular to a rock mass fissure key point recognition and skeleton reconstruction method based on a multi-task network. Background Art

[0002] Rock mass is a complex geological body formed in the long geological tectonic movement. The cracks in it, as a major form of rock mass structural surface, are non-continuous and non-uniform, which has a great influence on the mechanical mechanism and properties of the rock mass. The design of geotechnical engineering construction methods is largely based on the survey results of the engineering geological background, which is closely related to the geological survey of the development of rock mass cracks. Rock mass images are obtained through non-contact close-range photography technology, and the rock mass crack area is accurately extracted from the image using computer algorithms. On this basis, the identification of the key points of the cracks and the skeleton reconstruction process are completed. Combining the results of rock mass crack identification and skeletonization and calculating the rock mass crack characterization parameters can serve tasks such as rock mass stability evaluation and engineering construction risk assessment.

[0003] Generally speaking, the requirements for crack skeleton extraction and reconstruction mainly include: ensuring the connectivity of the refined curve, retaining the detailed features of the image, retaining the curve breakpoints, etc. According to these requirements, the early skeletonization algorithms were designed based on rules, such as the fast parallel refinement algorithm, which gradually deleted the edge pixels in the image in an iterative manner until the refinement purpose was achieved, or introduced a preprocessing stage before applying the standard refinement algorithm to improve the refinement accuracy. This type of refinement algorithm can effectively extract the crack skeleton results, but it is very sensitive to the noise in the image. In addition, it can only be extracted for binary images that have completed crack identification, and it is easy to produce interruptions or redundancy during the refinement process. In recent years, with the development of deep learning, convolutional neural networks have shown great potential in skeletonization tasks. CNN is introduced in this field to solve the problems of skeleton interruption or redundancy in traditional rule-driven methods.

[0004] In the prior art, the application of data-driven deep learning methods has made great breakthroughs in improving the accuracy of crack skeletonization, but the results obtained after skeletonization are not sufficient to be directly used in the calculation process of multiple crack characterization parameters. The current method uses a two-step processing method to apply image morphological processing algorithms to the skeletonization results, manually set key point templates, and then detect the coordinates of the key points of the crack skeleton to achieve the crack reconstruction process. This method requires two independent serial processing steps and manual template point setting. The processing efficiency is low and the key point results obtained in complex crack scenes have many redundant points. In addition, this two-stage processing method does not consider the correlation between the skeleton line extraction task and the key point detection task. The deviation of the skeleton line extraction result in the previous step will inevitably lead to inaccurate subsequent key point extraction results. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method for rock fracture key point identification and skeleton reconstruction based on a multi-task network to solve the problems raised in the above-mentioned background technology. The present invention constructs a multi-task network and utilizes two parallel skeleton line extraction branches and key point identification branches to realize an end-to-end process. Moreover, through the mutual constraints of point and line extraction tasks, the accuracy of key point identification and rock fracture reconstruction is effectively improved while improving efficiency.

[0006] In order to achieve the above object, the present invention is implemented by the following technical scheme: a method for identifying key points of rock fractures and reconstructing skeletons based on a multi-task network, comprising the following steps:

[0007] S1. Obtain rock mass fracture image data, perform preprocessing and random enhancement operations on the image, and use annotation tools to annotate the key points and skeleton line true values ​​of the fracture data, divide the data into three parts: training set, validation set and test set, and complete the construction of the rock mass fracture sample data set;

[0008] S2. Using the constructed rock fracture sample dataset as input, a series of convolution, pooling, activation, downsampling and other operations are performed on the fracture dataset through the convolutional neural network encoder structure to extract the fracture feature information in the data;

[0009] S3, using a multi-task branch structure to decode the feature information extracted by the encoder, the multi-task structure includes two branches: skeleton line detection and key point detection. Through the decoding process of the two branches, the crack skeleton line prediction result is obtained, and the crack key point heat map is generated at the same time;

[0010] S4. Screen the candidate points in the key point heat map to obtain three types of key point results, including endpoints, inflection points, and intersection points. Use the skeleton line prediction results to guide the connection process of the key points and obtain the rock mass fracture skeleton reconstruction results.

[0011] S5, judging long cracks according to the bifurcation angles formed by different line segments in the skeleton reconstruction result, judging line segments with large bifurcation angles as long cracks and merging them to obtain skeleton line segmentation results;

[0012] S6. Based on the identification of key fracture points, skeleton reconstruction and segmentation results, the occurrence parameters such as fracture length, inclination, and distribution density of the rock mass are calculated.

[0013] Furthermore, in the process of constructing the rock fracture sample dataset, the rock fracture data obtained by field photography with drones is used as the main data source, and the open source fracture image data collected by crawling is used as a supplement, and the preprocessing and random image enhancement operations are performed in superposition.

[0014] Furthermore, a sliding window of a certain size is used to perform a cropping operation on the image to obtain slice data, and the slice data is divided into a training set, a test set, and a validation set in a certain ratio for subsequent model training and prediction processes.

[0015] Furthermore, the multi-task structure performs a decoding operation on the feature information extracted by the encoder, including two parallel branches of key point detection and skeleton line detection.

[0016] Furthermore, the key point detection branch performs a decoding operation on the crack feature information obtained by the encoder, and the target key point types are defined as three categories: endpoints, inflection points, and intersections, where intersections include X-type intersections and Y-type intersections. The decoded feature map contains three channels, each channel predicts a type of key point, sets a threshold to screen the key point clusters on each heat map, and obtains the final candidate points. The coordinate offset correction is performed on all candidate points on each feature map, and the coordinate points are mapped to the original image position to obtain the final key point prediction result.

[0017] Furthermore, the skeleton line detection branch also performs a decoding operation on the crack feature information obtained by the encoder, and obtains the crack skeleton line prediction result by regression.

[0018] Furthermore, the loss values ​​of the network predicted key points and the real key points, as well as the loss values ​​of the predicted skeleton line and the real skeleton line are calculated, and the two branches are combined and the joint loss is calculated for feedback, so that the model can converge in multiple training processes, so as to realize the mutual constraint between the two branch tasks.

[0019] Furthermore, the long crack determination process in S5 includes the following steps:

[0020] S5.1. Use the crack line segments obtained by network reconstruction as input, find the crack segments with the same points, and adjust the direction of the crack vector to ensure the consistent angle.

[0021] S5.2. Calculate the angles of crack segments with the same points, merge the crack segments with large vector angles into three-point long cracks, and output the crack segments with small angles directly without processing.

[0022] Furthermore, the occurrence parameter calculation process in S6 includes the following steps:

[0023] S6.1. Based on the skeleton line results obtained by network extraction, the coordinate position of the key point is used as the boundary, and the length of a single crack is calculated using the differential method. The length of each micro-segmented crack unit is accumulated to obtain the total length of a single crack.

[0024] S6.2. Based on the skeleton line reconstruction and segmentation results, the apparent inclination angle of a single crack is calculated using a linear regression algorithm combined with a circular function.

[0025] S6.3. Based on the results of the fracture skeleton reconstruction, the fracture distribution density parameter is obtained by calculating the number of fractures per unit area using a statistical method.

[0026] Beneficial effects of the present invention:

[0027] 1. The present invention can achieve multi-task end-to-end output with high processing efficiency: for the skeletonization results of rock fractures, by constructing a multi-task network, skeleton line regression and key point heat map generation operations are directly performed on the fracture features learned in the encoder, and the results are extracted in an end-to-end manner, which effectively improves the processing efficiency.

[0028] 2. The present invention takes into account the relevance of the two tasks, and the key point detection and skeleton reconstruction are more accurate. The two branches are integrated into the multi-task network for end-to-end output, and the joint loss of the two tasks is calculated for feedback. The model converges during multiple training processes, point detection and line extraction constrain each other, and the line generation results can guide the point connection process. The final key point detection and skeleton line reconstruction results are more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of a method for identifying key points of rock mass fractures and reconstructing skeletons based on a multi-task network according to the present invention;

[0030] Figure 2 A schematic diagram of a multi-task network structure provided by an embodiment of the present invention;

[0031] Figure 3A schematic diagram of a process for determining a long crack provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0033] See also Figures 1 to 3 The present invention provides the following technical solution: a method for identifying key points of rock fractures and reconstructing skeletons based on a multi-task network, comprising the following steps:

[0034] S1. Prepare data, obtain rock fracture image data, preprocess and randomly enhance the image before inputting it into the network. Use annotation tools to annotate the key points and skeleton line true values ​​of the fracture data, divide the data into three parts: training set, validation set and test set, and complete the construction of the rock fracture sample set.

[0035] In this embodiment, the rock fracture data obtained by the field shooting of the drone is used as the main data source, and the open source fracture data collected by crawling is used as a supplement, and the preprocessing and random image enhancement operations are superimposed. In order to construct a sample set suitable for key point and skeleton line detection, the open source annotation tool Labelme is used to annotate the key points and skeleton lines in the image data. The original image is cropped in a unified size, and the cropping size can be set to 512×512. The sample set can be divided into a training set and a test set according to a certain ratio. The ratio of the training set, the validation set, and the test set can be set to 6:3:1, 60% of the data is used for training, 30% of the data is used for model validation, and 10% of the data is used to test the performance of the model.

[0036] S2. Using the constructed rock fracture sample dataset as input, a series of convolution, pooling, activation, downsampling and other operations are performed on the rock fracture sample dataset in the encoder of the multi-task network to extract the feature information of the rock fracture skeleton lines and key points in the dataset.

[0037] In this embodiment, the encoder is formed by stacking a continuous multi-layer feature extraction structure, each layer of which is composed of three parts: continuous convolution, normalization, and activation. Through a series of convolution, pooling, activation, downsampling and other operations, the key points of rock fractures and skeleton line feature information in the data set are extracted.

[0038] S3. A multi-task branch structure is used to decode the feature information extracted by the encoder. The multi-task structure includes two branches: skeleton line detection and key point detection. Through the decoding process of the two branches, the crack skeleton line prediction result is obtained, and the crack key point heat map is generated at the same time.

[0039] In this embodiment, the key point detection branch performs a decoding operation on the crack feature information obtained by the encoder, and the target key point types are defined as three categories: endpoints, inflection points, and intersections, where the intersections include X-type intersections and Y-type intersections. The decoded feature map contains three channels, each channel for the prediction of a type of key point.

[0040] In this embodiment, the input image I∈R is of height H and width W. H×W×3 , for each image, there is a corresponding true value heat map such that Y∈[0,1] H×W The purpose of this process is to generate a corresponding candidate key point heat map Y′∈[0,1] H×W , which represents the vertex of each instance, predicting Y′=1 corresponds to the key point position detected, and predicting Y′=0 corresponds to the background detected. For each true key point p∈R 2 , the real key point mapping is guided by the Gaussian kernel function, and the corresponding kernel function formula is where σ p 2 is the object size adaptive standard deviation. During the training process, the predicted key point coordinates are constrained to be within the radius of the true value position. Therefore, the training object is set to the loss L with the corrected focus. keypoint , to ensure the balance between the positive and negative positions, the formula definition is set as follows in this embodiment:

[0041]

[0042] Among them, α and β are hyperparameters in the focal loss, N is the number of key points, and in this embodiment, α is set to 2 and β is set to 4. Set a threshold to screen the key point clusters on each heat map to obtain the final candidate points. Perform coordinate offset correction on all candidate points on each feature map and map the coordinate points to the original image position. The resolution of the heat map output result of the key point detection branch in the network is 1 / 4 of the original image. It is necessary to find the maximum position of the predicted information cluster for each key point as the predicted key point position, and map it to the original image to obtain the key point coordinates on the original image.

[0043] The skeleton line detection branch also decodes the crack feature information obtained by the encoder and obtains the crack skeleton line prediction result by regression. The loss value between the network predicted key points and the real key points and the loss value between the predicted skeleton line and the real skeleton line are calculated. The two branches are combined and the joint loss L is calculated and fed back, where L = αL keypoint +βL skeleton , the model is converged during multiple training processes to achieve the mutual constraints between the two branch tasks.

[0044] S4. The candidate points in the key point heat map are screened to obtain three types of key point results, including endpoints, inflection points, and intersection points. The skeleton line prediction results are used to guide the connection process of the key points to obtain the rock mass fracture skeleton reconstruction results.

[0045] In this embodiment, the process of skeleton line guiding key point connection specifically includes the following steps: (1) all crack key points are removed, and the crack skeleton line forms a plurality of disconnected line segments; (2) two removed crack key points are randomly selected for backfilling; (3) it is determined whether the two backfilled key points are connected. If they are connected, it can be proved that the two key points are adjacent and can be included in the crack reconstruction result; (4) the two previously backfilled key points are removed, and the three steps (2), (3) and (4) are repeated until all key points are backfilled in pairs; (5) the reconstruction result is output.

[0046] S5. Determine long cracks according to the bifurcation angles formed by different line segments in the skeleton reconstruction result, determine the line segments with large bifurcation angles as long cracks and merge them to obtain the skeleton line segmentation result.

[0047] In this embodiment, the process of determining long cracks is divided into the following steps: using the crack line segments obtained by network reconstruction as input, finding crack segments with the same points, and adjusting the direction of the crack vector to ensure consistent angles, calculating the angles of the crack segments with the same points, and merging cracks with large vector angles into three-point long cracks. The angle threshold here is set to 150 degrees in this example, and crack segments with angles less than the threshold angle are directly output without processing.

[0048] S6. Based on the identification of key fracture points, skeleton reconstruction and segmentation results, the occurrence parameters such as fracture length, inclination, and distribution density of the rock mass are calculated.

[0049] In this embodiment, the crack length is obtained through the main skeleton line, and the true length is gradually approached by differential accumulation. First, the distance from the mth point to the m+1th point is calculated, and then the distance between each two points is integrated to gradually approach the true length of the crack. The specific formula is:

[0050]

[0051] Among them, D m is the distance between point m and point m+1; i m 、i m+1 is the horizontal vector of points m and m+1; q m ,q m+1 is the vertical vector of points m and m+1.

[0052] In this embodiment, the crack width can be comprehensively calculated through the crack skeleton line extraction results and the crack identification results, and the average width is used as the crack width parameter for characterization. The crack contour points are connected with rays at the center of the skeleton point and 15 degrees apart to form 10 line segments. The lengths of the 10 line segments are counted, and the minimum length is recorded as the crack width at the skeleton point. The specific formula is:

[0053] ω i =min(ω i -30° ,ω i -15° ,ω i -0° ,ω i 15° ,ω i 30° )

[0054]

[0055] Among them, ω i is the minimum length of the line segment at point i of the crack, and ω′ is the average value of the gap width at multiple locations of the crack.

[0056] The basic principles and main features of the present invention and the advantages of the present invention are shown and described above. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-mentioned exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention.

[0057] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for identifying key points and reconstructing skeletons of rock mass fractures based on multi-task networks. It is characterized in that The following steps are involved: S1. Obtain rock mass fracture image data, perform preprocessing and random enhancement operations on the image, and use annotation tools to annotate the key points and skeleton line true values ​​of the fracture data, divide the data into three parts: training set, validation set and test set, and complete the construction of the rock mass fracture sample data set; S2. Using the constructed rock fracture sample dataset as input, a series of convolution, pooling, activation, downsampling and other operations are performed on the fracture dataset through the convolutional neural network encoder structure to extract the fracture feature information in the data; S3, using a multi-task branch structure to decode the feature information extracted by the encoder, the multi-task structure includes two branches: skeleton line detection and key point detection. Through the decoding process of the two branches, the crack skeleton line prediction result is obtained, and the crack key point heat map is generated at the same time; S4. Screen the candidate points in the key point heat map to obtain three types of key point results, including endpoints, inflection points, and intersection points. Use the skeleton line prediction results to guide the connection process of the key points and obtain the rock mass fracture skeleton reconstruction results. S5, judging long cracks according to the bifurcation angles formed by different line segments in the skeleton reconstruction result, judging line segments with large bifurcation angles as long cracks and merging them to obtain skeleton line segmentation results; S6. Based on the identification of key fracture points, skeleton reconstruction and segmentation results, the occurrence parameters such as fracture length, inclination, and distribution density of the rock mass are calculated.

2. According to the method for identifying key points and reconstructing skeletons of rock mass fractures based on a multi-task network according to claim 1, Features: In the process of constructing the rock fracture sample dataset, the rock fracture data obtained by field photography with drones is used as the main data source, and the open source fracture image data collected by crawling is used as a supplement, which is then superimposed for preprocessing and random image enhancement operations.

3. According to claim 2, a method for identifying key points and reconstructing skeletons of rock mass fractures based on a multi-task network, Features: The image is cropped using a sliding window of a certain size to obtain slice data, and the slice data is divided into a training set, a test set, and a validation set in a certain ratio for subsequent model training and prediction processes.

4. The method for identifying key points and reconstructing skeletons of rock mass fractures based on a multi-task network according to claim 1, Features: The multi-task structure performs a decoding operation on the feature information extracted by the encoder, and includes two parallel branches: key point detection and skeleton line detection.

5. The method for identifying key points and reconstructing skeletons of rock mass fractures based on a multi-task network according to claim 4, Features: The key point detection branch performs a decoding operation on the crack feature information obtained by the encoder. The target key point types are defined as three categories: endpoints, inflection points, and intersections, where intersections include X-type intersections and Y-type intersections. The decoded feature map contains three channels, each channel is used to predict a type of key point, and a threshold is set to screen the key point clusters on each heat map to obtain the final candidate points. The coordinate offset correction is performed on all candidate points on each feature map, and the coordinate points are mapped to the original image position to obtain the final key point prediction result.

6. The method for identifying key points and reconstructing skeletons of rock mass fractures based on a multi-task network according to claim 5, Features: The skeleton line detection branch also performs a decoding operation on the crack feature information obtained by the encoder, and obtains the crack skeleton line prediction result by regression.

7. The method for identifying key points and reconstructing skeletons of rock mass fractures based on a multi-task network according to claim 6, Features: Calculate the loss value between the network's predicted key points and the true key points, as well as the loss value between the predicted skeleton line and the true skeleton line. Combine the two branches and calculate the joint loss for feedback. Make the model converge during multiple training processes to achieve the mutual constraint between the two branch tasks.

8. The method for identifying key points and reconstructing skeletons of rock mass fractures based on a multi-task network according to claim 1, Features: The long crack determination process in S5 comprises the following steps: S5.

1. Use the fracture line segments obtained by network reconstruction as input, find fracture segments with the same points, and adjust the fracture vector direction to ensure the angle is consistent; S5.

2. Calculate the angles of crack segments with the same points, merge the crack segments with large vector angles into three-point long cracks, and output the crack segments with small angles directly without processing.

9. The method for identifying key points and reconstructing skeletons of rock mass fractures based on a multi-task network according to claim 1, It is characterized in that The occurrence parameter calculation process in S6 includes the following steps: S6.

1. Based on the skeleton line results obtained by network extraction, the coordinate position of the key point is used as the boundary, and the length of a single crack is calculated by differential method. The length of each micro-segmented crack unit is accumulated to obtain the total length of a single crack; S6.

2. Based on the skeleton line reconstruction and segmentation results, the apparent inclination angle of a single crack is calculated according to the linear regression algorithm combined with the circular function; S6.

3. Based on the results of the fracture skeleton reconstruction, the fracture distribution density parameter is obtained by calculating the number of fractures per unit area using a statistical method.

Images