Coal mine tunnel deformation detection basic model based on self-supervised learning and detection method

Abstract

The invention relates to a coal mine tunnel deformation detection basic model based on self-supervised learning and a detection method, the model comprises a tunnel deformation point cloud data generation module, a data enhancement module and a tunnel deformation intelligent detection module which are connected in sequence, and the tunnel deformation intelligent detection module comprises a self-supervised learning module and a deformation detection module; the method comprises the following steps: generating roadway point cloud data with different deformation types through a simulation method by utilizing the roadway deformation point cloud data generation module; performing deformation processing on the deformed roadway point cloud data by using a data enhancement module; constructing a self-supervised learning module for automatically extracting features from the unlabeled point cloud data; a deformation detection module is constructed, and accurate identification of a roadway deformation area and evaluation of the deformation degree are achieved; and the roadway deformation intelligent detection module is used for automatically identifying and evaluating a roadway deformation area. According to the model and the method, the accuracy and the efficiency of roadway deformation detection can be effectively improved, and an efficient and intelligent solution can be provided for safety monitoring of the roadway.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud target detection and intelligent monitoring of tunnel deformation, and specifically relates to a basic model and a detection method for coal mine tunnel deformation detection based on self-supervised learning. Background Art

[0002] In recent years, with the rapid development of artificial intelligence, the Internet of Things, 5G communications, edge computing, and robotics, intelligent mine monitoring technology has made significant progress, and the application of artificial intelligence and deep learning technologies has shown great potential in the field of mine monitoring. The rapid development of technologies such as multimodal data fusion, real-time environmental perception, and geological structure prediction has not only improved the safety and efficiency of mine production, but also made automated and precise mine monitoring possible. In terms of mine safety monitoring, three-dimensional point cloud processing and sensor data analysis methods based on deep learning have achieved remarkable results in tasks such as environmental monitoring, equipment health assessment, and structural deformation detection. Through these intelligent monitoring technologies, potential risks such as geological structure changes, equipment failures, and tunnel deformation in mines can be detected and predicted in real time, and early warnings can be issued in a timely manner, thereby effectively avoiding the occurrence of disasters, ensuring the safety of miners, and improving production efficiency.

[0003] In intelligent mine monitoring, commonly used sensor types include laser radar (LiDAR), high-definition cameras, infrared thermal imaging sensors, ground-based radar (GPR), etc. Each sensor plays its own unique role in different scenarios. Laser radar (LiDAR) has obvious advantages over image sensors. LiDAR can provide accurate three-dimensional point cloud data, which can directly reflect spatial depth information and is suitable for three-dimensional monitoring of tunnel deformation, terrain changes, etc., while image sensors can only provide two-dimensional images and lack depth perception capabilities. In addition, LiDAR is not affected by lighting conditions and factors such as dust and moisture in the mine environment, and can work stably in dark or harsh environments, while image sensors will have problems such as image blur and loss of details in low-light or interference environments. In terms of data processing, LiDAR can provide high-density point cloud data, which can accurately capture small deformations and perform three-dimensional modeling, while images require complex calculations to derive depth information. Therefore, LiDAR has more advantages in mine tunnel deformation detection and spatial monitoring, and can provide more accurate and stable monitoring results.

[0004] Point cloud data used for intelligent monitoring of mines are mainly collected through geological radar or lidar equipment. These data contain high-precision three-dimensional spatial information of the mine environment. However, the point cloud data of coal mine tunnels involve spatially dense three-dimensional coordinate information, and the deformation at different time points is relatively subtle. Labeling the deformation information of each point is very complex and labor-intensive. At the same time, the standardized labeling method is not mature. Therefore, the use of self-supervised learning methods can effectively reduce the burden of manual labeling and avoid relying on complex and immature standardized labeling processes. Self-supervised learning can identify and model subtle deformations by automatically learning the spatiotemporal characteristics of point cloud data from unlabeled data, and generate high-quality training data in an unsupervised manner, greatly improving the efficiency and accuracy of data processing. In addition, self-supervised learning can automatically adjust the model to adapt to different tunnel deformation modes, thereby enhancing the generalization ability of the model and showing strong robustness in the face of practical problems such as missing data and noise interference. In addition, since the deformation of the tunnel occurs slowly and the frequency of collection is limited by mine safety and equipment, the number of data samples is seriously insufficient. Therefore, using 3D point cloud deformation enhancement technology to simulate and generate tunnel point cloud deformation data has become an effective solution. By deforming the existing 3D point cloud data to different degrees, different geological changes, tunnel deformation and other scenarios can be simulated, thereby expanding the diversity of the data set. Such technologies include geometric transformations such as rotation, translation, scaling, and adding noise to the point cloud data to simulate the settlement of the tunnel, crack expansion, or deformation caused by geological disasters. In addition, through methods based on physical models or finite element analysis (FEA), combined with data collected by sensors such as lidar, the deformation of the point cloud can be further enhanced to simulate and generate more realistic training data. These deformation-enhanced data not only help improve the generalization ability of deep learning models in actual environments, but also improve the accuracy and reliability of tunnel deformation detection in the absence of data. These models can quickly and accurately identify abnormal changes in the mining environment through deep learning and point cloud processing technology, and thus provide reliable risk assessment and early warning.

[0005] In summary, the three-dimensional point cloud deformation detection method of coal mine tunnels represents the frontier field of the combination of intelligent mine monitoring and artificial intelligence technology. It has great potential and can effectively improve the accuracy of mine environmental monitoring and risk prediction, providing mine managers with more efficient and safe tools to timely warn of potential geological deformation and equipment failures, thereby ensuring the safety of miners and optimizing production processes. Research and application in this field are at a critical stage of the development of intelligent mines, and further technological breakthroughs and application promotion are urgently needed. Summary of the invention

[0006] In response to the above-mentioned problems of the prior art, the present invention provides a basic model and detection method for coal mine tunnel deformation detection based on self-supervised learning. The model has a simple structure, can effectively improve the intelligence level of tunnel deformation detection, can effectively improve the accuracy of mine environment monitoring and risk prediction, and can provide efficient and intelligent solutions for the safety monitoring of coal mine tunnels; this method can effectively solve the problems of scarce underground tunnel deformation data and complex point cloud data labeling, can reduce dependence on manual labeling, can effectively improve the accuracy and efficiency of tunnel deformation detection, and can significantly enhance the ability of coal mine safety monitoring.

[0007] In order to achieve the above-mentioned object, the present invention provides a basic model for coal mine roadway deformation detection based on self-supervised learning, including a roadway deformation point cloud data generation module, a data enhancement module and a roadway deformation intelligent detection module;

[0008] The tunnel deformation point cloud data generation module is used to generate deformed three-dimensional point cloud data based on the annotated coal mine tunnel original point cloud data and according to the tunnel geometry and different deformation types, and then send the three-dimensional point cloud data to the data enhancement module;

[0009] The data enhancement module is used to perform deformation processing on the three-dimensional point cloud data to different degrees, and at the same time, introduce noise and disturbance to realistically simulate the actual data to expand the diversity of the data set, and obtain an extended data set, and then send the extended data set to the self-supervised learning module;

[0010] The lane deformation intelligent detection module includes a self-supervised learning module and a deformation detection module:

[0011] The self-supervised learning module is used to automatically extract features from unlabeled point cloud data in the extended data set by using variational autoencoder contrastive learning and cross entropy loss supervision, and send the obtained feature data to the deformation detection module;

[0012] The deformation detection module is used to accurately identify the deformation area of ​​the tunnel according to the characteristic data and accurately evaluate the severity of the deformation.

[0013] Preferably, the different deformation types include sinking deformation, extrusion deformation, expansion deformation and tilting deformation.

[0014] Preferably, the data deformation processing includes rotation transformation, translation transformation, scaling transformation and noise addition.

[0015] In the present invention, through the setting of the tunnel deformation point cloud data generation module, tunnel point cloud data with different deformation types can be generated by simulation methods, so that a large amount of simulated three-dimensional tunnel deformation data can be obtained, and sufficient sample data can be obtained; through the setting of the data enhancement module, the original point cloud data set can be fully enhanced by rotation, translation, scaling, and noise addition operations, providing richer and more diverse samples for the training model, which is conducive to obtaining a detection model with higher detection accuracy. Through the setting of the self-supervised learning module, it is convenient to automatically and quickly extract technical features related to tunnel deformation from unlabeled point cloud data, and then provide reliable technical support for the subsequent intelligent detection process. Through the setting of the deformation detection module, the tunnel deformation area and deformation situation can be accurately identified according to the feature data, and the deformation program evaluation operation can be performed. The model has a simple structure, which can effectively improve the intelligence level of tunnel deformation detection, can effectively improve the accuracy of mine environment monitoring and risk prediction, and provide an efficient and intelligent solution for the safety monitoring of coal mine tunnels.

[0016] The present invention also provides a detection method of a basic model for coal mine tunnel deformation detection based on self-supervised learning, which adopts a basic model for coal mine tunnel deformation detection based on self-supervised learning and includes the following steps:

[0017] Step 1: Generate roadway point cloud data with different deformation types by simulation method using roadway deformation point cloud data generation module;

[0018] For the annotated original point cloud data P of coal mine tunnels, according to the tunnel geometry and different deformation types, combined with the physical model and numerical calculation, the deformed tunnel point cloud data P is obtained using formula (1): f ; Where P = {(x i ,y i ,z i )|i=1,2,3,....,N},(x i ,y i ,z i ) represents the coordinates of the i-th 3D point in the lane, and N is the total number of points in the point cloud;

[0019] P f =P·T+b (1);

[0020] In the formula, f represents the deformation type, T represents the transformation matrix related to the deformation type, and b represents the translation offset;

[0021] Step 2: Use the data enhancement module to deform the deformed roadway point cloud data;

[0022] Deformed roadway point cloud data P fEnhanced processing is performed, and at the same time, noise and disturbance are introduced to realistically simulate the actual data. Formula (2) is used to obtain the enhanced point cloud data P aug ;

[0023] P aug =(P f ·R)+t·(1+ε) (2);

[0024] Where R is the enhancement transformation matrix; t is the translation vector; ε represents the noise added randomly, and (1+ε) represents the scaling factor of the noise change added by randomization;

[0025] Step 3: Build an intelligent detection module for tunnel deformation and automatically identify and evaluate the tunnel deformation area;

[0026] S31: Build a self-supervised learning module to automatically extract features from unlabeled point cloud data;

[0027] The variational autoencoder and contrastive learning are combined to form a variational autoencoder contrastive learning network structure, which is used to perform self-supervised feature learning on unlabeled point cloud data; the variational autoencoder is used to extract the potential feature representation of the point cloud data, and through contrastive learning, the distribution of the positive sample features extracted by the encoder is converged to be similar, while distinguishing the distribution between positive samples and negative samples; at the same time, a cross entropy loss function is introduced to supervise the training process, and a generative adversarial mechanism is introduced to optimize the generated potential features through contrastive learning, so that the model can effectively capture the essential features in the point cloud data. At the same time, the cross entropy loss is calculated by the positive samples generated by two different data enhancements to retain more invariant features and promote the consistency of distribution between positive samples; the trained self-supervised learning network is used as the backbone network of the roadway deformation intelligent detection module to extract deep features related to roadway deformation from the input roadway point cloud data;

[0028] S32: construct a deformation detection module to achieve accurate identification of the deformation area of ​​the roadway and evaluation of the deformation degree;

[0029] Multi-layer fully connected layers and convolutional layers are used to generate 3D candidate frames, and the 3D candidate frames are refined as the detection head of the lane deformation intelligent detection module to achieve accurate identification of lane deformation areas and evaluation of deformation degrees based on feature representation;

[0030] S33: Automatically identify and evaluate the deformation area of ​​the tunnel using the tunnel deformation intelligent detection module;

[0031] An intelligent tunnel deformation detection module is composed of a self-supervised learning module and a deformation detection module. A three-dimensional sensor arranged in the tunnel is used to automatically scan the tunnel and generate point cloud data at a set sampling frequency. The point cloud data is input into the self-supervised learning module. The self-supervised learning module is used to automatically extract depth features related to tunnel deformation from the point cloud data. The obtained feature data is input into the deformation detection module. The deformation detection module is used to accurately identify the tunnel deformation area and evaluate the deformation degree. Finally, the identified deformation area and an evaluation report are output.

[0032] As a preferred embodiment, in step 3 S31, the specific process of using the variational autoencoder contrastive learning network structure to perform self-supervised feature learning on unlabeled point cloud data is as follows:

[0033] S31-1: For the point cloud data obtained through data enhancement, two point cloud samples p and q are extracted each time for contrastive learning. The encoder for contrastive learning is G(·). After the sample passes through the encoder, the feature f is obtained. p and f q ;

[0034] S31-2: Using two single-layer fully connected networks F μ (·) and F σ (·) respectively transform the features f p and f q Projected into a high-dimensional invariant space, each point cloud sample generates two vectors μ p ,σ p and μ q ,σ q , let the potential features of the two point cloud samples obey a Gaussian distribution with a mean of μ and a standard deviation of σ:

[0035] S31-3: Use KL divergence as a regularization term to constrain the feature distribution of all samples to a standard normal distribution with a mean of 0 and a standard deviation of 1: D KL (N(μ,σ), N(0,1)); According to formula (3), KL divergence is used to calculate the distance between the Gaussian distributions of two sample features, and samples of the same category are constrained to have similar Gaussian distributions, while the distributions of different categories are made as far apart as possible;

[0036]

[0037] In the formula, P(x) and Q(x) are two sample features f p and f q Gaussian distribution of

[0038] S31-4: Calculate the distance between two Gaussian distributions using the symmetric JS divergence according to formula (4) to calculate the distribution similarity between the positive samples generated by two random data augmentations, and map the distance between the two samples to between 0 and 1;

[0039]

[0040] Where D KL (·) is to calculate the KL divergence between two distributions. The value of JS divergence is between 0 and 1. The smaller the value, the more similar the two distributions are.

[0041] S31-5: According to formula (5), the feature distributions of samples of the same category are made similar, and the feature distributions of samples of different categories are made as far apart as possible;

[0042]

[0043] In the formula, N represents the number of samples in the batch, i represents the i-th sample, k represents the k-th sample, P and Q are the Gaussian distributions of the two sample features, and D JS (·) means calculating the JS divergence between two distributions;

[0044] S31-6: Obtain the feature distributions P(x) and Q(x) of two samples p and q from the variational autoencoder contrastive learning network structure, and sample from P(x) and Q(x) to generate the embedding g p and g q ; will g p and g q Input into the decoder D(·) to generate new point clouds p′ and q′ to introduce randomness and make the generated point cloud data diverse;

[0045] S31-7: Normalize the coordinates of the point clouds p′ and q′ to the range [0,1] and discretize them into a fixed-size grid to convert the point cloud into a probability distribution; at the same time, convert the input point clouds p and q into a probability distribution;

[0046] S31-8: Calculate the cross entropy loss between the probability distribution of the generated point cloud and the probability distribution of the input point cloud according to formula (6) to evaluate the generation quality of the point cloud;

[0047]

[0048] Where P(i) is the probability value of the i-th grid cell in the probability distribution of the input point cloud, and Q(i) is the probability value of the i-th grid cell in the probability distribution of the generated point cloud;

[0049] S31-9: According to formula (7), the cross entropy loss between the probability distribution of the generated point cloud and the probability distribution of the input point cloud is minimized to ensure that the generated red frame cloud is highly consistent with the input point cloud in terms of probability distribution;

[0050] min(CE(P,P′)+CE(Q,Q′))(7);

[0051] Where CE(P,P′) is the cross entropy loss between the input point cloud P and the generated point cloud P′, and CE(Q,Q′) is the cross entropy loss between the input point cloud Q and the generated point cloud Q′.

[0052] As a preferred embodiment, in step 3 S32, by segmenting the point cloud into foreground points and background points, the search space is reduced, the efficiency of generating candidate boxes is improved, and a small number of high-quality 3D candidate boxes are directly generated from the original point cloud, providing a basis for subsequent bounding box refinement; the specific process is as follows:

[0053] S32-11: For the input point cloud X={x 1 ,x 2 ,····,x N}, where each point x i =(x,y,z,h,w,l,θ), (x,y,z) is the coordinate of the center point of the point cloud, (h,w,l) is the length, width and height, θ is the direction angle; the feature F = E(X) is extracted through the trained self-supervised learning network, where F = {f 1 ,f 2 ,····,f N} represents the feature corresponding to each point;

[0054] S32-12: Predict the probability of foreground points through a simple fully connected network, where the input is the feature f of each point i , the output is the probability p that the point is a foreground point i , as shown in formula (8);

[0055] p i =σ(W s f i +b i ) (8);

[0056] Where σ(·) is the Sigmoid activation function, which is used to map the output value to the range [0,1] to represent the probability, and W s and b s They are the weight and bias parameters of the fully connected network respectively;

[0057] S32-13: For each foreground point, bin-based regression loss is used to estimate the parameters of the 3D candidate box, the surrounding area is divided into multiple discrete bins, and the classification loss and residual regression loss are used to estimate the center position. Since the distribution of target objects in the vertical direction is usually more concentrated and the range of variation is small, the loss in the y-axis direction is not calculated;

[0058] First, the target center (x p ,z p ) relative to the foreground point p, and discretize it into bins to obtain the straight value bin distribution L along the x-axis and z-axis directions respectively bin (p) x , L bin (p) z ;

[0059]

[0060] Where (x(p),z(p)) is the coordinate of the foreground point p, (x p ,z p ) is the coordinate of the calculated center point, S is the search range set between 0 and 3, and δ is the bin size set between 100 and 255;

[0061] Secondly, according to formula (11) and formula (12), the residual regression res(p) of the target center is calculated in the determined bin. x 、res(p) z ;

[0062]

[0063] Where C is a normalization constant, which is set to 50 to control the range of the residual.

[0064] As a preferred embodiment, in step 3 S32, more accurate bounding box parameters are learned by standardizing the coordinate system and global semantic features, and the 3D candidate box generated in the previous step is further refined to improve the detection accuracy;

[0065] S32-21: Determine each 3D candidate box b i , and slightly expand its scope to include more contextual information, as shown in formula (13);

[0066]

[0067] In the formula, (x i ,y i ,z i ) is the coordinate of the center point of the point cloud, (h i ,wi ,l i ) are the dimensions length, width and height, θ is the direction angle, and η is a constant for the expansion range, which is set to 0.5;

[0068] S32-22: For each point p, determine whether it is within the expanded candidate box. If so, retain the point and its features. The pooled point features include the point coordinates, reflection intensity, segmentation mask and global features. The pooled point coordinates are converted to the standard coordinate system to eliminate rotation and position changes.

[0069] S32-23: The definition of the standard coordinate system, the origin is located at the center of the candidate box; the local x-axis and z-axis are parallel to the ground plane, the x-axis points to the head direction of the candidate box, and the z-axis is perpendicular to the x-axis; the y-axis is consistent with the y-axis of the LiDAR coordinate system;

[0070] S32-24: Combine the converted local spatial features and global semantic features f(p), encode them into a unified feature vector through a fully connected layer, use these features to refine the bounding box and predict the confidence, and finally, use formula (14) to refine the bounding box using bin-based regression loss;

[0071]

[0072] Where L is the total loss of 3D bounding box refinement, B is the set of all 3D candidate boxes, |B| is the number of candidate boxes, Fcls(·) is the classification loss function used to calculate the difference between the predicted confidence and the true label, and prob i is the prediction confidence of the i-th candidate box, label i The true label of the i-th candidate box, B pos is the set of positive sample candidate boxes, |B pos | is the number of positive sample candidate boxes, is the bin classification loss of the i-th positive sample candidate box, It is the residual regression loss within the bin of the i-th positive sample candidate box.

[0073] As a preferred embodiment, in step 1, the tunnel deformation point cloud data P generated by simulation is obtained through sinking deformation, squeezing deformation, expansion deformation and tilting deformation operations. f ;

[0074] Among them, during the sinking deformation process, the sinking deformation is simulated by translation along the z-axis and the tunnel deformation point cloud data P is generated. f , T 1 is the transformation matrix of the underlying deformation, s is the lower level quantity, which is a random value set between 0 and 50;

[0075] During the extrusion deformation process, the extrusion deformation is simulated by scaling in the x-axis and y-axis directions and generating the roadway deformation point cloud data P f , P f =P·T 2 , T 2 is the transformation matrix of the extrusion deformation, α is the squeezing factor, which is a random value set between 0 and 1;

[0076] During the expansion deformation process, the entire point cloud is enlarged in proportion to simulate the expansion deformation and generate the roadway deformation point cloud data P f , P f =P·T 3 , T 3 is the transformation matrix of the extended deformation, β is the expansion factor, which is a random value set between 0 and 1;

[0077] During the tilt deformation process, the tilt deformation is simulated by rotation and the roadway deformation point cloud data P is generated. f , P f =P·T 4 , T 4 is the transformation matrix of the tilt deformation, θ is the tilt angle, which is a random value set between 0 and 30 degrees.

[0078] As a preferred embodiment, in step 2, the deformed roadway point cloud data Pf is enhanced by rotation transformation, translation transformation, scaling transformation and noise addition, wherein P f ={(x i ′,y i ′,z i ′)|i=1,2,3,....,N 0},(x i ′,y i ′,z i ′) represents the coordinates of the i-th 3D point in the generated lane point cloud data, N 0 is the total number of points in the generated roadway point cloud;

[0079] In the process of rotation transformation, the point cloud is rotated around the z-axis, and each point in the point cloud (x i ′,y i ′,z i ′) After rotation, the new point coordinates are The rotation transformation formula is: Among them, θ 1 is the rotation angle, R is the rotation matrix,

[0080] During the translation transformation, translation is performed by adding translation amounts on the x, y, and z axes. The translation transformation formula is: Where t is the translation vector, t=(t x ,t y ,t z ), t x ,t y ,t z Represents the translation along the x, y, and z axes respectively;

[0081] During the scaling transformation, each coordinate of the point cloud is multiplied by a scaling factor s. x ,s y ,s z To achieve scaling, the scaling transformation formula is:

[0082] In the process of adding noise, a Gaussian distributed random noise N(μ,σ 2 ), the noise adding formula is: Here, μ is the mean and σ is the standard deviation.

[0083] The present invention proposes a detection method for a basic model of coal mine tunnel deformation detection based on self-supervised learning, aiming to solve the problems of data scarcity and difficulty in labeling in coal mine tunnel deformation monitoring. First, three-dimensional point cloud data with diversity is generated by simulating different types of tunnel deformation, and the richness of the data and the robustness of the model are further improved by combining data enhancement technology. Then, using a self-supervised learning network, through variational autoencoder contrast learning and cross entropy loss supervision modules, features are automatically extracted from unlabeled point cloud data, avoiding the complex manual labeling process. On this basis, a tunnel deformation detection network is constructed, and the detection head is used to realize the accurate identification of the tunnel deformation area and the evaluation of the deformation degree.

[0084] Compared with the prior art, the present invention has the following advantages:

[0085] 1. The present invention adopts a self-supervised learning method to learn features through unlabeled three-dimensional point cloud data, which greatly reduces the need for a large amount of labeled data. Compared with the traditional method that relies on a large amount of manually labeled data for training, the present invention can complete accurate and efficient detection of roadway deformation based on a small amount of labeled data, reducing the cost and workload of data annotation.

[0086] 2. Through the deep feature extraction capability of the self-supervised learning network, the present invention can more accurately capture the details and structural changes of the tunnel deformation. Combined with the high-quality features obtained by self-supervised learning, the present invention can effectively deal with various complex situations in the tunnel environment, improve the accuracy and robustness of deformation detection, and perform well in tunnels of different types and sizes.

[0087] 3. The present invention uses a small amount of labeled data sets to fine-tune the self-supervised learning network, which can quickly adapt to different roadway environments and deformation types in practical applications. This method not only improves the adaptability of the model, but also ensures its real-time performance and accuracy in practical scenarios, allowing the network to be efficiently deployed and updated online.

[0088] In summary, this technology not only reduces the reliance on a large amount of labeled data, but also improves the generalization ability and detection accuracy of the model in complex environments, providing an efficient and intelligent solution for the safety monitoring of coal mine tunnels, which has important practical application value. This method can effectively solve the problems of scarce deformation data of underground tunnels and complex point cloud data annotation, thereby improving the accuracy and efficiency of deformation detection, reducing the reliance on manual annotation, and significantly improving the ability of coal mine safety monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 It is a principle block diagram of the model part of the present invention;

[0090] Figure 2 is a flow chart of the method portion of the present invention;

[0091] Figure 3 It is a schematic diagram of the structure of the self-supervised learning function on the downstream detection task in the method part of the present invention;

[0092] Figure 4 It is a schematic diagram of the structure of the variational autoencoder contrast learning network in the method part of the present invention;

[0093] Figure 5 It is a schematic diagram of the structure of the cross entropy loss supervision network in the method part of the present invention;

[0094] Figure 6 It is a schematic diagram of the structure of the deformation detection network in the method part of the present invention. DETAILED DESCRIPTION

[0095] The invention provides a detection method of a basic model for coal mine roadway deformation detection based on self-supervised learning, which includes a roadway deformation point cloud data generation module, a data enhancement module, a self-supervised learning module and a deformation detection module. First, the deformation roadway point cloud data generation module generates deformed three-dimensional point cloud data according to the roadway geometry and different deformation types, such as sinking, extrusion, tilting, expansion, etc., by a traditional simulation data generation method, and introduces noise and disturbance to simulate the actual data realistically. Then, the data enhancement module performs spatial transformations such as rotation and translation on the generated point cloud data to increase the diversity of data and improve the robustness of the model. Then, the self-supervised learning module automatically extracts deformation features from a large amount of unlabeled point cloud data by constructing a self-supervised learning network, and fine-tunes the network so that it can be effectively used for roadway deformation detection. The fine-tuned network is embedded in the roadway deformation detection network in the deformation detection module as a backbone network, and the network is applied to the actual coal mine underground roadway three-dimensional point cloud for deformation detection, automatically identifying the deformation area and evaluating the severity of the deformation.

[0096] The architecture flow chart of the present invention is as follows Figure 1 and Figure 2 In order to more clearly illustrate the objectives, technical solutions and advantages of the present invention, the following will be combined with the attached Figures 1 to 6 And specific implementation steps are provided to further illustrate the present invention comprehensively and clearly.

[0097] The present invention provides a basic model for coal mine tunnel deformation detection based on self-supervised learning, including a tunnel deformation point cloud data generation module, a data enhancement module and a tunnel deformation intelligent detection module;

[0098] The tunnel deformation point cloud data generation module is used to generate deformed three-dimensional point cloud data based on the annotated coal mine tunnel original point cloud data and according to the tunnel geometry and different deformation types, and then send the three-dimensional point cloud data to the data enhancement module;

[0099] The data enhancement module is used to perform deformation processing on the three-dimensional point cloud data to different degrees, and at the same time, introduce noise and disturbance to realistically simulate the actual data to expand the diversity of the data set, and obtain an extended data set, and then send the extended data set to the self-supervised learning module;

[0100] The lane deformation intelligent detection module includes a self-supervised learning module and a deformation detection module:

[0101] The self-supervised learning module is used to automatically extract features from unlabeled point cloud data in the extended data set by using variational autoencoder contrastive learning and cross entropy loss supervision, and send the obtained feature data to the deformation detection module;

[0102] The deformation detection module is used to accurately identify the deformation area of ​​the tunnel according to the characteristic data and accurately evaluate the severity of the deformation.

[0103] Preferably, the different deformation types include sinking deformation, extrusion deformation, expansion deformation and tilting deformation.

[0104] Preferably, the data deformation processing includes rotation transformation, translation transformation, scaling transformation and noise addition.

[0105] In the present invention, through the setting of the tunnel deformation point cloud data generation module, tunnel point cloud data with different deformation types can be generated by simulation methods, so that a large amount of simulated three-dimensional tunnel deformation data can be obtained, and sufficient sample data can be obtained; through the setting of the data enhancement module, the original point cloud data set can be fully enhanced by rotation, translation, scaling, and noise addition operations, providing richer and more diverse samples for the training model, which is conducive to obtaining a detection model with higher detection accuracy. Through the setting of the self-supervised learning module, it is convenient to automatically and quickly extract technical features related to tunnel deformation from unlabeled point cloud data, and then provide reliable technical support for the subsequent intelligent detection process. Through the setting of the deformation detection module, the tunnel deformation area and deformation situation can be accurately identified according to the feature data, and the deformation program evaluation operation can be performed. The model has a simple structure, which can effectively improve the intelligence level of tunnel deformation detection, can effectively improve the accuracy of mine environment monitoring and risk prediction, and provide an efficient and intelligent solution for the safety monitoring of coal mine tunnels.

[0106] The present invention also provides a detection method of a basic model for coal mine tunnel deformation detection based on self-supervised learning, which adopts a basic model for coal mine tunnel deformation detection based on self-supervised learning and includes the following steps:

[0107] Step 1: Generate roadway point cloud data with different deformation types by simulation method using roadway deformation point cloud data generation module;

[0108] The point cloud data of simulated roadway deformation is generated by the existing coal mine roadway point cloud data. The deformation roadway point cloud data generation module generates roadway point cloud data with different deformation types through simulation methods. For the annotated coal mine roadway original point cloud data P, according to the roadway geometric structure and different deformation types, such as sinking, extrusion, tilting, expansion, etc., combined with the physical model and numerical calculation, the deformation roadway point cloud data P is obtained by using formula (1): f ; Where P = {(x i ,y i ,z i )|i=1,2,3,....,N},(x i ,y i,z i ) represents the coordinates of the i-th 3D point in the lane, and N is the total number of points in the point cloud;

[0109] P f =P·T+b (1);

[0110] In the formula, f represents the deformation type, T represents the transformation matrix related to the deformation type, and b represents the translation offset;

[0111] After the above deformation processing, simulated tunnel deformation three-dimensional point cloud data can be generated.

[0112] Step 2: Use the data enhancement module to deform the deformed roadway point cloud data;

[0113] Deformed roadway point cloud data P f Enhanced processing is performed, and at the same time, noise and disturbance are introduced to realistically simulate the actual data. Formula (2) is used to obtain the enhanced point cloud data P aug ; Data enhancement operations can transform point clouds by rotation, translation, scaling, etc., thereby increasing data diversity and improving the robustness of the model. At the same time, noise information is added to simulate the errors in the actual acquisition process, increasing the realism of the training data. Through these enhancement techniques, the trained model can better adapt to complex actual environments and improve its generalization ability on real data.

[0114] P aug =(P f ·R)+t·(1+ε) (2);

[0115] Where R is the enhancement transformation matrix, including rotation, translation, scaling and noise addition; t is the translation vector; ε represents the noise added randomly, which can be Gaussian noise or discrete noise; (1+ε) represents the scaling factor of the noise change added by randomization;

[0116] Through the above operations, the generated enhanced dataset P aug There will be more deformation samples.

[0117] Step 3: Build an intelligent detection module for tunnel deformation and automatically identify and evaluate the tunnel deformation area;

[0118] S31: Build a self-supervised learning module to automatically extract features from unlabeled point cloud data;

[0119] In order to solve the problem of difficulty in labeling 3D point cloud data, this paper uses a method combining variational autoencoder and contrastive learning while adding cross entropy loss supervision to construct a 3D point cloud self-supervised learning network. The network mainly uses contrastive learning method to learn the feature representation of point cloud data, and combines variational autoencoder contrastive learning and cross entropy loss supervision module to design the network structure to enhance the robustness and generalization ability of the model. The goal of the self-supervised learning network is to learn a scalable encoder E(·) that can be effectively transferred to perform downstream detection tasks on other datasets, such as Figure 3 As shown. In order to achieve this object without labels, a specific pre-task needs to be designed to help the point cloud encoder extract enough information from the data to improve its encoding ability. Contrastive learning is a widely used pre-task, which is a special kind of self-supervised learning. Different from the encoding and decoding scheme of generative learning, it focuses on learning the common features between instances of the same class and distinguishing the differences between different classes.

[0120] The variational autoencoder and contrastive learning are combined to form a variational autoencoder contrastive learning network structure. The characteristic of variational autoencoder contrastive learning is to constrain the feature distribution rather than the feature value corresponding to each sample in the latent space. The variational autoencoder contrastive learning network structure is used to perform self-supervised feature learning on unlabeled point cloud data; the variational autoencoder is used to extract the potential feature representation of the point cloud data, and through contrastive learning, the distribution of the positive sample features extracted by the encoder is converged to be similar, while distinguishing the distribution between positive and negative samples; the purpose of variational contrastive learning is to perform self-supervised feature learning on point cloud data while avoiding the hard constraints on feature values ​​in traditional contrastive learning, thereby improving the generalization ability of the encoder. The variational autoencoder contrastive learning network structure is as follows Figure 4 As shown in Figure 2. Different from the classic contrastive learning method that directly learns the feature values ​​of samples, variational contrastive learning improves the robustness of the model by constraining the feature distribution in the latent space.

[0121] Contrastive learning may cause the model to over-focus on specific key parts of the sample, affecting the generalization ability. For this reason, generative supervision is introduced to retain invariant features. Traditional self-reconstruction methods focus on global coordinate features, which in turn reduces the generalization ability of the point cloud feature extractor. In order to solve this problem, the cross-entropy loss function is introduced to supervise the training process, and a generative adversarial mechanism is introduced to optimize the generated latent features through contrastive learning, so that the model can effectively capture the essential features in the point cloud data, thereby improving the performance of self-supervised learning. At the same time, the cross-entropy loss is calculated for the positive samples generated by two different data enhancements to retain more invariant features and promote the consistency of distribution between positive samples; the goal of cross-entropy loss supervision is to maximize the similarity of positive sample pairs through the generator and maintain the distinguishability of samples of different categories in the latent space. The cross-entropy loss supervision network structure is as follows Figure 5 As shown in the figure, the trained self-supervised learning network is used as the backbone network of the tunnel deformation intelligent detection module to extract the deep features related to the tunnel deformation from the input tunnel point cloud data. Specifically, it is responsible for extracting the deep features from the input tunnel point cloud data. These features effectively capture the spatial layout, structural changes and deformation patterns of the tunnel.

[0122] S32: construct a deformation detection module to achieve accurate identification of the deformation area of ​​the roadway and evaluation of the deformation degree;

[0123] The self-supervised learning network model can learn robust feature representations from unlabeled point cloud data. These feature representations can capture the geometric and topological information of the point cloud and provide a high-quality feature basis for subsequent deformation detection tasks. After the pre-training of the self-supervised learning network is completed, the weights of the encoder E(·) are saved. These weights will be used as the initialization weights for subsequent tasks. The pre-trained self-supervised learning encoder E(·) is used as the backbone network of the deformation detection network, responsible for extracting high-dimensional feature representations from the input point cloud. On the basis of the backbone network, a detection head is added to detect the deformation area of ​​the lane from the feature representation. The detection head is mainly divided into two stages: the generation of 3D candidate boxes and the refinement of 3D candidate boxes.

[0124] After training the self-supervised learning network, a deep feature representation that can effectively extract the features of point cloud data is obtained. Next, the feature representations obtained by self-supervised learning are applied to downstream tasks to build a lane deformation detection network. The goal of this detection network is to accurately identify and detect lane deformation based on the extracted point cloud features. The deformation detection network structure is as follows Figure 6 As shown. In order to improve the accuracy and robustness of the network and enhance the network's expressiveness, multi-layer fully connected layers (MLP) and convolutional layers are used to generate 3D candidate boxes, and the 3D candidate boxes are refined as the detection head of the lane deformation intelligent detection module, which is used to accurately identify the lane deformation area and evaluate the deformation degree based on feature representation; the detection head uses classification or regression methods to determine whether the lane has deformed, and further identifies the type and degree of deformation to perform the lane deformation detection task. Finally, after optimizing the loss function of the overall network and combining the features obtained by self-supervised learning, the detection head can accurately identify the deformation of the lane and perform accurate evaluation.

[0125] S33: Automatically identify and evaluate the deformation area of ​​the tunnel using the tunnel deformation intelligent detection module;

[0126] An intelligent tunnel deformation detection module is composed of a self-supervised learning module and a deformation detection module. A three-dimensional sensor arranged in the tunnel is used to automatically scan the tunnel and generate point cloud data at a set sampling frequency. The point cloud data is input into the self-supervised learning module. The self-supervised learning module is used to automatically extract depth features related to tunnel deformation from the point cloud data. The obtained feature data is input into the deformation detection module. The deformation detection module is used to accurately identify the tunnel deformation area and evaluate the deformation degree. Finally, the identified deformation area and an evaluation report are output.

[0127] As a preferred embodiment, in step 3 S31, the specific process of using the variational autoencoder contrastive learning network structure to perform self-supervised feature learning on unlabeled point cloud data is as follows:

[0128] S31-1: For the point cloud data obtained through data enhancement, two point cloud samples p and q are extracted each time for contrastive learning. The encoder for contrastive learning is G(·). After the sample passes through the encoder, the feature f is obtained. p and f q , then constrain the features generated by the encoder to obey the Gaussian distribution;

[0129] S31-2: Using two single-layer fully connected networks F μ (·) and F σ (·) respectively transform the features f p and f q Projected into a high-dimensional invariant space, each point cloud sample generates two vectors μ p ,σ p and μ q ,σ q , let the potential features of the two point cloud samples obey a Gaussian distribution with a mean of μ and a standard deviation of σ:

[0130] S31-3: Use KL divergence as a regularization term to constrain the feature distribution of all samples to a standard normal distribution with a mean of 0 and a standard deviation of 1: D KL (N(μ,σ), N(0,1)); According to formula (3), KL divergence is used to calculate the distance between the Gaussian distributions of two sample features, and samples of the same category are constrained to have similar Gaussian distributions, while distributions of different categories are made as far apart as possible, which helps the model generate samples similar to the training set;

[0131]

[0132] In the formula, P(x) and Q(x) are two sample features f p and f q Gaussian distribution of

[0133] S31-4: In order to maintain the distinguishing ability of the feature extractor, it is not desirable for all samples to follow the same distribution. At the same time, in contrastive learning, it is necessary to calculate the distribution similarity between the positive samples generated by two random data enhancements, and the similarity should be order-independent. Therefore, according to formula (4), the symmetric JS divergence is used to calculate the distance between the two Gaussian distributions to calculate the distribution similarity between the positive samples generated by two random data enhancements, and the distance between the two samples is mapped to between 0 and 1;

[0134]

[0135] Where D KL (·) is to calculate the KL divergence between two distributions. The value of JS divergence is between 0 and 1. The smaller the value, the more similar the two distributions are.

[0136] S31-5: According to formula (5), the feature distributions of samples of the same category are made similar, and the feature distributions of samples of different categories are made as far apart as possible;

[0137]

[0138] In the formula, N represents the number of samples in the batch, i represents the i-th sample, k represents the k-th sample, P and Q are the Gaussian distributions of the two sample features, and D JS (·) means calculating the JS divergence between two distributions;

[0139] S31-6: Obtain the feature distributions P(x) and Q(x) of two samples p and q from the variational autoencoder contrastive learning network structure, and sample from P(x) and Q(x) to generate the embedding g p and g q ; will g p and g q Input to the decoder D(·) to generate new point clouds p′ and q′. By sampling from the feature distribution, randomness can be introduced and the generated point cloud data can be diverse, which helps the model learn the potential distribution of the input point cloud rather than just a single reconstruction. The decoder converts the feature embedding back to the point cloud space and generates an output similar to the input point cloud. This is a key step in evaluating the model's generation ability.

[0140] S31-7: In order to represent the point cloud in a comparable form, the coordinates of the point clouds p′ and q′ are normalized to the range [0,1] and discretized into a fixed-size grid to convert the point cloud into a probability distribution. The probability distribution can intuitively represent the density and distribution of the point cloud, which is convenient for calculating the difference between the generated point cloud and the input point cloud. At the same time, the input point clouds p and q are also converted into probability distributions;

[0141] S31-8: Calculate the cross entropy loss between the probability distribution of the generated point cloud and the probability distribution of the input point cloud according to formula (6) to evaluate the generation quality of the point cloud;

[0142]

[0143] Where P(i) is the probability value of the i-th grid cell in the probability distribution of the input point cloud, and Q(i) is the probability value of the i-th grid cell in the probability distribution of the generated point cloud;

[0144] S31-9: Cross entropy loss directly measures the similarity between the generated point cloud and the input point cloud. According to formula (7), by minimizing the cross entropy loss, it can be ensured that the generated point cloud is highly consistent with the input point cloud in terms of probability distribution. Finally, the goal of the entire cross entropy loss supervision module is achieved by minimizing the cross entropy loss between the probability distribution of the generated point cloud and the probability distribution of the input point cloud;

[0145] min(CE(P,P′)+CE(Q,Q′))(7);

[0146] Where CE(P,P′) is the cross entropy loss between the input point cloud P and the generated point cloud P′, and CE(Q,Q′) is the cross entropy loss between the input point cloud Q and the generated point cloud Q′.

[0147] As a preferred embodiment, in step 3 S32, by segmenting the point cloud into foreground points and background points, the search space is reduced, the efficiency of generating candidate boxes is improved, and a small number of high-quality 3D candidate boxes are directly generated from the original point cloud, providing a basis for subsequent bounding box refinement; the specific process is as follows:

[0148] S32-11: For the input point cloud X={x 1 ,x 2 ,····,x N}, where each point x i =(x,y,z,h,w,l,θ), (x,y,z) is the coordinate of the center point of the point cloud, (h,w,l) is the length, width and height, θ is the direction angle; the feature F = E(X) is extracted through the trained self-supervised learning network, where F = {f 1 ,f 2 ,····,f N} represents the feature corresponding to each point;

[0149] S32-12: Predict the probability of foreground points through a simple fully connected network, where the input is the feature f of each point i , the output is the probability p that the point is a foreground point i , as shown in formula (8);

[0150] pi =σ(W s f i +b i ) (8);

[0151] Where σ(·) is the Sigmoid activation function, which is used to map the output value to the range [0,1] to represent the probability, and W s and b s They are the weight and bias parameters of the fully connected network respectively;

[0152] S32-13: For each foreground point, bin-based regression loss is used to estimate the parameters of the 3D candidate box, the surrounding area is divided into multiple discrete bins, and the classification loss and residual regression loss are used to estimate the center position. Since the distribution of target objects in the vertical direction is usually more concentrated and the range of variation is small, the loss in the y-axis direction is not calculated;

[0153] First, the target center (x p ,z p ) relative to the foreground point p, and discretize it into bins to obtain the straight value bin distribution L along the x-axis and z-axis directions respectively bin (p) x , L bin (p) z ;

[0154]

[0155] Where (x(p),z(p)) is the coordinate of the foreground point p, (x p ,z p ) is the coordinate of the calculated center point, S is the search range set between 0 and 3, and δ is the bin size set between 100 and 255;

[0156] Secondly, according to formula (11) and formula (12), the residual regression res(p) of the target center is calculated in the determined bin. x 、res(p) z ;

[0157]

[0158] Where C is a normalization constant, which is set to 50 to control the range of the residual.

[0159] As a preferred embodiment, in step 3 S32, more accurate bounding box parameters are learned by standardizing the coordinate system and global semantic features, and the 3D candidate box generated in the previous step is further refined to improve the detection accuracy;

[0160] S32-21: Determine each 3D candidate box b i , and slightly expand its scope to include more contextual information, as shown in formula (13);

[0161]

[0162] In the formula, (x i ,y i ,z i ) is the coordinate of the center point of the point cloud, (h i ,w i ,l i ) are the dimensions length, width and height, θ is the direction angle, and η is a constant for the expansion range, which is set to 0.5;

[0163] S32-22: For each point p, determine whether it is within the expanded candidate box. If so, retain the point and its features. The pooled point features include the point coordinates, reflection intensity, segmentation mask and global features. The pooled point coordinates are converted to the standard coordinate system to eliminate rotation and position changes.

[0164] S32-23: The definition of the standard coordinate system, the origin is located at the center of the candidate box; the local x-axis and z-axis are parallel to the ground plane, the x-axis points to the head direction of the candidate box, and the z-axis is perpendicular to the x-axis; the y-axis is consistent with the y-axis of the LiDAR coordinate system;

[0165] S32-24: Combine the converted local spatial features and global semantic features f(p), encode them into a unified feature vector through a fully connected layer, use these features to refine the bounding box and predict the confidence, and finally, use formula (14) to refine the bounding box using bin-based regression loss;

[0166]

[0167] Where L is the total loss of 3D bounding box refinement, B is the set of all 3D candidate boxes, |B| is the number of candidate boxes, Fcls(·) is the classification loss function used to calculate the difference between the predicted confidence and the true label, and prob i is the prediction confidence of the i-th candidate box, label i The true label of the i-th candidate box (positive sample is 1, negative sample is 0), B pos is the set of positive candidate boxes (i.e., candidate boxes with a large IoU with the true bounding box), |B pos | is the number of positive sample candidate boxes, is the bin classification loss of the i-th positive sample candidate box, is the residual regression loss within the bin of the i-th positive sample candidate box. This combined method fully utilizes the feature extraction capability of the self-supervised learning network and the powerful detection capability of the deformation detection network, significantly improving the performance of lane deformation detection.

[0168] As a preferred embodiment, in step 1, the tunnel deformation point cloud data P generated by simulation is obtained through sinking deformation, squeezing deformation, expansion deformation and tilting deformation operations. f ;

[0169] Among them, during the sinking deformation process, the sinking deformation is simulated by translation along the z-axis and the tunnel deformation point cloud data P is generated. f , T 1 is the transformation matrix of the underlying deformation, s is the lower level quantity, which is a random value set between 0 and 50;

[0170] During the extrusion deformation process, the extrusion deformation is simulated by scaling in the x-axis and y-axis directions and generating the roadway deformation point cloud data P f , P f =P·T 2 , T 2 is the transformation matrix of the extrusion deformation, α is the squeezing factor, which is a random value set between 0 and 1;

[0171] During the expansion deformation process, the entire point cloud is enlarged in proportion to simulate the expansion deformation and generate the roadway deformation point cloud data P f , P f =P·T 3 , T 3 is the transformation matrix of the extended deformation, β is the expansion factor, which is a random value set between 0 and 1;

[0172] During the tilt deformation process, the tilt deformation is simulated by rotation and the roadway deformation point cloud data P is generated. f , P f =P·T 4 , T 4 is the transformation matrix of the tilt deformation, θ is the tilt angle, which is a random value set between 0 and 30 degrees;

[0173] Through the above process, a large amount of simulated three-dimensional tunnel deformation data is obtained based on the original point cloud data through batch transformation processing, thus having sufficient sample data.

[0174] As a preferred embodiment, in step 2, the deformed roadway point cloud data Pf is enhanced by rotation transformation, translation transformation, scaling transformation and noise addition, wherein Pf ={(x i ′,y i ′,z i ′)|i=1,2,3,....,N 0},(x i ′,y i ′,z i ′) represents the coordinates of the i-th 3D point in the generated lane point cloud data, N 0 is the total number of points in the generated roadway point cloud; the main purpose of the data enhancement module is to increase the diversity of the data by performing multiple transformations on the generated roadway point cloud data, thereby improving the robustness and generalization ability of the model.

[0175] In the process of rotation transformation, the point cloud is rotated around the z-axis, and each point in the point cloud (x i ′,y i ′,z i ′) After rotation, the new point coordinates are The rotation transformation formula is: Among them, θ 1 is the rotation angle, R is the rotation matrix, This rotation operation can realize the rotation of the point cloud in the z-axis plane, and by randomly selecting θ 1 Can generate rotation data of different angles, θ 1 A random value set between 0 and 30 degrees

[0176] During the translation transformation, translation is performed by adding translation amounts on the x, y, and z axes. The translation transformation formula is: Where t is the translation vector, t=(t x ,t y ,t z ), t x ,t y ,t z Represents the translation along the x, y, and z axes respectively; the translation is randomly generated and ranges from 0 to 50.

[0177] During the scaling transformation, each coordinate of the point cloud is multiplied by a scaling factor s. x ,s y ,s z To achieve scaling, the scaling transformation formula is: This scaling operation can achieve uniform enlargement or reduction of the point cloud and non-uniform scaling using different scaling factors on the x, y, and z axes, where the scaling factor s x ,s y ,s z The size of is set as a random value between 0 and 1.

[0178] Point cloud data is usually affected by noise, so it is necessary to introduce noise in the data enhancement process. Noise can simulate the errors in the actual acquisition process, such as the accuracy error of the scanner, environmental interference, etc. This paper mainly uses Gaussian noise to enhance the data set. In the process of adding noise, a Gaussian distributed random noise N(μ,σ 2 ), N(0,σ 2 ) is a Gaussian noise with a mean of 0 and a standard deviation of σ, which can simulate small random fluctuations in the point cloud due to sensor errors or other factors. The noise addition formula is:

[0179] Here, μ is the mean and σ is the standard deviation.

[0180] In the data enhancement module, the original point cloud dataset is fully enhanced through the above-mentioned rotation, translation, scaling, and noise addition operations, providing richer and more diverse samples for the training model.

[0181] In summary, the detection method of the basic model of coal mine tunnel deformation detection based on self-supervised learning proposed in the present invention forms a complete set of intelligent monitoring solutions by simulating deformation data generation, data enhancement, self-supervised learning network construction and optimization of deformation detection network. This method not only effectively solves the problems of scarcity and complex labeling of coal mine tunnel deformation data, but also significantly improves the generalization ability and detection accuracy of the model through self-supervised learning technology. In addition, through the effective combination of data enhancement and variational autoencoder contrast learning, it can adapt to different types of tunnel deformation and show good robustness and adaptability. In practical applications, this method can significantly reduce the cost of manual labeling and significantly improve monitoring efficiency, providing an efficient and reliable technical means for coal mine safety monitoring. With the development of intelligent mines, the present invention is expected to be widely promoted and applied in more mine scenarios, and can provide a strong technical guarantee for safe production in mines.

Claims

1. A basic model for coal mine tunnel deformation detection based on self-supervised learning, characterized in that: It includes a roadway deformation point cloud data generation module, a data enhancement module and a roadway deformation intelligent detection module; The tunnel deformation point cloud data generation module is used to generate deformed three-dimensional point cloud data based on the annotated coal mine tunnel original point cloud data and according to the tunnel geometry and different deformation types, and then send the three-dimensional point cloud data to the data enhancement module; The data enhancement module is used to perform deformation processing on the three-dimensional point cloud data to different degrees, and at the same time, introduce noise and disturbance to realistically simulate the actual data to expand the diversity of the data set, and obtain an extended data set, and then send the extended data set to the self-supervised learning module; The lane deformation intelligent detection module includes a self-supervised learning module and a deformation detection module: The self-supervised learning module is used to automatically extract features from unlabeled point cloud data in the extended data set by using variational autoencoder contrastive learning and cross entropy loss supervision, and send the obtained feature data to the deformation detection module; The deformation detection module is used to accurately identify the deformation area of ​​the tunnel according to the characteristic data and accurately evaluate the severity of the deformation.

2. A basic model for coal mine tunnel deformation detection based on self-supervised learning according to claim 1, characterized in that: The different deformation types include sinking deformation, squeezing deformation, expanding deformation and tilting deformation.

3. A basic model for coal mine tunnel deformation detection based on self-supervised learning according to claim 1, characterized in that: The data deformation processing includes rotation transformation, translation transformation, scaling transformation and noise addition.

4. A detection method for a basic model for coal mine tunnel deformation detection based on self-supervised learning, using a basic model for coal mine tunnel deformation detection based on self-supervised learning as claimed in any one of claims 1 to 3, characterized in that: The steps include: Step 1: Generate roadway point cloud data with different deformation types by simulation method using roadway deformation point cloud data generation module; For the annotated original point cloud data P of coal mine tunnels, according to the tunnel geometry and different deformation types, combined with the physical model and numerical calculation, the deformed tunnel point cloud data P is obtained using formula (1): f ; Where P = {(x i ,y i ,z i )|i=1,2,3,....,N},(x i ,y i ,z i ) represents the coordinates of the i-th 3D point in the lane, and N is the total number of points in the point cloud; P f =P·T+b (1); In the formula, f represents the deformation type, T represents the transformation matrix related to the deformation type, and b represents the translation offset; Step 2: Use the data enhancement module to deform the deformed roadway point cloud data; Deformed roadway point cloud data P f Enhanced processing is performed, and at the same time, noise and disturbance are introduced to realistically simulate the actual data. Formula (2) is used to obtain the enhanced point cloud data P aug ; P aug =(P f ·R)+t·(1+ε) (2); Where R is the enhancement transformation matrix; t is the translation vector; ε represents the noise added randomly, and (1+ε) represents the scaling factor of the noise change added randomly; Step 3: Build an intelligent detection module for tunnel deformation and automatically identify and evaluate the tunnel deformation area; S31: Build a self-supervised learning module to automatically extract features from unlabeled point cloud data; The variational autoencoder and contrastive learning are combined to form a variational autoencoder contrastive learning network structure, which is used to perform self-supervised feature learning on unlabeled point cloud data; the variational autoencoder is used to extract the potential feature representation of the point cloud data, and through contrastive learning, the distribution of the positive sample features extracted by the encoder is converged to be similar, while distinguishing the distribution between positive samples and negative samples; at the same time, a cross entropy loss function is introduced to supervise the training process, and a generative adversarial mechanism is introduced to optimize the generated potential features through contrastive learning, so that the model can effectively capture the essential features in the point cloud data. At the same time, the cross entropy loss is calculated by the positive samples generated by two different data enhancements to retain more invariant features and promote the consistency of distribution between positive samples; the trained self-supervised learning network is used as the backbone network of the roadway deformation intelligent detection module to extract deep features related to roadway deformation from the input roadway point cloud data; S32: construct a deformation detection module to achieve accurate identification of the deformation area of ​​the roadway and evaluation of the deformation degree; 3D candidate boxes are generated using multiple layers of fully connected layers and convolutional layers, and the 3D candidate boxes are refined as the detection head of the lane deformation intelligent detection module to achieve accurate identification of lane deformation areas and evaluation of deformation degrees based on feature representation; S33: Automatically identify and evaluate the deformation area of ​​the tunnel using the tunnel deformation intelligent detection module; An intelligent tunnel deformation detection module is composed of a self-supervised learning module and a deformation detection module. A three-dimensional sensor arranged in the tunnel is used to automatically scan the tunnel and generate point cloud data at a set sampling frequency. The point cloud data is input into the self-supervised learning module. The self-supervised learning module is used to automatically extract depth features related to tunnel deformation from the point cloud data. The obtained feature data is input into the deformation detection module. The deformation detection module is used to accurately identify the tunnel deformation area and evaluate the deformation degree. Finally, the identified deformation area and an evaluation report are output.

5. The detection method of the basic model for coal mine tunnel deformation detection based on self-supervised learning according to claim 4 is characterized in that: In step 3 S31, the specific process of using the variational autoencoder contrastive learning network structure to perform self-supervised feature learning on unlabeled point cloud data is as follows: S31-1: For the point cloud data obtained through data enhancement, two point cloud samples p and q are extracted each time for contrastive learning. The encoder for contrastive learning is G(·). After the sample passes through the encoder, the feature f is obtained. p and f q ; S31-2: Using two single-layer fully connected networks F μ (·) and F σ (·) respectively transform the features f p and f q Projected into a high-dimensional invariant space, each point cloud sample generates two vectors μ p ,σ p and μ q ,σ q , let the potential features of the two point cloud samples obey a Gaussian distribution with a mean of μ and a standard deviation of σ: S31-3: Use KL divergence as a regularization term to constrain the feature distribution of all samples to a standard normal distribution with a mean of 0 and a standard deviation of 1: D KL (N(μ,σ), N(0,1)); According to formula (3), KL divergence is used to calculate the distance between the Gaussian distributions of two sample features, and samples of the same category are constrained to have similar Gaussian distributions, while the distributions of different categories are made as far apart as possible; In the formula, P(x) and Q(x) are two sample features f p and f q Gaussian distribution of S31-4: Calculate the distance between two Gaussian distributions using the symmetric JS divergence according to formula (4) to calculate the distribution similarity between the positive samples generated by two random data augmentations, and map the distance between the two samples to between 0 and 1; Where D KL (·) is to calculate the KL divergence between two distributions. The value of JS divergence is between 0 and 1. The smaller the value, the more similar the two distributions are. S31-5: According to formula (5), the feature distributions of samples of the same category are made similar, and the feature distributions of samples of different categories are made as far apart as possible; In the formula, N represents the number of samples in the batch, i represents the i-th sample, k represents the k-th sample, P and Q are the Gaussian distributions of the two sample features, and D JS (·) means calculating the JS divergence between two distributions; S31-6: Obtain the feature distributions P(x) and Q(x) of two samples p and q from the variational autoencoder contrastive learning network structure, and sample from P(x) and Q(x) to generate the embedding g p and g q ; will g p and g q Input into the decoder D(·) to generate new point clouds p′ and q′ to introduce randomness and make the generated point cloud data diverse; S31-7: Normalize the coordinates of the point clouds p′ and q′ to the range [0,1] and discretize them into a fixed-size grid to convert the point cloud into a probability distribution; at the same time, convert the input point clouds p and q into a probability distribution; S31-8: Calculate the cross entropy loss between the probability distribution of the generated point cloud and the probability distribution of the input point cloud according to formula (6) to evaluate the generation quality of the point cloud; Where P(i) is the probability value of the i-th grid cell in the probability distribution of the input point cloud, and Q(i) is the probability value of the i-th grid cell in the probability distribution of the generated point cloud; S31-9: According to formula (7), the cross entropy loss between the probability distribution of the generated point cloud and the probability distribution of the input point cloud is minimized to ensure that the generated red frame cloud is highly consistent with the input point cloud in terms of probability distribution; min(CE(P,P′)+CE(Q,Q′))(7); Where CE(P,P′) is the cross entropy loss between the input point cloud P and the generated point cloud P′, and CE(Q,Q′) is the cross entropy loss between the input point cloud Q and the generated point cloud Q′.

6. The detection method of the basic model for coal mine tunnel deformation detection based on self-supervised learning according to claim 5 is characterized in that: In step 3 S32, by segmenting the point cloud into foreground points and background points, the search space is reduced, the efficiency of generating candidate boxes is improved, and a small number of high-quality 3D candidate boxes are directly generated from the original point cloud, providing a basis for subsequent bounding box refinement; the specific process is as follows: S32-11: For the input point cloud X = {x1, x2, ····, x N }, where each point x i =(x,y,z,h,w,l,θ), (x,y,z) is the coordinate of the center point of the point cloud, (h,w,l) is the length, width and height, θ is the direction angle; the feature F = E(X) is extracted through the trained self-supervised learning network, where F = {f1,f2,····,f N } represents the feature corresponding to each point; S32-12: Predict the probability of foreground points through a simple fully connected network, where the input is the feature f of each point i , the output is the probability p that the point is a foreground point i , as shown in formula (8); p i =σ(W s f i +b i ) (8); Where σ(·) is the Sigmoid activation function, which is used to map the output value to the range [0,1] to represent the probability, and W s and b s They are the weight and bias parameters of the fully connected network respectively; S32-13: For each foreground point, bin-based regression loss is used to estimate the parameters of the 3D candidate box, the surrounding area is divided into multiple discrete bins, and the classification loss and residual regression loss are used to estimate the center position. Since the distribution of target objects in the vertical direction is usually more concentrated and the range of variation is small, the loss in the y-axis direction is not calculated; First, the target center (x p ,z p ) relative to the foreground point p, and discretize it into bins to obtain the straight value bin distribution L along the x-axis and z-axis directions respectively bin (p) x , L bin (p) z ; Where (x(p),z(p)) is the coordinate of the foreground point p, (x p ,z p ) is the coordinate of the calculated center point, S is the search range set between 0 and 3, and δ is the bin size set between 100 and 255; Secondly, according to formula (11) and formula (12), the residual regression res(p) of the target center is calculated in the determined bin. x 、res(p) z ; Where C is a normalization constant, which is set to 50 to control the range of the residual.

7. The detection method of the basic model for coal mine tunnel deformation detection based on self-supervised learning according to claim 6 is characterized in that: In step 3 S32, more accurate bounding box parameters are learned by standardizing the coordinate system and global semantic features, and the 3D candidate box generated in the previous step is further refined to improve the detection accuracy; S32-21: Determine each 3D candidate box b i , and slightly expand its scope to include more contextual information, as shown in formula (13); In the formula, (x i ,y i ,z i ) is the coordinate of the center point of the point cloud, (h i ,w i ,l i ) are the dimensions length, width and height, θ is the direction angle, and η is a constant for the expansion range, which is set to 0.5; S32-22: For each point p, determine whether it is within the expanded candidate box. If so, retain the point and its features. The pooled point features include the point coordinates, reflection intensity, segmentation mask and global features. The pooled point coordinates are converted to the standard coordinate system to eliminate rotation and position changes. S32-23: The definition of the standard coordinate system, the origin is located at the center of the candidate box; the local x-axis and z-axis are parallel to the ground plane, the x-axis points to the head direction of the candidate box, and the z-axis is perpendicular to the x-axis; the y-axis is consistent with the y-axis of the LiDAR coordinate system; S32-24: Combine the converted local spatial features and global semantic features f(p), encode them into a unified feature vector through a fully connected layer, use these features to refine the bounding box and predict the confidence, and finally, use formula (14) to refine the bounding box using bin-based regression loss; Where L is the total loss of 3D bounding box refinement, B is the set of all 3D candidate boxes, |B| is the number of candidate boxes, Fcls(·) is the classification loss function used to calculate the difference between the predicted confidence and the true label, and prob i is the prediction confidence of the i-th candidate box, label i The true label of the i-th candidate box, B pos is the set of positive sample candidate boxes, |B pos | is the number of positive sample candidate boxes, is the bin classification loss of the i-th positive sample candidate box, It is the residual regression loss within the bin of the i-th positive sample candidate box.

8. The detection method of the basic model for coal mine tunnel deformation detection based on self-supervised learning according to claim 7 is characterized in that: In step 1, the tunnel deformation point cloud data P is obtained by sinking deformation, squeezing deformation, extending deformation and tilting deformation operations. f ; Among them, during the sinking deformation process, the sinking deformation is simulated by translation along the z-axis and the tunnel deformation point cloud data P is generated. f , T1 is the transformation matrix of the lower layer deformation, s is the lower level quantity, which is a random value set between 0 and 50; During the extrusion deformation process, the extrusion deformation is simulated by scaling in the x-axis and y-axis directions and generating the roadway deformation point cloud data P f , P f =P·T2, T2 is the transformation matrix of extrusion deformation, α is the squeezing factor, which is a random value set between 0 and 1; During the expansion deformation process, the entire point cloud is enlarged in proportion to simulate the expansion deformation and generate the roadway deformation point cloud data P f , P f =P·T3, T3 is the transformation matrix of the extended deformation, β is the expansion factor, which is a random value set between 0 and 1; During the tilt deformation process, the tilt deformation is simulated by rotation and the roadway deformation point cloud data P is generated. f , P f =P·T4, T4 is the transformation matrix of tilt deformation, θ is the tilt angle, which is a random value set between 0 and 30 degrees.

9. The detection method of the basic model for coal mine tunnel deformation detection based on self-supervised learning according to claim 8 is characterized in that: In step 2, the deformed roadway point cloud data Pf is enhanced by rotation transformation, translation transformation, scaling transformation and noise addition, where P f ={(x i ′,y i ′,z i ′)|i=1,2,3,....,N0},(x i ′,y i ′,z i ′) represents the coordinates of the i-th three-dimensional point in the generated lane point cloud data, and N0 is the total number of points in the generated lane point cloud; In the process of rotation transformation, the point cloud is rotated around the z-axis, and each point in the point cloud (x i ′,y i ′,z i ′) After rotation, the new point coordinates are The rotation transformation formula is: Among them, θ1 is the rotation angle, R is the rotation matrix, During the translation transformation, translation is performed by adding translation amounts on the x, y, and z axes. The translation transformation formula is: Where t is the translation vector, t=(t x ,t y ,t z ), t x ,t y ,t z Respectively represent the translation along the x, y, and z axes; During the scaling transformation, each coordinate of the point cloud is multiplied by a scaling factor s. x ,s y ,s z To achieve scaling, the scaling transformation formula is: In the process of adding noise, a Gaussian distributed random noise N(μ,σ 2 ), the noise adding formula is: Here, μ is the mean and σ is the standard deviation.

Images