Coal rock mass failure precursor recognition method and system based on infrared and deep learning

Abstract

The application provides a coal rock mass failure precursor identification method and system based on infrared and deep learning, and belongs to the technical field of material damage identification. The method comprises the following steps: acquiring an original infrared thermal image sequence in a loading process of a coal rock combination body, obtaining a time series denoising infrared temperature matrix DITM after region of interest extraction and noise suppression processing; extracting a damage infrared matrix through adaptive threshold segmentation, and calculating a time series damage infrared energy response parameter DIER; constructing a multi-channel input sample, inputting the sample into a deep learning segmentation model, calculating a thermal anomaly area ratio TAR according to a segmentation result; and inputting a joint feature sequence composed of the time series damage infrared energy response parameter DIER and the thermal anomaly area ratio TAR into an anomaly detection model to identify an abnormal sample, so as to realize automatic identification of a key precursor before macro instability of the coal rock combination body.

Description

Technical Field

[0001] This application relates to the field of material damage and failure identification technology, and in particular to a method and system for identifying precursors of coal and rock mass failure based on infrared and deep learning. Background Technology

[0002] Existing rock damage early warning technologies mainly rely on traditional monitoring methods such as stress, strain, acoustic emission, and displacement. While these methods can reflect the material damage evolution process, they often suffer from insufficient spatial resolution, insensitivity to local early anomalies, and difficulty in directly characterizing the expansion process of thermal anomalies on the sample surface. Infrared thermography, on the other hand, offers advantages such as non-contact operation, visualization, real-time performance, and ease of large-scale monitoring. It can capture early signs of rock damage from changes in surface temperature fields, thus possessing significant engineering application potential in rock mechanics and disaster early warning. However, most current research based on infrared thermography remains at the stage of single-index analysis, typically extracting only simple statistics such as average temperature, infrared radiation temperature, variance, and infrared counts. Although these can reflect certain changes in thermal response, they are insufficient to comprehensively and quantitatively characterize the complex state of damage evolution, leading to inadequate accuracy and timeliness in early warning identification.

[0003] In existing technologies, the literature "Wang J, Chang Y, He M. Image Processing basedInfrared Precursor Recognition for NPR Cable Anchored Rock with InclinationAngles[J]. Journal of Rock Mechanics and Geotechnical Engineering, 2026" proposes a failure prediction method for anchored composite rocks. This method employs two independent isolated forest models to detect anomalies in the temporal evolution sequence of the damaged area and the temporal crack length, and then weights and fuses these models to predict the precursors of failure in the anchored composite rocks. However, due to the significant differences in thermal and mechanical properties between coal-rock assemblies and anchored composite rocks, their damage patterns in thermal infrared images differ considerably. Directly applying the failure prediction scheme for anchored composite rocks to the scenario of identifying precursors of failure in coal-rock assemblies is insufficient to effectively identify early precursors.

[0004] On the other hand, traditional infrared image processing methods typically rely on threshold segmentation, manual experience-based judgment, or simple image enhancement, which are insufficient for identifying weak, discrete, discontinuous, and poorly defined temperature anomaly regions. This is especially true in coal-rock assemblages, where significant differences in thermal and mechanical properties between coal and rock lead to strong heterogeneity in the temperature field during loading. Anomaly regions may exhibit complex characteristics such as local initiation, gradual expansion, discontinuous distribution, and regional merging. Using conventional image processing or manual identification methods is not only inefficient but also highly subjective, lacks repeatability, and is difficult to standardize. Furthermore, research and experiments have shown that relying solely on traditional image segmentation methods (such as machine learning model segmentation) makes it difficult to accurately distinguish between background thermal fluctuations and truly damaging anomaly zones, easily leading to misjudgments or missed detections. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for identifying precursors of coal and rock mass failure based on infrared and deep learning, so as to solve or alleviate the problems existing in the prior art.

[0006] To achieve the above objectives, this application provides the following technical solution: This application provides a method for identifying precursors of coal and rock mass failure based on infrared and deep learning, including: Step S1: Obtain the original infrared thermographic sequence during the loading process of the coal-rock composite. After region of interest extraction and noise suppression, the time-series denoised infrared temperature matrix (DITM) is obtained. Step S2: Based on the time-series denoised infrared temperature matrix, the damage infrared matrix is ​​extracted by adaptive threshold segmentation, and the time-series damage infrared energy response parameter DIER (Damage Infrared Energy Response) is calculated in combination with the thermal radiation law to characterize the energy enhancement features of damage evolution. Step S3: Construct multi-channel input samples, which include at least the following: the temporal denoised infrared temperature matrix DITM, the temperature increment matrix TIM, the coal-rock mask CRM, the region-based normalized temperature increment matrix RN-TIM, and the temporal variation feature TVF. The multi-channel input samples are input into a deep learning segmentation model to obtain pixel-level segmentation results of the temperature anomaly region, and the thermal abnormal-area ratio (TAR) is calculated to characterize the spatial expansion features of damage evolution. Step S4: The time-series damage infrared energy response parameter DIER and the thermal anomaly region proportion TAR are combined to form a joint feature sequence, which is then input into the anomaly detection model to identify abnormal samples that deviate from the normal evolutionary trajectory in the joint feature space, determine the critical precursor frame, and realize the identification of damage precursors.

[0007] Furthermore, step S2 specifically includes: The optimal segmentation threshold for each frame in the temporal denoised infrared temperature matrix (DITM) is calculated using the maximum inter-class variance method. Pixels with temperature values ​​less than the optimal segmentation threshold in each frame are set to zero, while the temperature values ​​of the remaining pixels are retained to obtain the damaged infrared matrix, abbreviated as IMD (Infrared Matrix of Damage). Based on the Stefan-Boltzmann law, the radiation energy of each pixel in the damaged infrared matrix is ​​calculated, and the radiation energy of all pixels is accumulated to obtain the total frame radiation energy. The temporal damage infrared energy response parameter DIER is calculated by combining the damaged infrared matrix IMD, the infrared camera sampling time interval, and the actual physical surface area corresponding to the region of interest.

[0008] Preferably, the temperature increment matrix TIM is calculated as follows: Obtain the reference frame; For each frame in the temporally denoised infrared temperature matrix DITM, the temperature field of that frame is compared with the temperature of the corresponding pixel in the reference frame to calculate the difference, thus obtaining the temperature increment matrix TIM.

[0009] Preferably, in step S3, the coal-rock mask CRM is generated based on the coal-rock partitioning structure of the coal-rock composite sample and is used to identify the coal / rock zone to which the pixel belongs. The regional normalized temperature increment matrix RN-TIM is constructed based on the temperature increment matrix TIM and the coal-rock mask CRM. The construction process includes: using the coal-rock mask CRM to divide the temperature increment matrix TIM into a coal area data subset and a rock area data subset; calculating the statistical characteristic values ​​of the coal area data subset and the rock area data subset respectively to form statistical characteristic value pairs; and using the statistical characteristic values ​​to independently normalize the temperature increment data in the corresponding data subset to obtain the regional normalized temperature increment matrix RN-TIM.

[0010] Preferably, the time-varying characteristic TVF is calculated as follows: Based on the temporal sequence of the region normalized temperature increment matrix RN-TIM, the numerical difference of corresponding pixels between adjacent frames is calculated. The numerical difference is used as the time-series variation feature (TVF) to characterize the rate of change of temperature anomaly response over time.

[0011] Preferably, the deep learning segmentation model is a U-Net network model with embedded pyramid pooling modules; The U-Net network model includes a downsampling path, an upsampling path, and a bottleneck layer connecting the two. The pyramid pooling module is set in the bottleneck layer and is used to perform multi-scale pooling operations on the feature map output by the bottleneck layer and fuse context information. The output layer of the deep learning segmentation model is used to generate a pixel-level probability map representing the distribution of temperature anomaly regions.

[0012] Preferably, the method for calculating the proportion of thermal anomaly region (TAR) includes: Based on the pixel-level probability map output by the deep learning segmentation model, a binary anomaly region mask is obtained by setting a confidence threshold. The number of pixels representing the abnormal region in the binary abnormal region mask is counted, and the ratio of the number of pixels to the total number of effective pixels in the region of interest is calculated to obtain the thermal abnormal region proportion (TAR).

[0013] Preferably, step S4 includes: An isolated forest model is used as the anomaly detection model to fit the joint feature sequence; Calculate the anomaly score of each sample point in the joint feature sequence in the joint feature space, and identify abnormal sample points that deviate from the normal evolutionary trajectory based on the anomaly score; The infrared thermal image frame number corresponding to the earliest occurrence of the abnormal sample point in time sequence is determined as the critical precursor frame. The damage warning time is calculated based on the frame number of the critical precursor frame and the sampling frequency of the infrared thermal image sequence.

[0014] Preferably, Step S1 includes: The temperature field region corresponding to the coal-rock composite sample is extracted from each frame of the original infrared thermal image sequence to obtain the time-series region of interest temperature field matrix; An adaptive median filtering algorithm is used to suppress impulse noise in the temperature field matrix of the time-series region of interest to obtain the temperature field matrix after impulse noise suppression. Gaussian noise suppression is applied to the temperature field matrix after impulse noise suppression using a block-matching three-dimensional filtering algorithm to obtain the time-series denoised infrared temperature matrix DITM.

[0015] This embodiment provides a coal and rock mass failure precursor identification system based on infrared and deep learning, including: The data acquisition unit is configured to acquire the original infrared thermal image sequence during the loading process of the coal-rock composite, and after region of interest extraction and noise suppression processing, obtain the time-series denoised infrared temperature matrix DITM. The energy extraction unit is configured to extract the damage infrared matrix based on the time-series denoised infrared temperature matrix through adaptive threshold segmentation, and calculate the time-series damage infrared energy response parameter DIER in combination with the thermal radiation law to characterize the energy enhancement features of damage evolution. The spatial extraction unit is configured to construct multi-channel input samples, which include at least the following: the temporal denoised infrared temperature matrix, the temperature increment matrix (TIM), the coal and rock mask (CRM), the regional normalized temperature increment matrix (RN-TIM), and the temporal variation feature (TVF). The multi-channel input samples are input into a deep learning segmentation model to obtain pixel-level segmentation results of the temperature anomaly region, and the thermal anomaly region ratio (TAR) is calculated to characterize the spatial expansion features of damage evolution. The damage precursor identification unit is configured to construct a joint feature sequence by combining the temporal damage infrared energy response parameter DIER and the thermal anomaly region proportion TAR, input it into the anomaly detection model, identify abnormal samples that deviate from the normal evolution trajectory in the joint feature space, determine the critical precursor frame, and realize damage precursor identification.

[0016] Beneficial effects: This application provides a proposed dual-route infrared precursor identification framework. By constructing an energy evolution path (DIER) and a spatial extension path (TAR) respectively, and inputting the two into an anomaly detection model, it can extract energy response information related to damage evolution from the infrared temperature field, and accurately segment and quantify the spatial evolution process of the temperature anomaly region. This enables the automatic identification of key precursor moments before the macroscopic instability of coal-rock assemblages, while improving the accuracy, stability and interpretability of its damage prediction. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the original temperature field data from the experiment.

[0018] Figure 2 This is a schematic diagram of the manual annotation process, where (a) the shooting area, (b) the original infrared thermal image (1300 frames), (c) the Labelme software interface, and (d) the annotation results (130 frames).

[0019] Figure 3 This is a schematic diagram of the BPPM-Unet model structure.

[0020] Figure 4 This is a schematic diagram of the processing flow of the isolated forest model.

[0021] Figure 5 This is a schematic diagram of the time-series denoised infrared temperature matrix (DITM), where (a)~(j) represent different peak stresses. DITM percentages (15.39% to 99.92%).

[0022] Figure 6 According to Figure 5 A schematic diagram of the damage infrared matrix (IMD) obtained by DITM extraction.

[0023] Figure 7 This is a schematic diagram of the two-stage training loss-iteration curve of BPPM-Unet, where (a) is the first stage and (b) is the second stage.

[0024] Figure 8 This is a schematic diagram of the temperature anomaly region segmentation results of the deep learning model BPPM-UNet, where (a)~(j) represent different peak stresses. Results of temperature anomaly region segmentation at percentages (15.39% to 99.92%).

[0025] Figure 9 This is a schematic diagram of the precursor point prediction results of the isolated forest model, where (a) is the curve of anomaly score changing over time, and (b) is the curve of DIER changing over time.

[0026] Figure reference numerals: 1-sample, 2-region of interest (ROI). Detailed Implementation

[0027] Studies have found that existing methods for predicting precursors of anchored composite rock failure cannot be directly applied to predicting the failure of coal-rock assemblies. This is because the damage in anchored composite rocks mainly manifests as anchor-rock interface slippage and tensile cracking, exhibiting clear geometric boundaries and linear characteristics. However, in coal-rock assemblies, due to significant differences in the thermal and mechanical properties of coal and rock, the temperature field exhibits strong heterogeneity during loading, and anomalous regions may display complex characteristics such as local initiation, gradual expansion, discontinuous distribution, and regional fusion. The technical solution of this application aims to construct a dual-route infrared precursor identification framework for damage characterization and failure prediction of coal-rock assemblies during uniaxial compression. The basic idea of ​​this scheme is to extract precursor information from two dimensions: the infrared energy evolution of damage and the spatial evolution of temperature anomaly regions unique to coal-rock assemblies. Then, an anomaly detection model is used to jointly analyze these two dimensions to identify key precursor moments.

[0028] The embodiments of this application will now be described with reference to the accompanying drawings.

[0029] This embodiment provides a method for identifying precursors of coal and rock mass failure based on infrared and deep learning. This method can be executed by a computer device, which can be a cloud server or an edge device. Figures 1-4 As shown, the method includes: Step S1: Obtain the original infrared thermographic sequence during the loading process of the coal-rock composite. After region of interest (ROI) extraction and noise suppression, the time-series denoised infrared temperature matrix (DITM) is obtained.

[0030] Further, step S1 includes: The temperature field region corresponding to the coal-rock composite sample is extracted from each frame of the original infrared thermographic sequence to obtain the time-series region of interest temperature field matrix; An adaptive median filter algorithm is used to suppress impulse noise in the temperature field matrix of the temporal region of interest, resulting in a temperature field matrix after impulse noise suppression. The Gaussian noise suppression was performed on the temperature field matrix after impulse noise suppression using the Block-Matching and 3D Filtering (BM3D) algorithm to obtain the time-series denoised infrared temperature matrix DITM.

[0031] In this embodiment, step S1 mainly involves the acquisition and preprocessing of raw data. First, an infrared thermal imaging acquisition system is used to record the infrared thermal image sequence of the coal-rock assembly during the loading process. This sequence consists of multiple frames of infrared images arranged in chronological order, with each frame corresponding to a two-dimensional temperature field matrix, where the elements represent the temperature values ​​of the corresponding pixels.

[0032] Considering that the original images often contain non-sample areas such as loading equipment and background environment, these areas are prone to introducing environmental temperature fluctuations and airflow interference, affecting the accuracy of subsequent feature extraction. Therefore, using a pre-defined region of interest (ROI) extraction algorithm, based on the spatial coordinates of the coal-rock composite sample in the field of view, a temperature field region containing only the sample surface is extracted from each frame of the original infrared thermographic sequence, resulting in a temporal ROI temperature field matrix. This processing effectively eliminates environmental interference from the background area, ensuring the spatial consistency of subsequent analysis data.

[0033] Furthermore, to eliminate infrared camera sensor noise and mixed noise introduced during signal transmission, a joint noise reduction strategy combining adaptive median filtering and block-matching 3D filtering is employed to process the temperature field matrix of the temporal region of interest. Specifically, an adaptive median filtering algorithm is first used for the first-stage filtering process to preserve the edge gradient information of the temperature anomaly region.

[0034] Subsequently, a block-matching 3D filtering algorithm is used to perform a second-stage filtering process on the temperature field matrix after the first-stage filtering. This algorithm leverages the spatial self-similarity of infrared images, effectively suppressing Gaussian noise by collaboratively filtering similar blocks in the 3D transform domain. Since early-stage thermal anomaly signals in coal-rock composite failure typically exhibit low amplitude, small spatial scale, and blurred boundaries, the block-matching 3D filtering algorithm can smooth background noise while preserving the image's detailed texture and subtle temperature rise information to the greatest extent possible, avoiding the loss of early precursor features due to over-smoothing.

[0035] After the above two-stage joint denoising process, a time-series denoised infrared temperature matrix is ​​obtained. This matrix improves the signal-to-noise ratio while preserving the true temperature distribution characteristics of the coal-rock composite surface, providing a reliable data foundation for the accurate calculation of damage infrared energy response parameters and the high-precision inference of the deep learning segmentation model in subsequent steps.

[0036] Step S2: Based on the time-series denoised infrared temperature matrix, the damage infrared matrix is ​​extracted by adaptive threshold segmentation, and the time-series damage infrared energy response parameter DIER (Damage Infrared Energy Response) is calculated in combination with the thermal radiation law to characterize the energy enhancement features of damage evolution.

[0037] In this embodiment, step S2 extracts damage features based on the temporal denoised infrared temperature matrix. The optimal segmentation threshold for each frame in the temporal denoised infrared temperature matrix DITM is calculated using the maximum inter-class variance method (Otsu image segmentation algorithm). Pixels with temperature values ​​less than the optimal segmentation threshold in each frame are set to zero, while the temperature values ​​of the remaining pixels are retained to obtain the infrared damage matrix, abbreviated as IMD (Infrared Matrix of Damage). Based on the Stefan–Boltzmann law, the radiation energy of each pixel in the damaged infrared matrix is ​​calculated, and the radiation energy of all pixels is accumulated to obtain the total radiation energy of the frame. By combining the damage infrared matrix IMD with the sampling time interval of the infrared camera and the actual physical surface area corresponding to the region of interest, the temporal damage infrared energy response parameter DIER is calculated.

[0038] In this embodiment, step S2 extracts damage features based on the temporally denoised infrared temperature matrix. First, an adaptive threshold segmentation method is used to process each frame of the temporally denoised infrared temperature matrix. Specifically, the optimal segmentation threshold for the current frame is calculated using the maximum inter-class variance method. Pixels with temperature values ​​less than the optimal segmentation threshold are set to zero, while pixels with temperature values ​​greater than or equal to the optimal segmentation threshold are retained, resulting in the damage infrared matrix. Through this processing method, low-temperature background regions are removed, retaining only high-temperature anomaly regions related to damage development.

[0039] Subsequently, the radiant energy of each pixel in the damaged infrared matrix was calculated based on the Stefan–Boltzmann law. The radiant energy of all non-zero pixels in the damaged infrared matrix was accumulated to obtain the total frame radiant energy. To eliminate the influence of sampling frequency and sample size on the energy value, the temporal damaged infrared energy response parameter DIER was calculated by combining the damaged infrared matrix IMD, the infrared camera sampling time interval, and the actual physical surface area corresponding to the region of interest.

[0040] The temporal damage infrared energy response parameter DIER varies with loading time to form a sequence, which is used to characterize the energy enhancement characteristics during the damage evolution of the coal-rock assemblage. As the damage intensifies, the area of ​​the high-temperature anomaly region on the surface expands and the temperature rises, causing the parameter value to show an upward trend. By monitoring the variation of this parameter, the evolution of the damage thermal response intensity over time can be reflected from an energy perspective.

[0041] Step S3: Construct multi-channel input samples, which include at least: temporal denoised infrared temperature matrix, temperature increment matrix (TIM), coal and rock mask (CRM), regional normalized temperature increment matrix (RN-TIM), and temporal variation feature (TVF). Input the multi-channel input samples into the deep learning segmentation model to obtain pixel-level segmentation results of the temperature anomaly region, and calculate the thermal anomaly region ratio (TAR) to characterize the spatial expansion features of damage evolution.

[0042] Step S3 aims to use a deep learning model to identify temperature anomaly regions and quantify their spatial expansion characteristics.

[0043] First, a multi-channel input sample is constructed, which contains a data matrix with five channels.

[0044] The first channel is a temporally denoised infrared temperature matrix (DITM). The DITM is generated by using the processing result obtained in step S1 to provide the absolute temperature distribution information at the current moment.

[0045] The second channel is the temperature increment matrix (TIM), which is calculated as follows: A reference frame is obtained; for each frame in the temporally denoised infrared temperature matrix (DITM), the temperature field of that frame is compared with the temperature of the corresponding pixel in the reference frame to calculate the difference, thus obtaining the temperature increment matrix (TIM). Specifically, the first frame at the start of the experiment is selected as the reference frame, and the temperature field of the current frame is compared with the temperature of the corresponding pixel in the reference frame to calculate the difference, which reflects the temperature change matrix.

[0046] The third channel is the Coal-Rock Mask CRM, which is generated based on the coal-rock partitioning structure of the coal-rock composite sample. In other words, it is generated based on the material partitioning structure of the coal-rock composite sample. The pixel values ​​in the matrix indicate whether the corresponding position belongs to the coal area or the rock area.

[0047] The fourth channel is the Regional Normalized Temperature Increment Matrix (RN-TIM), which is constructed based on the Temperature Increment Matrix (TIM) and the Coal-Rock Mask (CRM). Specifically, the temperature increment matrix is ​​divided into coal area data subsets and rock area data subsets using the coal-rock mask. Statistical characteristic values ​​(such as mean and variance) are calculated for each subset, and the data within each subset is normalized using these statistical characteristic values ​​(e.g., the normalized temperature increment values ​​for each subset are calculated using the Z-Score normalization method). The normalized coal area data and rock area data are then resynthesized according to the spatial location of the coal-rock mask (CRM) to obtain the RN-TIM. Normalization eliminates the interference of different baseline temperatures caused by differences in thermal properties between coal and rock on anomaly identification.

[0048] The fifth channel is the temporal variation feature (TVF), which is a temporal sequence based on the region normalized temperature increment matrix (RN-TIM). It is obtained by calculating the numerical difference between corresponding pixels between adjacent frames and is used to characterize the temporal rate of change of temperature response.

[0049] The constructed five-channel input samples are fed into the deep learning segmentation model. The deep learning segmentation model is a U-Net network model (BPPM-UNet model) embedding a Pyramid Pooling Module (PPM). The U-Net network model includes a downsampling path, an upsampling path, and a bottleneck layer connecting the two. The Pyramid Pooling Module is placed in the bottleneck layer to perform multi-scale pooling operations on the feature map output by the bottleneck layer and fuse contextual information. The output layer of the deep learning segmentation model is used to generate pixel-level probability maps representing the distribution of temperature anomaly regions.

[0050] In this embodiment, the downsampling path extracts deep features of the image step by step through convolution and pooling operations, while the upsampling path restores the feature map resolution through deconvolution operations and fuses it with the shallow features of the downsampling path.

[0051] The pyramid pooling module is located in the bottleneck layer to perform multi-scale pooling operations on the feature map output by the bottleneck layer and fuse contextual information. By introducing pooling operations at different scales, the model can acquire contextual information under different receptive fields. For weak, discrete, or discontinuous temperature anomaly regions that may appear in infrared images of coal-rock assemblages, multi-scale contextual information helps the model understand the relationship between local anomalies and the global background, improving the segmentation effect of blurred boundary regions.

[0052] The deep learning segmentation model encodes and decodes the multidimensional features of the input, and the output layer generates a pixel-level probability map representing the distribution of temperature anomaly regions. The value of each pixel in the probability map represents the probability that the location belongs to a temperature anomaly region. In subsequent processing, a binary mask is generated based on the probability map, and the proportion of thermal anomaly regions is calculated, thereby completing the spatial quantization of temperature anomaly regions.

[0053] Specifically, by setting a confidence threshold, the probability map is converted into a binary anomaly region mask, where pixels with a value of 1 represent regions identified as temperature anomalies. The number of pixels representing anomalies in the binary anomaly region mask is then counted, and the ratio of this ratio to the total number of valid pixels (pixels with a value of 1) within the region of interest is calculated to obtain the thermal anomaly region proportion (TAR).

[0054] The thermal anomaly region proportion (TAR) forms a time-series sequence over loading time, used to characterize the spatial expansion features of damage evolution. As damage develops, the area of ​​the surface high-temperature anomaly region changes, and this ratio changes accordingly. By monitoring the evolution of the thermal anomaly region proportion, the initiation, expansion, and fusion processes of temperature anomalies can be reflected spatially, providing spatial characteristic evidence for subsequent precursor identification.

[0055] Step S4: The time-series damage infrared energy response parameter DIER and the thermal anomaly region proportion TAR are combined to form a joint feature sequence, which is then input into the anomaly detection model to identify anomalous samples that deviate from the normal evolutionary trajectory in the joint feature space, determine the critical precursor frame, and realize the identification of damage precursors.

[0056] Further, step S4 includes: using an isolation forest model as an anomaly detection model to fit the joint feature sequence; calculating the anomaly score of each sample point in the joint feature sequence in the joint feature space, and identifying anomalous sample points that deviate from the normal evolutionary trajectory based on the anomaly score; determining the infrared thermal image frame number corresponding to the earliest anomalous sample point in time as the critical precursor frame; and calculating the damage warning time based on the frame number of the critical precursor frame and the sampling frequency of the infrared thermal image sequence.

[0057] Step S4 aims to identify precursors of damage using the feature parameters extracted in the preceding steps.

[0058] First, the temporal damage infrared energy response parameters and the proportion of thermal anomaly regions are paired in chronological order to form a joint feature sequence. Each sample point in the joint feature sequence contains the damage infrared energy response value and the proportion of thermal anomaly regions at the same time, reflecting the energy enhancement characteristics and spatial expansion characteristics of damage evolution, respectively.

[0059] An isolated forest model was used as the anomaly detection model. The joint feature sequence was input into the model for fitting, and the anomaly score of each sample point in the joint feature space was calculated. The anomaly score reflects the degree to which the sample point deviates from the normal evolutionary trajectory; the higher the score, the more the thermal response state at that moment deviates from the normal loading process.

[0060] Anomaly samples are identified based on anomaly scores. The infrared thermographic frame number corresponding to the earliest appearing anomaly sample point in time series is determined as the critical precursor frame. This frame number marks a key stage before the coal-rock assemblage enters macroscopic instability. Based on the frame number of the critical precursor frame and the sampling frequency of the infrared thermographic sequence, the damage warning time can be further calculated.

[0061] This embodiment uses DIER and TAR to characterize the damage state from two different dimensions. DIER is called the energy evolution path, and TAR is called the spatial expansion path. Together, they form a dual-path infrared precursor identification framework. Through the dual-path design, DIER characterizes the damage intensity from a thermodynamic energy perspective, while TAR characterizes the damage range from a geometric spatial perspective. The two complement each other in the feature space, enabling the model to simultaneously capture various damage evolution modes, such as sudden energy increases without area changes or area expansion with gradual temperature rise, thus improving the coverage of precursor identification. In particular, the spatial expansion path uses the proportion of thermal anomaly regions instead of the traditional crack length to directly quantify the area evolution of two-dimensional hot spots. This is more consistent with the physical morphology of coal and rock damage and avoids information loss caused by forcibly simplifying planar damage into linear features, thereby improving the identification accuracy for specific coal and rock damage modes.

[0062] Furthermore, the DIER and TAR parameters extracted via dual routes both possess clear physical meanings (energy and area). When the model determines a certain moment to be a critical precursor frame, technicians can retrospectively examine the changes in energy value and area ratio at that moment to understand whether the warning is based on a sudden increase in energy or spatial expansion. Compared to end-to-end black-box classification models, this feature design provides stronger interpretability of results, helping engineers to conduct secondary verification of warning results by incorporating mechanical mechanisms.

[0063] The technical solution of this application will be further explained below with reference to experiments. As an example, the technical solution of this application can be implemented according to the following steps: Step 1: Construct a dual-path infrared precursor identification framework, wherein the dual paths include an energy evolution path (also known as the first path) and a spatial expansion path (also known as the second path).

[0064] In the first approach, background interference and noise suppression were first applied to over 4000 original infrared thermogram sequences obtained from the experiment to obtain a denoised infrared temperature matrix (DITM). Then, an adaptive thresholding method was used to extract the high-temperature anomaly portion related to damage development from the DITM, yielding the damage infrared matrix (IMD). Based on this matrix, the damage infrared energy response parameter (DIER) was constructed. This parameter describes the overall infrared energy level corresponding to the anomalous thermal zones during loading, essentially reflecting the change in damage thermal response intensity over time, thus characterizing the damage evolution of the coal-rock assemblage from an energy perspective.

[0065] In the second approach, a deep learning model, BPPM-UNet, is constructed for segmenting temperature anomaly regions. For example... Figure 3As shown, the model uses U-Net as the main network and introduces a pyramid pooling module (PPM) at the bottleneck layer to enhance the ability to extract contextual information under different receptive fields, thereby improving the ability to identify weak, discrete and discontinuous temperature anomaly regions.

[0066] Furthermore, the parameter configuration of the pyramid pooling module is: PPM4[1,2,3,6], that is, PPM contains 4 levels of pooling branches. The pooling grid sizes corresponding to the 4 branches are set to 1, 2, 3 and 6 respectively, forming a multi-scale design. Through multi-scale pooling operations, the model can simultaneously aggregate global context information and local detail information of different ranges, thereby enhancing the feature extraction capability of temperature anomaly regions of different sizes and discrete distributions.

[0067] To enable the model to not only focus on surface temperature amplitude but also understand temperature rise variations, material regional differences, and temporal evolution information, a five-channel input was designed, including a temporally denoised infrared temperature matrix (DITM), a temperature increment matrix (TIM), a coal-rock mask matrix (CRM), a region-normalized temperature increment matrix (RN-TIM), and a temporal variation feature matrix (TVF). Specifically, TIM describes the temperature increment relative to the reference frame, CRM explicitly represents the spatial distribution of coal and rock regions, RN-TIM eliminates interference from baseline differences between coal and rock regions, and TVF reflects the temporal rate of change of anomalous thermal responses. Based on this five-channel input, the model can segment anomalous thermal areas in infrared images frame by frame, outputting pixel-level probability maps and binary masks, and further calculate the thermal anomaly region proportion (TAR), which characterizes the area proportion of the anomaly region in each frame.

[0068] After completing the two aforementioned routes, DIER and TAR are input as joint features into the IsolationForest model. DIER(q) reflects the energy enhancement process of the damage thermal response, while TAR(q) reflects the spatial expansion process of the temperature anomaly region. The two have a clear complementary relationship, where q represents the frame number in the infrared thermographic sequence. By identifying anomalous samples deviating from the normal evolutionary trajectory in the feature space formed by the two, the earliest anomalous frame can be automatically determined and defined as the critical precursor frame. Further combining the infrared sampling frequency or stress-time curve, the corresponding forecasting time can be obtained, i.e., the advance warning time, thereby achieving quantitative identification of key precursor moments before the macroscopic instability of the coal-rock assemblage.

[0069] Thus, this application has constructed a complete methodological system from raw infrared thermal image data processing, multi-channel feature construction, deep learning anomaly segmentation, infrared feature quantization, to anomaly detection and precursor recognition.

[0070] Step Two: Specific Implementation Process.

[0071] like Figure 1 As shown, a coal-rock composite sample was used as the research object, and a destructive test was carried out using uniaxial compression loading. The infrared thermography sampling frequency was 5 Hz, and the original infrared sequence contained approximately 1300 frames. During the test, the time-series infrared temperature field data of the sample surface was continuously acquired using an infrared thermography imager, while auxiliary information such as stress and acoustic emission was recorded simultaneously.

[0072] For a successful uniaxial compression experiment of coal and rock mass, the original experimental data files (such as ATS files) recorded synchronously during the experiment are first played back. The infrared thermal imaging acquisition system's built-in software is used to "export the infrared raw data as a CSV temperature field", resulting in thousands of temperature field matrix files (stored in .csv file format). Each CSV file contains 484×512 temperature values ​​according to the experimental settings, that is, the size of the temperature field matrix is ​​484×512.

[0073] The raw temperature field data from the experiment was imported into the `data` section of the Python project. ROI (Region of Interest) extraction was used to extract the rock sample area from the infrared camera's acquisition region, resulting in a time-series ROI temperature field matrix of size 80×150. To eliminate interference from background noise such as ambient temperature and airflow during the ROI temperature field matrix acquisition process, the average surface temperature of a reference rock under the same experimental conditions was used as a temperature correction parameter to perform preliminary noise reduction on the time-series ROI temperature field matrix.

[0074] Secondly, to reduce the unavoidable Gaussian noise and impulse noise introduced by the infrared camera into the temperature field matrix during acquisition, an adaptive median filtering algorithm and a BM3D denoising algorithm are used to filter the time-series ROI temperature field matrix, resulting in a denoised infrared temperature matrix DITM, as shown below. Figure 5 As shown.

[0075] The temporal ROI temperature field DITM obtained from the preceding image denoising step is used as input to the adaptive thresholding segmentation method (Otsu image segmentation algorithm). By calculating the optimal segmentation threshold for each time step, the image background (regions smaller than the segmentation threshold) is set to 0, and the damage infrared matrix IMD is extracted. Figure 6 As shown.

[0076] Based on the Stefan–Boltzmann law, the infrared thermal radiation energy of the coal and rock mass surface characterized by IMD is calculated, and this energy value is combined with the sampling time interval of the infrared camera and the actual physical surface area of ​​the rock to obtain the time-series damage infrared energy response parameter DIER, which characterizes the damage on the rock surface.

[0077] Specifically, the radiant power density of each non-zero pixel is calculated according to the following formula: , In the formula, For the qth frame pixel The radiant power density is used to characterize the infrared thermal radiation energy of that pixel. This is the Stefan–Boltzmann constant. For material emissivity, For pixels The temperature is given, and i and j represent the coordinates of the pixel in the IMD matrix of that frame. Then, the radiant power density of all non-zero pixels in the damaged infrared matrix IMD is accumulated, and the total radiant power of the current frame is obtained based on the actual physical area represented by a single pixel (which can be determined according to the resolution). DIER for each frame is calculated using the following formula: , In the formula, f s Let A be the sampling time interval of the infrared camera, and let N and M be the actual physical surface area corresponding to the region of interest. N and M represent the total number of pixels in the x and y directions of the IMD matrix, respectively. Calculate the DIER for each frame to form the temporally damaged infrared energy response parameter DIER(q).

[0078] Based on this, the dataset required for the deep learning model is constructed. Specifically, this includes generating a TIM based on DITM, manually constructing a CRM based on the spatial distribution of coal and rock, standardizing the TIM in the coal and rock areas separately according to the CRM to obtain RN-TIM, and calculating the time derivative of RN-TIM to obtain TVF. Subsequently, DITM, TIM, CRM, RN-TIM, and TVF are combined by channel to form a five-channel input sample.

[0079] At the same time, several representative frames were selected from the original infrared sequence and manually labeled (e.g.) Figure 2 As shown, sample 1 is first cropped into region of interest (ROI) 2 to obtain the original infrared thermal image (about 1300 frames). Then, the temperature anomaly zone boundary is drawn using Labelme annotation software, and the corresponding pixel-level binary mask (i.e., annotation result, about 130 frames) is generated as the label data required for supervised training.

[0080] Next, a two-stage training strategy was adopted to train BPPM-UNet. In the first stage, supervised training was performed using only manually labeled samples to establish an initial mapping relationship between multi-channel infrared features and anomalous temperature regions, resulting in an initial segmentation model. In the second stage, the model trained in the first stage was used to predict unlabeled infrared frames, generating pseudo-labels and probability maps. The manually labeled samples and highly reliable pseudo-labels were then used together for model refinement training to expand the coverage of training samples and enhance the model's ability to identify anomalous regions across the entire time series of infrared data. Figure 7 As shown. After training, the trained model is used to segment abnormal regions frame by frame in all infrared sequences (results are shown in...). Figure 8 As shown), output pixel-level probability map. and binary outlier mask Based on this, the thermal anomaly area percentage (TAR) of each frame is calculated as TAR(q), where... These are pixel coordinates.

[0081] Finally, the DIER(q) and TAR(q) obtained frame by frame are combined to form a joint feature sequence, which is then input into the Isolation Forest model for anomaly detection. The execution flow is as follows: Figure 4 As shown.

[0082] The model identifies the earliest anomalous samples deviating from the normal evolutionary trajectory in the feature space, determines the critical precursor frame, and further calculates the corresponding damage warning time. For example... Figure 9 As shown in (a), the anomaly score is the output of the Isolation Forest model, and therefore this value is the result of the calculation of the joint features of DIER and TAR. Before the red dot, the anomaly score mainly fluctuates in the negative range (between -0.2 and 0). In the Isolation Forest algorithm, this represents that the sample point is in the normal distribution area, that is, the coal and rock are in the normal loading and deformation stage. At frame 3183 (corresponding to the red dot), the anomaly score suddenly jumps into the positive range. At this moment, the joint features (energy + space) deviate significantly from the previous normal evolution trajectory in the feature space and are judged as an anomaly by the model. Figure 9 (b) shows that before the red dot, DIER remains low and fluctuates little, indicating that the infrared radiation energy on the surface of the coal-rock composite is stable for most of the loading time, and damage evolution is slow. At the red dot, DIER shows a sharp vertical upward jump, indicating that the accumulation of damage inside the coal-rock has reached a critical point, microcracks rapidly expand, leading to a sharp release of surface thermal radiation energy, which is a physical signal that damage is about to occur (i.e., a precursor to damage). The time from the critical precursor frame to the macroscopic damage moment is called the warning time, and thus the warning window is defined. Figure 9The interval indicated by "Ⅲ" in the diagram is used to calculate the damage warning time based on the difference between the time point of the critical precursor frame and the time of macroscopic damage. The warning advance time is 25.20 seconds.

[0083] Thus, this embodiment completes the entire implementation process from infrared thermal image acquisition, quantitative analysis of damage thermal response, identification of abnormal areas, to extraction of key precursors. This method can provide a physically meaningful, highly automated, and interpretable technical solution for predicting the failure of coal-rock assemblages and for infrared early warning.

[0084] In summary, the infrared and deep learning-based method for identifying precursors of coal and rock mass failure provided in this application can extract energy response information related to damage evolution from the infrared temperature field and accurately segment and quantify the spatial evolution process of temperature anomaly regions. Through joint analysis of energy response and spatial expansion, it overcomes the limitation of a single indicator in comprehensively characterizing the damage state, achieving a dual representation of the damage evolution process and providing richer feature evidence for identifying precursors of coal and rock mass failure. This scheme effectively combines traditional digital image processing, deep learning anomaly segmentation, and anomaly detection methods, establishing a complete technical chain from raw infrared thermal images to key precursor identification. Data flow between each stage is clear, and feature transmission is coherent, ensuring the integrity and logic of the data processing flow. Automatic identification of key precursor moments before macroscopic instability based on joint feature sequences reduces the subjectivity and uncertainty of manual experience-based judgment, improving the automation and efficiency of the early warning process. The feature parameters extracted by this method have clear physical meaning, and the anomaly detection results can be traced back to specific energy and spatial changes, improving prediction performance and interpretability, facilitating understanding and verification by engineers, and meeting the needs of disaster early warning fields for method reliability and standardized application.

[0085] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for identifying precursors of coal and rock mass failure based on infrared and deep learning, characterized in that, Includes the following steps: Step S1: Obtain the original infrared thermographic sequence during the loading process of the coal-rock composite. After region of interest extraction and noise suppression, the temporal denoised infrared temperature matrix DITM is obtained. Step S2: Based on the time-series denoised infrared temperature matrix DITM, the damage infrared matrix is ​​extracted by adaptive threshold segmentation, and the time-series damage infrared energy response parameter DIER is calculated in combination with the thermal radiation law to characterize the energy enhancement features of damage evolution. Step S3: Construct multi-channel input samples, which include at least: the temporal denoised infrared temperature matrix DITM, the temperature increment matrix TIM, the coal and rock mask CRM, the regional normalized temperature increment matrix RN-TIM, and the temporal variation feature TVF; The multi-channel input samples are input into a deep learning segmentation model to obtain pixel-level segmentation results of the temperature anomaly region, and the thermal anomaly region ratio (TAR) is calculated to characterize the spatial expansion features of damage evolution. Step S4: The time-series damage infrared energy response parameter DIER and the thermal anomaly region proportion TAR are combined to form a joint feature sequence, which is then input into the anomaly detection model to identify abnormal samples that deviate from the normal evolutionary trajectory in the joint feature space, determine the critical precursor frame, and realize the identification of damage precursors.

2. The method according to claim 1, characterized in that, Step S2 specifically includes: The optimal segmentation threshold for each frame in the temporally denoised infrared temperature matrix DITM is calculated using the maximum inter-class variance method. Pixels with temperature values ​​less than the optimal segmentation threshold in each frame are set to zero, while the temperature values ​​of the remaining pixels are retained to obtain the damaged infrared matrix IMD. Based on the Stefan-Boltzmann law, the radiation energy of each pixel in the damaged infrared matrix is ​​calculated, and the radiation energy of all pixels is accumulated to obtain the total frame radiation energy. The temporal damage infrared energy response parameter DIER is calculated by combining the damaged infrared matrix IMD, the infrared camera sampling time interval, and the actual physical surface area corresponding to the region of interest.

3. The method according to claim 1, characterized in that, The temperature increment matrix TIM is calculated as follows: Obtain the reference frame; For each frame in the temporally denoised infrared temperature matrix DITM, the temperature field of that frame is compared with the temperature of the corresponding pixel in the reference frame to calculate the difference, thus obtaining the temperature increment matrix TIM.

4. The method according to claim 3, characterized in that, In step S3, The coal-rock mask CRM is generated based on the coal-rock partitioning structure of the coal-rock composite sample and is used to identify the coal zone / rock zone to which the pixel belongs. The regional normalized temperature increment matrix RN-TIM is constructed based on the temperature increment matrix TIM and the coal-rock mask CRM. The construction process includes: using the coal-rock mask CRM to divide the temperature increment matrix TIM into a coal area data subset and a rock area data subset; calculating the statistical characteristic values ​​of the coal area data subset and the rock area data subset respectively to form statistical characteristic value pairs; and using the statistical characteristic values ​​to independently normalize the temperature increment data in the corresponding data subset to obtain the regional normalized temperature increment matrix RN-TIM.

5. The method according to claim 4, characterized in that, The calculation method for the time-varying characteristic TVF is as follows: Based on the temporal sequence of the region normalized temperature increment matrix RN-TIM, the numerical difference of corresponding pixels between adjacent frames is calculated. The numerical difference is used as the time-series variation feature (TVF) to characterize the rate of change of temperature anomaly response over time.

6. The method according to claim 1, characterized in that, The deep learning segmentation model is a U-Net network model with an embedded pyramid pooling module; The U-Net network model includes a downsampling path, an upsampling path, and a bottleneck layer connecting the two. The pyramid pooling module is set in the bottleneck layer and is used to perform multi-scale pooling operations on the feature map output by the bottleneck layer and fuse context information. The output layer of the deep learning segmentation model is used to generate a pixel-level probability map representing the distribution of temperature anomaly regions.

7. The method according to claim 6, characterized in that, The calculation method for the proportion of thermal anomaly area (TAR) includes: Based on the pixel-level probability map output by the deep learning segmentation model, a binary anomaly region mask is obtained by setting a confidence threshold. The number of pixels representing the abnormal region in the binary abnormal region mask is counted, and the ratio of the number of pixels to the total number of effective pixels in the region of interest is calculated to obtain the thermal abnormal region proportion (TAR).

8. The method according to claim 1, characterized in that, Step S4 includes: An isolated forest model is used as the anomaly detection model to fit the joint feature sequence; Calculate the anomaly score of each sample point in the joint feature sequence in the joint feature space, and identify abnormal sample points that deviate from the normal evolutionary trajectory based on the anomaly score; The infrared thermal image frame number corresponding to the earliest occurrence of the abnormal sample point in time sequence is determined as the critical precursor frame. The damage warning time is calculated based on the frame number of the critical precursor frame and the sampling frequency of the infrared thermal image sequence.

9. The method according to claim 1, characterized in that, Step S1 includes: The temperature field region corresponding to the coal-rock composite sample is extracted from each frame of the original infrared thermal image sequence to obtain the time-series region of interest temperature field matrix; An adaptive median filtering algorithm is used to suppress impulse noise in the temperature field matrix of the time-series region of interest to obtain the temperature field matrix after impulse noise suppression. Gaussian noise suppression is applied to the temperature field matrix after impulse noise suppression using a block-matching three-dimensional filtering algorithm to obtain the time-series denoised infrared temperature matrix DITM.

10. A coal and rock mass failure precursor identification system based on infrared and deep learning, characterized in that, include: The data acquisition unit is configured to acquire the original infrared thermal image sequence during the loading process of the coal-rock composite, and after region of interest extraction and noise suppression processing, obtain the time-series denoised infrared temperature matrix DITM. The energy extraction unit is configured to extract the damage infrared matrix based on the time-series denoised infrared temperature matrix through adaptive threshold segmentation, and calculate the time-series damage infrared energy response parameter DIER in combination with the thermal radiation law to characterize the energy enhancement features of damage evolution. The spatial extraction unit is configured to construct multi-channel input samples, which include at least the following: the temporally denoised infrared temperature matrix DITM, the temperature increment matrix TIM, the coal and rock mask CRM, the region normalized temperature increment matrix RN-TIM, and the temporal variation feature TVF. The multi-channel input samples are input into a deep learning segmentation model to obtain pixel-level segmentation results of the temperature anomaly region, and the thermal anomaly region ratio (TAR) is calculated to characterize the spatial expansion features of damage evolution. The damage precursor identification unit is configured to construct a joint feature sequence by combining the temporal damage infrared energy response parameter DIER and the thermal anomaly region proportion TAR, input it into the anomaly detection model, identify abnormal samples that deviate from the normal evolution trajectory in the joint feature space, determine the critical precursor frame, and realize damage precursor identification.

Images