Drilling cuttings dynamic sensing method and system for deep coal mine disaster early warning

Abstract

The application discloses a kind of drilling cuttings dynamic perception method and system for deep coal mine disaster early warning, system includes video acquisition and space-time synchronization module, light attention image enhancement module, U-Net drilling cuttings target segmentation module, transfer learning feature coding module, multi-modal time sequence fusion module, LSTM dynamic perception and early warning module, self-evolution incremental learning module;The application is through the light image enhancement technology of fusion channel and space attention, effectively suppresses deep coal mine underground dust, mud, low light and other complex interference, substantially improves drilling cuttings image quality, provides reliable data basis for subsequent feature extraction and perception.Based on transfer learning and ResNet field self-adaptive fine-tuning, make full use of the general feature capability of pre-training model, without large-scale underground labeled sample, high-precision drilling cuttings visual feature coding can be realized, solve the problem of insufficient model generalization ability in small sample scene.

Description

Technical Field

[0001] This invention relates to safety monitoring and video image processing technology in deep coal mines, specifically a dynamic sensing method and system for drill cuttings for early warning of disasters in deep coal mines. Background Technology

[0002] As coal mining progresses into deeper strata, the conditions of high ground stress, complex geological structures, and high gas pressure significantly increase, leading to frequent dynamic disasters such as coal and gas outbursts and rock bursts. These have become core hidden dangers restricting the safe and efficient mining of deep coal mines. Drill cuttings, as the coal and rock medium directly emitted during drilling, provide crucial and intuitive information regarding their morphology, particle size, emission volume, and temporal evolution, reflecting precursors of disasters such as abnormal ground stress, abrupt geological changes, and gas enrichment. Therefore, dynamic sensing of drill cuttings has become an important technical approach for disaster early warning in deep coal mines.

[0003] Existing drill cuttings monitoring and early warning technologies suffer from several inherent problems: Traditional drill cuttings monitoring often relies on manual weighing and contact sensor acquisition, which suffers from data lag, large human error, and the inability to continuously and dynamically perceive data, making it difficult to meet the real-time early warning needs of deep coal mines; Video image-based drill cuttings monitoring methods are constrained by complex underground working conditions, such as poor lighting, high dust concentration, and severe mud obstruction, resulting in low signal-to-noise ratios and significant loss of details in the original video images, directly leading to the failure of subsequent feature extraction and early warning decisions; Underground drill cuttings labeled samples are scarce, and directly training deep learning models is prone to overfitting and insufficient generalization ability, making it difficult to adapt to the differences in working conditions in different mining areas and strata; Existing methods mostly focus on static feature extraction of drill cuttings, lacking modeling of the temporal evolution of drill cuttings, and cannot distinguish between normal working condition fluctuations and disaster precursor anomalies, resulting in high false alarm and false negative rates; At the same time, existing technologies have not formed an end-to-end system of video enhancement-target segmentation-feature encoding-temporal perception-intelligent early warning-self-evolutionary optimization, with each module operating independently, resulting in insufficient overall early warning reliability and adaptability. Summary of the Invention

[0004] The primary objective of this invention is to provide a dynamic sensing method for drill cuttings for early warning of disasters in deep coal mines, in order to solve the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A dynamic sensing method for drill cuttings for early warning of disasters in deep coal mines, the key of which includes the following steps:

[0007] S1: Acquire video of drill cuttings ejection and perform fixed frame extraction to label the images with timestamps and drilling progress, generating a spatiotemporal drill cuttings image sequence.

[0008] S2: Input the spatiotemporal drill cuttings image sequence into a lightweight attention CNN network, and achieve image enhancement through dual attention feature fusion to output high-quality drill cuttings images;

[0009] S3: Input the high-quality drill cuttings image into the U-Net network to complete pixel-level segmentation, generate a drill cuttings binary mask, and extract the clean drill cuttings target image;

[0010] S4: Based on the pre-trained ResNet model, perform domain adaptive fine-tuning to convert the clean drill cuttings target image into a temporal visual feature vector sequence;

[0011] S5: Synchronously collect drilling condition data, concatenate it with the time-series visual feature vector sequence according to timestamps and standardize it to construct a multimodal time-series feature sequence;

[0012] S6: Input the multimodal temporal feature sequence into the LSTM network to complete dynamic feature prediction, and calculate the residual between the predicted value and the true value;

[0013] S7: Generates dynamic early warning thresholds based on sliding window residuals, and outputs disaster early warning signals after multi-step self-verification and confirmation.

[0014] To optimize the relevant model parameters, the above method may further include the following steps:

[0015] S8: Store high-confidence verification data into the experience pool, and use the joint loss function to perform incremental fine-tuning of the model to achieve self-evolutionary optimization of the system.

[0016] Optionally, the lightweight attention CNN network includes a multi-feature fusion unit and a model training unit, wherein:

[0017] Multi-feature fusion unit: Construct a lightweight CNN network that fuses channel attention and spatial attention, and perform Squeeze, Excitation, and Rescale operations on the convolutional feature maps in sequence;

[0018] 3) Squeeze operation: Compresses the spatial information of each channel into a scalar;

[0019] 4) Excitation operation: The channel dependencies are learned through two fully connected layers and an activation function, and the channel weights are output.

[0020] 5) Rescale operation: The weights are applied to the original feature map to enhance important features and suppress noisy features;

[0021] Model training unit: With perceptual loss as the optimization objective, it minimizes the error between the enhanced image and the clear image, and learns the mapping from low-quality image to high-quality image.

[0022] Optionally, the U-Net network includes a segmentation mapping unit and a hybrid loss optimization unit, wherein:

[0023] Segmentation Mapping Unit: Constructs a U-Net encoder-decoder structure, learns non-linear mapping, and transforms high-quality drill cuttings images into pixel-level binary masks; the mask is defined at the pixel level as follows: drill cuttings region pixels are assigned a value of 1, and background region pixels are assigned a value of 0.

[0024] Hybrid Loss Optimization Unit: A hybrid loss function consisting of binary cross-entropy loss and Dice loss is used to solve the problem of imbalance between drill cuttings and background categories;

[0025] Binary cross-entropy loss: ;

[0026] Dice loss: ;

[0027] in, To compensate for Dice loss, optimize the region overlap. For real labels, To predict probabilities, For smoothing, N is the number of pixels.

[0028] Optionally, step S4 specifically includes:

[0029] S41: Load a ResNet-50 model pre-trained on the ImageNet dataset, whose residual blocks satisfy: ,in, Input for residual blocks, For output, It is the residual function;

[0030] S42: Retain the ResNet convolutional layer weights, reconstruct the top-level classifier, and perform differential learning rate fine-tuning using a clean drill cuttings target image, with cross-entropy loss as the optimization objective:

[0031]

[0032] in This represents the batch sample size. This represents the number of drill cuttings morphology categories. For genuine and unique hot tags, To predict probabilities;

[0033] S43: Remove the classification layer, extract the activation values ​​of the global average pooling layer as the feature vector, and calculate the feature vector of a single frame image as follows: ;

[0034] in, To remove the classification head of the ResNet-50 feature extractor, For the fine-tuned model parameters, For the first The clean drill cuttings target image after frame segmentation; finally, the temporal visual feature vector sequence is obtained.

[0035] Optionally, drilling condition data is collected synchronously in step S5, and the condition group at a single moment is: ,in, For timestamps, For drilling progress, To push for pressure, This refers to the drilling rig's rotational speed.

[0036] Optionally, step S6 specifically includes:

[0037] S61: Input the multimodal time-series feature sequence into the LSTM network, and predict the feature value at the next time step based on historical window data. The mean squared error is used as the loss function;

[0038] S62: Calculate the Euclidean distance between the predicted value and the true value to obtain the residual;

[0039] S63: Generating adaptive thresholds based on the statistical properties of sliding window residuals:

[0040]

[0041] in, It is an adjustable parameter. and The residuals are the mean and standard deviation within the sliding window; when the residuals exceed the adaptive threshold, the system triggers a preliminary warning.

[0042] S64: Perform continuous multi-step residual monitoring for preliminary warnings, and confirm the actual disaster warning only when the residuals are continuously abnormal.

[0043] Optionally, in step S8, the self-verified high-confidence time-series samples and labels are stored in the experience pool, and then an incremental learning sample set is constructed by sampling from the experience pool. The sample format is: , indicating the first A time series sample, by The standardized feature vectors at each time step constitute the structure. It is the corresponding binary tag. This is normal. For anomalies, n is the batch size for incremental learning; incremental fine-tuning is performed using a joint loss function, which is: Perform incremental fine-tuning, where To predict losses, For classifying losses, and To balance the weighted hyperparameters of the two losses; where the prediction loss function is:

[0044]

[0045] Classification loss function:

[0046]

[0047] The predictive loss constraint model learns the dynamic patterns of drill cuttings; the classification loss constraint model distinguishes between normal and abnormal patterns. For the first The predicted time-series features of a time-series sample at time step t. For the first Predicted labels for each time series sample.

[0048] Based on the above method, the present invention also provides a dynamic sensing system for drill cuttings for early warning of disasters in deep coal mines, which is used to implement the method described above. The key features are that it includes a video acquisition and spatiotemporal synchronization module, a lightweight attention image enhancement module, a U-Net drill cuttings target segmentation module, a transfer learning feature encoding module, a multimodal temporal fusion module, and an LSTM dynamic sensing and early warning module.

[0049] The video acquisition and spatiotemporal synchronization module is used to continuously acquire video of drill cuttings ejection and perform fixed frame extraction and spatiotemporal annotation, outputting a spatiotemporal drill cuttings image sequence with timestamps and drilling progress.

[0050] The lightweight attention image enhancement module is connected to the output of the video acquisition and spatiotemporal synchronization module, and uses a lightweight CNN network that fuses channel and spatial attention to achieve image noise suppression and detail enhancement.

[0051] The U-Net drill cuttings target segmentation module is connected to the output of the lightweight attention image enhancement module. It uses the U-Net semantic segmentation network to achieve pixel-level separation of drill cuttings foreground and background, outputs a drill cuttings binary mask sequence and extracts a clean drill cuttings target image.

[0052] The transfer learning feature encoding module is connected to the output of the U-Net drill cuttings target segmentation module. Based on the pre-trained ResNet model, it performs domain adaptive fine-tuning and transforms the clean drill cuttings target image into a temporal visual feature vector sequence.

[0053] The multimodal temporal fusion module is connected to the transfer learning feature encoding module and the borehole condition acquisition unit, respectively, and splices and standardizes the temporal visual feature vector sequence with the borehole condition data to construct a multimodal temporal feature sequence;

[0054] The LSTM dynamic perception and early warning module is connected to the output of the multimodal time series fusion module. It realizes the time series prediction of drill cuttings features based on the LSTM network and completes disaster early warning by combining dynamic threshold and self-verification mechanism.

[0055] Optionally, the system may also include a self-evolving incremental learning module, which is connected to the LSTM dynamic perception and early warning module and the transfer learning feature encoding module, respectively, to achieve incremental fine-tuning and self-evolution of the model based on high-confidence data and joint loss function.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] This invention utilizes a lightweight image enhancement technique that integrates channel and spatial attention to effectively suppress complex interferences such as dust, mud, and low light in deep coal mines, significantly improving the quality of drill cuttings images and providing a reliable data foundation for subsequent feature extraction and perception. Based on transfer learning and adaptive fine-tuning in the ResNet domain, it fully leverages the general feature capabilities of pre-trained models, achieving high-precision visual feature encoding of drill cuttings without the need for large-scale underground labeled samples, thus solving the problem of insufficient model generalization ability in small sample scenarios. By modeling the dynamic evolution of drill cuttings using an LSTM temporal network, combined with adaptive dynamic thresholds and a multi-step self-verification mechanism, it effectively distinguishes between normal operating condition fluctuations and disaster precursor anomalies, significantly reducing the false alarm and false negative rates of early warnings. A complete end-to-end system is constructed, encompassing video acquisition, image enhancement, target segmentation, feature encoding, temporal fusion, dynamic early warning, and self-evolutionary optimization. Each module works collaboratively, eliminating performance losses caused by module fragmentation. Through incremental learning and joint loss optimization, the system continuously absorbs new data and optimizes model parameters during operation, constantly improving its adaptability across geological formations and mining areas, enhancing early warning reliability, and extending the system's effective lifespan. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating a specific embodiment of the method of the present invention;

[0059] Figure 2 This is a technical roadmap for drill cuttings video image enhancement and target extraction according to the present invention;

[0060] Figure 3 This is a comparison image showing the image enhancement effect before and after the present invention;

[0061] Figure 4 This is a diagram of the U-Net segmentation network model of the present invention;

[0062] Figure 5 This is a comparison image of the image before and after segmentation according to the present invention;

[0063] Figure 6 This is a roadmap for the generation of drill cuttings time-series feature vector sequences according to the present invention.

[0064] Figure 7 This is a structural diagram of the reconstructed head of the new classifier of the present invention;

[0065] Figure 8 This is a diagram of the dynamic sensing model for drill cuttings according to the present invention;

[0066] Figure 9 This is a diagram of the LSTM network structure of the present invention;

[0067] Figure 10 This is a system module architecture diagram;

[0068] Figure 11 This is a comparison image of each stage of the experimental procedure in a specific embodiment of the present invention;

[0069] Figure 12 The above is a simulated operating condition parameter change curve in a specific embodiment of the present invention;

[0070] Figure 13 This is a specific embodiment of the invention that simulates the risk probability and alarm results under abnormal conditions;

[0071] Figure 14 This is a comparison chart of the early warning effects at different time periods in a specific embodiment of the present invention. Detailed Implementation

[0072] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0073] like Figure 1 As shown, this embodiment provides a dynamic sensing method for drill cuttings for early warning of disasters in deep coal mines, including the following steps:

[0074] S1: Acquire video of drill cuttings ejection and perform fixed frame extraction to label the images with timestamps and drilling progress, generating a spatiotemporal drill cuttings image sequence.

[0075] S2: Input the spatiotemporal drill cuttings image sequence into a lightweight attention CNN network, and achieve image enhancement through dual attention feature fusion to output high-quality drill cuttings images;

[0076] S3: Input the high-quality drill cuttings image into the U-Net network to complete pixel-level segmentation, generate a drill cuttings binary mask, and extract the clean drill cuttings target image;

[0077] S4: Based on the pre-trained ResNet model, perform domain adaptive fine-tuning to convert the clean drill cuttings target image into a temporal visual feature vector sequence;

[0078] S5: Synchronously collect drilling condition data, concatenate it with the time-series visual feature vector sequence according to timestamps and standardize it to construct a multimodal time-series feature sequence;

[0079] S6: Input the multimodal temporal feature sequence into the LSTM network to complete dynamic feature prediction, and calculate the residual between the predicted value and the true value;

[0080] S7: Generates dynamic early warning thresholds based on sliding window residuals, and outputs disaster early warning signals after multi-step self-verification and confirmation;

[0081] S8: Store high-confidence verification data into the experience pool, and use the joint loss function to perform incremental fine-tuning of the model to achieve self-evolutionary optimization of the system.

[0082] In practice, an industrial camera is used to continuously capture video streams of the drill cuttings ejection process in the deep coal mine drilling area. The capture angle covers the entire area of ​​the drill cuttings outlet to ensure that the entire process of drill cuttings ejection is completely recorded. Then, frame sampling is performed at fixed intervals according to a preset sampling frequency, breaking the continuous video stream into discrete image frames. The sampling interval of each frame strictly follows... , To ensure temporal consistency, a video sampling frequency is set. Then, a unique timestamp is assigned to each frame, and the drilling footage data is synchronously read. The timestamps are then linked to the drilling footage to complete spatiotemporal annotation, generating a traceable and aligned sequence of drill cuttings images. This sequence serves as the original input for all subsequent processing steps, ensuring the spatiotemporal regularity of the data.

[0083] pass Figure 1 and Figure 2 It can be seen that the lightweight attention CNN network includes a multi-feature fusion unit and a model training unit, wherein:

[0084] Multi-feature fusion unit: Construct a lightweight CNN network that fuses channel attention and spatial attention, and perform Squeeze, Excitation, and Rescale operations on the convolutional feature maps in sequence;

[0085] 1) Squeeze operation: Compresses the spatial information of each channel into a scalar;

[0086] 2) Excitation operation: The channel dependencies are learned through two fully connected layers and an activation function, and the channel weights are output.

[0087] 3) Rescale operation: The weights are applied to the original feature map to enhance important features and suppress noisy features;

[0088] Model training unit: With perceptual loss as the optimization objective, it minimizes the error between the enhanced image and the clear image, and learns the mapping from low-quality image to high-quality image.

[0089] After receiving the original image sequence, the lightweight attention-based image enhancement module activates the multi-feature fusion unit. First, it performs a convolution operation on the image to generate a multi-dimensional feature map. The process then sequentially executes three channel attention operations: Squeeze, Excitation, and Rescale. The Squeeze operation compresses the spatial dimension information of each channel into a single scalar using global pooling, completing a global description of the channel features. The Excitation operation learns the importance weights of different channel features through two fully connected layers and a non-linear activation function, distinguishing effective drill cuttings features from dust and mud noise features. The Rescale operation weights the learned channel weights back onto the original feature map, enhancing key features such as drill cuttings edges, granularity, and texture, as well as suppressing noise interference. Simultaneously, the model training unit loads pre-set clear drill cuttings image samples, using perceptual loss as the optimization objective, and iteratively updates the lightweight CNN network parameters to adapt the enhanced model to complex downhole lighting and high-dust conditions. The final output is a high-quality drill cuttings image sequence with clear details and thorough noise suppression. The comparison of the effects before and after image enhancement is shown below. Figure 3 As shown.

[0090] pass Figure 2 , Figure 4 and Figure 5 It can be seen that the U-Net network includes a segmentation mapping unit and a hybrid loss optimization unit;

[0091] Segmentation Mapping Unit: Constructs the U-Net encoder-decoder structure and learns nonlinear mappings. This will enhance the image. Convert to pixel-level binary mask The mask at pixel (x,y) is defined as follows: pixels in the cuttings region are assigned a value of 1, and pixels in the background region are assigned a value of 0.

[0092]

[0093] Among them, the mask The mathematical definition at (x,y) is:

[0094] ;

[0095] Hybrid Loss Optimization Unit: Employs a hybrid loss function consisting of binary cross-entropy loss and Dice loss to address the imbalance between drill cuttings and background categories; Binary Cross-Entropy Loss:

[0096]

[0097] Dice loss: ;

[0098] in, Dice loss is used to optimize region overlap. For real labels, To predict probabilities, For smoothing terms, Figure 5 The image before and after segmentation can be seen to show the difference.

[0099] pass Figure 6 As can be seen, step S4 specifically includes:

[0100] S41: Load a ResNet-50 model pre-trained on the ImageNet dataset, whose residual blocks satisfy: ,in, Input for residual blocks, For output, It is the residual function;

[0101] S42: Retain the ResNet convolutional layer weights, reconstruct the top-level classifier, and perform differential learning rate fine-tuning using a clean drill cuttings target image, with cross-entropy loss as the optimization objective:

[0102]

[0103] in This represents the batch sample size. This represents the number of drill cuttings morphology categories. For genuine and unique hot tags, To predict probabilities;

[0104] S43: Remove the classification layer, extract the activation values ​​of the global average pooling layer as the feature vector, and calculate the feature vector of a single frame image as follows: ;

[0105] in, To remove the classification head of the ResNet-50 feature extractor, For the fine-tuned model parameters, For the first The clean drill cuttings target image after frame segmentation; finally, the temporal visual feature vector sequence is obtained.

[0106] The transfer learning feature encoding module receives the segmented clean drill cuttings target image sequence. First, a ResNet-50 model pre-trained on a large public dataset is loaded by the pre-trained model loading unit. This leverages the general visual feature extraction capabilities provided by the pre-trained weights to avoid the training difficulties caused by small downhole samples. Residual block structure. To prevent gradient vanishing during deep network training and improve the depth of model feature extraction, a domain-adaptive fine-tuning unit is used to reconstruct the top-level ResNet classifier. The reconstructed version is as follows: Figure 7 As shown, the original classification head is then removed to adapt to the requirements of drill cuttings morphology feature extraction. A smaller learning rate is used to fine-tune the model's convolutional layers and the new classifier, with cross-entropy loss (LCE) as the optimization objective, shifting the model from general visual feature extraction to drill cuttings-specific visual feature extraction. After fine-tuning, the feature vector generation unit removes the final classification layer and extracts the activation values ​​of the global average pooling layer as the feature vector for a single frame image. This vector densely encodes key visual information such as drill cuttings grain size, texture, edges, and area. The feature vectors of consecutive frames are arranged in chronological order to generate a complete temporal visual feature sequence of drill cuttings.

[0107] The multimodal time-series fusion module activates the operating condition synchronization unit to read the drilling equipment's operating parameters in real time, including the drilling progress at the corresponding timestamp. Drilling rig propulsion pressure Drilling rig speed The operating condition data and visual feature vectors are strictly aligned according to time stamps. The sequence stitching unit combines the visual feature vectors at a single moment. By directly concatenating the three operating condition data, a multi-dimensional time series input can be constructed. It integrates visual precursor information with equipment operating status information to improve perception dimensions and early warning reliability. The standardized unit performs standardization processing on the stitched time sequence, through... Eliminating the scale difference between the high-dimensional values ​​of visual features and the physical dimensions of operating parameters accelerates the convergence speed of subsequent time-series models and improves training stability.

[0108] pass Figure 8 and Figure 9 It can be seen that the LSTM dynamic sensing and early warning module receives standardized multimodal time series sequences. The time series prediction unit, based on the gating mechanism of the LSTM network, learns the long-term dependencies and dynamic evolution patterns of drill cuttings features during normal drilling. Using historical time series window data as input, it predicts the normal state of drill cuttings features at the next moment. The mean squared error loss (LMSE) is employed to minimize the error between the predicted and actual values, thereby improving prediction accuracy. The residual calculation unit calculates the Euclidean distance between the predicted and actual observations in real time to obtain the residuals. This residual directly quantifies the degree of deviation between the current drilling status and the normal mode; the greater the deviation, the higher the risk of disaster. The dynamic threshold unit is based on the mean of the residuals within the recent sliding window. with standard deviation Generate adaptive threshold To replace fixed thresholds and adapt to normal fluctuations under different formations and drilling parameters, when The system triggers an initial warning. The self-verifying warning unit initiates a continuous multi-step residual monitoring mechanism to continuously track the changes in residuals over multiple time steps. If the residuals remain above the threshold or continue to increase, it is determined to be a precursor to a real disaster and an warning signal is output. If the residuals quickly fall back to the normal range, it is determined to be a transient disturbance and the warning is canceled, significantly reducing the system's false alarm rate.

[0109] Finally, the self-evolving incremental learning module continuously operates during system operation. The experience pool storage unit receives high-confidence data confirmed by the self-verification early warning unit in real time, including time-series samples of normal operating conditions and disaster anomalies. The samples are accompanied by accurate binary labels, forming a high-quality experience dataset. The incremental dataset construction unit periodically samples from the experience pool in batches to construct an incremental learning sample set. , This ensures the temporal integrity and label accuracy of the incremental data. The joint loss optimization unit loads the incremental dataset and employs the joint loss function. Simultaneous incremental fine-tuning of the LSTM early warning model and the ResNet feature encoding model: prediction loss To ensure the accuracy of the model's time-series predictions, classification loss Improve the model's ability to distinguish between normal and abnormal patterns, and balance the coefficients. , The system is dynamically adjusted based on the complexity of the working conditions. After incremental fine-tuning, the model parameters are updated and the original parameters are overwritten, enabling the system to continuously learn the characteristics and patterns of new working conditions and new strata during long-term operation. This continuously optimizes the feature extraction capability and the accuracy of early warning decisions, achieving self-evolutionary optimization without human intervention and improving the system's generalization adaptability under different deep coal mining areas and geological conditions.

[0110] Based on the above method, this embodiment also discloses a dynamic sensing system for drill cuttings for early warning of disasters in deep coal mines, such as... Figure 10 As shown, it includes a video acquisition and spatiotemporal synchronization module, a lightweight attention image enhancement module, a U-Net drill cuttings target segmentation module, a transfer learning feature encoding module, a multimodal temporal fusion module, an LSTM dynamic perception and early warning module, and a self-evolving incremental learning module;

[0111] The video acquisition and spatiotemporal synchronization module is used to continuously acquire video of drill cuttings ejection and perform fixed frame extraction and spatiotemporal annotation, outputting a spatiotemporal drill cuttings image sequence with timestamps and drilling progress.

[0112] The lightweight attention image enhancement module is connected to the output of the video acquisition and spatiotemporal synchronization module, and uses a lightweight CNN network that fuses channel and spatial attention to achieve image noise suppression and detail enhancement.

[0113] The U-Net drill cuttings target segmentation module is connected to the output of the lightweight attention image enhancement module. It uses the U-Net semantic segmentation network to achieve pixel-level separation of drill cuttings foreground and background, outputs a drill cuttings binary mask sequence and extracts a clean drill cuttings target image.

[0114] The transfer learning feature encoding module is connected to the output of the U-Net drill cuttings target segmentation module. Based on the pre-trained ResNet model, it performs domain adaptive fine-tuning and transforms the clean drill cuttings target image into a temporal visual feature vector sequence.

[0115] The multimodal temporal fusion module is connected to the transfer learning feature encoding module and the borehole condition acquisition unit, respectively, and splices and standardizes the temporal visual feature vector sequence with the borehole condition data to construct a multimodal temporal feature sequence;

[0116] The LSTM dynamic perception and early warning module is connected to the output of the multimodal temporal fusion module. It realizes the temporal prediction of drill cuttings features based on the LSTM network and completes disaster early warning by combining dynamic thresholds and self-verification mechanisms.

[0117] The self-evolving incremental learning module is connected to the LSTM dynamic perception and early warning module and the transfer learning feature encoding module, respectively, and realizes the incremental fine-tuning and self-evolution of the model based on high-confidence data and joint loss function.

[0118] Experimental performance analysis

[0119] To further illustrate the effectiveness of the drill cuttings dynamic sensing method described in this invention, a controlled software simulation environment was constructed based on existing drill cuttings videos. The videos were sampled at fixed intervals of 5fps and divided into training time periods, time isolation intervals, and subsequent verification time periods according to the source time sequence. Dynamic sensing used 10 consecutive frames as a historical time sequence window and simultaneously constructed three simulated operating conditions for each sampling interval: footage, propulsion pressure, and drilling rig speed.

[0120] To address image degradation caused by low light, dust, blur, and noise, a lightweight attention enhancement network is employed for image restoration. The enhancement network has an input size of 224×224, a training batch size of 16, and an initial learning rate of 1×10⁻⁶. The network was trained for 15 epochs, with reconstruction loss and perceptual loss weights of 0.7 and 0.3, respectively. The U-Net segmentation network had 48 channels, and the binary cross-entropy loss and Dice loss weights were 0.3 and 0.7, respectively. Peak signal-to-noise ratio (PSNR) and structural similarity score (SSIM) were used to evaluate the consistency between the enhanced image and the reference image; higher values ​​indicated better image reconstruction quality. Figure 11 As shown, the PSNR of the degraded input image is 14.6134 dB and the SSIM is 0.5988. After attention enhancement, these values ​​are improved to 22.7406 dB and 0.9044, respectively, indicating that the enhancement module can improve the image quality and structural information preservation under controlled degradation conditions.

[0121] Furthermore, a software-generated granular pixel mask is used as a simulation reference for target segmentation. The Dice coefficient and Intersection over Union (IoU) are used to measure the degree of overlap between the predicted region and the reference mask; higher values ​​indicate more complete target region extraction. The Dice and IoU of the traditional Otsu method are 0.0538 and 0.0276, respectively. After applying U-Net, they reach 0.7734 and 0.6306, respectively. After combining attention enhancement and U-Net, they reach 0.7826 and 0.6429, respectively. Figure 11 It can be seen that traditional thresholding methods are easily affected by complex backgrounds and local bright areas, while the enhancement-segmentation process described above can complete the extraction of the main target region under the current controlled simulation conditions, providing target region information for subsequent visual feature encoding.

[0122] To simulate the working condition response when drilling conditions change, the simulated footage, feed pressure, and rotation speed are kept to fluctuate accordingly during the normal phase. Within a preset simulated abnormal range, a coordinated change is set in which the footage decreases, the feed pressure increases, and the drilling rig rotation speed decreases. The change process is as follows: Figure 12 As shown, visual features correspond to three simulated operating conditions according to sampling time. After dual-branch temporal coding and feature fusion, the abnormal risk probability is output. The fusion model has 51486 parameters, and the decision threshold is determined only by the preceding training time block and remains fixed during subsequent confirmation.

[0123] The early warning effect was evaluated using Accuracy, Precision, Recall, F1 score, and False Alarm Rate (FAR). Accuracy represents the proportion of correctly identified samples out of all samples; Precision represents the proportion of alarm samples that actually belong to the simulated anomaly; Recall represents the proportion of preset simulated anomaly samples that were correctly detected; F1 score is the harmonic mean of Precision and Recall; and FAR represents the proportion of normal samples that were falsely identified as anomalies. After determining the model structure and decision threshold, further review and validation were performed on the latter part of the source video, resulting in 40 valid time-series samples. The decision threshold was fixed at 0.8774. Figure 13 As shown, the risk probability during the normal phase is generally lower than the judgment threshold. After entering the preset simulated abnormal zone, the risk probability increases significantly and an alarm is triggered. The time block has Accuracy of 0.9250, Precision of 0.9167, Recall of 0.8462, F1 of 0.8800, and FAR of 0.0370. Both simulated abnormal events received alarm responses.

[0124] To observe the stability of the results across different time segments of the same video, two further rolling verifications with time isolation were performed. In the first verification, the fusion model achieved an F1 score of 0.9474 and a FAR score of 0.0476; in the second verification, the F1 score was 0.8667 and the FAR score was 0.0741. The results are as follows: Figure 14 As shown. Combining the above image enhancement, target segmentation, and dynamic early warning results, it can be seen that under controlled software simulation conditions with single video source time segmentation, the image enhancement, target extraction, vision-work condition temporal fusion, and risk judgment processes described in this invention can run continuously, and can form corresponding risk differentiation and alarm output for preset simulated abnormal states, thereby further illustrating the feasibility and corresponding technical effects of the technical solution of this invention.

[0125] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dynamic sensing method for drill cuttings for early warning of disasters in deep coal mines, characterized in that, Includes the following steps: S1: Acquire video of drill cuttings ejection and perform fixed frame extraction to label the images with timestamps and drilling progress, generating a spatiotemporal drill cuttings image sequence. S2: Input the spatiotemporal drill cuttings image sequence into a lightweight attention CNN network, and achieve image enhancement through dual attention feature fusion to output high-quality drill cuttings images; S3: Input the high-quality drill cuttings image into the U-Net network to complete pixel-level segmentation, generate a drill cuttings binary mask, and extract the clean drill cuttings target image; S4: Based on the pre-trained ResNet model, perform domain adaptive fine-tuning to convert the clean drill cuttings target image into a temporal visual feature vector sequence; S5: Synchronously collect drilling condition data, concatenate it with the time-series visual feature vector sequence according to timestamps and standardize it to construct a multimodal time-series feature sequence; S6: Input the multimodal temporal feature sequence into the LSTM network to complete dynamic feature prediction, and calculate the residual between the predicted value and the true value; S7: Generates dynamic early warning thresholds based on sliding window residuals, and outputs disaster early warning signals after multi-step self-verification and confirmation.

2. The method for dynamic sensing of drill cuttings for early warning of disasters in deep coal mines according to claim 1, characterized in that, It also includes the following steps: S8: Store high-confidence verification data into the experience pool, and use the joint loss function to perform incremental fine-tuning of the model to achieve self-evolutionary optimization of the system.

3. The method for dynamic sensing of drill cuttings for early warning of disasters in deep coal mines according to claim 1 or 2, characterized in that, The lightweight attention CNN network includes a multi-feature fusion unit and a model training unit, wherein: Multi-feature fusion unit: Construct a lightweight CNN network that fuses channel attention and spatial attention, and perform Squeeze, Excitation, and Rescale operations on the convolutional feature maps in sequence; 1) Squeeze operation: Compresses the spatial information of each channel into a scalar; 2) Excitation operation: The channel dependencies are learned through two fully connected layers and an activation function, and the channel weights are output. 3) Rescale operation: The weights are applied to the original feature map to enhance important features and suppress noisy features; Model training unit: With perceptual loss as the optimization objective, it minimizes the error between the enhanced image and the clear image, and learns the mapping from low-quality image to high-quality image.

4. The method for dynamic sensing of drill cuttings for early warning of disasters in deep coal mines according to claim 3, characterized in that, The U-Net network includes a segmentation mapping unit and a hybrid loss optimization unit, wherein: Segmentation Mapping Unit: Constructs a U-Net encoder-decoder structure, learns non-linear mapping, and transforms high-quality drill cuttings images into pixel-level binary masks; the mask is defined at the pixel level as follows: drill cuttings region pixels are assigned a value of 1, and background region pixels are assigned a value of 0. Hybrid Loss Optimization Unit: A hybrid loss function consisting of binary cross-entropy loss and Dice loss is used to solve the problem of imbalance between drill cuttings and background categories.

5. The method for dynamic sensing of drill cuttings for early warning of disasters in deep coal mines according to claim 1 or 4, characterized in that, Step S4 specifically includes: S41: Load a ResNet-50 model pre-trained on the ImageNet dataset, whose residual blocks satisfy: ,in, Input for residual blocks, For output, It is the residual function; S42: Retain the ResNet convolutional layer weights, reconstruct the top-level classifier, and perform differential learning rate fine-tuning using a clean drill cuttings target image, with cross-entropy loss as the optimization objective: in This represents the batch sample size. This represents the number of drill cuttings morphology categories. For genuine and unique hot tags, To predict probabilities; S43: Remove the classification layer, extract the activation values ​​of the global average pooling layer as the feature vector, and calculate the feature vector of a single frame image as follows: ; in, To remove the classification head of the ResNet-50 feature extractor, For the fine-tuned model parameters, For the first The clean drill cuttings target image after frame segmentation; finally, the temporal visual feature vector sequence is obtained.

6. The method for dynamic sensing of drill cuttings for early warning of disasters in deep coal mines according to claim 5, characterized in that, In step S5, drilling condition data is collected synchronously. The condition group at a single moment is as follows: ,in, For timestamps, For drilling progress, To push for pressure, This refers to the drilling rig's rotational speed.

7. The method for dynamic sensing of drill cuttings for early warning of disasters in deep coal mines according to claim 5, characterized in that, Step S6 specifically includes: S61: Input the multimodal time-series feature sequence into the LSTM network and predict the feature value at the next time step based on historical window data. The mean squared error is used as the loss function; S62: Calculate the Euclidean distance between the predicted value and the true value to obtain the residual; S63: Generating adaptive thresholds based on the statistical properties of sliding window residuals: in, It is an adjustable parameter. and The residuals are the mean and standard deviation within the sliding window; when the residuals exceed the adaptive threshold, the system triggers a preliminary warning. S64: Perform continuous multi-step residual monitoring for preliminary warnings, and confirm the actual disaster warning only when the residuals are continuously abnormal.

8. The method for dynamic sensing of drill cuttings for early warning of disasters in deep coal mines according to claim 6 or 7, characterized in that, In step S8, the self-verified high-confidence time-series samples and labels are stored in the experience pool, and then an incremental learning sample set is constructed by sampling from the experience pool. The sample format is: , indicating the first A time series sample, by The standardized feature vectors at each time step constitute the structure. It is the corresponding binary tag. This is normal. For anomalies, n is the batch size for incremental learning; incremental fine-tuning is performed using a joint loss function, which is: Perform incremental fine-tuning, where To predict losses, For classifying losses, and To balance the weighted hyperparameters of the two losses; where: Prediction loss function: ; Classification loss function: ; The predictive loss constraint model learns the dynamic patterns of drill cuttings; the classification loss constraint model distinguishes between normal and abnormal patterns. For the first The predicted time-series features of a time-series sample at time step t. For the first Predicted labels for each time series sample.

9. A dynamic drill cuttings sensing system for early warning of disasters in deep coal mines, used to implement the method described in any one of claims 1-8, characterized in that, It includes a video acquisition and spatiotemporal synchronization module, a lightweight attention image enhancement module, a U-Net drill cuttings target segmentation module, a transfer learning feature encoding module, a multimodal temporal fusion module, and an LSTM dynamic perception and early warning module; The video acquisition and spatiotemporal synchronization module is used to continuously acquire video of drill cuttings ejection and perform fixed frame extraction and spatiotemporal annotation, outputting a spatiotemporal drill cuttings image sequence with timestamps and drilling progress. The lightweight attention image enhancement module is connected to the output of the video acquisition and spatiotemporal synchronization module, and uses a lightweight CNN network that fuses channel and spatial attention to achieve image noise suppression and detail enhancement. The U-Net drill cuttings target segmentation module is connected to the output of the lightweight attention image enhancement module. It uses the U-Net semantic segmentation network to achieve pixel-level separation of drill cuttings foreground and background, outputs a drill cuttings binary mask sequence and extracts a clean drill cuttings target image. The transfer learning feature encoding module is connected to the output of the U-Net drill cuttings target segmentation module. Based on the pre-trained ResNet model, it performs domain adaptive fine-tuning and transforms the clean drill cuttings target image into a temporal visual feature vector sequence. The multimodal temporal fusion module is connected to the transfer learning feature encoding module and the borehole condition acquisition unit, respectively, and splices and standardizes the temporal visual feature vector sequence with the borehole condition data to construct a multimodal temporal feature sequence; The LSTM dynamic perception and early warning module is connected to the output of the multimodal time series fusion module. It realizes the time series prediction of drill cuttings features based on the LSTM network and completes disaster early warning by combining dynamic threshold and self-verification mechanism.

10. The drill cuttings dynamic sensing system for early warning of disasters in deep coal mines according to claim 9, characterized in that: The system also includes a self-evolving incremental learning module, which is connected to the LSTM dynamic perception and early warning module and the transfer learning feature encoding module, respectively, and realizes incremental fine-tuning and self-evolution of the model based on high-confidence data and joint loss function.

Images