Coal mine gangue hill abnormal change identification and early warning method, system, medium and product

Abstract

A coal mine gangue hill abnormal change identification and early warning method, system, medium and product relate to the field of coal mine safety supervision. The method comprises: performing spatial registration on corresponding images of current remote sensing inspection data and benchmark remote sensing inspection data to obtain spatially aligned benchmark time phase and current time phase; constructing initial difference features according to the characteristics of the benchmark time phase and the current time phase; injecting the initial difference features into the current time phase features to obtain change perception visual features, determining candidate targets based on the change perception visual features, and determining visual detection scores of the candidate targets; calculating scene semantic review scores, three-dimensional deformation review scores of landslide candidate targets, and / or calculating gas anomaly review scores and event evolution scores of ignition candidate targets to calculate risk scores, and outputting early warning results according to the risk scores. The implementation of the above technical scheme improves the accuracy and stability of the identification of coal mine gangue hill landslide and ignition events.

Description

Technical Field

[0001] This application relates to the field of coal mine safety supervision, and in particular to a method, system, medium and product for identifying and warning of abnormal changes in coal mine gangue piles. Background Technology

[0002] Coal mine gangue piles are artificial accumulations formed by the long-term stockpiling of coal gangue during coal mining and washing. Gangue piles are typically characterized by their height, steep slopes, loose material, internal oxidation and heat release, significant risk of spontaneous combustion, and severe erosion during the rainy season. Under long-term stockpiling, rainfall erosion, natural weathering, and internal oxidation, they are prone to landslides, collapses, accumulation at the toe of the slope, abnormal heat, spontaneous combustion, smoke, open flames, and the release of harmful gases.

[0003] Current methods for monitoring coal gangue piles mainly include manual inspections, fixed camera monitoring, single-point sensor monitoring, periodic surveying, and ordinary drone inspections. Manual inspections suffer from incomplete coverage, low efficiency, and personnel safety risks; fixed cameras are limited by their field of view and cannot cover blind spots at the top, back, and bottom of the slope; while single-point sensors can detect local temperature or gas anomalies, they are difficult to pinpoint the source and spatial extent of the anomalies; traditional surveying has a long cycle and cannot detect short-term anomalies in a timely manner. Multi-source data on drone platforms often suffer from problems such as inconsistent timestamps, inaccurate sensor extrinsic parameters, spatial mismatch between images and point clouds, and difficulty in correlating gas sampling locations with visual candidate regions, making it difficult for visual evidence, point cloud evidence, and gas evidence to point to the same risk target. Summary of the Invention

[0004] This application provides a method, system, medium, and product for identifying and warning of abnormal changes in coal mine gangue piles, which improves the accuracy and stability of identifying coal mine gangue pile landslides and fires.

[0005] Firstly, this application provides a method for identifying and providing early warning of abnormal changes in coal mine gangue piles, the method comprising: The baseline remote sensing inspection data and the current remote sensing inspection data of the coal mine gangue pile monitoring area are obtained. The current remote sensing inspection data includes visible light images and infrared thermal images. The corresponding images of the current remote sensing inspection data and the baseline remote sensing inspection data are spatially registered to obtain the spatially aligned baseline time phase and current time phase. Multi-level visual features of the reference time phase and the current time phase are extracted respectively. For the same level, initial difference features are constructed based on the reference time phase features of the reference time phase, the current time phase features of the current time phase, directional difference features, and absolute difference features. The directional difference features and the absolute difference features are the difference features between the reference time phase features and the current time phase features. A cross-attention mechanism is used to inject the initial difference features into the current temporal features to obtain change-aware visual features. Based on the change-aware visual features, candidate targets are determined, and the visual detection scores of the candidate targets are determined. The candidate targets include landslide candidate targets and / or fire candidate targets. Calculate the scene semantic verification score based on the semantic region where the candidate targets are located and the spatial relationship between the candidate targets; The landslide candidate targets are mapped to the lidar point cloud space, and the three-dimensional deformation verification score is calculated using the point cloud elevation change. And / or, for the fire candidate targets, the gas anomaly verification score is calculated by calling the gas sensor data in the same spatiotemporal neighborhood. The event evolution score is calculated based on the mask crossover ratio, area change rate, and review score change trend of the candidate target in multiple consecutive inspections. The risk score is calculated by integrating the visual detection score, the scene semantic verification score, the three-dimensional deformation verification score or the gas anomaly verification score, and the event evolution score, and an early warning result is output based on the risk score.

[0006] By employing the aforementioned technical solution, and constructing initial difference features that include both directional and absolute differences, and then injecting these features into the current temporal features using cross-attention, it is possible to effectively suppress false changes caused by non-risk factors such as illumination, shadows, post-rain humidity, and changes in aerial photography angles, thus highlighting genuine landslide and fire areas. By integrating visual detection scores, scene semantic verification scores, LiDAR 3D deformation verification scores (for landslides) or gas anomaly verification scores (for fires), and event evolution scores, multi-level verification is achieved, progressing from "observing changes" to "reasonableness judgment" and then to "physical / chemical evidence confirmation," significantly improving the accuracy and reliability of anomaly identification. By calculating the mask cross-union ratio, area change rate, and verification score trends of candidate targets in multiple consecutive inspections, an event evolution score is obtained. This score can capture deteriorating trends such as crack expansion, hot spot expansion, and rising gas concentrations, enabling early warning and dynamic risk level assessment.

[0007] In some embodiments, constructing initial difference features based on the reference time phase features of the reference time phase, the current time phase features of the current time phase, directional difference features, and absolute difference features specifically includes: The initial differential features are constructed using the following formula: ; ; in, i The first level represents the multi-level visual features. i Each level The first phase of the reference time i Layer visual features, Indicates the first phase of the current time. i Layer visual features, Indicates directional differences. Indicates absolute differences. Cat (·) indicates a channel splicing operation. Represents a difference projection network. Indicates the first i Initial differences in the hierarchy.

[0008] By employing the above technical solution, the baseline features, current features, directional differences, and absolute differences are simultaneously preserved through channel splicing. This enables the subsequent network to perceive both the direction and intensity of changes, thus more accurately distinguishing between genuine anomalies and noise interference. The combined features of directional and absolute differences weaken the response to symmetrical changes such as illumination, shadows, and humidity, thereby reducing the false alarm rate. The constructed initial difference features serve as the keys and values ​​for the subsequent attention mechanism, containing both original features and difference information. This allows attention to focus on truly physically meaningful areas of change, improving the robustness of change perception.

[0009] In some embodiments, the step of injecting the initial differential features into the current temporal features using a cross-attention mechanism to obtain change-perceived visual features includes: The enhanced features are calculated using the following formula: ; in, Indicates the first i Enhancement features at each level, Indicates the first i Difference injection coefficients at each level Attn (·) denotes the attention function, which is defined by the following formula: ; in, d The dimension of the feature vector. Q For querying the matrix, K The key matrix, V For value matrices, The following formula is used to perform multi-scale fusion calculation of the enhanced features at multiple levels to perceive changes in visual features: ; in, F chg Visual features that indicate change perception. Indicates the first phase of the current time. k A high-level semantic feature, k This is a preset high-level feature index. This represents the set of levels participating in the integration. Indicates the first i Adaptive weights at each level This indicates element-wise multiplication.

[0010] By employing the aforementioned technical solution and utilizing a cross-attention mechanism, the relevance weights between the difference region and the current feature are dynamically calculated using the current temporal feature as the query and the initial difference feature as the key and value. This allows the network to focus on regions highly correlated with real abnormal changes, effectively suppressing background noise and irrelevant spurious changes. The enhanced features injected with differences at multiple levels are weighted and fused with high-level semantic features, preserving local texture details of landslide targets such as slope cracks and slope toe accumulation, as well as global semantic information of fire targets such as hot spot expansion and smoke diffusion, thus improving the detection capability of abnormal targets of different scales and types. A residual connection structure (current feature plus attention-enhanced feature) is used to retain the original current temporal features while injecting difference information, avoiding the loss of original information caused by feature overmodulation, making model training more stable and improving generalization ability.

[0011] In some embodiments, the visual detection score includes a landslide visual detection score and a fire visual detection score, and the visual detection score for determining the candidate target includes: The visual detection score for landslides is calculated using the following formula: ; in, This represents the average response value of the response map of freshly exposed gangue within the candidate region. This represents the average response value of the crack response map within the candidate region. This represents the average response value of the newly added accumulation response map within the candidate region at the toe of the slope. This represents the average response value of the slump texture response map within the candidate region. a1, a2, a3, and a4 are preset weight coefficients, corresponding to the contribution degree of each response map. Norm(·) represents the normalization function. The fire visual detection score is calculated using the following formula: ; in, This represents the average response value of the infrared thermal anomaly response map within the candidate region. This represents the average response value of the smoke response map within the candidate region. This represents the average response value of the open flame response map within the candidate region. This represents the average response value of the blackened region response map within the candidate region. b1, b2, b3, and b4 are preset weight coefficients, corresponding to the contribution degree of each response map.

[0012] Using the above technical solutions, landslide detection integrates four types of responses: fresh exposed soil, cracks, slope toe accumulation, and landslide texture. Fire detection integrates four types of responses: thermal anomalies, smoke, open flames, and blackening. Weighted summation allows different pieces of evidence to corroborate each other, avoiding misjudgments based on a single clue (such as relying solely on thermal anomalies to suggest the source is a vehicle or equipment), thus improving detection accuracy. The average response value of each response map within the candidate region is normalized and outputs a score between 0 and 1, reflecting both the probability of an anomaly and the implied response intensity of the anomaly area, providing quantifiable visual evidence for subsequent risk classification. The preset weighting coefficients a1~a4 and b1~b4 can be adjusted according to the geological conditions, seasonal changes, or historical experience of different waste rock piles. For example, the weight of slope toe accumulation is increased during the rainy season, and the weight of thermal anomalies is increased during the dry season, giving the method scene adaptability.

[0013] In some embodiments, the scene semantic verification score includes a landslide relationship verification score and a fire relationship verification score. The calculation of the scene semantic verification score based on the semantic region where the candidate targets are located and the spatial relationships between the candidate targets specifically includes: The landslide relationship verification score is calculated using the following formula: ; in, The semantic region matching score represents the candidate landslide target. The score indicates the consistency of the landslide direction. This represents the stability score of landslide candidate targets across consecutive time phases. r 1 , r 2 , r 3 Preset weighting coefficients; The fire relationship verification score is calculated using the following formula: ; in, This represents the semantic region matching score of the candidate fire target. The spatial consistency score represents the relationship between thermal anomalies and visible signs of combustion. This represents the stability score of the ignition candidate target across consecutive time phases. q 1 , q 2 , q 3 These are preset weighting coefficients.

[0014] Using the above technical solutions, landslide verification requires that anomalies in the upper part (cracks, exposed surfaces) and the accumulation at the toe of the slope be consistent with the direction of sloping. Fire verification requires that thermal anomalies spatially overlap with smoke, open flames, or blackened areas, thereby eliminating false alarms that do not conform to the physical laws of landslides / spontaneous combustion in waste rock piles. A temporal stability score is introduced, requiring candidate targets to maintain a certain mask cross-union ratio in continuous time phases, effectively suppressing transient interference caused by short-term shadows, dust, reflections, or single-frame sensor noise. The semantic region matching, spatial relationship verification, and temporal stability dimensions are weighted separately and then normalized, making the contribution of each component of the verification score transparent and traceable, facilitating on-site engineers to adjust the weights according to the characteristics of the mining area.

[0015] In some embodiments, mapping the landslide candidate target to the lidar point cloud space and calculating the three-dimensional deformation verification score using point cloud elevation changes specifically includes: Elevation changes are calculated using the following formula: ; in, This represents the current elevation model generated after registering the current time-phase lidar point cloud with the reference time-phase lidar point cloud. Represents the benchmark elevation model; The threshold for significant change is calculated using the following formula: ; in, This represents the quantile corresponding to the confidence level. This represents the standard deviation of the benchmark point cloud measurement error. This represents the standard deviation of the current point cloud measurement error. This represents the standard deviation of the point cloud registration error; The slope loss volume is calculated using the following formula: ; in, R slope This represents the semantic region of the slope, where ΔA represents the area per unit grid. O j Indicates candidate landslide targets; The volume of material accumulated at the toe of the slope is calculated using the following formula: ; in, R toe Indicates the semantic region of the foot of the slope; The volume consistency score is calculated using the following formula: ; in, To prevent small quantities from being reduced to zero; The three-dimensional deformation verification score is calculated using the following formula: ; in, c 1 , c 2 , c 3 These are preset weighting coefficients.

[0016] By employing the above technical solution, elevation models of the preceding and following phases are generated using lidar point clouds. This directly calculates the slope deficit volume and the toe accumulation volume, transforming visually detected changes such as cracks and exposed soil into quantifiable evidence of soil displacement. This effectively eliminates false alarms caused by purely visual spurious changes such as shadows and color variations. A volume consistency score is introduced to penalize situations where the difference between the deficit volume and the accumulation volume is too large, consistent with the physical law of landslide material where "upper deficit volume ≈ lower accumulation volume." This makes the three-dimensional verification results more consistent with actual geological processes, improving the reliability of the early warning. A significant change threshold is set, integrating the baseline point cloud error, the current point cloud error, and the registration error. Only deformations exceeding the statistical confidence interval are included, preventing minor spurious deformations caused by point cloud measurement noise or registration deviations from being misjudged as landslides.

[0017] In some embodiments, calculating the gas anomaly verification score by calling gas sensor data within the same spatiotemporal neighborhood includes: The gas smoothing concentration is calculated using the following formula: ; in, Indicates the current time t gas smoothing concentration, This indicates the smoothed concentration of the gas at the previous moment. This indicates the gas concentration at the current moment. As a smoothing factor, q Indicates the type of gas. q ∈{CO, CH4, SO2, H2S, O2}; Gas statistical outliers are calculated using the following formula: ; in, This represents the historical average concentration of gas type q. Indicates the standard deviation of historical concentrations. Indicates prevention of zero small quantities; The rate of increase in gas concentration is calculated using the following formula: ; Where △t represents the time interval between two consecutive inspections; The gas anomaly review score is calculated using the following formula: ; in, Indicates the weight of the gas category. Indicates the statistical outlier threshold. Indicates the weight of the upward trend. This represents the rate of increase threshold.

[0018] By employing the above technical solution and simultaneously calculating and statistically analyzing outliers and concentration rise rates, a review can be triggered as long as the current concentration has not reached the absolute threshold and a continuous upward trend exists. This approach can capture concealed spontaneous combustion and early oxidation and heating stages, enabling early warning. Multiple gases, including carbon monoxide, methane, sulfur dioxide, hydrogen sulfide, and oxygen, are processed separately and weighted according to the spontaneous combustion patterns of coal gangue piles. This ensures that the review score comprehensively reflects the characteristics of multiple gases, avoiding misjudgments caused by occasional fluctuations of a single gas. An exponential moving average is used to smooth the concentration series, and historical mean and standard deviation are combined for normalization statistics. This effectively suppresses short-term fluctuations caused by instantaneous sensor noise, sudden wind changes, and other environmental factors, improving the stability and reliability of gas review.

[0019] In a second aspect, embodiments of this application provide a computer system including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the steps of the method described in any possible implementation of the first aspect.

[0020] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method described in any possible implementation of the first aspect.

[0021] Fourthly, embodiments of this application provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the method described in any possible implementation of the first aspect.

[0022] It is understood that the computer system provided in the second aspect, the storage medium provided in the third aspect, and the computer program product provided in the fourth aspect are all used to execute the method provided in this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0023] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By constructing directional difference features and absolute difference features, and combining the cross-attention mechanism to inject the initial difference features into the current temporal features, the change perception visual features can suppress visual differences caused by non-risk factors such as changes in illumination, shadow movement, differences in humidity after rain, and deviations in aerial photography angles. It only enhances the response of areas related to real landslides or fires, thereby significantly reducing false anomaly alarms. 2. Visually detected landslide candidate targets are mapped to the LiDAR point cloud space. The elevation changes in the point cloud are used to quantitatively calculate the slope deficit volume, slope toe accumulation volume, and volume consistency, forming a three-dimensional deformation verification score. For fire candidate targets, gas sensor data from the same spatiotemporal neighborhood is used to calculate gas concentration anomalies and rise rates, forming a gas anomaly verification score. This targeted verification mechanism based on candidate target spatial anchoring solves the problem that multi-source sensor data is difficult to point to the same risk target, enabling visual evidence and physical / chemical evidence to corroborate each other, significantly improving the reliability of anomaly identification. 3. By performing semantic analysis on the monitoring area, regions such as slope area, slope foot area, and main body area of ​​the pile are divided, and the scene semantic verification score is calculated based on the semantic region where the candidate target is located and its spatial relationship. This verification mechanism utilizes the inherent physical laws of coal gangue landslides and spontaneous combustion to eliminate a large number of candidate targets that do not conform to common sense; 4. By recording the mask crossover ratio, area change rate, and verification score trends of candidate targets in multiple consecutive inspections, an event evolution score is calculated. This score can reflect deteriorating trends such as crack expansion, increased exposed area, increased slope toe accumulation, hot spot expansion, and continuous increase in gas concentration, thus enabling early warnings to be issued before the risk reaches the level of a visible disaster, and supporting dynamic adjustment of risk levels. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the method for identifying and issuing early warnings of abnormal changes in coal mine gangue piles in this embodiment of the application. Figure 2 This is a schematic diagram of an exemplary hardware structure of a computer system in an embodiment of this application. Detailed Implementation

[0025] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0026] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0027] The following is combined Figure 1 The method of the embodiments of this application will be described below.

[0028] Please see Figure 1 This is a flowchart illustrating the method for identifying and issuing early warnings of abnormal changes in coal mine gangue piles in this embodiment of the application. Figure 1 As shown, the method includes the following steps: S101. Obtain the baseline remote sensing inspection data and the current remote sensing inspection data of the coal mine gangue mountain monitoring area. The current remote sensing inspection data includes visible light images and infrared thermal images. Spatially register the corresponding images of the current remote sensing inspection data and the baseline remote sensing inspection data to obtain the spatially aligned baseline time phase and current time phase. S102. Extract the multi-level visual features of the reference time phase and the current time phase respectively, and for the same level, construct initial difference features based on the reference time phase features of the reference time phase, the current time phase features of the current time phase, the directional difference features, and the absolute difference features. The directional difference features and the absolute difference features are the difference features between the reference time phase features and the current time phase features. S103. The initial difference features are injected into the current temporal features using a cross-attention mechanism to obtain change-aware visual features. Candidate targets are determined based on the change-aware visual features, and the visual detection scores of the candidate targets are determined. The candidate targets include landslide candidate targets and / or fire candidate targets. S104. Calculate the scene semantic verification score based on the semantic region where the candidate target is located and the spatial relationship between the candidate targets; S105. Map the landslide candidate target to the lidar point cloud space, calculate the three-dimensional deformation verification score using the point cloud elevation change, and / or, for the fire candidate target, call the gas sensor data in the same spatiotemporal neighborhood to calculate the gas anomaly verification score. S106. Calculate the event evolution score based on the mask crossover ratio, area change rate, and review score change trend of the candidate target in multiple consecutive inspections. S107. The visual detection score, the scene semantic verification score, the three-dimensional deformation verification score or the gas anomaly verification score, and the event evolution score are integrated to calculate the risk score, and the warning result is output based on the risk score.

[0029] Baseline remote sensing inspection data refers to the image data of the waste rock pile collected by low-altitude remote sensing from UAVs in previous inspections (e.g., the first inspection), while current remote sensing inspection data refers to the data collected during this inspection. Current remote sensing inspection data includes one or more of the following: visible light imagery, infrared thermal imaging imagery, lidar point cloud data, and UAV pose data. Visible light imagery is used to identify exposed areas, cracks, landslide textures, newly accumulated deposits at the toe of the waste rock pile slope, smoke, open flames, and blackened areas; infrared thermal imaging imagery is used to identify thermal anomalies, hot spot expansion, and suspected spontaneous combustion areas; lidar point cloud data is used for three-dimensional deformation verification of landslide candidate areas; gas sensor data is used for gas anomaly verification of fire or spontaneous combustion candidate areas; and UAV pose data is used for multi-source data time synchronization, spatial registration, and spatial anchoring of candidate targets. To eliminate the positional shift of the same geographical area on the image caused by deviations in the UAV go-around flight path, changes in gimbal angle, or differences in flight attitude, this application embodiment spatially registers the corresponding images (i.e., visible light to visible light, infrared to infrared) of the current remote sensing inspection data and the reference remote sensing inspection data. Spatial registration can be achieved using methods such as feature point matching (e.g., SIFT, ORB), homography transformation, orthorectification, or geographic coordinate mapping combined with UAV pose data. After registration, a spatially aligned reference time phase and the current time phase are obtained, ensuring a one-to-one correspondence between the two time phase data under the same pixel coordinates or geographic coordinates, laying the foundation for subsequent pixel-by-pixel or region-by-region difference comparisons.

[0030] For spatially aligned baseline temporal visual data (including multi-channel images stitched together from visible and infrared channels) and current temporal visual data, respectively, a visual encoding network is input to extract multi-level visual features. This visual encoding network can employ a structure combining lightweight convolutional networks (such as MobileNet) and visual Transformers (such as ViT) to balance local texture details and global semantic relationships. The network outputs feature maps at multiple levels, including baseline temporal features and current temporal features. For the same feature level, this embodiment constructs initial difference features. Specifically, it combines baseline temporal features, current temporal features, directional difference features, and absolute difference features. The directional difference features express the direction of change from the baseline state to the current state (e.g., pixel value increase or decrease); the absolute difference features express the intensity of the change. By stitching the above four components along the channel dimension and passing them through a difference projection network (e.g., composed of convolutional layers or fully connected layers), initial difference features for the corresponding level are generated. This initial difference feature simultaneously contains information about the original features, the direction of change, and the intensity of change, which can initially distinguish between real physical changes and pseudo-changes such as illumination and shadows.

[0031] To further enhance the feature response of regions exhibiting real anomalies and suppress irrelevant noise, this embodiment employs a cross-attention mechanism to inject initial differential features into the current temporal features. Specifically, for each feature level, the current temporal features are used as the query, and the initial differential features as the key and value. Cross-attention weights are calculated, and then the initial differential features are weighted and summed using these weights to obtain the differential-enhanced features. Finally, the differential-enhanced features are added to the original current temporal features (residual connection) to obtain the feature injected with differentials at that level. Through residual connections, both the original information of the current temporal phase and the attention-enhanced information related to real changes are preserved. Furthermore, multi-scale fusion (e.g., weighted fusion of deep semantic features and shallow detail features) is performed on the enhanced features at multiple levels to obtain the final change-perceived visual features. This feature exhibits a high response to real anomalies such as landslides (e.g., cracks, exposure, landslide textures) and fires (e.g., thermal anomalies, smoke, open flames, blackening), while suppressing pseudo-changes such as changes in illumination, post-rain humidity differences, and changes in aerial photography angles. Based on change-aware visual features, the system performs candidate target detection, instance segmentation, or semantic segmentation to extract landslide candidate targets and / or fire candidate targets. For example, lightweight target detection networks (such as the YOLO series) or instance segmentation networks (such as Mask R-CNN) can be used to output the bounding boxes, masks, and corresponding confidence scores of the candidate targets. This confidence score is the visual detection score, which reflects the probability of detecting landslide or fire anomalies from the visual image.

[0032] Landslides and fires at coal mine waste rock piles exhibit specific spatial distribution patterns: landslides typically occur on slopes, with cracks and exposed areas at the top and new deposits at the toe, showing spatial consistency along the slope's downward direction; fires or spontaneous combustion usually occur in the main body of the waste rock pile, and areas of thermal anomalies often overlap or are adjacent to areas of smoke, open flames, or blackening. To utilize these patterns to filter out unreasonable candidate targets, this application embodiment performs scene semantic analysis on the monitoring area. A scene semantic region for the waste rock pile is constructed in advance or in real-time, including at least one of the following: main body of the waste rock pile, slope area, toe area, top area, road area, drainage area, and stable background area. The semantic region can be obtained through manual annotation, GIS boundary import, UAV orthophoto segmentation, or 3D terrain-assisted analysis. For each candidate target, a scene semantic verification score is calculated based on its semantic region and the spatial relationships between candidate targets. For example, for landslide candidate targets, a higher verification score is obtained if the target is located in the slope area or toe area, and there is consistency between the upper anomaly (such as cracks or exposed rock) and the lower accumulation along the slope direction; conversely, a lower verification score is obtained if the candidate target is located in the road area or a stable background area. For fire candidate targets, a higher verification score is obtained if the thermal anomaly area overlaps or is adjacent to the smoke, open flame, or blackening area, and is located in the main body area of ​​the pile; a lower verification score is obtained if the thermal anomaly is located in the road vehicle area or drainage area. Through the above semantic verification, a large number of visual false detections that do not conform to the physical laws of coal gangue risk can be eliminated.

[0033] To further verify the results of visual inspection and semantic verification, this application introduces lidar point cloud data and gas sensor data for targeted verification. For landslide candidate targets: the candidate targets are mapped to the lidar point cloud space through point cloud-image registration. Specifically, using pre-calibrated camera and lidar extrinsic parameters and camera intrinsic parameters, the point cloud is projected onto the image plane, or the candidate region in the image is back-projected onto the 3D point cloud, thereby obtaining the 3D point set corresponding to the candidate target. Then, the lidar point cloud of the current time phase is registered with the lidar point cloud of the reference time phase to generate the current elevation model and the reference elevation model, and the elevation change of each grid point is calculated. Based on the elevation change, the slope deficit volume (the part with reduced elevation) and the slope toe accumulation volume (the part with increased elevation) in the candidate area can be calculated. Based on these 3D deformation parameters, a 3D deformation verification score is obtained. This score reflects whether soil displacement has actually occurred in the candidate area, rather than a pseudo-change caused only by changes in image texture. For fire candidate targets: gas sensor data within the same inspection time window and the same spatial neighborhood (i.e., within a certain radius around the candidate target) are called. Gas sensors can be mounted on drones to detect the concentrations of gases such as carbon monoxide, methane, sulfur dioxide, hydrogen sulfide, and oxygen. The system calculates a gas anomaly verification score based on a comparison of the current gas concentration with historical benchmarks (statistical anomalies) and the rate of increase in gas concentration. A higher score is awarded if the concentration of spontaneously combustible gases such as carbon monoxide near the fire candidate target shows an increasing trend; conversely, a lower score is awarded if no gas anomaly is observed. The above target verification steps can be used individually (when only lidar or gas sensors are available) or simultaneously to improve the integrity of the evidence chain.

[0034] Anomalies identified in a single inspection may be subject to transient interference (such as short-term shadows, dust, single-frame reflections, or sensor noise). To determine whether the risk persists or is worsening, embodiments of this application perform spatiotemporal relationship modeling and event evolution analysis on candidate targets. Specifically, the state of each candidate target in multiple consecutive inspections (e.g., the first K inspections) is recorded, including the mask (i.e., the target region outline), position, area, and confidence level. For the current candidate target, the mask intersection-over-union ratio (IoU) between it and the same target in historical time phases (matched by position and mask overlap) is calculated to assess the spatiotemporal stability of the target; the area change rate is calculated to determine whether the anomalous region is expanding (e.g., crack extension, exposure expansion, hot spot expansion, increased slope toe accumulation, etc.); simultaneously, the changing trend of the verification score (3D deformation verification score or gas anomaly verification score) is tracked to determine whether the physical / chemical evidence is strengthening. Based on the above information, the event evolution score is calculated. If a candidate target maintains high mask stability, continuously expands its area, and increases its verification score in consecutive phases, its evolution score is high, indicating that the event is developing and the risk is escalating; conversely, if the target appears only once and then disappears, its evolution score is low.

[0035] This application embodiment weights and fuses visual detection scores, scene semantic verification scores, 3D deformation verification scores (for landslides) or gas anomaly verification scores (for fires), and event evolution scores to calculate a risk score. The risk score is a quantitative indicator based on multi-dimensional evidence, with a value range normalized to 0-1. Based on preset risk grading thresholds (e.g., below T1 is normal, T1-T2 is moderate risk, T2-T3 is significant risk, and above T3 is major risk), the system determines the risk level of the current monitoring area. Simultaneously, it outputs structured early warning results, including: abnormal state (normal / abnormal); event type (landslide / fire); risk level (I / II / III / Normal); abnormal location (e.g., the south slope and toe area); evidence description (briefly describing visual, semantic, verification, and evolutionary evidence); and handling recommendations (e.g., recommending on-site verification of slope stability or immediate verification of CO concentration and implementation of isolation measures). The aforementioned warning results can be pushed to the coal mine safety supervision platform in text, JSON format, or graphical interface form, thereby achieving closed-loop management of gangue pile risks.

[0036] In some embodiments, constructing initial difference features based on the reference time phase features of the reference time phase, the current time phase features of the current time phase, directional difference features, and absolute difference features specifically includes: The initial differential features are constructed using the following formula: ; ; in, iThe first level represents the multi-level visual features. i Each level The first phase of the reference time i Layer visual features, Indicates the first phase of the current time. i Layer visual features, Indicates directional differences. Indicates absolute differences. Cat (·) indicates a channel splicing operation. Represents a difference projection network. Indicates the first i Initial differences in the hierarchy.

[0037] For each feature level i The system performs the following operations: inputs the reference time phase characteristics. Current temporal characteristics directional differences and absolute difference characteristics By splicing along the channel dimension, combined features are obtained. . For one H i × W i × C i tensor, H i , W i , C i These represent the height, width, and number of channels of the feature map at this level. Dimensions and same. This indicates that the difference between the current feature and the baseline feature is calculated element by element. The sign of the feature reflects the direction of change. For example, if a pixel is darker in the baseline phase and brighter in the current phase, the difference is positive, and vice versa. This can be used to distinguish between material increase (such as slope foot accumulation) and material decrease (such as slope loss). This represents the absolute value of the difference between each element, reflecting only the intensity of the change and ignoring the direction. This feature is less sensitive to symmetrical changes such as lighting and shadows, but it can highlight the magnitude of the true change. Cat (·) indicates a concatenation operation along the channel dimension, resulting in a concatenated array. The number of channels is 4× C i Combined features Input to a difference projection network , obtained the i Initial difference features of the hierarchy The structure of a difference projection network can be one or more convolutional layers (e.g., two 3×3 convolutional layers followed by batch normalization and ReLU activation), or a small multilayer perceptron consisting of fully connected layers. The network's role is to reduce the dimensionality of the concatenated high-dimensional combined features and map them to a more compact feature space, while simultaneously learning the nonlinear interactions between different difference components. The dimensionality of the initial difference features output is typically the same as the dimensionality of the current phase. i The visual features of each layer are identical to facilitate subsequent cross-attention injection operations. Through this construction method, the initial difference features simultaneously encode the original feature information, the direction of change, and the intensity of change. Compared to directly using a single difference feature (such as using only the difference value or only the absolute difference value), this provides a richer and more discriminative information basis for subsequent change perception, effectively reducing spurious change responses caused by non-risk factors such as lighting and shadows, and highlighting genuine changes in landslide and fire areas. In a specific embodiment of this invention, for the third level (i=3) of the feature pyramid... and The dimensions are 64×64×128, after stitching The dimensions are 64×64×512; the difference projection network uses two 3×3 convolutions. The first convolution reduces the number of channels from 512 to 256, and the second convolution reduces it to 128, outputting... Its dimensions are 64×64×128, and Consistent.

[0038] In some embodiments, the step of injecting the initial differential features into the current temporal features using a cross-attention mechanism to obtain change-perceived visual features includes: The enhanced features are calculated using the following formula: ; in, Indicates the first i Enhancement features at each level, Indicates the first i Difference injection coefficients at each level Attn (·) denotes the attention function, which is defined by the following formula: ; in, d The dimension of the feature vector. Q For querying the matrix, K The key matrix, V For value matrices, The following formula is used to perform multi-scale fusion calculation of the enhanced features at multiple levels to perceive changes in visual features: ; in, F chgVisual features that indicate change perception. Indicates the first phase of the current time. k A high-level semantic feature, k This is a preset high-level feature index. This represents the set of levels participating in the integration. Indicates the first i Adaptive weights at each level This indicates element-wise multiplication.

[0039] For each feature level i The system employs a cross-attention mechanism to inject initial differential features into the current temporal features. Indicates the first i The differential injection coefficients at each level are preset hyperparameters (e.g., floating-point numbers between 0.1 and 1.0). These coefficients control the injection intensity of attention-enhanced features, preventing overmodulation that could lead to the loss of original feature information. In one implementation, It can be set as a learnable parameter that adjusts automatically during network training. Q This is a query matrix, here representing the current temporal features. In actual calculations, the spatial locations are usually flattened, and each location's... C i A dimensional feature vector is used as a query. K Let be the key matrix, which is used here as the initial difference feature. V The value matrix is ​​also taken as the initial difference feature. d The dimension of the feature vector, i.e. C i Divided by d This is to scale the dot product result and prevent gradients from entering. Softmax The saturation region. Softmax (·) Normalize by row to make the sum of attention weights equal to 1. The above cross-attention calculation process can be explained as follows: using the current temporal features as a reference, calculate the relevance weights of each position to all positions in the initial difference features, and then perform a weighted summation of the initial difference features based on these weights to obtain the change enhancement information related to the current feature. Since the initial difference features already contain the difference information between the baseline and the current feature, this mechanism can adaptively enhance those difference regions that are highly related to the current feature (i.e., regions of real physical change), while suppressing differences that are unrelated to the current feature (such as noise or spurious changes). By adding the attention-enhanced features to the original current features through residual connections, the original local texture details are preserved while incorporating the difference enhancement information, making the feature representation more robust.

[0040] In real-world scenarios, landslides and fire targets have different scales: cracks may be only tens of centimeters wide, while thermal anomaly areas may cover tens of square meters. To capture both fine-grained and coarse-grained features simultaneously, embodiments of this application perform multi-scale fusion of enhanced features at multiple levels to obtain the final change-aware visual features. Indicates the first phase of the current time. k A high-level semantic feature, k This is a pre-defined index for high-level features. High-level features have a larger receptive field and stronger semantic information, such as feature maps output by deep networks. These features serve as the basis for fusion, providing global contextual information. This represents the set of levels participating in the fusion, which can be all levels or a selected number of intermediate levels. For example, in one implementation, These correspond to shallow, intermediate, and deep features, respectively. Indicates the first i Each layer of adaptive weights is a coefficient tensor learned by the network or dynamically generated through an attention mechanism. This weight determines the contribution of the feature from that layer to each spatial location and each channel. The advantages of this multi-scale fusion are: shallow features retain high-resolution spatial details (such as crack edges and plume outlines), which helps to accurately locate candidate target boundaries; deep features provide semantic information (such as whether it is the main body of a pile), which helps to distinguish target categories. By adjusting the adaptive weights, the network can automatically select the most favorable feature scale for different regions, thereby improving the accuracy and recall of candidate target detection. In a specific embodiment of this invention, a three-layer feature pyramid is used: Layer 1 (i=1): feature map size 128×128, 64 channels; Layer 2 (i=2): feature map size 64×64, 128 channels; Layer 3 (i=3): feature map size 32×32, 256 channels. k=3 (high-level semantic features) is chosen. , and The changes-aware visual features are calculated using a lightweight channel attention module. The fused features have a size of 32×32×256. These features are then fed into a candidate target detection head (such as the YOLO detection head or the RPN of Mask R-CNN) for classification and localization, outputting landslide or fire candidate targets and their visual detection scores. Through the aforementioned cross-attention injection and multi-scale fusion, the resulting changes-aware visual features exhibit high responsiveness to real landslide and fire areas and significantly suppress pseudo-changes such as illumination, shadows, post-rain humidity, and changes in aerial photography angle, thus providing a high-quality feature base for subsequent candidate target detection.

[0041] In some embodiments, the visual detection score includes a landslide visual detection score and a fire visual detection score, and the visual detection score for determining the candidate target includes: The visual detection score for landslides is calculated using the following formula: ; in, This represents the average response value of the response map of freshly exposed gangue within the candidate region. This represents the average response value of the crack response map within the candidate region. This represents the average response value of the newly added accumulation response map within the candidate region at the toe of the slope. This represents the average response value of the slump texture response map within the candidate region. a1, a2, a3, and a4 are preset weight coefficients, corresponding to the contribution degree of each response map. Norm(·) represents the normalization function. The fire visual detection score is calculated using the following formula: ; in, This represents the average response value of the infrared thermal anomaly response map within the candidate region. This represents the average response value of the smoke response map within the candidate region. This represents the average response value of the open flame response map within the candidate region. This represents the average response value of the blackened region response map within the candidate region. b1, b2, b3, and b4 are preset weight coefficients, corresponding to the contribution degree of each response map.

[0042] For landslide events, this application focuses on four types of visual anomaly cues: freshly exposed gangue, cracks, newly accumulated debris at the toe of the slope, and landslide texture. These four cues correspond to typical visual changes before, during, and after a gangue landslide. The system processes the change-perceived visual features using four independent detection heads (e.g., four convolutional layers, each outputting a single-channel response map) to generate response maps for freshly exposed gangue. P bare ( x , y Crack response diagram P crk ( x , y ), response diagram of newly added deposits at the toe of the slope P toe ( x , y ) and slip texture response map P sld ( x , yThe pixel value in each response map represents the confidence level (between 0 and 1) that a corresponding anomalous clue exists at that location. For a landslide candidate target region O (e.g., a mask region obtained through instance segmentation), the system calculates the average response value of the above four response maps within that region: ; ; ; ; Then, the landslide visual detection score is obtained through weighted summation and normalization. a1, a2, a3, and a4 are preset weight coefficients, corresponding to the contribution levels of fresh exposed rock, cracks, slope toe accumulation, and landslide texture, respectively. These weights can be set according to the specific geological conditions of the waste rock pile and historical landslide cases. For example, during the rainy season, slope toe accumulation may be more significant, so a3 can be appropriately increased; during the dry season, cracks and exposed rock may be more prominent, so a1 and a2 can be increased. In a typical implementation, a1 = 0.3, a2 = 0.3, a3 = 0.2, a4 = 0.2, and the sum is 1.

[0043] For fire events (including spontaneous combustion), this application focuses on four types of visual anomaly cues: infrared thermal anomalies, smoke, open flames, and blackened (ablated) areas. These four cues correspond to key visual features before ignition, in the early stages of ignition, and during the sustained combustion phase, respectively. The system processes the change-sensing visual features using four additional detection heads to generate infrared thermal anomaly response maps. Smoke response diagram Open flame response diagram and response map of blackened areas Infrared thermal anomaly response maps are typically derived from feature extraction in the infrared thermal imaging branch, while smoke, open flames, and blackening are derived from the visible light imaging branch. For a candidate fire region O, the average response value of the four response maps within that region is calculated: ; ; ; ; Then, a fire visual detection score is obtained through weighted summation and normalization. b1, b2, b3, and b4 are preset weight coefficients, corresponding to the contribution levels of thermal anomalies, smoke, open flames, and blackened areas, respectively. Since thermal anomalies are often the earliest signal of spontaneous combustion, and open flames are key evidence confirming a fire, b1 and b3 are usually assigned higher weights in practical applications. For example, in one implementation, b1 = 0.4, b2 = 0.2, b3 = 0.3, and b4 = 0.1. Smoke and blackening serve as auxiliary cues to enhance credibility.

[0044] In some embodiments, the scene semantic verification score includes a landslide relationship verification score and a fire relationship verification score. The calculation of the scene semantic verification score based on the semantic region where the candidate targets are located and the spatial relationships between the candidate targets specifically includes: The landslide relationship verification score is calculated using the following formula: ; in, The semantic region matching score represents the candidate landslide target. The score indicates the consistency of the landslide direction. This represents the stability score of landslide candidate targets across consecutive time phases. r 1 , r 2 , r 3 Preset weighting coefficients; The fire relationship verification score is calculated using the following formula: ; in, This represents the semantic region matching score of the candidate fire target. The spatial consistency score represents the relationship between thermal anomalies and visible signs of combustion. This represents the stability score of the ignition candidate target across consecutive time phases. q 1 , q 2 , q 3 These are preset weighting coefficients.

[0045] Before performing the verification, the monitoring area of ​​the waste rock pile was first subjected to scene semantic analysis, dividing it into several semantic regions, including at least the main body area, slope area, slope toe area, slope top area, road area, drainage area, and background stable area. These regions can be obtained through manual annotation, GIS boundary import, UAV orthophoto segmentation, or 3D terrain-assisted analysis. Each region has a clear geographical meaning and risk correlation. For example, the slope area is the main area where landslides occur; the slope toe area is the accumulation area of ​​landslide material; the main body area is the area with the highest risk of spontaneous combustion; the road area and background stable area are generally not related to landslides or spontaneous combustion.

[0046] For landslide candidate targets, this application embodiment performs relationship verification from three dimensions: semantic region matching degree, landslide direction consistency, and temporal stability. The formula for calculating the landslide relationship verification score is as follows: ; in, This represents the semantic region matching score of landslide candidate targets. This score measures whether the semantic region where the candidate target is located matches the expected distribution of landslide events. For example, cracks, exposed surfaces, and landslide textures have higher matching scores when located in slope areas; newly added deposits at the toe of the slope have higher matching scores when located in the toe area; and lower matching scores if the above anomalies appear in road areas, drainage areas, or background stable areas. This score represents the consistency score of the landslide direction. It measures whether there is spatial consistency between the upper anomalous area (cracks, exposed areas, etc.) and the lower deposited area along the slope's sliding direction. The calculation formula is as follows: ; in, C up The centroid (two-dimensional or three-dimensional coordinates) of the upper anomalous region; C toe The centroid of the depositional area at the toe of the slope; v g It is the unit vector in the direction of slope sliding, which can be calculated based on the digital elevation model of the waste rock pile; A directional deviation threshold (e.g., 0.5 meters or a cosine threshold for the directional angle) is used to tolerate a certain degree of offset; This is a scaling parameter that controls the sensitivity of the score; For activation functions, the Sigmoid function is typically chosen, mapping the calculation results to the (0, 1) interval. When ( C toe - C up )and v g When the directions are consistent and the projection length exceeds the threshold, the score is close to 1; when the directions are opposite or the projection length is very small, the score is close to 0. This represents the temporal stability score of landslide candidate targets. This score measures the mask intersection-over-union (IoU) stability of candidate targets across multiple consecutive inspections, and can be taken as the average IoU value of the K most recent inspections that successfully matched the current target. The temporal stability score can suppress false targets caused by short-term shadows, dust, or single-frame false detections. r 1 , r 2 , r 3 The preset weight coefficients correspond to the contributions of semantic matching, directional consistency, and temporal stability, respectively.

[0047] For ignition candidate targets, this application embodiment also performs relationship verification from three dimensions: semantic region matching degree, spatial consistency between thermal anomalies and visible combustion signs, and temporal stability. The formula for calculating the ignition relationship verification score is as follows: ; in, This represents the semantic region matching score of the candidate target for fire. Fire events typically occur in the main body of the reactor structure, and sometimes in the slope area (if internal oxidation causes temperature rise). Road areas, drainage areas, and areas with stable backgrounds have a lower probability of ignition. The system calculates this score based on the overlap between the candidate target and each semantic region, combined with a preset matching weight table (e.g., 1.0 for the main body of the reactor structure, 0.6 for the slope area, and 0.0 or lower for other areas). This score represents the spatial consistency between thermal anomalies and visible signs of combustion. It measures the degree of spatial overlap or proximity between areas of infrared thermal anomalies and areas of smoke, open flame, or blackening. The calculation formula is as follows: ; in, A T Indicates infrared thermal anomaly regions (extracted from infrared thermal imaging response maps by threshold segmentation); A smk Indicates the smoke area (extracted from the visible light smoke response map); A flm Indicates the area with open flame; A blk ∪ represents the blackened (ablation) region; ∪ represents the union operation; ∩ represents the intersection operation; Area (·) Calculate the pixel area or geographic area of ​​the region; To prevent false alarms from being zero, the score ranges from [0, 1]. If the thermal anomaly area is completely covered by visible signs of combustion, the score is close to 1; if the thermal anomaly area is far from any visible smoke, open flame, or blackened area, the score is close to 0. This check can effectively eliminate false alarms caused by non-combustion heat sources such as abandoned equipment, vehicle engines, and sunlight reflection. This represents the temporal stability score of a candidate target on fire. Similar to landslide, this score measures the persistence and mask overlap of targets such as thermal anomalies and smoke across multiple consecutive inspections, and is used to suppress false detections of instantaneous heat sources (such as passing vehicles) or single-frame smoke.

[0048] In some embodiments, mapping the landslide candidate target to the lidar point cloud space and calculating the three-dimensional deformation verification score using point cloud elevation changes specifically includes: Elevation changes are calculated using the following formula: ; in, This represents the current elevation model generated after registering the current time-phase lidar point cloud with the reference time-phase lidar point cloud. Represents the benchmark elevation model; The threshold for significant change is calculated using the following formula: ; in, This represents the quantile corresponding to the confidence level. This represents the standard deviation of the benchmark point cloud measurement error. This represents the standard deviation of the current point cloud measurement error. This represents the standard deviation of the point cloud registration error; The slope loss volume is calculated using the following formula: ; in, R slope This represents the semantic region of the slope, where ΔA represents the area per unit grid. O j Indicates candidate landslide targets; The volume of material accumulated at the toe of the slope is calculated using the following formula: ; in, R toe Indicates the semantic region of the foot of the slope; The volume consistency score is calculated using the following formula: ; in, To prevent small quantities from being reduced to zero; The three-dimensional deformation verification score is calculated using the following formula: ; in, c 1 , c 2 , c 3 These are preset weighting coefficients.

[0049] The lidar point cloud L acquired in the current time phase t Register the point cloud with the reference time-phase lidar point cloud L0. Registration can be performed using iterative nearest neighbor (ICP) or feature-based registration methods to align the two point clouds in the same geographic coordinate system. After registration, generate digital elevation models (DEMs) for the current time phase. Digital elevation model of reference time phase ,in( x , y The region () represents the geographic coordinates of a regular grid, with each grid corresponding to a unit area ΔA (e.g., 1m × 1m). Then, the elevation change is calculated grid by grid: ; when When, it indicates that the ground surface at that location has risen (possibly due to deposition); when When this occurs, it indicates that the surface has lowered (possibly due to erosion or slippage loss).

[0050] Since both lidar measurement and point cloud registration have errors, to avoid minor elevation fluctuations being misjudged as landslide deformation, this application embodiment sets a threshold for statistically significant changes. LoD : ; in, This represents the quantile corresponding to the confidence level. This represents the standard deviation of the benchmark point cloud measurement error, with the same unit as elevation (e.g., meters), and is usually given by the nominal accuracy of the lidar equipment. This represents the standard deviation of the current point cloud measurement error, which also originates from the equipment's nominal accuracy. This represents the standard deviation of point cloud registration error, which can be obtained through statistical analysis of the residuals between corresponding points after registration. It is only when the absolute value of the elevation change exceeds... LoD Only when the mesh is significantly deformed is it considered to have undergone significant deformation; otherwise, it is considered noise or registration residual and is not included in subsequent volume calculations.

[0051] For a certain landslide candidate target O j (The area typically retained after visual detection and semantic verification) The system compares it with the semantic region of the slope. R slope Calculate the slope deficit volume within the intersection of the semantic region at the slope toe. R toe Calculate the accumulation volume at the toe of the slope within the intersection. This represents a grid that is simultaneously located within both the slope semantic region and the landslide candidate target. This represents the overlapping mesh located within the semantic region at the slope toe and inside the candidate target. Theoretically, for closed or semi-closed slope systems, the material deficit on the slope surface should be approximately equal to the material deposited at the slope toe (ignoring loss or compression). This embodiment utilizes this physical constraint to calculate the volume consistency score.

[0052] This represents the absolute value of the difference between the deficit volume and the accumulated volume. This represents the total volume change.

[0053] The exponential function exp() maps the score to the interval (0, 1). A score of 1 is given when the loss equals the accumulation volume; the greater the difference between the two, the closer the score approaches 0. This score reflects whether the three-dimensional deformation conforms to the physical law of material conservation in landslides, and can further distinguish between genuine landslides (which usually have high consistency) and elevation changes caused by other reasons (such as local collapse without corresponding accumulation, or soil piling without corresponding loss). The three indicators of loss volume, accumulation volume, and volume consistency are weighted and fused to obtain the lidar three-dimensional deformation verification score.

[0054] In some embodiments, calculating the gas anomaly verification score by calling gas sensor data within the same spatiotemporal neighborhood includes: The gas smoothing concentration is calculated using the following formula: ; in, Indicates the current time t gas smoothing concentration, This indicates the smoothed concentration of the gas at the previous moment. This indicates the gas concentration at the current moment. As a smoothing factor, q Indicates the type of gas. q ∈{CO, CH4, SO2, H2S, O2}; Gas statistical outliers are calculated using the following formula: ; in, This represents the historical average concentration of gas type q. Indicates the standard deviation of historical concentrations. Indicates prevention of zero small quantities; The rate of increase in gas concentration is calculated using the following formula: ; Where △t represents the time interval between two consecutive inspections; The gas anomaly review score is calculated using the following formula: ; in, Indicates the weight of the gas category. Indicates the statistical outlier threshold. Indicates the weight of the upward trend. This represents the rate of increase threshold.

[0055] During drone inspections, onboard gas sensors can detect the concentrations of various gases associated with spontaneous combustion of coal gangue piles in real time. The gas types discussed in this application's embodiments are the set of gases addressed. q ∈{CO, CH4, SO2, H2S, O2}. The current inspection time is... t The type of gas output by the sensor q The original concentration was Because gas sensors are affected by factors such as airflow disturbances and sensor noise, instantaneous readings may fluctuate. To obtain stable concentration estimates, embodiments of this application employ exponential moving averages to smooth the concentration sequence. It is a smoothing factor, with a value range of (0, 1). The larger the value, the higher the weight of the current measurement, and the faster the response. The smaller the value, the stronger the smoothing effect and the better the noise reduction. This is the smoothed concentration value from the previous inspection time (or the previous time step). For the first inspection (t=0), we can let... Alternatively, historical averages can be used. Exponential moving averages can suppress transient spike noise while preserving trend information about concentration changes.

[0056] To determine whether the current gas concentration is significantly higher than historical normal levels, this invention calculates and statistically analyzes outliers. Z q This value compares the current smoothed concentration with the historical baseline (mean). and standard deviation Compare them. When Z q Greater than the preset threshold At that time, the gas was considered to have statistical anomalies, possibly indicating spontaneous combustion oxidation activity. A single instance of elevated concentration might be a temporary disturbance (such as a small amount of smoke blown in by the wind), while a sustained upward trend better reflects the development of spontaneous combustion. Therefore, this application's embodiments calculate the rate of increase in gas concentration. .when Exceeding the preset rise rate threshold (For example, when the rate of increase in CO is >1 ppm / day), it indicates that the gas concentration is continuously deteriorating and the risk score should be increased.

[0057] For any candidate target Record its state in consecutive time phases: ; in, This represents the candidate target mask at time t. Indicates the type of candidate target. Indicates the center location of the target. Indicates the target area. Indicates the confidence level of the target.

[0058] The stability of the candidate target in continuous time phases is: ; in, Indicates the cross-union ratio between candidate target masks. The time weight is represented by K, and the time window length is represented by K. This score is used to suppress transient interferences such as short-term shadows, short-term dust, single-frame reflections, and vehicle heat sources. The higher the score, the more stable the shape and position of the candidate target is across multiple inspections, and the more likely it is a genuine, persistent anomaly rather than an occasional disturbance.

[0059] The expansion rate of the candidate target area is: ; Indicates the target area at the current time phase. This indicates the area of ​​the same target during the last inspection. When the exposed area on the slope, the accumulation area at the foot of the slope, the thermal anomaly area, or the blackened area continues to expand, the system increases the risk level of the corresponding event.

[0060] Evidence for the evolution of landslide events is presented as follows: ; in, Indicates the first t The average response score of freshly exposed gangue during the next inspection (corresponding to) ), Indicates the first t The average crack response score during the second inspection (corresponding to) ), Indicates the first t The average response score of the newly added accumulation at the toe of the slope during the next inspection (corresponding to) ), Indicates the first t The average value of the slip texture response score during the second inspection (corresponding to) ), Indicates the first t The laser radar 3D deformation verification score during the second inspection (corresponding to) ).

[0061] set up: ; It can be If any one of the following is true, then the landslide event evolution score is: ; ; This represents the incremental change in the fresh exposure response score (positive values ​​are taken, and only the direction of deterioration is considered). Similarly, the score increments correspond to cracks, slope toe deposits, and landslide textures, respectively; ; This represents the increment of the lidar review score; Indicates the consistency score of landslide direction; u 1 ~ u 6 The pre-set weighting coefficients reflect the importance of each evolutionary factor in the deterioration of landslide risk. This step is used to determine whether cracks, exposure, slope toe accumulation, landslide texture, and three-dimensional deformation increase over time, and whether they conform to the landslide evolution direction. The higher the score, the more the visual signs and three-dimensional deformation evidence of the landslide continuously increase over time, conforming to the sliding direction pattern, and the risk is rapidly increasing.

[0062] Evidence of the evolution of fire events is presented as follows: ; in, Indicates the first t The average response score of infrared thermal anomalies during the second inspection (corresponding to) ), Indicates the area expansion rate of the thermal anomaly region. Indicates the first t The average smoke response score during each inspection (corresponding to) ), Indicates the first t The average response score of the blackened (ablation) area during the next inspection (corresponding to) ), Indicates the first t The gas anomaly review score during the second inspection (corresponding to) ).

[0063] set up: ; It can be If any one of the following is true, then the fire event evolution score is: ; This step is used to determine whether thermal anomalies, smoke, blackening, and gas anomalies are continuously increasing, thereby improving the reliability of early spontaneous combustion and fire warnings.

[0064] For any candidate target The final target confidence level is calculated based on visual detection, semantic matching, relationship verification, sensor verification, time stability, and event evolution trend. ; in, Indicates the visual difference perception detection score (i.e. or ), Represents the semantic matching score (i.e. or ), Indicates the scene relationship verification score (i.e. or ), Indicates the sensor's targeted verification score (in the case of a landslide). When it catches fire ), This represents the time stability score. Indicates the event evolution score (for landslides). When it catches fire ). w 1 ~ w 6 The preset weighting coefficients reflect the contribution of each dimension to the final confidence level.

[0065] The landslide event risk score is: ; in, Indicates the visual score of the landslide. This indicates the score for the landslide relationship assessment. This indicates the score for lidar review. This represents the stability score of a landslide candidate over continuous time phases. This indicates the score for the evolution of the landslide event. This indicates the preset weighting coefficient.

[0066] The fire risk score is: ; in, Indicates the visual score for the fire. This indicates the score for the fire relationship verification. This indicates the score for the gas sensor review. This represents the stability score of the fire candidate across consecutive time phases. This indicates the score for the evolution of the fire event. This indicates the preset weighting coefficient.

[0067] The overall risk score is: ; The risk level is: ; Among them, I represents major risk, II represents significant risk, III represents general risk, and Normal represents no obvious abnormalities. T 1 , T 2 , T 3 It can be set based on historical data and risk acceptance criteria.

[0068] The above describes the method for identifying and warning of abnormal changes in coal mine gangue piles in the embodiments of this application. The following describes in detail the computer system in the embodiments of this application, in conjunction with the above method for identifying and warning of abnormal changes in coal mine gangue piles.

[0069] Please see Figure 2 This is a schematic diagram of an exemplary hardware structure of a computer system in an embodiment of this application.

[0070] In some embodiments, the computer system 200 includes a computer device, which may be a terminal device. The computer device includes a processor 201, a memory 202, a sensor module 203, a communication module 204, an input device 205, and an output device 206 connected via a system bus. The processor 201 of the computer device provides computing and control capabilities. The memory 202 of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database is used to store data.

[0071] Those skilled in the art will understand that Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0072] In some embodiments of this application, a computer-readable storage medium is provided, including instructions that, when executed on the computer system 200, cause the computer system 200 to perform the coal mine gangue pile abnormal change identification and early warning method of the embodiments of this application.

[0073] In some embodiments of this application, a computer program product is also provided, which, when run on a computer system 200, causes the computer system 200 to execute the coal mine gangue pile abnormal change identification and early warning method in the embodiments of this application.

[0074] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0075] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0076] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for identifying and issuing early warnings of abnormal changes in coal mine gangue piles, characterized in that, include: The baseline remote sensing inspection data and the current remote sensing inspection data of the coal mine gangue pile monitoring area are obtained. The current remote sensing inspection data includes visible light images and infrared thermal images. The corresponding images of the current remote sensing inspection data and the baseline remote sensing inspection data are spatially registered to obtain the spatially aligned baseline time phase and current time phase. Multi-level visual features of the reference time phase and the current time phase are extracted respectively. For the same level, initial difference features are constructed based on the reference time phase features of the reference time phase, the current time phase features of the current time phase, directional difference features, and absolute difference features. The directional difference features and the absolute difference features are the difference features between the reference time phase features and the current time phase features. A cross-attention mechanism is used to inject the initial difference features into the current temporal features to obtain change-aware visual features. Based on the change-aware visual features, candidate targets are determined, and the visual detection scores of the candidate targets are determined. The candidate targets include landslide candidate targets and / or fire candidate targets. Calculate the scene semantic verification score based on the semantic region where the candidate targets are located and the spatial relationship between the candidate targets; The landslide candidate targets are mapped to the lidar point cloud space, and the three-dimensional deformation verification score is calculated using the point cloud elevation change. And / or, for the fire candidate targets, the gas anomaly verification score is calculated by calling the gas sensor data in the same spatiotemporal neighborhood. The event evolution score is calculated based on the mask crossover ratio, area change rate, and review score change trend of the candidate target in multiple consecutive inspections. The risk score is calculated by integrating the visual detection score, the scene semantic verification score, the three-dimensional deformation verification score or the gas anomaly verification score, and the event evolution score, and an early warning result is output based on the risk score.

2. The method according to claim 1, characterized in that, The construction of initial difference features based on the reference time phase characteristics of the reference time phase, the current time phase characteristics of the current time phase, the direction difference characteristics, and the absolute difference characteristics specifically includes: The initial differential features are constructed using the following formula: ； ； in, i The first level represents the multi-level visual features. i Each level The first phase of the reference time i Layer visual features, Indicates the first phase of the current time. i Layer visual features, Indicates directional differences. Indicates absolute differences. Cat (·) indicates a channel splicing operation. Represents a difference projection network. Indicates the first i Initial differences in the hierarchy.

3. The method according to claim 2, characterized in that, The step of injecting the initial differential features into the current temporal features using a cross-attention mechanism to obtain change-perceived visual features includes: The enhanced features are calculated using the following formula: ； in, Indicates the first i Enhancement features at each level, Indicates the first i Difference injection coefficients at each level Attn (·) represents the attention function, which is defined by the following formula: ； in, d The dimension of the feature vector. Q For querying the matrix, K The key matrix, V For value matrices, The following formula is used to perform multi-scale fusion calculation of the enhanced features at multiple levels to perceive changes in visual features: ； in, F chg Visual features representing the perception of change. Indicates the first phase of the current time. k A high-level semantic feature, k This is a preset high-level feature index. This represents the set of levels participating in the integration. Indicates the first i Adaptive weights at each level This indicates element-wise multiplication.

4. The method according to claim 1, characterized in that, The visual detection score includes a landslide visual detection score and a fire visual detection score, and the visual detection score for determining the candidate target includes: The visual detection score for landslides is calculated using the following formula: ； in, This represents the average response value of the response map of freshly exposed gangue within the candidate region. This represents the average response value of the crack response map within the candidate region. This represents the average response value of the newly added accumulation response map within the candidate region at the toe of the slope. This represents the average response value of the slump texture response map within the candidate region. a1, a2, a3, and a4 are preset weight coefficients, corresponding to the contribution degree of each response map. Norm(·) represents the normalization function. The fire visual detection score is calculated using the following formula: ； in, This represents the average response value of the infrared thermal anomaly response map within the candidate region. This represents the average response value of the smoke response map within the candidate region. This represents the average response value of the open flame response map within the candidate region. This represents the average response value of the blackened region response map within the candidate region. b1, b2, b3, and b4 are preset weight coefficients, corresponding to the contribution degree of each response map.

5. The method according to claim 1, characterized in that, The scene semantic verification score includes a landslide relationship verification score and a fire relationship verification score. The calculation of the scene semantic verification score based on the semantic region where the candidate targets are located and the spatial relationships between the candidate targets specifically includes: The landslide relationship verification score is calculated using the following formula: ； in, The semantic region matching score represents the candidate landslide target. The score indicates the consistency of the landslide direction. This represents the stability score of landslide candidate targets across consecutive time phases. r 1 , r 2 , r 3 Preset weighting coefficients; The fire relationship verification score is calculated using the following formula: ； in, This represents the semantic region matching score of the candidate fire target. The spatial consistency score represents the relationship between thermal anomalies and visible signs of combustion. This represents the stability score of the ignition candidate target across consecutive time phases. q 1 , q 2 , q 3 These are preset weighting coefficients.

6. The method according to claim 1, characterized in that, The process of mapping the landslide candidate targets to the lidar point cloud space and calculating the three-dimensional deformation verification score using point cloud elevation changes specifically includes: Elevation changes are calculated using the following formula: ； in, This represents the current elevation model generated after registering the current time-phase lidar point cloud with the reference time-phase lidar point cloud. Represents the benchmark elevation model; The threshold for significant change is calculated using the following formula: ； in, This represents the quantile corresponding to the confidence level. This represents the standard deviation of the benchmark point cloud measurement error. This represents the standard deviation of the current point cloud measurement error. This represents the standard deviation of the point cloud registration error; The slope loss volume is calculated using the following formula: ； in, R slope This represents the semantic region of the slope, where ΔA represents the area per unit grid. O j Indicates candidate landslide targets; The volume of material accumulated at the toe of the slope is calculated using the following formula: ； in, R toe Indicates the semantic region of the foot of the slope; The volume consistency score is calculated using the following formula: ； in, To prevent small quantities from being reduced to zero; The three-dimensional deformation verification score is calculated using the following formula: ； in, c 1 , c 2 , c 3 These are preset weighting coefficients.

7. The method according to claim 1, characterized in that, The calculation of the gas anomaly verification score by calling gas sensor data from the same spatiotemporal neighborhood includes: The gas smoothing concentration is calculated using the following formula: ； in, Indicates the current time t gas smoothing concentration, This indicates the smoothed concentration of the gas at the previous moment. This indicates the gas concentration at the current moment. As a smoothing factor, q Indicates the type of gas. q ∈{CO, CH4, SO2, H2S, O2}; Gas statistical outliers are calculated using the following formula: ； in, This represents the historical average concentration of gas type q. Indicates the standard deviation of historical concentrations. Indicates prevention of zero small quantities; The rate of increase in gas concentration is calculated using the following formula: ； Where △t represents the time interval between two consecutive inspections; The gas anomaly review score is calculated using the following formula: ； in, Indicates the weight of the gas category. Indicates the statistical outlier threshold. Indicates the weight of the upward trend. This represents the rate of increase threshold.

8. A computer system comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

Images