Mine data fusion and anomaly intelligent mining method, system and equipment and medium

Abstract

The invention discloses a mine data fusion and anomaly intelligent mining method, system and device and a medium, and belongs to the technical field of data processing.The method comprises the steps that multi-source heterogeneous data of a mine is collected, and space-time alignment processing is conducted on the multi-source heterogeneous data to obtain a standardized data set; based on the standardized data set, a mine three-dimensional digital twinborn body is constructed, and the digital twinborn body comprises a geometric layer representing a static structure, an equipment layer representing a dynamic equipment state and an environment layer representing an environment monitoring field; based on the digital twinborn body, constructing a multi-layer space-time diagram G = (V, ES, ET, EE); mining and analyzing the multi-layer space-time diagram by using a multi-layer space-time correlation model based on a hybrid intelligent algorithm cluster to obtain an abnormal score of each algorithm; and performing weighted fusion on the abnormal scores of the algorithms to generate a comprehensive abnormal score, and when the comprehensive abnormal score exceeds a preset threshold value, triggering an alarm and outputting a causal link and time-space distribution information of an abnormal event. According to the invention, intelligent mining of multi-source data fusion and spatial-temporal distribution rules is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data processing, and particularly relates to a mine data fusion and abnormal intelligent mining method, system, device and medium. BACKGROUND

[0002] At present, the mining industry is undergoing intelligent transformation with data driving as the core. In the production process, a large amount of multi-source heterogeneous data is generated, covering multi-dimensional information such as geology, equipment, environment, video, etc. However, due to the wide range of sources, different formats, and non-uniform time and space benchmarks, a serious "data island" phenomenon is formed, resulting in low efficiency of traditional data fusion methods, and the data utilization rate is generally less than 40%, which cannot provide high-quality data basis for mine safety production and efficient operation.

[0003] In the aspect of abnormal monitoring, the existing technology mainly relies on single-point alarm based on fixed threshold or traditional single machine learning algorithm. These methods are difficult to capture the dynamic evolution law and chain reaction of abnormality in time and space. Especially for the implicit risks such as slope slip and rock layer micro-deformation, there are inherent defects such as response lag, high false alarm and missed alarm rate, inability to locate the initial point of abnormality and trace the propagation path, etc., which makes the mine safety supervision always face the passive situation of "treating the disease" rather than "preventing the disease".

[0004] Therefore, an innovative solution capable of realizing multi-source data deep fusion and abnormal spatio-temporal distribution intelligent mining is urgently needed to solve the problems of low data fusion efficiency, low abnormal detection accuracy, and insufficient spatio-temporal distribution law mining in the existing technology. SUMMARY

[0005] To solve the above problems, the application provides a mine data fusion and abnormal intelligent mining method, system, device and medium, which realizes the fusion of multi-source data, automatic identification of abnormal events and intelligent mining of spatio-temporal distribution law.

[0006] To achieve the above purpose, the application provides the following technical solutions: In a first aspect, the application embodiment provides a mine data fusion and abnormal intelligent mining method, which comprises: Collecting multi-source heterogeneous data of the mine, and performing spatio-temporal alignment processing on the multi-source heterogeneous data to obtain a standardized data set; Based on the standardized data set, a three-dimensional digital twin of the mine is constructed; wherein the digital twin includes a geometry layer representing static structure, an equipment layer representing dynamic equipment state, and an environment layer representing environment monitoring field; Based on the digital twin, a multi-layer spatio-temporal graph G=(V,E S ,E T ,E E); wherein, V represents a node set, E S represents a spatial adjacency edge, E T represents a time evolution edge, E E represents an event trigger edge. The multi-layer spatio-temporal graph is mined and analyzed by using a multi-layer spatio-temporal correlation model based on a hybrid intelligent algorithm cluster to obtain abnormal scores corresponding to each algorithm. The abnormal scores of each algorithm are fused by weighting to generate a comprehensive abnormal score, and when the comprehensive abnormal score exceeds a preset threshold, an alarm is triggered and the causal link and spatio-temporal distribution information of an abnormal event are output.

[0007] In a second aspect, an embodiment of the present application provides a mine data fusion and abnormal intelligent mining system, the system comprising: A collection and standardization module is configured to collect multi-source heterogeneous data of a mine and perform spatio-temporal alignment processing on the multi-source heterogeneous data to obtain a standardized data set. A digital twin construction module is configured to construct a three-dimensional digital twin of the mine based on the standardized data set, wherein the digital twin comprises a geometry layer representing a static structure, a device layer representing a dynamic device state, and an environment layer representing an environment monitoring field. A spatio-temporal graph construction module is configured to construct a multi-layer spatio-temporal graph G=(V, E S , E T , E E ) based on the digital twin, wherein V represents a node set, E S represents a spatial adjacency edge, E T represents a time evolution edge, and E E represents an event trigger edge. A collaborative intelligent mining module is configured to mine and analyze the multi-layer spatio-temporal graph by using a multi-layer spatio-temporal correlation model based on a hybrid intelligent algorithm cluster to obtain abnormal scores corresponding to each algorithm. A fusion decision and visualization module is configured to fuse the abnormal scores of each algorithm by weighting to generate a comprehensive abnormal score, and when the comprehensive abnormal score exceeds a preset threshold, an alarm is triggered and the causal link and spatio-temporal distribution information of an abnormal event are output.

[0008] In a third aspect, an embodiment of the present application further provides an electronic device comprising a memory, a processor, and a computer program or instructions stored in the memory, wherein the processor executes the computer program or instructions to implement the aforementioned mine data fusion and abnormal intelligent mining method.

[0009] In a fourth aspect, an embodiment of the present application further provides a computer storage medium having a computer program or instructions stored therein, wherein the computer program or instructions are executed by a processor to implement the aforementioned mine data fusion and abnormal intelligent mining method.

[0010] In a fifth aspect, the embodiments of the present application further provide a computer program product comprising computer programs or instructions, which, when executed by a processor, implement the foregoing mine data fusion and abnormal intelligent mining method.

[0011] Compared with the prior art, the present application has the following advantages: 1. The present application can fully utilize the advantages of digital twin technology, realize efficient fusion and deep mining of mine multi-source heterogeneous data, accurately identify abnormal spatio-temporal distribution patterns, and provide strong support for mine production decision and safety management; 2. The present application realizes efficient fusion and unified spatio-temporal representation of mine multi-source heterogeneous data by constructing a high-fidelity digital twin and a multi-layer spatio-temporal graph model, which fundamentally solves the problem of data isolation and difficulty in linkage in traditional supervision; further relying on hybrid intelligent algorithms such as spatio-temporal graph neural network and event causal reasoning, the present application realizes accurate identification, early warning and propagation path tracing of mine abnormal events, significantly improves the active prevention and control ability of safety risks; finally, through simulation deduction and optimization decision driven by digital twin, the present application effectively improves the mining efficiency, reduces the operation cost and energy consumption, and comprehensively promotes the transformation and upgrading of mine production to intelligence, green and safety; the intelligent analysis results are intuitively presented through a visual interface, providing clear decision support for management personnel.

[0012] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and obtained by the structures indicated in the specification, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0014] Figure 1 A flowchart of a mine data fusion and abnormal intelligent mining method according to an embodiment of the present application is shown; Figure 2 A detailed flowchart of a mine data fusion and abnormal intelligent mining method according to an embodiment of the present application is shown; Figure 3 A training flowchart of a multi-layer spatio-temporal correlation model in an embodiment of the present application is shown; Figure 4A structural schematic diagram of a mine data fusion and abnormal intelligent mining system is shown. Figure 5 A structural schematic diagram of an electronic device is shown. DETAILED DESCRIPTION

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely explain the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0016] To solve the problems in the prior art, the embodiments of the present application disclose a mine data fusion and abnormal intelligent mining method, as shown in Figure 1 and Figure 2 , comprising the following steps. Step S1: collecting multi-source heterogeneous data of a mine, and performing spatio-temporal alignment processing on the multi-source heterogeneous data to obtain a standardized data set; Step S2: constructing a mine three-dimensional digital twin based on the standardized data set; wherein the digital twin comprises a geometry layer representing a static structure, a device layer representing a dynamic device state, and an environment layer representing an environment monitoring field; Step S3: constructing a multi-layer spatio-temporal graph G=(V, E S , E T , E E ) based on the digital twin; wherein V represents a node set, E S represents a spatial adjacency edge, E T represents a time evolution edge, and E E represents an event trigger edge; Step S4: performing mining analysis on the multi-layer spatio-temporal graph by using a multi-layer spatio-temporal correlation model based on a hybrid intelligent algorithm cluster to obtain abnormal scores corresponding to each algorithm; Step S5: performing weighted fusion on the abnormal scores of each algorithm to generate a comprehensive abnormal score, and when the comprehensive abnormal score exceeds a preset threshold, triggering an alarm and outputting a causal link and spatio-temporal distribution information of an abnormal event.

[0017] In some specific embodiments, step S1: collecting multi-source heterogeneous data of a mine, and performing spatio-temporal alignment processing on the multi-source heterogeneous data to obtain a standardized data set, comprises the following contents. (1) Data collection: The video data, sensor time series data, device log data, geographic map data and positioning data of the mine are obtained by data collection of the mine through cameras, various sensors, device logs, geographic maps and personnel / vehicle positioning systems.

[0018] Specifically, the data collection of the mine is performed through cameras, various sensors, device logs, geographic maps and personnel / vehicle positioning systems, and the camera frame sequence , the time series readings of the sensor set , the device log stream , the geographic map and three-dimensional terrain data , and the personnel / vehicle positioning stream , i.e. obtaining the multi-source heterogeneous data. (2) Metadata definition: Write uniform metadata for each piece of data in the multi-source heterogeneous data; wherein the metadata includes timestamp, device ID and local spatial coordinates, data type and quality index. (3) Time synchronization and space calibration:

[0019] In terms of time, the collection devices of the multi-source heterogeneous data are time-synchronized and corrected by NTP, GPS or RTK technology to realize the time index unification of the multi-source heterogeneous data and obtain the time-aligned multi-source heterogeneous data. Specifically, the clock of each collection device is corrected by network time protocol (NTP), global positioning system (GPS) or real-time kinematic (RTK), and the time deviation and time uncertainty are recorded to align the time of all data with uniform time index; wherein the time uncertainty is retained as a confidence factor during the alignment process for subsequent confidence calculation.

[0020] In space, based on the time-aligned multi-source heterogeneous data, the local coordinates of each data source are converted to the pre-defined digital twin global coordinate system by rigid transformation formula to generate twin global coordinates (i.e. converting the local coordinate system of each device to the pre-defined digital twin global coordinate system), and obtain the space-time aligned multi-source heterogeneous data; wherein the rigid transformation formula is:

[0021] pglobal=R·plocal+c, wherein pglobal represents the twin global coordinates of each data source, R represents the rotation matrix, plocal represents the local coordinates of each data source, and c represents the translation vector.

[0022] ​(4) Data cleaning and feature preprocessing: Lightweight front-end processing is performed on video data, and optical flow features, foreground masks, and target detection box information of video frame sequences are extracted at edge computing nodes; Denoising processing is performed on sensor time series data (such as using a low-pass filter algorithm to remove high-frequency noise), abnormal values are identified and removed based on statistical methods, and missing data is completed using linear interpolation; wherein the missing values can be marked and a missing mask is set to facilitate the multi-layer spatio-temporal correlation model to handle incomplete input data during training / inference; Natural language parsing and event extraction are performed on device log data, and unstructured log text is converted into a structured standardized event label sequence; Grid processing and spatial indexing are performed on geographic map data, and continuous geographic information is converted into discrete indexed grid cells; At a uniform time step t, the multi-source heterogeneous data after data cleaning and feature preprocessing is aggregated to generate a unified observation vector z(t), and the observation vector z(t) is labeled with data source and quality score to obtain a standardized data set; The unified observation vector z(t) includes a gridded environmental monitoring field, a device state vector, target detection information, a sparse event sequence, and positioning information; and the source and quality score corresponding to each unified observation vector are saved.

[0023] In some specific embodiments, step S2: based on the standardized data set, a three-dimensional digital twin of the mine is constructed, including the following contents: Based on the standardized data set, a three-layer virtual entity is built on the twin platform to form the digital twin.

[0024] The digital twin includes a geometry layer representing static structures, a device layer representing dynamic device states, and an environment layer representing environmental monitoring fields; The geometry layer uses three-dimensional grids and voxels to model the terrain and tunnels of the mine, forming a static structure skeleton; The device layer mounts device models on the geometry layer and dynamically updates the running parameter state of the device, achieving virtual-real mapping at the device level; The environment layer generates a dynamically updated environmental monitoring field by interpolating and gridding the collected sensor time series data, achieving continuous spatial distribution expression of environmental parameters; wherein the environmental monitoring field includes gas concentration field, temperature field, and displacement field monitoring field.

[0025] In the system corresponding to the method, the time series state of each virtual entity is maintained Meanwhile, the digital twin module provides an API interface for the system's analysis layer to obtain the historical and real-time status of any node or region, and saves the accuracy and update time of each transformation in the digital twin as the confidence weight when inputting the model; the digital twin module enables multi-source heterogeneous data to be verified and replayed in a unified spatial semantics.

[0026] In some specific embodiments, step S3: Based on the digital twin, construct a multi-layer spatiotemporal graph G=(V,E) S E T E E ), including the following: The key entities in the digital twin are mapped to a set of nodes V; wherein the key entities include sensors, device points, and interest grids; Based on the node set V, three types of edge relationships are defined to characterize the multi-dimensional associations between key entities; wherein, the three types of edge relationships include spatial adjacency edges E. S Time evolution edge E T and event triggering edge E E By defining spatial adjacent edges E S Define the temporal evolution edge E by connecting nodes that are physically adjacent. T Define event-triggered edges E to connect the states of the same node at different time steps. E To characterize the potential causal relationships between events; Based on the node set V and the three types of edge set E S E T E E Construct the multi-layer spatiotemporal graph G=(V,E) S E T E E ); where each node v carries an attribute vector at time t. This attribute vector It originates from the unified observation vector z(t) in step S1.

[0027] Among them, in constructing a multi-layer spatiotemporal graph G=(V,E) S E T E E When performing a function, a weight and uncertainty are stored for each edge. The weight can be initialized based on distance, connectivity history, or expert rules.

[0028] In some specific embodiments, step S4: The multi-layer spatiotemporal graph is mined and analyzed using a multi-layer spatiotemporal correlation model based on a hybrid intelligent algorithm cluster to obtain anomaly scores corresponding to each algorithm, including the following: The multi-layer spatio-temporal correlation model is constructed by a machine learning model, and the mixed intelligent algorithm cluster at least includes a spatio-temporal graph neural network (ST-GNN), a convolutional long short-term memory network (ConvLSTM), a multivariate Hawkes process (HawkesProcess), and a spatio-temporal clustering algorithm (ST-DBSCAN); the multi-layer spatio-temporal correlation model respectively adopts the spatio-temporal graph neural network, the convolutional long short-term memory network, the multivariate Hawkes process, and the spatio-temporal clustering algorithm to perform parallel analysis on the multi-layer spatio-temporal graph, and respectively outputs corresponding abnormal feature scores.

[0029] In some specific embodiments, the multi-layer spatio-temporal correlation model respectively adopts the spatio-temporal graph neural network, the convolutional long short-term memory network, the multivariate Hawkes process, and the spatio-temporal clustering algorithm to perform parallel analysis on the multi-layer spatio-temporal graph, and respectively outputs corresponding abnormal feature scores, including the following contents: The multi-layer spatio-temporal correlation model includes a spatio-temporal feature learning sub-module, a topological dependence learning sub-module, an event trigger reasoning sub-module, and an abnormal spatio-temporal clustering sub-module, which correspond to the spatio-temporal graph neural network, the convolutional long short-term memory network, the multivariate Hawkes process, and the spatio-temporal clustering algorithm, respectively.

[0030] (1) Topological dependence learning sub-module: input the multi-layer spatio-temporal graph G into the spatio-temporal graph neural network, learn the topological propagation law between nodes through graph convolution and time series modeling, and output a topological anomaly score s GNN (v, t).

[0031] Input the multi-layer spatio-temporal graph G=(V, E S , E T , E E ) into the spatio-temporal graph neural network, based on the adjacency matrix composed of the spatial adjacency edges E S , aggregate the spatial adjacency nodes and their feature information of each node through graph convolution operation, and generate node embedding representation representing spatial dependence relationship; Build the node embedding representation into a feature sequence along the time dimension and input it into the time series modeling layer, and the gated recurrent unit in the spatio-temporal graph neural network learns the dynamic evolution law of the feature sequence, so as to fuse the spatial dependence relationship and the time dynamic feature, and output the spatio-temporal state representation of each node; Through the full connection layer and the activation function in the spatio-temporal graph neural network, the spatio-temporal state representation is mapped to the topological anomaly score s GNN (v, t) of each node at each time point.

[0032] Wherein, the hierarchical update process formula of the graph convolution operation is: , In the formula, h is the node embedding representation, and represents the first output feature matrix of all nodes in the layer, which is the input of the layer; denotes the adjacency matrix after adding the self-loop, , denotes the original adjacency matrix in the multi-layer spatio-temporal graph G, which encodes the spatial connection relationship between nodes; denotes the unit matrix, which is used to realize the addition of the self-loop; denotes the degree matrix, denotes the negative second power of the degree matrix; is a node embedding representation, which is the input feature matrix of all nodes in the layer; denotes the input feature matrix of all nodes in the layer; denotes the trainable weight matrix of the

[0033] The application adopts a spatio-temporal graph neural network to learn the topology distribution and long-term interaction of a multi-layer graph; the output of the spatio-temporal graph neural network in this step can be used for subsequent construction of a causal chain. The topology anomaly score sGNN(v,t) in this step is an anomaly score representing long-term interaction and abnormal propagation mode between nodes, and the topology anomaly score sGNN(v,t) quantifies the degree of abnormality exhibited by the node due to its specific position and historical state context in the spatio-temporal topology network.

[0034] (2) Spatio-temporal feature learning submodule: for the rasterized environment monitoring field or video sub-region in the digital twin, a convolutional long short-term memory network is used to capture its local spatio-temporal evolution features, and output a local anomaly score s ConvLSTM (v,t).

[0035] Specifically, for the rasterized environment monitoring field or video sub-region in the digital twin, a convolutional long short-term memory network is used to model the local spatio-temporal evolution feature sequence, and through the gating mechanism of its input gate, forget gate and output gate, local spatio-temporal evolution features such as gas concentration diffusion and slope displacement evolution are captured, and a local anomaly score s ConvLSTM (v,t) is output.

[0036] That is, in the convolutional long short-term memory network, the local grid or frame block is taken as the input, the memory update is realized using the gating equation of ConvLSTM, and the local spatio-temporal evolution feature sequence is output; In this embodiment, the convolutional long short-term memory network outputs the local anomaly score and the local spatio-temporal evolution feature sequence for use by subsequent modules.

[0037] (3) Event-triggered reasoning submodule: based on the event-triggered edge E E, the causal link is generated, and an event anomaly score s Hawkes (v,t) is output.

[0038] Specifically, based on the event trigger edge E E in the multi-layer spatio-temporal graph, a multivariate Hawkes process is used to calculate the conditional intensity function of the historical abnormal event sequence on the current event occurrence, to quantify the trigger intensity and causal probability between events, and to generate a trigger matrix a ij representing the abnormal propagation path. Hawkes (v,t), which is the main basis for causal chain reasoning. Hawkes (v,t) is output.

[0039] wherein, for the trigger matrix a ij : i represents the type of the event, i.e. the type of the source event triggering other events; j represents the type of the target event, i.e. the type of the target event triggered by other events; the numerical value of the matrix element a ij represents the intensity of triggering the occurrence of an event of type j after the occurrence of an event of type i per unit time; the larger the value of a ij , the stronger the causal trigger relationship from event type i to event type j.

[0040] (4) Abnormal spatio-temporal clustering submodule: a spatio-temporal density clustering algorithm is used to identify abnormal spatio-temporal clusters from the abnormal event point set, and a clustering anomaly score s Cluster (v,t) is output.

[0041] Specifically, for the abnormal event point set, a spatio-temporal density clustering algorithm is used for density clustering in the spatial and temporal dimensions; the spatial radius eps s , the time window eps t and the minimum sample number MinPts are defined as clustering parameters (with the determination rule defined as: a point P is a core if and only if the number of neighbors within the spatial radius eps s and the time window eps t is greater than or equal to the minimum sample number MinPts), and the hot spot area and high incidence period of abnormal occurrence are automatically identified, and a clustering anomaly score s Cluster (v,t) based on the cluster density and distribution rule is output.

[0042] wherein, the abnormal event point set is an intermediate product in the entire collaborative intelligent mining process, and the abnormal event point set is the output of the pre-analysis algorithm in the execution process of the other three algorithm modules (i.e. ST-GNN, ConvLSTM and Hawkes Proces).

[0043] (5) Parallel scoring output: the spatio-temporal graph neural network, the convolutional long short-term memory network, the multivariate Hawkes process and the spatio-temporal clustering algorithm are independently calculated and output the topology anomaly score, the local anomaly score, the event anomaly score and the clustering anomaly score in parallel.

[0044] (6) Weighted fusion: The comprehensive anomaly score is generated by weighted fusion of the anomaly scores of each algorithm by the following formula: , In the formula, represents the comprehensive anomaly score, represents the topology anomaly score, s ConvLSTM (v, t) represents the local anomaly score, s Hawkes (v, t) represents the event anomaly score, represents the clustering anomaly score, , , and respectively represent the weight of the corresponding anomaly score, and satisfy w1+w2+w3+w4=1. The weights are calibrated and dynamically adjusted by historical performance indicators or online sliding windows; represents the spatio-temporal coordinates, and t represents a unified time index, represents the key entities at different levels in the digital twin.

[0045] wherein, for the spatio-temporal coordinates , represents the key entities at different levels in the digital twin, mainly including the following two types: Type one, node entity, corresponding to the ST-GNN and Hawkes process; The object referred to usually represents a node in the spatio-temporal graph G, which can be mapped to a physical device and a logical area; when calculating , v represents a certain specific device node, and the score measures the abnormal risk of the device due to its topological relationship in the network; Type two, spatial position / grid, corresponding to the ConvLSTM; The object referred to represents a grid unit or a geographic coordinate in the environment layer of the digital twin; when calculating s ConvLSTM (v, t), v may refer to a 1m x 1m grid on a map, and the score measures the local abnormality of the temperature field and the gas concentration field at the grid position.

[0046] trigger an alarm when S(v, t) exceeds a preset threshold T; generate a suspected causal link based on a trigger matrix of the multivariate Hawkes process and a propagation vector in the spatio-temporal graph neural network; concatenate the suspected causal link in chronological order to form a propagation path of the abnormal event, and output the abnormal spatio-temporal distribution in the form of a heat map in the digital twin interface, while presenting the cluster, link and suggested action in the form of a chart and a table.

[0047] write the alarm information of each time into a knowledge base, the alarm information including position, time, confidence decomposition, abnormal event, causal link, treatment scheme, related equipment and personnel, generate an operation and maintenance list based on the alarm information, the propagation path of the abnormal event and the like to support inspection, ventilation adjustment and equipment maintenance, as well as rule mining and template report generation for subsequent abnormal reasoning. The confidence decomposition shows the sharing of the comprehensive abnormal score by each sub-module.

[0048] based on the historical abnormal event cases of the knowledge base, optimize and iterate the model parameters of the multilayer spatio-temporal correlation model or the weights of the weighted fusion.

[0049] In some specific embodiments, the method further comprises training the multilayer spatio-temporal correlation model, such as Figure 3 as shown, comprising: The multilayer spatio-temporal correlation model adopts a phased training process in the training stage, first independently trains two sub-modules corresponding to ConvLSTM and ST-GNN respectively, then jointly fine-tunes four sub-modules, and finally estimates Hawkes parameters and performs end-to-end joint adjustment of the four sub-modules.

[0050] In the training process, the multilayer spatio-temporal correlation model is optimized by minimizing a multi-task loss function; The multi-task loss function is: , In the formula, denotes the multi-task loss function, denotes the node state prediction loss term, denotes the event sequence likelihood loss term, denotes the clustering auxiliary loss term, denotes the regularization term, , , and denote hyperparameters for balancing the importance of each task term, respectively.

[0051] Among them, the node state prediction loss term is calculated by the mean square error between the node state prediction value output by the spatiotemporal graph neural network ST-GNN and the actual observation value, thereby supervising the multi-layer spatiotemporal correlation model to learn accurate node-level state evolution laws. The event sequence likelihood loss term maximizes the likelihood function of the event sequence generated by the multivariate Hawkes process, thereby supervising the multi-layer spatiotemporal correlation model to accurately characterize the triggering intensity and causal dependency between events. The clustering auxiliary loss term, based on the results of the spatiotemporal clustering algorithm, guides the learning model to learn feature representations with good clustering properties by maximizing the similarity of samples within the same cluster and minimizing the similarity of samples between different clusters. Regularization terms prevent overfitting of the model and its learning model by imposing L1 or L2 norm penalties on the model parameters θ, thereby improving its generalization ability.

[0052] The training of the multi-layer spatiotemporal correlation model is completed when the entire model converges.

[0053] In some specific embodiments, the method adopts an edge-cloud collaborative computing architecture, including: edge computing nodes are responsible for front-end processing of the video data, lightweight inference and real-time alarm triggering; cloud servers are responsible for model training, full-graph inference, historical data playback and knowledge base management of the digital twin.

[0054] This application employs a distributed message bus (Kafka) to ensure data flow and designs a fault tolerance strategy to ensure that critical modules / steps are replaced by backup nodes when a single node fails.

[0055] In some specific embodiments, the method further includes backtesting using historical anomalous events from the knowledge base, with key metrics including recall, precision, F1 accuracy, AUC (Area Under the ROC Curve, i.e., the area under the ROC curve), and spatial localization error; it also verifies whether the triggering pathways learned by Hawkes are consistent with expert knowledge, with verification performed separately for various scenarios (open slopes, underground tunnels, transportation networks).

[0056] Based on the same inventive concept, embodiments of this application disclose a mining data fusion and intelligent anomaly mining system, such as... Figure 4 As shown, the system includes: The data acquisition and standardization module is used to collect multi-source heterogeneous data from the mine and perform spatiotemporal alignment processing on the multi-source heterogeneous data to obtain a standardized dataset. a digital twin construction module configured to construct a three-dimensional digital twin of the mine based on the standardized dataset, wherein the digital twin comprises a geometry layer representing static structures, a device layer representing dynamic device states, and an environment layer representing an environment monitoring field; a spatio-temporal graph construction module configured to construct a multi-layer spatio-temporal graph G=(V,E S ,E T ,E E ) based on the digital twin, wherein V represents a node set, E S represents a spatial adjacency edge, E T represents a temporal evolution edge, and E E represents an event trigger edge; a collaborative intelligent mining module configured to mine and analyze the multi-layer spatio-temporal graph using a multi-layer spatio-temporal correlation model based on a hybrid intelligent algorithm cluster to obtain abnormal scores corresponding to each algorithm; a fusion decision and visualization module configured to perform weighted fusion on the abnormal scores of each algorithm to generate a comprehensive abnormal score, and when the comprehensive abnormal score exceeds a preset threshold, trigger an alarm and output a causal link and spatio-temporal distribution information of an abnormal event.

[0057] In some specific embodiments, the system further comprises a knowledge base construction and iterative learning module; The knowledge base construction and iterative learning module is configured to construct and update a mine-specific knowledge graph, and store abnormal events, causal links, and disposal schemes of each alarm into the knowledge base to support rule mining and template report generation for subsequent abnormal reasoning. Based on historical abnormal events of the knowledge base, the model parameters of the multi-layer spatio-temporal correlation model or the weights of the weighted fusion are optimized and iterated.

[0058] As to the system in the above embodiments, the specific manner in which each unit module performs operations has been described in detail in the embodiments of the method, and will not be described in detail here.

[0059] Based on the same inventive concept, the embodiments of the present application also provide an electronic device, the structure of which is shown in Figure 5 the memory, the processor, and a computer program stored in the memory, wherein the processor executes the computer program or instructions to implement the aforementioned mine data fusion and abnormal intelligent mining method.

[0060] Based on the same inventive concept, the embodiments of the present application also provide a computer storage medium, which stores a computer program or instructions, and the computer program or instructions are executed by a processor to implement the aforementioned mine data fusion and abnormal intelligent mining method.

[0061] Based on the same inventive concept, the embodiment of the present application also provides a computer program product comprising computer programs or instructions which, when executed by a processor, implement the aforementioned mine data fusion and abnormal intelligent mining method.

[0062] The method of the present embodiment is verified by the following content: 1. Multi-source data acquisition and preprocessing verification: (1) Data source verification: The data comes from various devices in the mine environment (such as sensors, drones, Internet of Things devices, etc.), covering device operating status, ore quality, geological data, environmental parameters (temperature and humidity, gas concentration, etc.), and other multi-dimensional information.

[0063] Verification method: Check the calibration records of data acquisition equipment, sensor accuracy (such as within ±0.5%), drone aerial resolution (such as 0.1 meters / pixel), etc.

[0064] Abnormal detection: Identify noise and outliers in the data through statistical analysis (such as Z-score, IQR) or machine learning (such as Isolation Forest), and verify the integrity of the preprocessed data (such as missing rate <1%). Examples of mine multi-source data types and collection parameters are shown in Table 1: Table 1 Multi-source data types and collection parameters

[0065] 2. Multi-source data fusion verification: (1) Data alignment and synchronization: Verify the timestamp synchronization (such as error <10ms) and spatial coordinate alignment (such as GPS positioning error <1 meter) of multi-source data.

[0066] Verification method: Use time series alignment algorithms (such as Dynamic Time Warping, DTW) and spatial coordinate conversion algorithms (such as Kalman filtering) for verification.

[0067] Data fusion effect: Generate a unified mine digital twin model through multi-modal data fusion (such as Internet of Things device sensor data + drone image + geological model), and verify its consistency.

[0068] Index: The matching degree (such as similarity >90%) of the model after data fusion and the real mine environment.

[0069] 3. Precision verification of digital twin model (i.e. digital twin): (1) Model construction and updating: Verify the construction process of multi-scale digital twin model (macroscopic geological structure + microscopic device state) and the real-time updating ability of the model when the environment changes.

[0070] (2) Verification method: Static verification: Compare the predicted mine structure with actual exploration data (e.g., geological layer thickness error <5%).

[0071] Dynamic verification: Test the model's real-time response time (e.g., <1 second) and prediction accuracy (e.g., fault prediction accuracy >95%) by simulating mine equipment failures or environmental mutations (e.g., heavy rain).

[0072] Nonlinear dynamics verification: Verify the model's ability to solve macro-micro coupling problems by combining the complex dynamic behavior of the mine system (e.g., ore flow, equipment vibration).

[0073] 4. Intelligent mining of abnormal spatio-temporal distribution verification: (1) Abnormal detection algorithm performance: Verify the effectiveness of abnormal mining algorithms based on machine learning (e.g., LSTM, Transformer) or deep learning (e.g., convolutional neural network CNN).

[0074] (2) Verification method: Offline verification: Train the model using historical data sets (e.g., mine accident records over the past 3 years) and test its recall rate (Recall), precision (Precision), and F1 score (e.g., F1>0.9).

[0075] Online verification: Deploy the model in a real mine environment and monitor its real-time detection capability for sudden abnormalities (e.g., equipment overheating, landslides).

[0076] (3) Spatio-temporal distribution analysis: Verify the spatio-temporal correlation of abnormal events (e.g., whether the same area equipment failure is related to geological changes) and generate a visual report.

[0077] Index: Accuracy of spatio-temporal correlation analysis (e.g., correlation rule support >80%).

[0078] The present application solves the problem of data format inconsistency and difficulty in correlation under traditional methods by unified spatio-temporal benchmarking and digital twin modeling, deeply fusing mine geology, equipment, environment, video, and other multi-source heterogeneous data in a unified three-dimensional spatial semantic, and improving data utilization from less than 40% in traditional methods to more than 90%, providing a solid and complete data foundation for advanced analysis. The present application compresses the fusion processing time of multi-source data from hours to minutes through efficient spatio-temporal alignment algorithm and edge-cloud collaborative computing architecture, and improves the data processing speed by 300%. This enables the value of data to be mined in real time or near real time, greatly improving the timeliness of supervision and decision-making.

[0079] The application can accurately capture implicit and slowly changing abnormalities (such as rock micro-deformation and slow gas leakage) through collaborative mining based on a hybrid intelligent algorithm cluster (ST-GNN, ConvLSTM, Hawkes Process, ST-DBSCAN, etc.). The accuracy of anomaly detection is significantly improved to more than 92%, and the false positive rate is less than 8%, which is much higher than the traditional method (≤75%). The application can intuitively display the spatio-temporal distribution and evolution process of the anomaly in the form of a heat map and a causal link diagram in the system digital twin interface, and the response time of the management personnel decision-making is shortened by 50%, which greatly improves the management efficiency and emergency handling capacity. The analysis results are presented through a visual interface, making complex spatio-temporal data easy to understand.

[0080] The application guarantees the continuous operation reliability of 99% by designing a fault-tolerant strategy to ensure that the key modules / steps are replaced by backup nodes when a single node fails. At the same time, by building a knowledge base and continuously learning from historical cases, the model parameters of the multi-layer spatio-temporal correlation model can be continuously optimized, and the long-term self-adaptive evolution ability is achieved. In summary, the application can help mine enterprises to reduce operating costs by 15%-20%, reduce carbon emissions by 10%-15%, and significantly protect personnel safety, which meets the requirements of green and intelligent mining development and provides strong technical support for sustainable and high-quality development of mines.

[0081] In summary, the application builds an intelligent closed loop of "perception-fusion-mining-decision", which not only surpasses the existing technology in technical indicators, but also brings substantial and significant value in safety, efficiency and economic benefits.

[0082] Although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions described in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. A mine data fusion and anomaly intelligent mining method, characterized in that, The method comprises: Collecting multi-source heterogeneous data of a mine, and performing spatio-temporal alignment processing on the multi-source heterogeneous data to obtain a standardized data set; Based on the standardized data set, a three-dimensional digital twin of the mine is constructed; wherein the digital twin comprises a geometric layer representing static structures, a device layer representing dynamic device states, and an environment layer representing environmental monitoring fields; Based on the digital twin, a multi-layer space-time graph G=(V,E S ,E T ,E E ) is constructed; wherein V represents a node set, E S represents a space adjacency edge, E T represents a time evolution edge, and E E represents an event trigger edge; Using a multi-layer spatio-temporal correlation model based on a hybrid intelligent algorithm cluster to mine and analyze the multi-layer spatio-temporal graph, and obtaining abnormal scores corresponding to each algorithm; The abnormal scores of each algorithm are weighted and fused to generate a comprehensive abnormal score. When the comprehensive abnormal score exceeds a preset threshold, an alarm is triggered and the causal link and spatio-temporal distribution information of the abnormal event are output.

2. The mine data fusion and anomaly intelligent mining method according to claim 1, characterized in that, The collection of multi-source heterogeneous data of a mine and the spatio-temporal alignment processing of the multi-source heterogeneous data to obtain a standardized data set comprises: Collecting the multi-source heterogeneous data and writing uniform metadata for each data; wherein the multi-source heterogeneous data includes video data, sensor time series data, device log data, geographic map data, and positioning data of the mine, and the metadata includes timestamp, device ID and local spatial coordinates, data type and quality indicators; Using NTP, GPS or RTK technology to perform time synchronization correction on the collection devices of the multi-source heterogeneous data to realize the time index unification of the multi-source heterogeneous data, and obtaining the multi-source heterogeneous data after time alignment; Based on the multi-source heterogeneous data after time alignment, the local coordinates of each data source are converted into twin global coordinates through a rigid body transformation formula, and the multi-source heterogeneous data after spatio-temporal alignment is obtained.

3. The mine data fusion and anomaly intelligent mining method according to claim 2, characterized in that, After completing the spatio-temporal alignment processing and before generating the standardized data, the method further comprises data cleaning and feature preprocessing of the multi-source heterogeneous data, comprising: Performing lightweight front-end processing on the video data, and extracting the optical flow features, foreground mask, and target detection box information of the video frame sequence on the edge computing node; Performing denoising processing and outlier rejection processing on the sensor time series data, and using linear interpolation to complete the missing data; Performing natural language analysis and event extraction on the device log data, and converting the unstructured log text into a structured standardized event label sequence; Performing rasterization processing on the geographic map data and establishing a spatial index to convert continuous geographic information into discrete indexed grid cells; Under a uniform time step t, the multi-source heterogeneous data after data cleaning and feature preprocessing is aggregated to generate a unified observation vector z(t), and the observation vector z(t) is labeled with data source and quality score to obtain a standardized data set; Wherein, the unified observation vector z(t) includes rasterized environmental monitoring fields, device state vectors, target detection information, sparse event sequences, and positioning information.

4. The mine data fusion and anomaly intelligent mining method according to claim 1, characterized in that, The construction of the three-dimensional digital twin of the mine based on the standardized data set comprises: Based on the standardized data set, a three-layer virtual entity is built on the twin platform to form the digital twin; Wherein, the geometric layer uses three-dimensional grids and voxels to model the terrain and roadway of the mine to form a static structure skeleton; The device layer mounts a device model on the geometric layer and dynamically updates the running parameter state of the device, thereby realizing virtual-real mapping at the device level. The environment layer generates a dynamically updated environment monitoring field by interpolating and rasterizing the collected sensor time series data, thereby realizing continuous spatial distribution expression of the environment parameters; wherein the environment monitoring field includes a gas concentration field, a temperature field, and a displacement field monitoring field.

5. The mine data fusion and anomaly intelligent mining method according to claim 1, characterized in that, constructing, based on the digital twin, a multi-layer spatiotemporal graph G=(V,E S ,E T ,E E ) comprising: Key entities in the digital twin are mapped to a node set V; wherein the key entities include sensors, device points, and interest grids; Define spatial adjacency edges E S Define temporal evolution edges E T Define event-triggered edges E E To represent potential causal relationships between events; based on the node set V and the three-class edge set E S , E T , E E , construct the multi-layer spatio-temporal graph G=(V, E S , E T , E E ); wherein each node carries an attribute vector h v (t) at time t, and the attribute vector h v (t) is derived from a unified observation vector z(t).

6. The mine data fusion and anomaly intelligent mining method according to claim 1, characterized in that, The multi-layer spatio-temporal correlation model based on the hybrid intelligent algorithm cluster is used to mine and analyze the multi-layer spatio-temporal graph, and abnormal scores corresponding to each algorithm are obtained, including: The hybrid intelligent algorithm cluster at least includes a spatio-temporal graph neural network ST-GNN, a convolutional long short-term memory network ConvLSTM, a multivariate Hawkes process, and a spatio-temporal clustering algorithm ST-DBSCAN; The multi-layer spatio-temporal correlation model respectively uses a spatio-temporal graph neural network, a convolutional long short-term memory network, a multivariate Hawkes process, and a spatio-temporal clustering algorithm to perform parallel analysis on the multi-layer spatio-temporal graph, and respectively outputs corresponding abnormal feature scores.

7. The mine data fusion and anomaly intelligent mining method according to claim 6, characterized in that, The multi-layer spatio-temporal correlation model respectively uses a spatio-temporal graph neural network, a convolutional long short-term memory network, a multivariate Hawkes process, and a spatio-temporal clustering algorithm to perform parallel analysis on the multi-layer spatio-temporal graph, and respectively outputs corresponding abnormal feature scores, including: inputting the multi-layer spatio-temporal graph G into a spatio-temporal graph neural network, learning the topological propagation law between nodes through graph convolution and time series modeling, and outputting a topological anomaly score s GNN (v,t); For the gridded environment monitoring field or video sub-area in the digital twin, a convolutional long short-term memory network is adopted to capture its local spatio-temporal evolution characteristics, and a local anomaly score s is output ConvLSTM (v,t); based on the event trigger edges E in the multi-layer spatio-temporal graph E , the causal link is generated by quantifying the causal trigger strength between events using a multivariate Hawkes process, and the event anomaly score s is output Hawkes (v, t); The abnormal event point set adopts a spatiotemporal density clustering algorithm to identify an abnormal spatiotemporal cluster, and outputs a clustering abnormality score s Cluster (v,t); The spatio-temporal graph neural network, the convolutional long short-term memory network, the multivariate Hawkes process, and the spatio-temporal clustering algorithm are independently calculated in parallel, and output the topological abnormal score, the local abnormal score, the event abnormal score, and the clustering abnormal score.

8. The mine data fusion and anomaly intelligent mining method according to claim 7, characterized in that, The inputting the multi-layer spatiotemporal graph G into a spatiotemporal graph neural network learns the topological propagation law between nodes through graph convolution and time series modeling, and outputs a topological anomaly score s GNN (v,t), comprising: inputting the multi-layer spatio-temporal graph G=(V, E S ,E T ,E E ) into a spatio-temporal graph neural network, aggregating spatial adjacent nodes and their feature information of each node through a graph convolution operation based on an adjacent matrix composed of the spatial adjacent edges E S , to generate a node embedding representation representing a spatial dependency relationship; The node embedding representation is constructed as a feature sequence along the time dimension and input to a time series modeling layer, and a gated recurrent unit in the spatio-temporal graph neural network learns the dynamic evolution law of the feature sequence, thereby fusing spatial dependency and temporal dynamic features, and outputting a spatio-temporal state representation of each node; The spatio-temporal state representation is mapped to a topology anomaly score s for each node at each time instant by a fully connected layer and an activation function in the spatio-temporal graph neural network GNN ( ,t).

9. The mine data fusion and anomaly intelligent mining method according to claim 1 or 7, characterized in that, The abnormal scores of each algorithm are weighted and fused to generate a comprehensive abnormal score, and when the comprehensive abnormal score exceeds a preset threshold, an alarm is triggered and the causal link and spatio-temporal distribution information of the abnormal event are output, including: The abnormal scores of each algorithm are weighted and fused to generate a comprehensive abnormal score as follows: ， wherein, denotes a comprehensive anomaly score, denotes a topological anomaly score, s ConvLSTM (v,t) denotes a local anomaly score, s Hawkes (v,t) denotes an event anomaly score, denotes a clustering anomaly score, , , and denote weights for the corresponding anomaly scores, denotes a spatiotemporal coordinate, t denotes a unified time index, denotes key entities at different levels in the digital twin; When S(v, t) exceeds a preset threshold T, an alarm is triggered; a causal link is generated based on the multivariate Hawkes process, the causal link is connected to form a propagation path of the abnormal event, and the abnormal spatio-temporal distribution is output in the form of a heat map on the digital twin interface.

10. The mine data fusion and anomaly intelligent mining method according to claim 1, characterized in that, The method further includes: A mine-specific knowledge graph is constructed and updated, and each time the abnormal event, the causal link, the confidence degree decomposition, and the disposal scheme of the alarm are stored in the knowledge base to support rule mining and template report generation for subsequent abnormal reasoning; Based on the historical abnormal event cases of the knowledge base, the model parameters of the multi-layer spatio-temporal correlation model or the weights of the weighted fusion are optimized and iterated.

11. The mine data fusion and anomaly intelligent mining method according to claim 6, characterized in that, The method further includes training the multi-layer spatio-temporal correlation model; In the training process, the multi-layer spatio-temporal correlation model is optimized by minimizing a multi-task loss function; The multi-task loss function is: ， wherein, denotes a multi-task loss function, denotes a node state prediction loss term, denotes an event sequence likelihood loss term, denotes a clustering auxiliary loss term, denotes a regularization term, , , and denote hyperparameters for balancing the importance of each task term, respectively.

12. A mine data fusion and anomaly intelligent mining system, characterized in that, The system comprises: The acquisition and standardization module is configured to acquire multi-source heterogeneous data of the mine and perform spatio-temporal alignment processing on the multi-source heterogeneous data to obtain a standardized data set. The digital twin construction module is configured to construct a three-dimensional digital twin of the mine based on the standardized data set, wherein the digital twin comprises a geometry layer representing static structures, a device layer representing dynamic device states, and an environment layer representing an environment monitoring field. A space-time graph construction module is configured to construct a multi-layer space-time graph G=(V,E S ,E T ,E E ) based on the digital twin, wherein V represents a node set, E S represents a space adjacency edge, E T represents a time evolution edge, and E E represents an event trigger edge. The collaborative intelligent mining module is configured to mine and analyze the multi-layer spatio-temporal graph by using a multi-layer spatio-temporal correlation model based on a hybrid intelligent algorithm cluster to obtain abnormal scores corresponding to each algorithm. The fusion decision and visualization module is configured to perform weighted fusion on the abnormal scores of each algorithm to generate a comprehensive abnormal score, and when the comprehensive abnormal score exceeds a preset threshold, trigger an alarm and output a causal link and spatio-temporal distribution information of an abnormal event.

13. An electronic device, comprising: The computer readable storage medium stores computer programs or instructions, and the computer programs or instructions are executed by the processor to implement the mine data fusion and abnormal intelligent mining method according to any one of claims 1-7.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer programs or instructions, and the computer programs or instructions are executed by the processor to implement the mine data fusion and abnormal intelligent mining method according to any one of claims 1-7.

Images