Intelligent management method and system for coal mine data

Abstract

The embodiment of the invention discloses a coal mine data intelligent management method and system, and the method is applied to a coal mine data storage and calculation integrated center, and comprises the steps: obtaining at least one of the following original coal mine data: coal mine data transmitted by underground control transfer equipment in real time, and coal mine data stored by mobile storage equipment; coal mine data transmitted by the coal mine geological disaster hidden danger transparency service system; performing multi-process processing on the obtained original coal mine data, managing the original coal mine data and processing the coal mine data; and the managed coal mine data is sent to a coal mine geological disaster hidden danger transparent business system. The technical scheme provided by the embodiment of the invention can be applied to a coal mine intelligent and transparent geological support system and a scene with high requirements of coal mine data new-quality productivity on coal mine data storage and calculation, and unified management and efficient application of the coal mine data can be realized.

Description

Technical Field

[0001] The present invention relates to the field of geophysical exploration technology, and in particular to an intelligent management method and system for coal mine data. Background Art

[0002] The coal industry plays an indispensable role in my country's energy structure. With the increasing social requirements for environmental protection, resource utilization efficiency, and safe production in coal mines with few people or no one, the development of intelligent coal mine technology is particularly urgent and important. At present, the transparent detection of geological hazards in coal mines faces many problems, such as unclear geophysical multi-source and multi-field response mechanisms for the sources of geological hazards in coal mines, low detection accuracy and small range of existing mine geophysical instruments, slow response to geological information feedback, low level of intelligent data processing, difficulty in dynamic detection, untimely early warning response, insufficient mine geological information, low accuracy of geological modeling, and difficulty in updating static model attributes.

[0003] In order to meet these challenges and promote the intelligent transformation of the coal mining industry, a scientific solution for coal mine data collection, storage, processing, analysis and interpretation is urgently needed to achieve mine geological transparency and realize green and intelligent precise mining of coal. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and system for intelligent management of coal mine data to achieve efficient collection, storage and real-time analysis of coal mine geological information, improve the transparency of coal mine geological information, and enhance the monitoring and early warning capabilities of geological disasters.

[0005] In a first aspect, an embodiment of the present invention provides a method for intelligent management of coal mine data, which is applied to a coal mine data storage and computing integrated center, and the method includes:

[0006] Obtain at least one of the following original coal mine data: coal mine data transmitted in real time by underground control transfer equipment, coal mine data stored in mobile storage devices, and coal mine data transmitted by a coal mine geological disaster hidden danger transparency business system;

[0007] Multi-process processing of the acquired raw coal mine data, management of raw coal mine data and processing of coal mine data;

[0008] The managed coal mine data is sent to the coal mine geological disaster hazard transparency business system.

[0009] Furthermore, manage the original coal mine data and process the coal mine data, including:

[0010] The original coal mine data is stored in the data lake in the smart lake warehouse according to the degree of hotness and coldness: the original coal mine data belonging to the hot data category is stored in the distributed file system server and cache server composed of solid-state hard disks; the original coal mine data belonging to the cold data category is compressed and stored in the distributed file system server composed of hard disk drives;

[0011] Coal mine data is managed and processed through the data warehouse in the smart lake warehouse.

[0012] Furthermore, cold data and thermal data The classification is determined according to the following formula:

[0013]

[0014] in, The calculation formula is as follows:

[0015]

[0016] is the access frequency of data D at the current time t, W is the set time sliding window size, is the number of accesses to data D at time i;

[0017] The calculation formula is as follows:

[0018]

[0019] is the hot data threshold at the current time t, is the hot data threshold at the previous time t-1, is the set smoothing coefficient, is the importance weight of the set data D, and β is the set weight smoothing coefficient.

[0020] Further, according to the following formula, from a variety of compression strategies Select a compression strategy in , compress the original coal mine data belonging to the cold data category:

[0021]

[0022] in, is the selected compression strategy, Compression strategy The computational cost of Compression strategy The compression efficiency, Compression strategy The execution time of Compression strategy During execution, the current system resource usage is monitored in real time. is the weight coefficient.

[0023] Furthermore, the data warehouse in the intelligent lake warehouse is used to manage and process coal mine data, including:

[0024] In the data warehouse operation data storage layer, the operation data generated by preprocessing the original coal mine data is obtained and stored;

[0025] At the data warehouse detail layer, a data model generated based on the operation data and meeting the business requirements for transparency of geological hazards in coal mines is obtained and stored;

[0026] At the data warehouse service layer, receive and respond to data access requests initiated by external access interfaces based on data stored in the data warehouse;

[0027] In the analytical data storage layer of the data warehouse, analytical data generated by multi-process processing of the original coal mine data is obtained and stored.

[0028] Furthermore, the raw coal mine data obtained by multi-process processing include:

[0029] Analyze the acquired original coal mine data to obtain at least one of the following acceleration data, which is used to generate storage data of the data warehouse in the intelligent lake warehouse: coal mine data category, association relationship between coal mine data, sharing mechanism of coal mine data, and access rights of coal mine data;

[0030] Among them, the correlation between coal mine data is maintained in real time based on the dynamic update algorithm of correlation relationship. In this algorithm:

[0031] The update formula of the correlation strength between coal mine data is as follows:

[0032]

[0033] For data and The strength of the association at the current time t; For data and The strength of association at the next time t+1; For data and Similarity function of ; is the preset balance coefficient; For data and The rate of change at the current time t; is the influence coefficient of the preset change rate;

[0034] The update formula of the correlation threshold between coal mine data is as follows:

[0035]

[0036] For the current moment The threshold value of correlation judgment is For the previous moment The threshold value of correlation judgment is is a preset update coefficient;

[0037] when Exceed When the coal mine data and There is a correlation between them.

[0038] Furthermore, the raw coal mine data obtained by multi-process processing include:

[0039] Through the management node of the computing center, the computing nodes in the computing pool are scheduled to perform computing tasks in a load-balanced manner to process the acquired raw coal mine data and / or intermediate processed coal mine data that meet the set high computing power requirements;

[0040] Among them, the allocation function of the computing task is as follows:

[0041]

[0042] is the allocation time of the jth computing task on the i-th computing node, is the computation weight of the jth computation task, is the computing power of the ith computing node, is the load prediction value of the i-th computing node at the next time t+1;

[0043] The load balancing adjustment formula is as follows:

[0044]

[0045] is the load ratio of the i-th computing node at the current time t; is the computing capacity or load capacity of the i-th computing node at the current time t; is the computing capacity or load capacity of the i-th computing node at the next time t+1 after load balancing adjustment; is the maximum value of the load ratios of all computing nodes at the current time t.

[0046] Furthermore, after sending the managed coal mine data to the coal mine geological disaster hidden danger transparency business system, the method further includes:

[0047] After receiving the abnormal information feedback sent by the coal mine geological disaster hidden danger transparency business system, notify the underground terminal.

[0048] In a second aspect, an embodiment of the present invention provides an intelligent coal mine data management system, which is applied to the intelligent coal mine data management method described in the first aspect above, wherein the intelligent coal mine data management system includes a coal mine data storage and computing integrated center, wherein the coal mine data storage and computing integrated center includes: a data acquisition module, used to acquire at least one of the following original coal mine data: coal mine data transmitted in real time by underground control transfer equipment, coal mine data stored by mobile storage devices, and coal mine data transmitted by a coal mine geological disaster hazard transparency business system; a data management module, used to process the acquired original coal mine data through multiple processes, manage the original coal mine data and process the coal mine data; a data usage module, used to send the managed coal mine data to the coal mine geological disaster hazard transparency business system.

[0049] Furthermore, the coal mine data intelligent management system also includes coal mine multi-field multi-source sensors, underground control transfer equipment and mobile storage equipment, wherein:

[0050] The first type of coal mine multi-field multi-source sensor is used to collect the first original coal mine data and transmit the first original coal mine data to the underground control transfer equipment in real time;

[0051] The second type of coal mine multi-field multi-source sensor is used to collect the second original coal mine data and regularly store the second original coal mine data in a mobile storage device. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A flow chart of a method for intelligent management of coal mine data provided in Embodiment 1 of the present invention;

[0054] Figure 2 A schematic diagram of a LRU algorithm for optimizing cold and hot data in an intelligent lake warehouse provided in the first embodiment of the present invention;

[0055] Figure 3 A functional structure diagram of a coal mine data storage and computing integrated center provided in Embodiment 2 of the present invention;

[0056] Figure 4 A schematic diagram of the hierarchical structure of an intelligent lake warehouse provided in the second embodiment of the present invention;

[0057] Figure 5 A schematic diagram of a computing center structure provided in Embodiment 2 of the present invention;

[0058] Figure 6 A schematic diagram of the structure of an intelligent coal mine data management system provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0059] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0060] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The technical solutions provided by the present invention are described in detail through various embodiments.

[0061] Embodiment 1

[0062] This embodiment provides a method for intelligent management of coal mine data, which can be executed by a coal mine data storage and computing center. Figure 1 The method specifically includes the following steps 101-103.

[0063] Step 101, obtaining at least one of the following original coal mine data: coal mine data transmitted in real time by underground control transfer equipment, coal mine data stored in a mobile storage device, and coal mine data transmitted by a coal mine geological disaster hazard transparency business system.

[0064] In this step, raw coal mine data refers to the basic, unprocessed or unanalyzed initial data collected during coal mining and related activities. These data are usually directly obtained through various underground monitoring instruments (such as monitoring equipment, sensors), surveys or observations. The types of raw coal mine data may include geological exploration data, environmental monitoring data, production operation data, safety monitoring data, resource utilization data and other auxiliary data. Raw coal mine data can be obtained from the following multiple sources to provide basic data support for subsequent intelligent management and analysis:

[0065] a. Coal mine data transmitted in real time by underground control transfer equipment. The data source can be collected in real time by underground monitoring instruments with high-reliability communication capabilities and sent to underground control transfer equipment;

[0066] b. Coal mine data stored in mobile storage devices are usually exported from underground monitoring instruments without reliable communication capabilities and stored in mobile storage devices (such as disks, USB flash drives, and hard disks);

[0067] c. Coal mine data transmitted by the coal mine geological disaster hazard transparency business system. This type of data can be derived from data collected by different business subsystems.

[0068] Step 102: multi-process processing of the acquired original coal mine data, management of the original coal mine data and processing of the coal mine data.

[0069] In this step, a series of continuous and different types of processing operations are performed on the acquired original coal mine data to achieve the set data analysis and mining goals. Among them, the processing operations may involve data access, data cleaning, data conversion, data integration, data analysis and modeling, data visualization, feedback and iteration, etc. The data obtained by the multi-process processing process can be called processed coal mine data, which is divided into intermediate processed coal mine data and final coal mine data. These data are obtained through a series of processing operations on the basis of the original coal mine data, and the data generated by each processing operation is the input of subsequent processing operations until support data that can be used for analysis or decision-making is finally generated. Coal mine data management may include storage, backup, update, access control, sharing control, etc. of coal mine data. Preferably, a hybrid storage mode of local and cloud services is adopted to store original coal mine data and processed coal mine data.

[0070] The embodiment of the present invention can adopt a "data + computing power" model to achieve division of labor and cooperation between the multi-process processing and management of coal mine data to ensure the efficient use of data resources. The coal mine data storage and computing integrated center includes two major parts: the intelligent lake warehouse and the computing center. The intelligent lake warehouse is responsible for managing two types of data: original coal mine data and processed coal mine data to ensure the quality, security and availability of the data. In terms of processing task allocation, the intelligent lake warehouse can perform some common and simple processing tasks, such as data integration and preliminary statistical analysis. For more complex data processing tasks, they are undertaken by the computing center. This division of labor can optimize resource utilization and ensure that complex computing tasks do not affect data storage and basic processing efficiency. Accordingly, the original coal mine data obtained by multi-process processing includes:

[0071] Through the computing center, the original coal mine data and / or intermediate processed coal mine data that meet the set high computing power requirements are processed.

[0072] In specific implementation, the management node of the computing center can be used to schedule the computing nodes in the computing pool to perform computing tasks in a load-balanced manner to process the acquired original coal mine data and / or intermediate processed coal mine data that meet the set high computing power requirements.

[0073] Exemplarily, managing the original coal mine data and processing the coal mine data includes: managing the original coal mine data through the data lake in the smart lake warehouse; managing and processing the coal mine data through the data warehouse in the smart lake warehouse. The following is a detailed description.

[0074] (1) Manage raw coal mine data through the data lake

[0075] A data lake is a centralized storage architecture that can receive and store raw coal mine data in various formats, including structured data, semi-structured data, and unstructured data, thereby efficiently managing and storing large amounts of data.

[0076] When managing raw coal mine data, the data lake can store the raw coal mine data by partitioning according to the degree of hotness and coldness. Specifically, an optimized LRU (Least Recently Use) algorithm can be used to distinguish between cold data and hot data in the raw coal mine data. Among them, hot data refers to data that is frequently read and processed; cold data refers to data with low access frequency and low usage rate. Figure 2 As shown, when a certain original coal mine data enters the lake for the first time, it will be placed at the tail of the data queue 1 with the lowest priority in the hot data storage area. When the original coal mine data is accessed later, according to the LRU algorithm rules, the data will be moved from the tail of the data queue 1 to the head. When the priority of the original coal mine data is further improved according to the number of accesses, the data will be moved to the tail of the data queue with a higher priority in the hot data storage area (the example in the figure is data queue 2 or data queue 3), and the data in the original data queue will be deleted. Similarly, when the priority of the original coal mine data is reduced according to the number of accesses, the data will be moved to the head of the low-priority data queue in the hot data storage area, and the data in the original data queue will be deleted. Data elimination in the hot data storage area is always carried out from the tail of the lowest priority data queue, and the eliminated tail data is added to the head of the historical data queue. If the data is accessed in the historical data queue, the priority of the data is recalculated and added to the data queue of the corresponding priority in the hot data storage area, otherwise it is completely eliminated from the hot data storage area according to the LRU algorithm and transferred to the cold data storage area.

[0077] Based on the above LRU algorithm, an improved adaptive hot and cold data partitioning algorithm is introduced to dynamically adjust the partitioning criteria of hot and cold data. The adaptive hot and cold data partitioning algorithm defines the sliding window function of the access frequency as follows:

[0078]

[0079] in, is the access frequency of data D at the current time t, W is the set time sliding window size, is the number of accesses to data D at the current time i.

[0080] Adaptive adjustment of threshold:

[0081]

[0082] in, is the hot data threshold at the current time t, is the hot data threshold at the previous time t-1, The smoothing coefficient is set to balance the impact of historical hot data threshold and current data. is the importance weight of the set data D, and β is the set weight smoothing coefficient.

[0083] Cold Data and thermal data The classification is determined according to the following formula:

[0084]

[0085] Preferably, the original coal mine data belonging to the hot data category is stored in a distributed file system HDFS server and a cache server composed of solid-state hard disks SSD; the original coal mine data belonging to the cold data category is compressed and stored in a distributed file system HDFS server composed of hard disk drives HDD to save storage space and cost for massive coal mine data. Exemplarily, a distributed file system server composed of solid-state hard disks can store all hot data and ensure that the most frequently accessed data is always in the cache server. When external data is accessed, the storage in the cache server is read first. Regarding data compression, the Deflate compression algorithm can be used to compress seismic exploration .segy format data, and the LZ78 algorithm can be used to compress unformatted or processed binary data, such as geological radar detection data .raw format data; Huffman coding can be used to compress ASCII text format .las format data of logging data, etc.

[0086] Among them, for the compression of the original coal mine data of the above cold data category, its compression strategy can be dynamically adjusted according to various factors such as storage space and processing requirements. For example, the following formula can be used to select from a variety of compression strategies Select a compression strategy in use:

[0087]

[0088] in, is the selected compression strategy, Compression strategy The computational cost of Compression strategy Compression efficiency (compression ratio), Compression strategy The execution time of Compression strategy Real-time monitoring of current system resource usage (CPU, memory, GPU, etc.) during execution. is the set weight coefficient.

[0089] Preferably, the weight coefficient It can also be optimized dynamically over time according to the following formula:

[0090]

[0091] Where L is the optimization objective function, which is pre-set according to the actual application requirements (such as space minimization or time minimization); For the current moment Used Weight coefficient, For the next moment Used Weight coefficient.

[0092] The above hot and cold storage mechanism has the following advantages:

[0093] Improve access efficiency: Hot data is stored in distributed file system servers and cache servers composed of solid-state hard drives. These storage media have faster read and write speeds, which can significantly improve the access efficiency of hot data.

[0094] Reduce costs: Cold data is stored in a distributed file system server composed of low-cost hard disk drives. By compressing storage, the storage space requirements are reduced, thereby reducing the overall storage cost.

[0095] Optimize resource utilization: By storing data with different access frequencies on different types of storage media, resources are effectively allocated, ensuring fast access to hot data while also reducing the cost of cold data storage;

[0096] Simplified data management: By using an optimized LRU algorithm to automatically distinguish between hot and cold data, the data management process is simplified and the need for manual intervention is reduced;

[0097] Enhanced data availability: Hot data is located in fast-access storage, ensuring high availability and low-latency access, while cold data, although less frequently accessed, can still be quickly retrieved through the distributed file system;

[0098] Improve data security: Through partitioned storage, different security measures can be taken for different types of data to improve the security of overall data.

[0099] (2) Manage and process coal mine data through data warehouse ① In the data warehouse operation data storage layer, the operation data generated by preprocessing the original coal mine data is obtained and stored.

[0100] Among them, preprocessing includes simple cleaning, deduplication, format conversion, etc.

[0101] ② In the data warehouse detail layer, obtain and store a data model generated based on the operation data that meets the business needs of transparency of coal mine geological disaster hazards.

[0102] Among them, the computing center can be called to use the existing big data and artificial intelligence technologies to clean, process, integrate, aggregate and perform other operations on the operation data storage layer to build a data model that meets the business needs of transparent geological disaster risks in coal mines.

[0103] ③ In the data warehouse service layer, receive and respond to data access requests initiated by external access interfaces based on data stored in the data warehouse.

[0104] The service layer defines the access interface for data warehouse storage data and provides various types of data services, such as data query, statistics, report generation, analysis, etc. Optionally, the intermediate processed coal mine data can be further processed through ETL operations and other methods to provide real-time data services for upper-level applications.

[0105] ④ In the analytical data storage layer of the data warehouse, analytical data generated by multi-process processing of the original coal mine data is obtained and stored.

[0106] Analytical data is a structured and standardized data generated by the original coal mine data through multiple processing processes such as cleaning, integration, transformation and modeling, which can reveal the potential patterns, trends and relationships in the data. Constructing and preserving specially optimized analytical data can realize efficient and convenient query of multi-source coal mine data, and also has the ability to respond quickly, providing data support for the transparent business of coal mine geological disaster hazards.

[0107] As a preferred implementation, the original coal mine data obtained by multi-process processing includes:

[0108] Analyze the acquired original coal mine data to obtain at least one of the following acceleration data, which is used to generate storage data for the data warehouse in the intelligent lake warehouse: coal mine data category, the relationship between coal mine data, the sharing mechanism of coal mine data, and the access rights of coal mine data.

[0109] Among them, the association relationship between coal mine data can be a data lineage relationship, which is obtained by parsing the original coal mine data using Apache Flink, Apache Spark, and Trino. The coal mine data lineage relationship covers the entire process from coal mine data acquisition to multi-process processing. In addition, the association relationship can also be a coal mine data knowledge graph established using the graph database Neo4J.

[0110] In the process of analyzing coal mine data, the correlation between data changes with time and space. A dynamic update algorithm for correlation based on a graph database can be introduced to maintain the correlation between data in real time. For example, in the dynamic update algorithm for correlation, the correlation strength and correlation threshold between coal mine data change with time, as follows:

[0111] The update formula of the correlation strength between coal mine data is as follows:

[0112]

[0113] in: For data and The strength of the association at the current time t; For data and The strength of association at the next time t+1; For data and Similarity function of ; The balance coefficient between historical data and current calculation results is preset; For data and The rate of change at the current time t; is the influence coefficient of the change rate, which is preset;

[0114] The formula for updating the relevance threshold is as follows:

[0115]

[0116] in, For the current moment The threshold value of correlation judgment is For the previous moment The threshold value of correlation judgment is is the preset update coefficient.

[0117] only Exceed Only when the connection is maintained in the graph database, that is, the coal mine data and There is a correlation between them.

[0118] Step 103: Send the managed coal mine data to the coal mine geological disaster hazard transparency business system.

[0119] In this step, the managed coal mine data can be regularly and actively broadcast to the coal mine geological disaster hidden danger transparency business system, and the specific broadcast object can be the intelligent platform in the system, which is used to display and update the coal mine geological disaster hidden danger transparency map. Of course, those skilled in the art should understand that after receiving the data access request of each subsystem in the coal mine geological disaster hidden danger transparency business system, the managed coal mine data corresponding to the request can be sent to the subsystem, and the embodiment of the present invention does not specifically limit this.

[0120] Based on the above scheme, as an optional implementation method, after step 103, it also includes: after receiving the abnormal information feedback sent by the coal mine geological disaster hidden danger transparency business system, notifying the underground terminal. Among them, the underground terminal refers to the terminal equipment used by the equipment or personnel operating underground in the coal mine, which may include explosion-proof mobile phones, explosion-proof tablets, explosion-proof notebooks, etc. By immediately notifying the underground terminal, it can ensure that miners are sufficiently alert to potential geological disasters and take corresponding preventive measures, thereby reducing the risk of accidents.

[0121] Embodiment 2

[0122] This embodiment provides a coal mine data storage and computing integrated center, which can be used to execute the coal mine data intelligent management method described in the embodiment of the present invention. Figure 3 The coal mine data storage and computing center specifically includes the following modules:

[0123] The data acquisition module 301 is used to acquire at least one of the following original coal mine data: coal mine data transmitted in real time by underground control transfer equipment, coal mine data stored in mobile storage devices, and coal mine data transmitted by a coal mine geological disaster hidden danger transparency business system;

[0124] The data management module 302 is used for multi-process processing of the acquired raw coal mine data, managing the raw coal mine data and processing the coal mine data;

[0125] The data use module 303 is used to send the managed coal mine data to the coal mine geological disaster hidden danger transparency business system.

[0126] Furthermore, the data management module 302 includes: a first management submodule, which is used to manage the original coal mine data through the data lake in the smart lake warehouse; and a second management submodule, which is used to manage and process the coal mine data through the data warehouse in the smart lake warehouse.

[0127] Furthermore, the first management submodule is used to manage the original coal mine data through the data lake in the smart lake warehouse, specifically including: storing the original coal mine data in partitions according to the degree of coldness and hotness through the data lake in the smart lake warehouse.

[0128] Furthermore, the first management submodule is used to store the original coal mine data by cold and hot partitions through the data lake in the intelligent lake warehouse, specifically including:

[0129] The original coal mine data belonging to the hot data category is stored in a distributed file system server and a cache server composed of solid state hard disks;

[0130] The original coal mine data belonging to the cold data category is compressed and stored in a distributed file system server composed of hard disk drives.

[0131] Furthermore, the second management submodule is used to manage and process coal mine data through the data warehouse in the intelligent lake warehouse, specifically including:

[0132] In the data warehouse operation data storage layer, the operation data generated by preprocessing the original coal mine data is obtained and stored;

[0133] At the data warehouse detail layer, a data model generated based on the operation data and meeting the business requirements for transparency of geological hazards in coal mines is obtained and stored;

[0134] At the data warehouse service layer, receive and respond to data access requests initiated by external access interfaces based on data stored in the data warehouse;

[0135] In the analytical data storage layer of the data warehouse, analytical data generated by multi-process processing of the original coal mine data is obtained and stored.

[0136] Furthermore, the data management module 302 also includes:

[0137] The data acceleration submodule is used to analyze the acquired original coal mine data to obtain at least one of the following acceleration data, which is used to generate storage data for the data warehouse in the intelligent lake warehouse: coal mine data categories, associations between coal mine data, sharing mechanisms for coal mine data, and access rights to coal mine data.

[0138] Furthermore, the data management module 302 is used to manage the original coal mine data and process the coal mine data, specifically including: using a hybrid storage mode of local and cloud services to store the original coal mine data and process the coal mine data.

[0139] Furthermore, the data management module 302 is used for multi-process processing of the acquired original coal mine data, specifically including: processing the original coal mine data and / or intermediate processed coal mine data that meet the set high computing power requirements through the computing center.

[0140] Furthermore, the data management module 302 is used to process the acquired raw coal mine data and / or intermediate processed coal mine data that meet the set high computing power requirements through the computing center, specifically including:

[0141] Through the management node of the computing center, the computing nodes in the computing pool are scheduled to perform computing tasks in a load-balanced manner to process the acquired original coal mine data and / or intermediate processed coal mine data that meet the set high computing power requirements.

[0142] In order to process coal mine data more efficiently, the computing tasks can be dynamically allocated based on the preset parallel processing optimization algorithm to improve processing efficiency. In this algorithm, the allocation function of computing tasks is as follows:

[0143]

[0144] in, is the allocation time of the jth computing task on the i-th computing node, is the computation weight of the jth computation task, is the computing power of the ith computing node, is the load prediction value of the i-th computing node at the next time t+1.

[0145] The load balancing adjustment formula is as follows:

[0146]

[0147] in, is the load ratio of the i-th computing node at the current time t (i.e., the current load of the node), indicating the actual load level of the i-th computing node at the current time t; dynamically adjust the load distribution of the computing nodes to achieve load balancing. is the computing capacity or load capacity of the i-th computing node at the current time t; It is the computing capacity or load capacity of the i-th computing node at the next time t+1 after load balancing adjustment. The maximum value among the load ratios of all computing nodes k at the current time t provides a standard for comparison with the load ratio of the i-th computing node to help determine the magnitude of the adjustment.

[0148] The above load balancing adjustment formula is used to update the computing power according to the current computing node's load ratio relative to the maximum load of other computing nodes. The computing power of a computing node is adjusted by increasing the current computing node's load ratio relative to the maximum load ratio. If the current computing node's load is close to the maximum load, its computing power will be increased more to help balance the load.

[0149] Furthermore, the coal mine data storage and computing integrated center also includes a notification module 304, which is used to: notify the underground terminal after receiving the abnormal information feedback sent by the coal mine geological disaster hidden danger transparency business system.

[0150] The above is an explanation of the coal mine data storage and computing center from the perspective of functional module implementation. In order to have a clearer understanding of the structure and function of the coal mine data storage and computing center, the following will further explain the coal mine data storage and computing center in detail from different levels of data flow and processing. The coal mine data storage and computing center includes intelligent lake warehouse and computing center. Figure 4 , the smart lake warehouse includes the following layers: (1) Coal mine multi-source data layer It consists of coal mine data in multiple subsystem databases (including MySQL, PostgreSQL, HBase, Redis, MongoDB, InfluxDB, Neo4j, PostGIS, Kdb, Elasticsearch, etc.) in the existing coal mine geological disaster hidden danger transparency business system, coal mine data stored in mobile storage devices (U disk, disk, hard disk, etc.) and real-time monitored coal mine data streams; (2) Data Collection Layer Through Apache Flink, Apache Spark, and Trino, data is accessed from the multi-source data layer of the coal mine. On the one hand, it accesses the real-time monitoring data stream input and mobile storage device storage data in the coal mine, and on the other hand, it accesses the data access library of the existing subsystems, ensuring the daily smooth operation of the existing system and the transition and upgrade of data. (3) Data Management Acceleration Layer Analyze the original coal mine data accessed by the data collection layer to obtain at least one of the following acceleration data, which is used to generate storage data for the data warehouse in the intelligent lake warehouse: coal mine data category (i.e., data content), association relationship between coal mine data (the example in the figure is data lineage), coal mine data sharing mechanism, and access rights to coal mine data; (4) Lake warehouse storage layer Establish a hybrid storage cluster of distributed file system HDFS and object storage AWS S3 to realize a hybrid storage mode of local and cloud services to store original coal mine data and accelerated data; (5) Resource management layer Resource scheduling and management of intelligent lake warehouses in coal mines can be realized through resource managers Yarn and Kubernetes; (6) Data Lake For details, please refer to the description of the data lake in the first embodiment, which will not be repeated here; (7) Data Warehouse Read the original coal mine data and acceleration data from the lake warehouse storage layer and access the data warehouse. The data warehouse includes the following four layers of operations: Operational Data Store (ODS), Data Warehouse Detail (DWD), Data Warehouse Service (DWS) and Analytical Data Store (ADS); please refer to the relevant description of Example 1 for details, which will not be repeated here; (8) Application layer First, it provides an interface to access the original coal mine data in the data lake, which can serve the upper-level applications for statistical analysis (such as gas monitoring), data mining and data fusion; second, it provides an analytical data access interface in the data warehouse, which can serve the transparent geological modeling of geological hazards, dynamic interpretation of all-factor multi-physical fields of geological hazards, multi-field and multi-attribute data aggregation and mining, coal mine detection and interpretation data sharing and other applications.

[0151] Computing centers, such as Figure 5 As shown, it may include data center A and data center B, and the two data centers play a disaster recovery role; each data center is composed of multiple computing pools (the example in the figure is 2), and each computing pool is composed of a management node and multiple computing nodes (the example in the figure is 3). The management node is responsible for allocating tasks to ensure the load balancing of the computing nodes in the data center, and the computing nodes are responsible for processing the original coal mine data and / or intermediate processed coal mine data that meet the set high computing power requirements, such as geological disaster hidden danger transparency, seismic exploration inversion interpretation and attribute analysis (calculation of seismic attributes, such as instantaneous amplitude, phase, frequency, etc.), seismic exploration data machine learning and artificial intelligence processing and interpretation applications.

[0152] The coal mine data storage and computing integrated center provided in this embodiment belongs to the same inventive concept as the aforementioned method embodiment. For technical details not described in this embodiment, please refer to the relevant description in the aforementioned method embodiment, and will not be repeated here.

[0153] Embodiment 3 This embodiment provides an intelligent management system for coal mine data, which is applied to the intelligent management method for coal mine data in the embodiment of the present invention. Figure 6 The intelligent coal mine data management system includes a coal mine data storage and computing center: an intelligent lake warehouse and a computing center.

[0154] Furthermore, the intelligent coal mine data management system may also include coal mine multi-field multi-source sensors, underground control transfer equipment and mobile storage devices (not shown in the figure), wherein: The first type of coal mine multi-field multi-source sensor is used to collect the first original coal mine data and transmit the first original coal mine data to the underground control transfer equipment in real time; The second type of coal mine multi-field multi-source sensor is used to collect the second original coal mine data and regularly store the second original coal mine data in a mobile storage device.

[0155] Among them, the coal mine multi-field multi-source sensor, as an underground monitoring instrument, can obtain multiple types of coal mine data from the following aspects: gravity, magnetism, electricity, earthquake, geology, surveying and mapping, etc. Specifically, microgravity measurement, transient electromagnetic exploration, high-density electrical exploration, geological radar detection, lidar SLAM scanning, 3D / 4D seismic exploration, microseismic monitoring, UWB inertial navigation positioning and other means can be used. The first type of coal mine multi-field multi-source sensor has high-reliability communication capabilities and can transmit data to underground control transfer equipment in real time via WIFI. The second type of coal mine multi-field multi-source sensor does not have high-reliability communication capabilities.

[0156] The underground control transfer equipment receives the original coal mine data transmitted by the multi-source sensors in the coal mine, processes and interprets it in real time, and stores the original coal mine data and the real-time interpretation data results in the buffer (the buffer is mainly used to avoid the data transmission pressure of the underground control transfer equipment caused by the real-time massive data), and the buffer is transmitted to the coal mine data storage and computing center in real time through optical fiber. In addition, the underground control transfer equipment can also control the work of mining equipment (such as coal mining machines).

[0157] In addition, the coal mine data storage and computing center can be connected to the smart platform of the coal mine geological disaster hazard transparency business system, and is used to: regularly and actively broadcast the managed coal mine data to the smart platform for the smart platform to display and update the coal mine geological disaster hazard transparency map; after receiving abnormal information feedback sent by the smart platform, notify the underground terminal.

[0158] The coal mine data storage and computing center can also be connected to the subsystem of the coal mine geological disaster hidden danger transparency business system, and is used to: receive coal mine data transmitted by the subsystem of the business system, perform multi-process processing and management on the coal mine data; and send managed coal mine data to the subsystem of the business system. Among them, the business subsystem may include one or more of the tunneling system, hierarchical control system, and comprehensive monitoring system.

[0159] In summary, the technical solution provided by the embodiment of the present invention can realize the scientific storage management, real-time dynamic processing, interpretation and practical application of multi-source heterogeneous data of coal mine geological disaster hazards.

[0160] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0161] In the embodiments of the present invention, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0162] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0163] In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0164] For the convenience of description, the above device is described by dividing the functions into various units / modules. Of course, when implementing the present invention, the functions of each unit / module can be implemented in the same or multiple software and / or hardware.

[0165] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0166] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for intelligent management of coal mine data, characterized in that: Applied to a coal mine data storage and computing integrated center, the method includes: Obtain at least one of the following original coal mine data: coal mine data transmitted in real time by underground control transfer equipment, coal mine data stored in mobile storage devices, and coal mine data transmitted by the coal mine geological disaster hidden danger transparency business system; Multi-process processing of the acquired raw coal mine data, management of raw coal mine data and processing of coal mine data; Send the managed coal mine data to the coal mine geological disaster hidden danger transparency business system.

2. The method according to claim 1, characterized in that Manage raw coal mine data and process coal mine data, including: Through the data lake in the intelligent lake warehouse, the original coal mine data is partitioned and stored according to the degree of hotness and coldness: the original coal mine data belonging to the hot data category is stored in the distributed file system server and cache server composed of solid-state drives; the original coal mine data belonging to the cold data category is compressed and stored in the distributed file system server composed of hard disk drives; Coal mine data is managed and processed through the data warehouse in the smart lake warehouse.

3. The method according to claim 2, characterized in that Cold Data and hot data The classification is determined according to the following formula: ； in, The calculation formula is as follows: ； is the access frequency of data D at the current time t, W is the set time sliding window size, is the number of accesses to data D at time i; The calculation formula is as follows: ； is the hot data threshold at the current time t, is the hot data threshold at the previous moment t-1, is the set smoothing coefficient, is the importance weight of the set data D, and β is the set weight smoothing coefficient.

4. The method according to claim 2, characterized in that According to the following formula, from multiple compression strategies Select a compression strategy , compress the original coal mine data belonging to the cold data category: ； in, is the selected compression strategy, Compression strategy The computational cost, Compression strategy The compression efficiency, Compression strategy The execution time of Compression strategy During execution, the current system resource usage is monitored in real time. is the weight coefficient.

5. The method according to claim 2, characterized in that Through the data warehouse in the intelligent lake warehouse, coal mine data is managed and processed, including: In the data warehouse operation data storage layer, the operation data generated by pre-processing the original coal mine data is obtained and stored; At the data warehouse detail layer, a data model generated based on the operation data and meeting the business requirements for transparency of coal mine geological disaster hazards is obtained and stored; At the data warehouse service layer, it receives and responds to data access requests initiated by external access interfaces based on data stored in the data warehouse; In the analytical data storage layer of the data warehouse, analytical data generated by multi-process processing of original coal mine data is obtained and stored.

6. The method according to claim 2, characterized in that The raw coal mine data obtained through multi-process processing includes: Analyze the acquired raw coal mine data to obtain at least one of the following acceleration data, which is used to generate storage data for the data warehouse in the intelligent lake warehouse: coal mine data category, association relationship between coal mine data, coal mine data sharing mechanism, and access rights to coal mine data; Among them, the correlation between coal mine data is maintained in real time based on the dynamic update algorithm of correlation relationship. In this algorithm: The formula for updating the correlation strength between coal mine data is as follows: ； For data and The strength of the association at the current time t; For data and The strength of the association at the next moment t+1; For data and Similarity function of is the preset balance coefficient; For data and The rate of change at the current time t; is the influence coefficient of the preset change rate; The formula for updating the correlation threshold between coal mine data is as follows: ； For the current moment The threshold value of correlation judgment, For the previous moment The threshold value of correlation judgment, is the pre-set update coefficient; when Exceed When coal mine data and There is a correlation between them.

7. The method according to any one of claims 1 to 6, characterized in that The raw coal mine data obtained through multi-process processing includes: Through the management node of the computing center, the computing nodes in the computing pool are scheduled to perform computing tasks in a load-balanced manner to process the acquired raw coal mine data and / or intermediate processed coal mine data that meet the set high computing power requirements; Among them, the allocation function of the computing task is as follows: ； is the allocation time of the jth computing task on the i-th computing node, is the computation weight of the jth computation task, is the computing power of the i-th computing node, is the load prediction value of the i-th computing node at the next time t+1; The load balancing adjustment formula is as follows: ； is the load ratio of the i-th computing node at the current time t; is the computing power or load capacity of the i-th computing node at the current time t; is the computing capacity or load capacity of the i-th computing node at the next time t+1 after load balancing adjustment; For all computing nodes at the current moment The maximum value among the load ratios.

8. The method according to claim 1, characterized in that After sending the managed coal mine data to the coal mine geological disaster hidden danger transparency business system, the method further includes: After receiving the abnormal information feedback sent by the coal mine geological disaster hidden danger transparency business system, the underground terminal is notified.

9. An intelligent coal mine data management system, characterized in that: Applied to the coal mine data intelligent management method according to claims 1-8, the coal mine data intelligent management system comprises a coal mine data storage and computing integrated center; Among them, the coal mine data storage and computing integrated center includes: a data acquisition module, used to obtain at least one of the following original coal mine data: coal mine data transmitted in real time by underground control transfer equipment, coal mine data stored by mobile storage devices, and coal mine data transmitted by the coal mine geological disaster hidden danger transparency business system; a data management module, used for multi-process processing of the acquired original coal mine data, management of the original coal mine data and processing of the coal mine data; a data usage module, used to send the managed coal mine data to the coal mine geological disaster hidden danger transparency business system.

10. The intelligent coal mine data management system according to claim 9, characterized in that: The intelligent coal mine data management system also includes coal mine multi-field multi-source sensors, underground control transfer equipment and mobile storage devices, among which: The first type of coal mine multi-field multi-source sensor is used to collect first original coal mine data and transmit the first original coal mine data to underground control transfer equipment in real time; The second type of coal mine multi-field multi-source sensor is used to collect the second original coal mine data and regularly store the second original coal mine data in a mobile storage device.

Images