Heterogeneous information classification prediction method and related equipment for coal mine disasters based on big data

Abstract

The present application discloses a method for hierarchical prediction of heterogeneous information of coal mine disasters based on big data and related equipment, the method comprising: obtaining monitoring data from different data sources; fusing the monitoring data to obtain a comprehensive data set; dividing the comprehensive data set into a data set and scattered data; mapping the data set to a storage node one-to-one; merging the data set with the scattered data to obtain a merged data set; performing big data analysis on the data set in the target storage node to obtain a coal mine disaster feature set; performing big data analysis on the merged data set to obtain a fused feature set; performing correlation analysis on the coal mine disaster feature set and the fused feature set to determine whether there are new coal mine disaster features; if so, expanding the storage node and the corresponding sub-computing node, and storing the new coal mine disaster features in the expanded storage node; inputting the coal mine disaster feature set and the new coal mine disaster features into a pre-trained coal mine disaster prediction model to obtain the coal mine disaster level.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of big data technology, and in particular to a method for hierarchical prediction of heterogeneous information of coal mine disasters based on big data and related equipment. Background Art

[0002] Coal mine disasters refer to various events that may occur during the coal mine production process and endanger the safety and health of miners, usually including gas explosions, coal dust explosions, fires, mine tremors, etc. In order to ensure the safety of miners and the normal production of coal mines, it is necessary to predict coal mine disasters. However, if coal mine disasters are to be predicted, it is often necessary to process data from different data sources at the same time, such as sensor data, video data, geological data, etc., to ensure the accuracy of coal mine disaster prediction as much as possible. At present, one computing node is usually used to process data from different data sources, and a trained prediction model is used to analyze and predict data from different data sources.

[0003] However, with the development of technology and the diversification of data collection methods, the amount of data required for coal mine disaster prediction and the complexity of data sources are gradually increasing. Due to the limited storage capacity and parallel processing capabilities of a single computing node, when the amount of data and computing requirements increase, the data processing and analysis speed of a single node will slow down, and a single node may not be able to process all data in a timely manner, resulting in incomplete data sources, affecting the accuracy of the prediction model, and thus causing low accuracy in coal mine disaster prediction.

[0004] There is currently no effective solution to the above problems. Summary of the invention

[0005] The embodiment of the present application provides a coal mine disaster heterogeneous information hierarchical prediction method and related equipment based on big data, which are used to effectively improve the accuracy of coal mine disaster prediction.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a method for hierarchical prediction of coal mine disaster heterogeneous information based on big data is provided, which is applied to an electronic device, wherein the electronic device is deployed with a Hadoop platform, the Hadoop platform includes a big data analysis component and a distributed data warehouse, the big data analysis component includes at least one main computing node and m sub-computing nodes, the distributed storage database includes storage nodes corresponding to the sub-computing nodes one by one, and m is a positive integer greater than or equal to 2, and the method includes:

[0008] Acquiring monitoring data from different data sources through the Hadoop platform, wherein the monitoring data includes sensor data, video surveillance data, equipment operation data, and geological exploration data;

[0009] Performing data fusion on the sensor data, the video monitoring data, the equipment operation data and the geological exploration data to obtain a comprehensive data set;

[0010] According to the node table structure of each storage node, the comprehensive data set is divided into p data sets and n scattered data, where p is less than or equal to m, and n is an integer greater than or equal to 0;

[0011] Mapping the p data sets to the m storage nodes one by one, so as to store the p data sets in the corresponding p target storage nodes;

[0012] Merging the mp data sets with the n scattered data to obtain a merged data set;

[0013] Using p target sub-computing nodes corresponding to the target storage nodes one by one among the m sub-computing nodes, performing big data analysis on the data sets in the corresponding target storage nodes, and obtaining p coal mine disaster feature sets;

[0014] Using the main computing node to perform big data analysis on the merged data set to obtain a fusion feature set;

[0015] Performing correlation analysis on the p coal mine disaster feature sets and the fusion feature set to determine whether there are new coal mine disaster features;

[0016] In the case of newly added coal mine disaster features, the storage node and the corresponding sub-computing node are expanded so that the number of storage nodes and the number of sub-computing nodes are both increased by 1, and the newly added coal mine disaster features are stored in the expanded storage node;

[0017] The p coal mine disaster feature sets and the newly added coal mine disaster features are input into a pre-trained coal mine disaster prediction model to obtain a coal mine disaster level.

[0018] In a possible implementation manner of the first aspect, the fusing the sensor data, the video monitoring data, the equipment operation data, and the geological exploration data to obtain a comprehensive data set includes:

[0019] respectively extracting semantic features of the sensor data, the video monitoring data, the equipment operation data, and the geological exploration data;

[0020] Performing semantic association on all the semantic features to obtain a plurality of associated semantic feature pairs and at least one scattered semantic feature;

[0021] Performing scene association on the plurality of the associated semantic feature pairs and the at least one scattered semantic feature to obtain the associated semantic feature pairs and the scattered semantic features of each coal mine scene;

[0022] The associated semantic feature pairs and the scattered semantic features of each coal mine scene are merged, wherein the comprehensive data set is used to characterize the associated semantic feature pairs and the scattered semantic features of all the coal mine scenes.

[0023] In another possible implementation of the first aspect, p target sub-computing nodes corresponding one-to-one to the target storage nodes are used among the m sub-computing nodes to perform big data analysis on the data sets in the corresponding target storage nodes to obtain p coal mine disaster feature sets, including:

[0024] Using the target sub-computing node, extracting at least one of the coal mine scenes from the corresponding target storage node, and obtaining the coal mine disaster semantics corresponding to each of the coal mine scenes;

[0025] Extract and summarize key features of the data set in the target storage node corresponding to the coal mine scene one by one to obtain p key feature sets;

[0026] In each of the key feature sets, coal mine disaster features are determined and summarized according to the corresponding coal mine disaster semantics to obtain a coal mine disaster feature set.

[0027] In another possible implementation manner of the first aspect, the using the main computing node to perform big data analysis on the merged data set to obtain a fusion feature set includes:

[0028] Acquire the time attribute and space attribute of each data in the merged data set;

[0029] According to the time attribute and the space attribute of each data, data fusion is performed on the data in the merged data set to obtain a plurality of fused data;

[0030] The main computing node is used to perform feature selection on all the fused data, and redundant features are screened out to obtain a fused feature set.

[0031] In another possible implementation manner of the first aspect, the main computing node is deployed with an evaluator, and the main computing node is used to perform feature selection on all the fused data and filter out redundant features to obtain a fused feature set, including:

[0032] Using the main computing node to extract data features of all the fused data;

[0033] The main computing node is used to generate a feature subset, and the first step is iteratively performed until a preset number of iterations is reached, and the optimal performance index of the evaluator in the iteration is determined, and the feature subset corresponding to the optimal performance index is used as the fusion feature set;

[0034] Wherein, the first step comprises:

[0035] Randomly selecting b data features from all the data features to add to the feature subset, where b is an integer greater than 1;

[0036] Using the feature subset to train the evaluator, obtaining the importance score of each sub-feature in the feature subset by the evaluator, and recording the performance index of the evaluator;

[0037] In the case where the importance score of the sub-feature is less than the first threshold, the sub-feature is taken as a redundant feature and is screened out to obtain an optimized feature subset.

[0038] In another possible implementation of the first aspect, performing correlation analysis on the p coal mine disaster feature sets and the fused feature set to determine whether there are new coal mine disaster features includes:

[0039] Merging the p coal mine disaster feature sets and the fusion feature set into a first data set;

[0040] Calculating a first support for each feature in the first data set, where the first support is used to characterize the frequency of occurrence of the feature in the first data set;

[0041] Filter and merge the features whose first support is greater than or equal to the first support threshold to obtain a frequent item set;

[0042] The second step is executed repeatedly until no updated frequent item sets can be generated, and the final frequent item sets are obtained;

[0043] Generate association rules from the final frequent item set, and calculate the confidence of each association rule;

[0044] An association rule corresponding to a confidence level greater than or equal to a confidence level threshold is used as a target association rule, and whether there is a new coal mine disaster feature is determined according to the target association rule;

[0045] Wherein, the second step comprises:

[0046] Randomly combining the features in the frequent item set to form a plurality of multinomial sets, and calculating the second support of each of the multinomial sets;

[0047] The multiple item sets whose second support is greater than or equal to the second support threshold are screened and merged to obtain updated frequent item sets.

[0048] In another possible implementation manner of the first aspect, the target association rule includes a conclusion feature, and determining whether there is a new coal mine disaster feature according to the target association rule includes:

[0049] If there is a feature in the conclusion feature that is not included in the p coal mine disaster feature sets, it is determined that the newly added coal mine disaster feature exists, wherein the feature that is not included in the p coal mine disaster feature sets is the newly added coal mine disaster feature.

[0050] In another possible implementation of the first aspect, the coal mine disaster prediction model is a gradient boosting regression tree.

[0051] In a second aspect, the present application provides a device for hierarchical prediction of coal mine disaster heterogeneous information based on big data, comprising:

[0052] a memory configured to store instructions; and

[0053] The processor is configured to call the instructions from the memory and implement the above-mentioned coal mine disaster heterogeneous information classification prediction method based on big data when executing the instructions.

[0054] In a third aspect, the present application provides an electronic device, including:

[0055] The above-mentioned coal mine disaster heterogeneous information hierarchical prediction device based on big data.

[0056] Through the above technical solution, the monitoring data of different data sources are obtained through the Hadoop platform, and data fusion is performed to obtain a comprehensive data set. Then, the comprehensive data set is divided into multiple data sets and scattered data according to the node table structure, and these data sets are mapped to the corresponding storage nodes. The data sets in the storage nodes are analyzed by the sub-computing nodes to obtain the coal mine disaster feature set, and the merged data set is analyzed to obtain the fusion feature set. Furthermore, it is determined whether there are new coal mine disaster features through association analysis, and in the case of new features, the storage nodes and sub-computing nodes are expanded to ensure the expansion of data processing capabilities and the integrity of data storage. Finally, the coal mine disaster feature set and the new features are input into the pre-trained coal mine disaster prediction model to obtain the coal mine disaster level. Not only does it effectively improve the speed of data processing and analysis, but it can also effectively ensure the integrity of the data source and the accuracy of the prediction model, thereby effectively improving the accuracy of coal mine disaster prediction.

[0057] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a flow chart of a method for hierarchical prediction of coal mine disaster heterogeneous information based on big data provided in an embodiment of the present application;

[0059] Figure 2 A schematic diagram of a mapping process provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of a data partitioning process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0062] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0063] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0064] Figure 1 The following schematically shows a flow chart of a method for hierarchical prediction of coal mine disaster heterogeneous information based on big data according to an embodiment of the present application. Figure 1As shown, an embodiment of the present application provides a method for hierarchical prediction of heterogeneous information of coal mine disasters based on big data, which is applied to an electronic device, and the electronic device is deployed with a Hadoop platform. The Hadoop platform includes a big data analysis component and a distributed data warehouse. The big data analysis component includes at least one main computing node and m sub-computing nodes. The distributed storage database includes storage nodes corresponding to the sub-computing nodes one by one, and m is a positive integer greater than or equal to 2. The method may include the following steps.

[0065] S110, acquiring monitoring data from different data sources through the Hadoop platform, wherein the monitoring data includes sensor data, video surveillance data, equipment operation data, and geological exploration data;

[0066] S120, fusing sensor data, video surveillance data, equipment operation data, and geological exploration data to obtain a comprehensive data set;

[0067] S130, dividing the comprehensive data set into p data sets and n scattered data according to the node table structure of each storage node, where p is less than or equal to m, and n is an integer greater than or equal to 0;

[0068] S140, mapping the p data sets to the m storage nodes one by one, so as to store the p data sets in the corresponding p target storage nodes;

[0069] S150, merging the mp data sets with the n scattered data to obtain a merged data set;

[0070] S160, using p target sub-computing nodes corresponding to the target storage nodes one by one among the m sub-computing nodes, performing big data analysis on the data sets in the corresponding target storage nodes, and obtaining p coal mine disaster feature sets;

[0071] S170, using the main computing node to perform big data analysis on the merged data set to obtain a fusion feature set;

[0072] S180, performing correlation analysis on the p coal mine disaster feature sets and the fusion feature set to determine whether there are new coal mine disaster features;

[0073] S190. In the case of newly added coal mine disaster features, expand the storage node and the corresponding sub-computing node so that the number of storage nodes and the number of sub-computing nodes are both increased by 1, and store the newly added coal mine disaster features in the expanded storage node;

[0074] S1100, inputting p coal mine disaster feature sets and newly added coal mine disaster features into a pre-trained coal mine disaster prediction model to obtain a coal mine disaster level.

[0075] The Hadoop platform is used to obtain monitoring data from different data sources, including sensor data, video surveillance data, equipment operation data, and geological exploration data. In specific implementations, sensor data can come from sensors such as temperature, humidity, and gas concentration installed at key locations in coal mines, video surveillance data comes from cameras installed inside and outside the mine, equipment operation data records the operating status and fault information of coal mine machinery and equipment, and geological exploration data includes information such as mine structure and geological faults. Through the distributed data collection function of the Hadoop platform, the latest data can be automatically obtained from these different data sources in real time and aggregated into a distributed storage database.

[0076] Data fusion is performed on sensor data, video surveillance data, equipment operation data and geological exploration data to obtain a comprehensive data set. The process of data fusion includes data preprocessing, format unification and data integration. Specifically, various types of data can be preprocessed first to remove noise, fill missing values, and align data by timestamp. Then, data in different formats are converted into a unified format, such as converting video data into a frame sequence and standardizing sensor data and equipment operation data. Finally, the preprocessed data are integrated into a comprehensive data set so that each data record contains sensor readings, video frame information, equipment status and geological information. In the specific implementation, data fusion can be achieved through the MapReduce programming model of the Hadoop platform.

[0077] Figure 3 A schematic diagram of a data partitioning process provided by an embodiment of the present application is shown. According to the node table structure of each storage node, the comprehensive data set can be divided into p data sets and n scattered data, where p is less than or equal to m, and n is an integer greater than or equal to 0. Figure 3 The circular area where data 7, data 1, and data 10 are located represents a data set, and scattered data refers to data 2, data 4, data 6, and data n.

[0078] In this embodiment, the node table structure of each storage node is used to characterize one or more types of coal mine disaster characteristics. For example, if the node table structure of storage node A is used to characterize the gas pressure type and gas outflow type in gas disasters, it indicates that the data stored in storage node A is data related to the gas pressure type and gas outflow type in gas disasters.

[0079] Specifically, the node table structure predefines the type of data stored in each storage node. For example, storage node A is used to store gas disaster data, including gas pressure and gas outflow; storage node B is used to store fire data, including temperature and smoke concentration. During the data partitioning process, the data features in the comprehensive data set are first analyzed, and the data that conforms to the node table structure is classified into the corresponding data set. Data that cannot be matched to any node table structure is marked as scattered data. This solution can ensure that the content of each data set is consistent with the table structure of its storage node.

[0080] Reference Figure 2 and Figure 3 , p data sets are mapped one by one to m storage nodes, so that the p data sets are stored in the corresponding p target storage nodes. This embodiment first determines which storage nodes have node table structures that match the data sets, and then stores the data sets in the corresponding storage nodes according to the matching relationship. For example, if data set 1 contains gas pressure and gas outflow data, it is mapped to storage node A; data set 2 contains temperature and smoke concentration data, it is mapped to storage node B. Through this mapping, the distributed storage capabilities of the Hadoop platform can be fully utilized to improve the efficiency of data storage and access.

[0081] The mp data sets are merged with the n scattered data to obtain a merged data set. The merged data set includes all types of data from different data sources, which can provide more dimensional information for subsequent big data analysis.

[0082] Using p target sub-computing nodes among the m sub-computing nodes that correspond one-to-one to the target storage nodes, perform big data analysis on the data sets in the corresponding target storage nodes to obtain p coal mine disaster feature sets. Each sub-computing node is responsible for processing a data set in a target storage node, and uses big data analysis techniques, such as machine learning algorithms, to conduct in-depth analysis of the data. For example, sub-computing node A analyzes the gas data in storage node A to extract features such as gas concentration change trends and gas pressure anomalies; sub-computing node B analyzes the temperature and smoke data in storage node B to identify fire hazards, etc. Through distributed computing, this embodiment can efficiently process large-scale data and extract valuable coal mine disaster features.

[0083] The main computing node is used to perform big data analysis on the merged data set to obtain a fusion feature set. The main computing node is responsible for processing the merged data set and extracting fusion features. For example, the main computing node can process the merged data set through feature selection technology in big data analysis to extract fusion features, and then obtain a fusion feature set, where the fusion feature set refers to a set of fusion features finally obtained after performing big data analysis on the merged data set. The fusion features are extracted and screened based on the merged data set and have a high correlation. The fusion feature set obtained in this embodiment can provide comprehensive and accurate basic data for coal mine disaster prediction and improve the performance and reliability of the prediction model.

[0084] The p coal mine disaster feature sets are associated with the fusion feature set to determine whether there are new coal mine disaster features. During the association analysis, new disaster features are identified by comparing and associating each feature set. New features may include previously undiscovered risk factors or new disaster patterns. This solution can update and expand the disaster feature library in a timely manner to ensure the accuracy and timeliness of the disaster prediction model.

[0085] In the case of newly added coal mine disaster characteristics, the storage nodes and the corresponding sub-computing nodes are expanded so that the number of storage nodes and the number of sub-computing nodes are both increased by 1, and the newly added coal mine disaster characteristics are stored in the expanded storage nodes. In specific implementation, it is necessary to first configure new storage nodes and sub-computing nodes, then import the newly added coal mine disaster characteristic data into the new storage nodes, and configure the corresponding sub-computing nodes to process these data. Through the dynamic expansion mechanism, it can flexibly respond to changes in data volume and data type, ensuring the scalability and adaptability of the system.

[0086] The p coal mine disaster feature sets and the newly added coal mine disaster features are input into the pre-trained coal mine disaster prediction model to obtain the coal mine disaster level. The prediction model is pre-trained based on historical data and expert knowledge, and can evaluate and predict the input feature data. First, all feature data are standardized to ensure that they meet the input requirements of the model. Then, the standardized feature data is input into the prediction model. The model outputs the prediction level of coal mine disasters, such as low, medium, and high risks, through calculation and analysis, to warn potential coal mine disasters in advance and guide mine safety management and emergency response.

[0087] Specifically, the coal mine disaster prediction model is a Gradient Boosting Regression Tree (GBRT) model. The Gradient Boosting Regression Tree model can gradually reduce errors and improve the accuracy of coal mine disaster prediction by integrating multiple decision trees. The training steps of the Gradient Boosting Regression Tree model are as follows:

[0088] S1. Initialize the gradient boosting regression tree model; S2. Optimize the hyperparameters of the gradient boosting regression tree model through random search, where the hyperparameters include learning rate, number of trees and maximum depth; S3. Use the training set to train the gradient boosting regression tree model, and use the validation set to evaluate the gradient boosting regression tree model; S4. Adjust the hyperparameters based on the evaluation results, and execute S2 to S4 in a loop until the performance of the gradient boosting regression tree model is optimal, thus completing the training of the gradient boosting regression tree model.

[0089] In summary, this embodiment uses the powerful data processing capabilities of the Hadoop platform to achieve hierarchical prediction of heterogeneous information on coal mine disasters. By integrating and analyzing monitoring data from different data sources, the characteristics of coal mine disasters can be extracted comprehensively and accurately. Through distributed computing and dynamic expansion mechanisms, the efficiency and flexibility of the system are ensured. Ultimately, through the prediction model, potential coal mine disasters can be identified and warned in advance, the level of mine safety management can be improved, and the safety of miners' lives can be guaranteed.

[0090] This embodiment obtains monitoring data from different data sources through the Hadoop platform, and performs data fusion to obtain a comprehensive data set. Then, the comprehensive data set is divided into multiple data sets and scattered data according to the node table structure, and these data sets are mapped to the corresponding storage nodes. The data set in the storage node is analyzed by the sub-computing node to obtain a coal mine disaster feature set, and the merged data set is analyzed to obtain a fusion feature set. Further, it is determined whether there are new coal mine disaster features through association analysis, and in the case of new features, the storage nodes and sub-computing nodes are expanded to ensure the expansion of data processing capabilities and the integrity of data storage. Finally, the coal mine disaster feature set and the new features are input into the pre-trained coal mine disaster prediction model to obtain the coal mine disaster level. Not only the speed of data processing and analysis is effectively improved, but also the integrity of the data source and the accuracy of the prediction model can be effectively ensured, thereby effectively improving the accuracy of coal mine disaster prediction.

[0091] In one implementation of this embodiment, data fusion is performed on sensor data, video monitoring data, equipment operation data and geological exploration data to obtain a comprehensive data set, including the following steps:

[0092] S210, extracting semantic features of sensor data, video surveillance data, equipment operation data, and geological exploration data respectively;

[0093] S220, semantically associating all semantic features to obtain a plurality of associated semantic feature pairs and at least one scattered semantic feature;

[0094] S230, performing scene association on the multiple associated semantic feature pairs and the at least one scattered semantic feature to obtain associated semantic feature pairs and scattered semantic features for each coal mine scene;

[0095] S240. Merge the associated semantic feature pairs and scattered semantic features of each coal mine scene, wherein the comprehensive dataset is used to characterize the associated semantic feature pairs and scattered semantic features of all coal mine scenes.

[0096] In the process of extracting semantic features from sensor data, video surveillance data, equipment operation data, and geological exploration data, different types of data need to be preprocessed first. For sensor data, meaningful features such as temperature, humidity, pressure, etc. can be extracted through signal processing techniques such as filtering and denoising. Specifically, Fourier transform can be used to convert time domain signals into frequency domain signals, from which frequency features can be extracted. For video surveillance data, computer vision techniques such as convolutional neural networks (CNNs) are used to analyze video frames and extract features such as object detection and motion trajectory. For example, using the pre-trained YOLO model, targets such as people and equipment in the video can be detected in real time, and their location information and motion trajectory can be extracted.

[0097] For equipment operation data, time series analysis methods, such as the autoregressive moving average model (ARMA), can be used to extract the changing patterns of equipment operation status and abnormal detection features. For example, by analyzing the vibration signal of the equipment, the normal operation status and fault status of the equipment can be identified. For geological exploration data, geological modeling and spatial analysis techniques, such as Kriging interpolation, can be used to extract geological strata, ore body distribution and other features. By interpolating geological data at different depths and locations, a geological profile can be generated, and the spatial distribution characteristics of the ore body can be extracted from it.

[0098] All extracted features are processed semantically, that is, the original data is converted into semantic features with practical significance. For example, temperature, humidity, etc. in sensor data can be converted into semantic labels such as "high temperature" and "low humidity"; object detection results in video surveillance data can be converted into semantic descriptions such as "personnel approaching dangerous areas"; abnormal detection results in equipment operation data can be converted into semantic information such as "equipment failure warning"; ore body distribution characteristics in geological exploration data can be converted into semantic labels such as "ore body concentration area". Through semantic processing, different types of data can have a unified semantic expression, which is convenient for subsequent semantic association and scene association processing.

[0099] In the process of semantically associating all the extracted semantic features, a semantic association model needs to be constructed first. The semantic association model can define the association rules between different semantic features based on the knowledge graph method. For example, a coal mine safety knowledge graph can be constructed, where nodes represent different semantic features, such as "high temperature", "low humidity", "personnel approaching dangerous areas", "equipment failure warning", etc., and edges represent the association relationship between features, such as there may be a causal relationship between "high temperature" and "equipment failure warning"; all semantic features are analyzed using a semantic association model. The specific implementation method includes the following steps: first, semantic matching is performed on each semantic feature to find its corresponding node in the knowledge graph; then, according to the association rules in the knowledge graph, other features related to the current feature are identified. For example, when the "high temperature" feature is detected, the "equipment failure warning" feature associated with it can be identified based on the knowledge graph. For each pair of associated features, an associated semantic feature pair is generated.

[0100] In actual operation, you can use rule-based reasoning engines, such as RDF4J, to implement semantic association analysis. The reasoning engine can automatically deduce the association relationship between features based on predefined rules. In addition, you can also use machine learning methods, such as association rule mining, to automatically learn the association relationship between features from historical data.

[0101] Features that do not have a clear association relationship in the knowledge graph are called scattered semantic features. Although scattered semantic features have no direct association, they may still be of great significance in specific scenarios. For example, "personnel approaching a dangerous area" may not be directly associated with other features in the current scenario, but it is still an important independent feature in coal mine safety monitoring. In other words, associated semantic feature pairs represent direct associations between features, and scattered semantic features represent independent important features.

[0102] In the process of scene association of multiple associated semantic feature pairs and at least one scattered semantic feature, it is first necessary to define the classification standard of coal mine scenes. Coal mine scenes can be classified according to different working areas, operating states, environmental conditions, etc. For example, coal mine scenes can be divided into different scenes such as "mining working face", "transportation tunnel", "equipment maintenance area", etc., each of which has specific environmental and operating characteristics.

[0103] Next, scene matching is performed on the semantic features in each scene. The specific implementation method includes the following steps: First, according to the scene classification standard, all semantic features are assigned to the corresponding scenes. For example, the "high temperature" feature may appear in the "mining working face" scene, and the "equipment failure warning" feature may appear in the "equipment maintenance area" scene. Then, cluster analysis is performed on the semantic features in each scene to identify the association patterns between the features. For example, in the "mining working face" scene, it may be found that there is a correlation between "high temperature" and "low humidity", while in the "equipment maintenance area" scene, it may be found that there is a correlation between "equipment failure warning" and "personnel approaching dangerous areas".

[0104] In actual operation, clustering algorithms, such as K-means, can be used to perform cluster analysis on the semantic features in each scene. Through cluster analysis, the association patterns between features can be identified, and the associated semantic feature pairs and scattered semantic features of each scene can be generated. For example, in the "mining face" scene, the "high temperature-low humidity" associated semantic feature pairs and the "ore body concentration area" scattered semantic features may be generated.

[0105] In addition, scene recognition models, such as image classification models based on convolutional neural networks (CNNs), can be used to perform scene recognition on video surveillance data. Through scene recognition models, the scene types in the video can be automatically identified and the corresponding semantic features can be assigned to the corresponding scenes. For example, by analyzing the video frames, it is possible to identify whether the current scene is a "mining face" or a "transportation tunnel" and assign the corresponding semantic features to the corresponding scene.

[0106] In the process of merging the associated semantic feature pairs and scattered semantic features of each coal mine scene, it is first necessary to build a framework for a comprehensive data set. The framework can define the feature storage structure under different scenarios based on data warehouse technology. For example, a table can be created for each coal mine scene, which contains all the associated semantic feature pairs and scattered semantic features of the scene. Next, the feature data of each scene is filled into the corresponding table. The specific implementation method includes: first, encoding the associated semantic feature pairs of each scene and converting them into a format suitable for storage. For example, the associated semantic feature pairs can be stored in JSON format, where each feature pair is represented as a key-value pair, the key represents the feature name, and the value represents the feature value; the scattered semantic features of each scene are encoded and converted into a format suitable for storage. For example, the scattered semantic features can be stored in CSV format, where each feature is represented as a row of data, and the column represents the feature name and value.

[0107] In actual operation, data integration tools can be used to integrate data from different sources into a comprehensive data set to automate the data extraction, transformation, and loading (ETL) process, convert data in different formats into a unified storage format, and load it into a comprehensive data set. For example, data flows can be defined through Apache NiFi to extract sensor data, video surveillance data, equipment operation data, and geological exploration data from different data sources, perform semantic processing and encoding, and then load them into a comprehensive data set.

[0108] In summary, a comprehensive dataset containing all coal mine scenarios can be constructed. The comprehensive dataset contains the associated semantic feature pairs and scattered semantic features of each scenario, which can be used to characterize the safety risks and operating status in different scenarios. The comprehensive dataset can provide comprehensive data support for subsequent coal mine disaster prediction and safety management, helping to better understand and respond to potential risks in coal mines.

[0109] This implementation method firstly converts different types of data into semantic features with practical significance through semantic feature extraction, which is convenient for subsequent processing and analysis. Then, through semantic association analysis, the association relationship between features can be identified, and multiple associated semantic feature pairs and scattered semantic features can be generated. Then, through scene association analysis, features can be assigned to different coal mine scenes to generate feature data for each scene. Finally, by merging feature data, a comprehensive data set containing all scenes can be constructed. This comprehensive data set can provide comprehensive data support for coal mine disaster prediction and safety management, help better understand and respond to potential risks in coal mines, and improve the safety and production efficiency of coal mines.

[0110] In one implementation of this embodiment, p target sub-computing nodes corresponding to the target storage nodes among the m sub-computing nodes are used to perform big data analysis on the data sets in the corresponding target storage nodes to obtain p coal mine disaster feature sets, including the following steps:

[0111] S310, using the target sub-computing node, extracting at least one coal mine scene from the corresponding target storage node, and obtaining the coal mine disaster semantics corresponding to each coal mine scene;

[0112] S320, extracting and summarizing key features of data sets in target storage nodes corresponding to coal mine scenarios one by one, and obtaining p key feature sets;

[0113] S330. In each key feature set, according to the corresponding coal mine disaster semantics, determine and summarize the coal mine disaster features to obtain a coal mine disaster feature set.

[0114] First, a target sub-computing node is used to extract at least one coal mine scene from the corresponding target storage node, and the coal mine disaster semantics corresponding to each coal mine scene are obtained. Specifically, the target sub-computing node is a computing unit used to extract data from the target storage node. The data set stored in the target storage node may include geological data of the operating area, operation records of miners, operating status of equipment, etc. Each coal mine scene can be regarded as a specific operating area or geographical location, and each scene has corresponding coal mine disaster semantics, which refers to the potential disaster types and characteristics related to the scene. For example, operating area 10 may face the risk of gas explosion, while operating area 20 may have the risk of landslide. The target sub-computing node can identify each coal mine scene and extract the coal mine disaster semantics related to it.

[0115] Next, extract and summarize the key features of the data set in the target storage node that corresponds to the coal mine scene one by one, and obtain p key feature sets. Key features refer to parameters or indicators that can significantly affect coal mine safety, such as gas concentration, geological stress, temperature change, etc. In order to extract key features, the target sub-computing node can extract and summarize the key features of the data set in the target storage node that corresponds to the coal mine scene one by one by means of data cleaning, data normalization, and feature selection. Data cleaning is used to remove noise and incomplete data to ensure the accuracy of the analysis results; data normalization is used to convert data of different dimensions into the same range for easy comparison and analysis; feature selection is used to screen out the most representative features from a large amount of data.

[0116] In each key feature set, according to the corresponding coal mine disaster semantics, the coal mine disaster features can be determined and summarized to obtain a coal mine disaster feature set. Specifically, the extracted key features can be combined with the coal mine disaster semantics to identify potential disaster types and their characteristics. For example, if the key feature set of a coal mine scene contains high gas concentration and high temperature changes, it may be associated with the risk of gas explosion; if the key feature set of a coal mine scene contains geological stress concentration and crack expansion, it may be associated with the risk of landslide. By matching the key features with the coal mine disaster semantics, the specific disaster characteristics of each coal mine scene can be determined and summarized into a coal mine disaster feature set.

[0117] This implementation method can effectively extract and summarize coal mine disaster characteristics by using target sub-computing nodes to perform big data analysis on the data set in the target storage node. First, by extracting coal mine scenes and obtaining coal mine disaster semantics, the potential disaster types in each operating area can be identified; by extracting and summarizing key features, key features can be screened out from a large amount of data; by combining key features with coal mine disaster semantics, coal mine disaster characteristics can be determined and summarized, providing an important reference basis for coal mine safety management, so as to comprehensively and accurately identify potential disaster risks in coal mines and improve the accuracy of coal mine disaster prediction.

[0118] In one implementation of this embodiment, a main computing node is used to perform big data analysis on the merged data set to obtain a fusion feature set, including the following steps:

[0119] S410, obtaining the time attribute and space attribute of each data in the merged data set;

[0120] S420, performing data fusion on the data in the merged data set according to the time attribute and the spatial attribute of each data, to obtain a plurality of fused data;

[0121] S430: Use the main computing node to perform feature selection on all fused data and filter out redundant features to obtain a fused feature set.

[0122] In this embodiment, each data point in the merged data set contains time attributes and space attributes. The time attribute refers to the specific timestamp when the data is collected, such as the gas concentration value recorded by a sensor at 15:30:00 on June 26, 2024. The spatial attribute refers to the specific location where the data is collected, such as the location coordinates of a specific area or equipment in a mine. In the specific implementation process, it is necessary to extract the time attributes and space attributes from each data record in the merged data set. You can use data parsing tools or write scripts to traverse each record in the data set, read the timestamp field and location field, and store them in a new data structure, such as a list of tuples containing time and space information.

[0123] After obtaining the time and space attributes of each data, the data can be fused according to the time and space attributes. The purpose of data fusion is to integrate different types of data according to time and space to form a more comprehensive view. For example, at the same time and location, there may be multiple sensor data such as gas concentration, temperature, humidity, as well as video monitoring data and equipment operation data. Through data fusion, different types of data can be integrated to form a fused data point containing all relevant information. Specifically, this can be achieved through time windows and space grids. Time windows refer to grouping data according to a certain time interval, such as treating data within each minute as a time window. The space grid divides the mine area into several small spatial units, such as each 1 square meter area. Then, all data in the same time window and the same space grid are integrated to form a fused data point.

[0124] After completing data fusion, it is necessary to perform feature selection on the fused data. The purpose of feature selection is to filter out the most useful features for coal mine disaster prediction from a large amount of fused data, while removing redundant and irrelevant features, thereby simplifying the data and improving analysis efficiency. In this embodiment, a feature selection algorithm on the main computing node, such as the chi-square test method, is used to evaluate the importance of each feature. By calculating the correlation between each feature and the coal mine disaster event, the most helpful features for disaster prediction are screened out. For features with low correlation or redundancy, they are removed. Finally, a fused feature set containing the most useful features is obtained, providing a high-quality data foundation for subsequent coal mine disaster prediction.

[0125] This implementation method can realize the comprehensive processing and analysis of the merged data set and extract a high-quality fusion feature set. First, by extracting the time and space attributes of each data point, the accurate positioning and time synchronization of the data can be ensured. Then, different types of data are fused to form fused data containing multi-dimensional information. Finally, the most useful features for coal mine disaster prediction are screened out to obtain a fused data set, thereby improving the quality of the data and the efficiency of analysis. The obtained fusion feature set can provide comprehensive and accurate basic data for coal mine disaster prediction and improve the performance and reliability of the prediction model.

[0126] In one implementation of this embodiment, an evaluator is deployed on the main computing node, and the main computing node is used to perform feature selection on all fused data and filter out redundant features to obtain a fused feature set, including the following steps:

[0127] S510, extracting data features of all fused data using a main computing node;

[0128] S520, using the main computing node to generate a feature subset, and iteratively executing the first step until a preset number of iterations is reached, and determining the optimal performance index of the evaluator in the iteration, and using the feature subset corresponding to the optimal performance index as the fusion feature set;

[0129] The first step includes:

[0130] S530, randomly selecting b data features from all data features to add to the feature subset, where b is an integer greater than 1;

[0131] S540, using the feature subset to train the evaluator, obtaining the importance score of each sub-feature in the feature subset by the evaluator, and recording the performance index of the evaluator;

[0132] S550: When there is a sub-feature whose importance score is less than the first threshold, the sub-feature is treated as a redundant feature and is screened out to obtain an optimized feature subset.

[0133] In this embodiment, the main computing node is used to extract data features of all fused data. The data features may be temperature, humidity, air pressure, gas concentration, equipment vibration frequency, etc. The main computing node is used to generate a feature subset, and the first step is iteratively executed until the preset number of iterations is reached, and the optimal performance index of the evaluator in the iteration is determined, and the feature subset corresponding to the optimal performance index is used as the fused feature set. In the specific implementation, the main computing node repeatedly generates feature subsets, and the feature subsets are evaluated by the evaluator to screen out the optimal feature subset. First, the main computing node randomly selects b data features to generate a feature subset. In each round of iteration, the main computing node inputs the feature subset into the evaluator to obtain the importance score of each sub-feature in the feature subset. Among them, the evaluator may be a machine learning model. According to the importance score and the performance index of the evaluator, the main computing node screens out redundant features and optimizes the feature subset until the preset number of iterations is reached, and determines the feature subset corresponding to the optimal performance index. Among them, the feature subset corresponding to the optimal performance index is the fused feature set.

[0134] In this embodiment, the value of b can be adjusted according to the specific situation. For example, 2 features are selected in the first round of iteration and 4 features are selected in the second round of iteration to ensure the diversity of feature subsets and avoid bias towards certain specific features.

[0135] The evaluator is trained using a feature subset to obtain the importance score of each sub-feature in the feature subset and record the performance indicators of the evaluator. Specifically, the training process includes taking the feature subset as input data and training the evaluator to predict the target variable. Through training, the evaluator will score the importance of each sub-feature, and the importance score reflects the importance of each feature to the prediction result. At the same time, the overall performance indicators of the evaluator will also be recorded, such as accuracy, precision, recall, etc., to measure the overall performance of the evaluator.

[0136] In the case where there is a sub-feature whose importance score is less than the first threshold, the sub-feature is treated as a redundant feature and screened out to obtain an optimized feature subset. Specifically, the feature subset is optimized by screening out features with low importance scores. The main computing node can screen out sub-features with scores lower than the first threshold based on the evaluator's score for each sub-feature. Sub-features with scores lower than the first threshold are redundant features and contribute less to the prediction results. By screening out redundant features, the dimensionality of the feature subset can be reduced, and the training efficiency and prediction performance of the evaluator can be improved. At the same time, it also helps to avoid overfitting and enhance the generalization ability of the model. Finally, the screened and optimized feature subset will be used as the input for the next round of iteration to continue feature selection and evaluation.

[0137] It should be noted that in the first round of iteration, redundant features are not removed, and the number of data features added in each round of iteration may be inconsistent. That is, in the first round of iteration, 2 data features may be randomly selected to be added to the feature subset, and in the second round of iteration, 4 data features may be randomly selected to be added to the feature subset, and this application does not limit this.

[0138] This implementation method can effectively extract the most representative fusion feature set by using the main computing node to perform feature selection on all fused data and screen out redundant features. First, by extracting the data features of the fused data, the characteristics and indicators of the data can be fully understood. Then, by generating feature subsets and iteratively executing evaluator training, the importance of each feature can be evaluated and the optimal feature subset can be screened out. Finally, by screening out redundant features and optimizing feature subsets, the training efficiency and prediction performance of the evaluator can be improved, and the most representative fusion features can be comprehensively and accurately identified, providing an important reference for subsequent analysis and application.

[0139] In one implementation of this embodiment, association analysis is performed on p coal mine disaster feature sets and the fusion feature set to determine whether there are new coal mine disaster features, including the following steps:

[0140] S610, merging the p coal mine disaster feature sets and the fusion feature set into a first data set;

[0141] S620, calculating a first support for each feature in the first data set, where the first support is used to characterize the frequency with which the feature appears in the first data set;

[0142] S630, filtering and merging features whose first support is greater than or equal to a first support threshold to obtain frequent itemsets;

[0143] S640, looping through the second step until no updated frequent item sets can be generated, and obtaining a final frequent item set;

[0144] S650, generating association rules from the final frequent item set, and calculating the confidence of each association rule;

[0145] S660, taking the association rule corresponding to the confidence level greater than or equal to the confidence level threshold as the target association rule, and determining whether there is a new coal mine disaster feature according to the target association rule;

[0146] The second step includes:

[0147] S670, randomly combining features in the frequent item sets to form multiple multinomial sets, and calculating the second support of each multinomial set;

[0148] S680 , filter and merge multi-item sets whose second support is greater than or equal to the second support threshold to obtain updated frequent item sets.

[0149] In this embodiment, p coal mine disaster feature sets and fusion feature sets are first merged into a first data set. Specifically, the coal mine disaster feature set includes various features that may cause disasters in coal mines, such as gas concentration, temperature, humidity, equipment failure rate, etc.; and the fusion feature set is the feature set obtained by feature selection and screening out redundant features in the above steps. By merging the two feature sets, a more comprehensive data set can be obtained, which includes all potential features and fusion features of coal mine disasters.

[0150] Next, calculate the first support of each feature in the first data set. Support refers to the frequency of a feature appearing in the entire data set, which is used to measure the importance of the feature. The specific calculation method is to count the number of times each feature appears in the data set and then divide it by the total number of data sets. For example, if a feature appears 200 times in 1000 data, the support is 200 / 1000=0.2. By calculating the support, you can identify features that appear more frequently in the data set.

[0151] Then, the features whose first support is greater than or equal to the first support threshold are screened and merged to obtain frequent item sets. Specifically, a support threshold is set, such as 0.1, and features whose support is greater than or equal to 0.1 are screened. Features whose support is greater than or equal to 0.1 are considered to be frequently occurring features and may have an important impact on coal mine disasters. Frequent features are merged to form a frequent item set. Frequent item sets refer to frequently occurring feature combinations that may represent certain potential associations. For example, if the risk of coal mine disasters increases when both gas concentration and temperature are above a certain threshold, then the combination of gas concentration and temperature is a frequent item set.

[0152] The second step is executed repeatedly until the updated frequent item set cannot be generated, and the final frequent item set is obtained. Specifically, the second step is to randomly combine the features in the frequent item set to form multiple polynomials, and calculate the second support of each polynomial. The calculation method of the second support is similar to the first support. The number of times each polynomial appears in the data set is counted and then divided by the total number of items in the data set. The polynomials whose support is greater than or equal to the second support threshold are screened out to obtain the updated frequent item set. The above process is repeated until a new frequent item set cannot be generated. Through the above iterative method, the most representative frequent item sets can be gradually screened out, and the frequent item sets contain potential correlation characteristics of coal mine disasters.

[0153] Generate association rules from the final frequent item set and calculate the confidence of each association rule. Association rules refer to the association relationship between features in the data set, such as "if the gas concentration is high, the temperature is also high". By generating association rules from the final frequent item set, the potential relationship between features can be identified. Confidence refers to the probability that a certain association rule is established in the data set, which is used to measure the reliability of the association rule. The specific calculation method is to count the number of times the association rule appears in the data set at the same time, and then divide it by the number of times the previous feature appears. For example, if "high gas concentration" appears 200 times in 1,000 data, and "high gas concentration and high temperature" appear 150 times at the same time, then the confidence of the association rule is 150 / 200=0.75. By calculating the confidence, association rules with higher confidence can be screened out.

[0154] The association rules corresponding to the confidence greater than or equal to the confidence threshold are taken as the target association rules, and whether there are new coal mine disaster characteristics is determined according to the target association rules. Specifically, the confidence threshold is a criterion for screening association rules, which is usually set to a higher value, such as 0.7 or 0.8. The association rules with confidence greater than or equal to the threshold are considered to be credible and can be used as target association rules. Through the target association rules, it can be determined whether there are new coal mine disaster characteristics.

[0155] This implementation method can effectively identify and determine newly added coal mine disaster features by performing correlation analysis on p coal mine disaster feature sets and the fusion feature set, thereby facilitating the improvement of the comprehensiveness of the features and more effectively improving the accuracy of coal mine disaster prediction.

[0156] In one implementation of this embodiment, the target association rule includes a conclusion feature, and determining whether there is a new coal mine disaster feature according to the target association rule includes the following steps:

[0157] S710. If there is a feature in the conclusion feature that is not included in the p coal mine disaster feature sets, it is determined that there is a new coal mine disaster feature, wherein the feature that is not included in the p coal mine disaster feature sets is the new coal mine disaster feature.

[0158] In the process of determining whether there are new coal mine disaster features, if the conclusion feature included in the target association rule is not included in the p coal mine disaster feature sets, it can be determined that there are new coal mine disaster features. Specifically, the target association rule has a high confidence, indicating that it has a strong correlation and credibility in the data set. Each association rule is usually expressed as "if condition A occurs, then result B may also occur", where condition A and result B are both features or feature combinations.

[0159] Assume that in the target association rule, a rule is expressed as "If the gas concentration is high and the temperature rises, the humidity increases." In this rule, high gas concentration and increased temperature are condition A, and increased humidity is result B. Next, it is necessary to check whether the features in result B are already included in the p coal mine disaster feature sets. The specific implementation method can be achieved by traversing each result feature in the target association rule and comparing it with the p coal mine disaster feature sets. If it is found that a certain result feature, such as increased humidity, does not appear in the p coal mine disaster feature sets, it can be determined that increased humidity is a newly added coal mine disaster feature.

[0160] Specifically, the determination of newly added coal mine disaster features can be achieved through the following steps: 1. Traverse each target association rule and extract the conclusion feature features. For example, assuming there is a target association rule "If the gas concentration is high and the temperature rises, the humidity increases", then extract the conclusion feature feature "increased humidity". 2. Compare the extracted conclusion feature features with the p coal mine disaster feature sets. Assuming that the p coal mine disaster feature sets include feature A, feature B, feature C, etc., but do not include "increased humidity", it means that "increased humidity" is a newly added feature. 3. Confirm the features that are not included. For example, during the comparison process, if it is found that "increased humidity" is not in the p coal mine disaster feature sets, it will be marked as a newly added coal mine disaster feature.

[0161] This embodiment can effectively identify new features that may exist in the data set but are not included in the original feature set. It can not only expand the coal mine disaster feature set, but also help identify potential new risk factors. For example, by identifying the newly added feature "increased humidity", its correlation with other features (such as gas concentration and temperature) can be further studied, and targeted preventive measures can be taken.

[0162] This implementation method can effectively identify and determine the newly added coal mine disaster features by analyzing the conclusion feature characteristics in the target association rules and comparing them with the existing coal mine disaster feature set. It can not only enrich the feature set, but also reveal potential important information in the data, provide more comprehensive support for coal mine safety monitoring and early warning, and further increase the accuracy of coal mine disaster prediction.

[0163] In a second aspect, the present application provides a device for hierarchical prediction of coal mine disaster heterogeneous information based on big data, comprising:

[0164] a memory configured to store instructions; and

[0165] The processor is configured to call instructions from the memory and implement the above-mentioned coal mine disaster heterogeneous information classification prediction method based on big data when executing the instructions.

[0166] In a third aspect, the present application provides an electronic device, including:

[0167] The above-mentioned coal mine disaster heterogeneous information hierarchical prediction device based on big data.

[0168] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned coal mine disaster heterogeneous information classification prediction method based on big data.

[0169] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0170] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0173] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0174] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0175] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0176] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0177] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A hierarchical prediction method for coal mine disaster heterogeneous information based on big data, It is characterized in that Applied to an electronic device, the electronic device is deployed with a Hadoop platform, the Hadoop platform includes a big data analysis component and a distributed data warehouse, the big data analysis component includes at least one main computing node and m sub-computing nodes, the distributed data warehouse includes storage nodes corresponding to the sub-computing nodes one by one, m is a positive integer greater than or equal to 2, the method includes: Acquiring monitoring data from different data sources through the Hadoop platform, wherein the monitoring data includes sensor data, video surveillance data, equipment operation data, and geological exploration data; Performing data fusion on the sensor data, the video monitoring data, the equipment operation data and the geological exploration data to obtain a comprehensive data set; According to the node table structure of each storage node, the comprehensive data set is divided into p data sets and n scattered data, where p is less than or equal to m, and n is an integer greater than or equal to 0; Mapping the p data sets to the m storage nodes one by one, so as to store the p data sets in the corresponding p target storage nodes; Merging the mp data sets with the n scattered data to obtain a merged data set; Using p target sub-computing nodes corresponding to the target storage nodes one by one among the m sub-computing nodes, performing big data analysis on the data sets in the corresponding target storage nodes, and obtaining p coal mine disaster feature sets; Using the main computing node to perform big data analysis on the merged data set to obtain a fusion feature set; Performing correlation analysis on the p coal mine disaster feature sets and the fusion feature set to determine whether there are new coal mine disaster features; In the case of newly added coal mine disaster features, the storage node and the corresponding sub-computing node are expanded so that the number of storage nodes and the number of sub-computing nodes are both increased by 1, and the newly added coal mine disaster features are stored in the expanded storage node; Inputting the p coal mine disaster feature sets and the newly added coal mine disaster features into a pre-trained coal mine disaster prediction model to obtain a coal mine disaster level; The adopting the main computing node to perform big data analysis on the merged data set to obtain a fusion feature set includes: Acquire the time attribute and space attribute of each data in the merged data set; According to the time attribute and the space attribute of each data, data fusion is performed on the data in the merged data set to obtain a plurality of fused data; Using the main computing node to perform feature selection on all the fused data, and screen out redundant features to obtain a fused feature set; The main computing node is deployed with an evaluator, and the main computing node is used to perform feature selection on all the fused data and filter out redundant features to obtain a fused feature set, including: Using the main computing node to extract data features of all the fused data; The main computing node is used to generate a feature subset, and the first step is iteratively performed until a preset number of iterations is reached, and the optimal performance index of the evaluator in the iteration is determined, and the feature subset corresponding to the optimal performance index is used as the fusion feature set; Wherein, the first step comprises: Randomly selecting b data features from all the data features to add to the feature subset, where b is an integer greater than 1; Using the feature subset to train the evaluator, obtaining the importance score of each sub-feature in the feature subset by the evaluator, and recording the performance index of the evaluator; In the case where the importance score of the sub-feature is less than the first threshold, the sub-feature is taken as a redundant feature and is screened out to obtain an optimized feature subset.

2. The method according to claim 1, It is characterized in that The data fusion of the sensor data, the video monitoring data, the equipment operation data and the geological exploration data to obtain a comprehensive data set includes: respectively extracting semantic features of the sensor data, the video monitoring data, the equipment operation data, and the geological exploration data; Performing semantic association on all the semantic features to obtain a plurality of associated semantic feature pairs and at least one scattered semantic feature; Performing scene association on the plurality of the associated semantic feature pairs and the at least one scattered semantic feature to obtain the associated semantic feature pairs and the scattered semantic features of each coal mine scene; The associated semantic feature pairs and the scattered semantic features of each coal mine scene are merged, wherein the comprehensive data set is used to characterize the associated semantic feature pairs and the scattered semantic features of all the coal mine scenes.

3. The method according to claim 2, It is characterized in that The method uses p target sub-computing nodes corresponding to the target storage nodes one by one among the m sub-computing nodes to perform big data analysis on the data sets in the corresponding target storage nodes to obtain p coal mine disaster feature sets, including: Using the target sub-computing node, extracting at least one of the coal mine scenes from the corresponding target storage node, and obtaining the coal mine disaster semantics corresponding to each of the coal mine scenes; Extract and summarize key features of the data set in the target storage node corresponding to the coal mine scene one by one to obtain p key feature sets; In each of the key feature sets, coal mine disaster features are determined and summarized according to the corresponding coal mine disaster semantics to obtain a coal mine disaster feature set.

4. The method according to claim 1, It is characterized in that The step of performing correlation analysis on the p coal mine disaster feature sets and the fusion feature set to determine whether there are new coal mine disaster features includes: Merging the p coal mine disaster feature sets and the fusion feature set into a first data set; Calculating a first support for each feature in the first data set, where the first support is used to characterize the frequency of occurrence of the feature in the first data set; Filter and merge the features whose first support is greater than or equal to the first support threshold to obtain a frequent item set; The second step is executed repeatedly until no updated frequent item sets can be generated, and the final frequent item sets are obtained; Generate association rules from the final frequent item set, and calculate the confidence of each association rule; An association rule corresponding to a confidence level greater than or equal to a confidence level threshold is used as a target association rule, and whether there is a new coal mine disaster feature is determined according to the target association rule; Wherein, the second step comprises: Randomly combining the features in the frequent item set to form a plurality of multinomial sets, and calculating the second support of each of the multinomial sets; The multiple item sets whose second support is greater than or equal to the second support threshold are screened and merged to obtain updated frequent item sets.

5. The method according to claim 4, It is characterized in that The target association rule includes conclusion features, and determining whether there are new coal mine disaster features according to the target association rule includes: If there is a feature in the conclusion feature that is not included in the p coal mine disaster feature sets, it is determined that the newly added coal mine disaster feature exists, wherein the feature that is not included in the p coal mine disaster feature sets is the newly added coal mine disaster feature.

6. The method according to claim 1, It is characterized in that The coal mine disaster prediction model is a gradient boosting regression tree.

7. A device for classifying and predicting heterogeneous information of coal mine disasters based on big data, It is characterized in that include: a memory configured to store instructions; as well as A processor is configured to call the instruction from the memory and to implement the coal mine disaster heterogeneous information hierarchical prediction method based on big data according to any one of claims 1 to 6 when executing the instruction.

8. An electronic device, It is characterized in that include: A device according to claim 7.

Images