Mine yield prediction method, system and device, and storage medium
The method addresses the limitations of traditional mine production prediction by employing machine learning models to handle complex mine environments, resulting in accurate and efficient predictions through data preprocessing, feature extraction, and ensemble learning.
Patent Information
- Application Number
- CN202411350561.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2024-12-27
AI Technical Summary
Existing mine production prediction methods rely on traditional statistical approaches that fail to account for the complexity of mines and changing geological environments, leading to inaccurate and unreliable predictions, especially when dealing with multi-source and multi-dimensional data.
A comprehensive mine production prediction method using machine learning models, including data collection, preprocessing, feature extraction, and model optimization, to create an ensemble learning model that considers the complexity of mines and multi-source data, utilizing Gradient Boosting Trees or Random Forest algorithms, and cross-validation for optimal model selection.
The method provides objective, accurate, and reliable mine production predictions by analyzing multi-dimensional data, reducing manual intervention, and improving prediction efficiency and accuracy.
Smart Images
Figure CN119202669A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of mining, and in particular to a mine production prediction method, system and equipment, and storage medium. Background Art
[0002] With the continuous tightening of global natural energy, the operation and management of mines have received more and more attention. In the operation of mines, the output forecast of mines plays a vital role in guiding mine production, optimizing resource allocation, reducing safety risks and other actual business scenarios.
[0003] At present, the prediction of mine production mainly relies on traditional statistical methods and empirical formulas. Although these methods can achieve production prediction to a certain extent, they often ignore the complexity of mines and the changeable geological environment, resulting in inaccurate prediction results and unable to provide effective support for mine production and decision-making. In addition, the existing prediction methods can usually only process a single type of data, and lack effective processing methods for multi-source and multi-dimensional data, further reducing the prediction accuracy and reliability. Therefore, it is of great significance to develop a complete set of new automated production prediction methods that can comprehensively consider the complexity of mines and multi-source data, and have high accuracy and reliability, in order to improve the scientificity, efficiency, practicality and convenience of mine operation and management activities. Summary of the invention
[0004] In order to solve one of the above-mentioned technical defects, the present application provides a mine production prediction method, system and equipment, and storage medium.
[0005] According to a first aspect of the present application, a mine production prediction method is provided, comprising:
[0006] According to the final target of the output forecast of the mine to be tested, the corresponding data collection requirements are matched in the data requirement table; the data collection requirements corresponding to each final target of the output forecast are recorded in the data requirement table;
[0007] According to the matched data collection requirements, the multi-dimensional raw data of the mine to be tested is collected; wherein the data collection requirements include: data type, data source, data collection scope, data collection frequency, data collection priority and data quality; the multi-dimensional raw data includes: geological data, production data and environmental data;
[0008] Preprocess the collected multi-dimensional raw data of the mine to be tested to form a data set to be tested;
[0009] According to the feature extraction target table, features related to the mine output to be tested are extracted from the data set to be tested to form a feature set to be tested; the feature extraction target table records various types of features to be extracted related to the corresponding final target of output prediction in various data sets;
[0010] According to the final goal of the output prediction of the mine to be tested and the characteristics of the feature set to be tested, a matching operation is performed in the model library to obtain the optimal prediction model for the output of the mine to be tested;
[0011] The feature set to be tested is input into the optimal prediction model to make yield prediction, and the prediction results are output.
[0012] As a preferred solution of the first aspect, the yield prediction method further includes:
[0013] Establish a model library, including:
[0014] Collect all raw data related to target mine production;
[0015] Preprocess all the collected raw data;
[0016] According to the different final production forecasting goals of the target mines, data is selected from the pre-processed raw data to form different data sets corresponding to different final production forecasting goals;
[0017] Prepare backup prediction models according to different final production prediction targets of the target mines and data characteristics of data sets corresponding to the final production prediction targets; wherein one final production prediction target corresponds to multiple different backup prediction models;
[0018] Extract features related to the target mine output from each data set to form a feature set corresponding to each data set, and divide each feature set into a training set, a validation set, and a test set;
[0019] Input each training set into each backup prediction model corresponding to each final target of output prediction for training, and use the corresponding verification set to verify each backup prediction model during the training process, and optimize each backup prediction model according to the verification result;
[0020] Use the corresponding test set to evaluate the performance of each trained backup machine learning model on the same training set; the evaluation indicators include accuracy, recall, F1 value, AUC-ROC curve and mean square error;
[0021] The backup prediction models with the best evaluation results are used as the optimal prediction models for achieving the corresponding final goal of production prediction;
[0022] Finally, all the optimal prediction models and the final target of production prediction, data set characteristics and feature set characteristics corresponding to each optimal prediction model are stored to form a model library.
[0023] After a series of tests and verifications, the ensemble learning model based on machine learning is the optimal prediction model for mine production prediction behaviors with different final goals. Preferably, the specific ensemble algorithm used is Gradient Boosting Trees (GBT) or Random Forest (RF), which have been proven to be able to handle complex data relationships and have good generalization and robustness. In the training process, the model parameters are further adjusted through cross-validation technology to optimize the prediction performance of the model.
[0024] As a preferred solution of the first aspect, the mine output prediction method further includes:
[0025] Analyze the characteristics of the data sets corresponding to the final targets of each output prediction stored in the model library, and obtain the influencing factors of each data in each data set in the corresponding optimal prediction model;
[0026] According to each influencing factor, a data requirement table is formed.
[0027] As a preferred solution of the first aspect, the mine output prediction method further includes:
[0028] Analyze the characteristics of the data set and the feature set stored in the model library respectively, and obtain the influencing factors of each feature in the corresponding data set;
[0029] According to each influencing factor, a feature extraction target table is formed.
[0030] As a preferred solution of the first aspect, one or more of the following steps are performed to extract features related to the target mine output from each data set:
[0031] Calculate the statistics of the features contained in the data set, and select the features whose statistics are within the preset range for extraction;
[0032] Use a weighted method to calculate the weights of word features in text data, and select features whose weight values are within a preset range for extraction;
[0033] Use dimensionality reduction methods to project high-dimensional data into low-dimensional space, retain the main features of the data, and extract the main features;
[0034] Train the deep learning model to automatically learn the feature representation of the data, use unsupervised learning methods to capture the intrinsic structure of the data, and extract the corresponding features in the data set based on the output results;
[0035] Use filter-based feature selection methods to select features that are highly correlated with the target variable and extract them;
[0036] Using a wrapper-based feature selection method, the importance of features is evaluated based on the performance of the model and important features are extracted.
[0037] As a preferred solution of the first aspect, preprocessing the collected multi-dimensional raw data of the mine to be tested specifically includes:
[0038] Clean the collected multi-dimensional raw data of the mine to be tested;
[0039] Convert the cleaned data to make all data have a unified format and unit;
[0040] All converted data can be integrated into an overall data set by centrally storing all data, establishing data mapping relationships between different data sources, or interconnecting the business logic / API interfaces / middleware between different application systems.
[0041] As a preferred solution of the first aspect, the collected multi-dimensional raw data of the mine to be tested is cleaned, specifically including:
[0042] Identify missing values in the data, use methods based on similar records or machine learning to predict missing values, and fill in the prediction results;
[0043] Identify outliers in the data and use binning to process them;
[0044] Identify duplicate values in the data and deduplicate the duplicate data;
[0045] Identify the error values in the data, correct the error data, or convert the error values into missing values and then perform the missing value processing flow.
[0046] According to a second aspect of the present application, a mine production prediction system is provided, comprising a module for implementing the above-mentioned mine production prediction method.
[0047] According to a third aspect of the present application, a device is provided, comprising:
[0048] Memory;
[0049] Processor; and
[0050] Computer programs;
[0051] Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the above-mentioned mine production prediction method.
[0052] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored; the computer program is executed by a processor to implement the above-mentioned mine production prediction method.
[0053] The mine production prediction method provided in the present application comprehensively considers the complexity of the mine and the complexity of multi-source data, and uses a machine learning model to conduct in-depth analysis and mining of the collected multi-dimensional data to discover hidden laws and trends, and finally obtain objective, accurate and reliable mine production prediction results; before making a formal prediction, the present application first uses massive data from various aspects of the mine to train and optimize various theoretically suitable models, and finally selects the model with the best performance as the final prediction model, so that when making a formal prediction, the efficiency of the prediction process is improved and the accuracy of the prediction results is guaranteed; and, the present application also considers that the data requirements and feature requirements of the production prediction results will be different when used for different final goals or application scenarios, so in the process of model building, the corresponding data requirements and feature requirements are mined for different final goals or application scenarios, so that when making a formal prediction, the system can collect the original data in a targeted manner according to the specific target requirements, and extract the required features from the original data in a targeted manner, further improving the prediction efficiency and accuracy of the entire system.
[0054] The mine production prediction process provided in this application is a complete and fully automatic prediction process, which reduces the reliance on manual intervention. The process is efficient and the results are accurate, scientific and reliable. It is of great significance to improve the scientificity, efficiency, practicality and convenience of mine operation and management activities.
[0055] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the contents indicated in the written description and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0057] Figure 1The first functional structure diagram of the mine production prediction system provided for this application;
[0058] Figure 2 A second functional structure diagram of the mine production prediction system provided for this application;
[0059] Figure 3 for Figure 2 Schematic diagram of the functional structure of the model building module;
[0060] Figure 4 A third functional structure diagram of the mine production prediction system provided for this application;
[0061] Figure 5 The fourth functional structure diagram of the mine production prediction system provided for this application;
[0062] Figure 6 for Figure 3 Schematic diagram of the functional structure of the second feature extraction module;
[0063] Figure 7 for Figure 1 Schematic diagram of the functional structure of the first preprocessing module;
[0064] Figure 8 for Figure 7 Schematic diagram of the functional structure of the cleaning module;
[0065] In the figure: 10 is a data demand matching module, 20 is a first acquisition module, 30 is a first preprocessing module, 40 is a first feature extraction module, 50 is a model matching module, 60 is a yield prediction module, 70 is a model building module, 80 is a first analysis module, 90 is a data demand table building module, 100 is a second analysis module, 110 is a feature extraction target table building module, 301 is a cleaning module, 302 is a conversion module, 303 is an integration module, 3011 is a missing value processing module, 3012 is an outlier processing module, 3013 is a duplicate value processing module, 3014 is Error value processing module, 701 is the second acquisition module, 702 is the second preprocessing module, 703 is the training data preparation module, 704 is the backup model preparation module, 705 is the second feature extraction module, 706 is the feature set division module, 707 is the model training module, 708 is the model evaluation module, 709 is the optimal model determination module, 710 is the model library establishment module; 7051 is the statistical module, 7052 is the weighting module, 7053 is the dimensionality reduction module, 7054 is the deep learning module, 7055 is the first feature selection module, and 7056 is the second feature selection module. DETAILED DESCRIPTION
[0066] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.
[0067] In view of some problems existing in the prior art:
[0068] In the first aspect, the embodiments of the present application provide a method for predicting mine production, which can be executed by a mine production prediction system, or by components configured inside the mine production prediction system, such as a chip, a chip system, etc., or by a logic module or software having some or all of the functions of the mine production prediction system. The present application does not limit this.
[0069] Exemplarily, the mine production prediction method includes:
[0070] According to the final target of the output forecast of the mine to be tested, the corresponding data collection requirements are matched in the data requirement table; the data collection requirements corresponding to each final target of the output forecast are recorded in the data requirement table;
[0071] According to the matched data collection requirements, the multi-dimensional raw data of the mine to be tested is collected; wherein the data collection requirements include: data type, data source, data collection scope, data collection frequency, data collection priority and data quality; the multi-dimensional raw data includes: geological data, production data and environmental data;
[0072] Specifically, the ultimate goal of production forecasting can be to improve production efficiency, reduce safety risks, optimize resource allocation, etc.; geological data can be deposit types, rock properties, geological structures, etc., production data can be historical production data, equipment performance data, safety monitoring data, etc., and environmental data can be weather data, etc. The above types of raw data can be collected through sensors, drones, satellite remote sensing, etc. In actual implementation, appropriate collection equipment should be selected according to the collection objectives. These collection equipment should have high precision, high reliability and long-term stability, and their installation location, layout and connection method should be fully considered during deployment to ensure the accuracy and real-time nature of the collection. The data type can be structured data (such as tabular form), semi-structured data (such as JSON, XML), unstructured data (such as text, images), etc.; the data source can be database, API, web crawler, etc.; the data collection scope can be time range, geographical range, specific type, theme, etc.; the data collection frequency can be daily, weekly or instant collection according to demand to ensure the timeliness and update of the data; the setting of data collection priority can ensure that the system will allocate collection resources to more important data first, and improve the effectiveness and efficiency of the entire system in processing data; data quality assessment can be achieved by setting indicators and standards (such as accuracy, completeness, consistency and reliability, etc.) to ensure that the collected data meets subsequent analysis needs.
[0073] Preprocess the collected multi-dimensional raw data of the mine to be tested to form a data set to be tested;
[0074] According to the feature extraction target table, features related to the output of the mine to be tested are extracted from the data set to be tested to form a feature set to be tested; the feature extraction target table records various types of features to be extracted in various data sets that are related to the corresponding final goal of output prediction; specifically, the features related to the output of the mine to be tested are meaningful features that can reflect the complexity and key information of the mine to be tested, such as: geological characteristics, mining difficulty, equipment efficiency, etc.
[0075] More specifically, after extracting features related to the output of the mine to be tested from the data set to be tested, it can also include:
[0076] Use feature transformation methods or dimensionality reduction methods to simplify the feature space to improve the interpretability and generalization ability of the prediction model.
[0077] In practical applications: feature transformation technology can use function transformation (such as logarithmic transformation, square transformation, square root transformation, etc., used to transform data into a form closer to normal distribution) or quantile transformation (converting data distribution into uniform distribution or specified quantile distribution); dimensionality reduction technology can use principal component analysis (abbreviated as PCA: a commonly used linear dimensionality reduction technology that reduces data dimensions by finding the principal component in the data (i.e. the direction of maximum variance)) or non-negative matrix factorization (abbreviated as NMF: used to decompose a non-negative matrix into the product of two non-negative matrices, suitable for fields where only non-negative signals exist). In addition to the above two technologies, other technologies can also be used: such as linear discriminant analysis (LDA), generalized discriminant analysis (GDA), autoencoders, etc., according to specific needs. Before simplifying the feature space, the data needs to be preprocessed: according to the specific application technology, the data is subjected to necessary operations such as standardization, normalization, and removal of missing values. When performing specific operations to simplify feature space, for feature transformation, the data is converted from the original space to the new space according to the selected function or quantile method; for dimensionality reduction techniques, such as PCA, the covariance matrix of the data is first calculated, and then the eigenvectors and eigenvalues of the covariance matrix are found, and the eigenvectors corresponding to the first k largest eigenvalues are selected as the new feature space. After completing the simplification of the feature space, the effect of the transformation or dimensionality reduction can be evaluated and optimized: the effect can be evaluated by comparing the performance of the original data and the transformed or reduced data on the target task (such as classification accuracy, regression error, etc.); according to the evaluation results, the parameters of the feature transformation or dimensionality reduction technology (such as the number of principal components k in PCA) are adjusted to optimize the performance. It should be noted that for the features after dimensionality reduction, the meaning and importance of each new feature need to be explained in order to better understand the data and model.
[0078] According to the final goal of the output prediction of the mine to be tested and the characteristics of the feature set to be tested, a matching operation is performed in the model library to obtain the optimal prediction model for the output of the mine to be tested;
[0079] The feature set to be tested is input into the optimal prediction model to make yield prediction, and the prediction results are output.
[0080] Based on the above scheme, the mine production prediction method provided in the embodiment of the present application comprehensively considers the complexity of the mine and the complexity of multi-source data, and uses a machine learning model to conduct in-depth analysis and mining of the collected multi-dimensional data, discover hidden laws and trends, and finally obtain objective, accurate and reliable mine production prediction results; and, the embodiment of the present application also considers that the data requirements and feature requirements of the production prediction results will be different when used for different final goals or application scenarios, so in the process of model building, the corresponding data requirements and feature requirements are mined for different final goals or application scenarios, so that in the formal prediction, the system can collect the original data according to the specific target requirements, and extract the required features from the original data, further improving the prediction efficiency and accuracy of the whole system. The mine production prediction process provided in the embodiment of the present application is a complete set of fully automatic prediction processes, which reduces the degree of dependence on manual intervention, has an efficient process, and the results are accurate, scientific and reliable, which is of great significance to improving the scientificity, efficiency, practicality and convenience of mine operation and management activities.
[0081] In some possible implementations of the first aspect, the yield prediction method further includes:
[0082] Establish a model library, including:
[0083] Collect all raw data related to the target mine’s production. In this step, first identify all data sources generated during the target mine’s operation that may be related to its production, such as sensor data, geological data, equipment logs, personnel records, etc. Second, formulate a data collection strategy, including data collection methods, data collection frequency, data storage methods, and data transmission methods, including:
[0084] The data collection method can be determined by the following methods:
[0085] (1) Conduct a comprehensive survey of the entire target group of the mine to obtain the most comprehensive data; or, based on statistical principles, select a sample from the target group of the mine for investigation and infer the overall situation of the mine through the sample data;
[0086] (2) By conducting on-site observations of mines and participating in production and operation activities (such as observing the production and operation behaviors and production processes of the target mines), recording relevant data, and then combining other methods (such as questioning mine staff and collecting inquiry data) to improve the reliability of information;
[0087] (3) Designing experimental plans to obtain specific information or conclusions by controlling experimental conditions;
[0088] (4) Using resources such as libraries and databases, search for relevant literature on mine data collection methods to obtain the required information. This is generally divided into two methods: manual search and computer search;
[0089] (5) Through technical means such as web crawlers and API interfaces, various information about mine data collection is obtained from the Internet, which is suitable for large-scale data collection scenarios with high real-time requirements.
[0090] The following aspects need to be considered when determining the frequency of data collection:
[0091] (1) Acquisition purpose: For dynamic experiments or real-time monitoring, a higher acquisition frequency is required to capture instantaneous features; for steady-state experiments or signal analysis, a lower acquisition frequency can be selected;
[0092] (2) Capacity of the data acquisition system: Determine the appropriate acquisition frequency based on the sampling range of the data acquisition system to avoid data loss or distortion caused by exceeding the system capacity;
[0093] (3) Signal characteristics: If the signal bandwidth is narrow and only the low-frequency components are of interest, a lower acquisition frequency can be selected; if the signal has higher frequency components or the high-frequency characteristics need to be finely analyzed, a higher acquisition frequency needs to be selected;
[0094] (4) Data storage and processing capabilities: Too high an acquisition frequency may result in excessive data volume, increasing the difficulty of storage and processing. It is necessary to reasonably select the acquisition frequency to reduce data storage and processing costs while meeting experimental requirements.
[0095] Data storage can be done in the following ways:
[0096] (1) Online storage: It can provide the best data access convenience and is suitable for data that needs to be frequently accessed and modified. You can choose storage devices such as large disk arrays to apply this method.
[0097] (2) Near-line storage: It is suitable for archiving less frequently used data. In actual applications, it is relatively cheap and can be applied by selecting storage devices such as automatic tape libraries.
[0098] (3) Offline storage: used for permanent or long-term data storage. The media does not need to be online or connected to the storage system. You can choose offline storage media such as tapes and mobile hard disks to apply this method.
[0099] (4) Off-site protection: This method is to send important data to other sites to prevent the impact of disasters and ensure data security.
[0100] The data transmission methods can be as follows:
[0101] (1) Wired transmission: Wired transmission uses network cables, optical fibers and other wired media for data transmission. The transmission speed is fast and stable, and is suitable for scenarios with large data volumes and high transmission speed requirements.
[0102] (2) Wireless transmission: Using wireless technologies such as Wi-Fi and Bluetooth for data transmission is flexible and convenient, and is suitable for scenarios such as mobile devices and remote monitoring;
[0103] (3) Encrypted transmission: Data is encrypted during the transmission process to ensure data security; it is suitable for scenarios where sensitive information needs to be transmitted or data integrity needs to be protected.
[0104] Preprocess all the collected raw data;
[0105] According to the different final production forecasting goals of the target mines, data is selected from the pre-processed raw data to form different data sets corresponding to different final production forecasting goals;
[0106] Prepare backup prediction models according to different final production prediction targets of the target mines and data characteristics of data sets corresponding to the final production prediction targets; wherein one final production prediction target corresponds to multiple different backup prediction models;
[0107] Features related to the target mine output are extracted from each data set to form feature sets corresponding to each data set, and each feature set is divided into a training set, a validation set, and a test set; among them, features related to the target mine output are meaningful features that can reflect the complexity and key information of the target mine, such as geological characteristics, mining difficulty, equipment efficiency, etc.;
[0108] Input each training set into each backup prediction model corresponding to each final target of output prediction for training, and use the corresponding verification set to verify each backup prediction model during the training process, and optimize each backup prediction model according to the verification result;
[0109] Use the corresponding test set to evaluate the performance of each trained backup machine learning model on the same training set; the evaluation indicators include accuracy, recall rate, F1 value, AUC-ROC curve and mean square error; the performance evaluation of the model involves many aspects, including model complexity, generalization ability, training time, prediction accuracy, interpretability and robustness.
[0110] The backup prediction models with the best evaluation results are used as the optimal prediction models for achieving the corresponding final goal of production prediction;
[0111] Finally, all the optimal prediction models and the final target of production prediction, data set characteristics and feature set characteristics corresponding to each optimal prediction model are stored to form a model library.
[0112] Based on the above scheme, before the formal prediction is carried out, the embodiment of the present application first uses the massive data of various aspects of the mine to train and optimize various theoretically suitable models, and finally selects the model with the best performance as the final prediction model. In this way, when the formal prediction is carried out, the efficiency of the prediction process is improved and the accuracy of the prediction results is guaranteed;
[0113] Optionally, the mine output prediction method further includes:
[0114] Analyze the characteristics of the data sets corresponding to the final targets of each output prediction stored in the model library, and obtain the influencing factors of each data in each data set in the corresponding optimal prediction model;
[0115] According to each influencing factor, a data requirement table is formed.
[0116] Optionally, the mine output prediction method further includes:
[0117] Analyze the characteristics of the data set and the feature set stored in the model library respectively, and obtain the influencing factors of each feature in the corresponding data set;
[0118] According to each influencing factor, a feature extraction target table is formed.
[0119] Optionally, one or more of the following steps are performed to extract features related to the target mine output from each data set:
[0120] Calculate the statistics (such as mean, median, mode, variance, etc.) of the features contained in the data set to capture the distribution characteristics of the data, and select features whose statistics are within the preset range for extraction;
[0121] Use weighting methods (such as TF-IDF, etc.) to calculate the weights of term features in text data to reflect the importance of terms in the document, and select features whose weight values are within a preset range for extraction;
[0122] Use dimensionality reduction methods (such as principal component analysis (PCA) and linear discriminant analysis (LDA)) to project high-dimensional data into a low-dimensional space, retain the main features of the data, and extract the main features;
[0123] Train deep learning models (such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs)) to automatically learn the feature representation of data, use unsupervised learning methods (such as autoencoders) to capture the intrinsic structure of data, and extract corresponding features from the data set based on the output results;
[0124] Use filter-based feature selection methods (such as chi-square test, mutual information, etc.) to select features that are highly correlated with the target variable and extract them;
[0125] Using wrapper-based feature selection methods, the importance of features is evaluated based on the performance of the model (such as decision tree, random forest, etc.), and important features are extracted.
[0126] In some possible implementations of the first aspect, preprocessing the collected multi-dimensional raw data of the mine to be tested specifically includes:
[0127] Clean the collected multi-dimensional raw data of the mine to be tested, including processing missing values, outliers, duplicate values and error values;
[0128] Convert the cleaned data to a unified format and unit to ensure data consistency and facilitate subsequent analysis and processing;
[0129] All converted data can be integrated into an overall data set by centrally storing all data, establishing data mapping relationships between different data sources, or interconnecting the business logic / API interfaces / middleware between different application systems.
[0130] Specifically, if a method of establishing a data mapping relationship between different data sources is adopted, after the data mapping relationship is established, it can also include: creating a global data view or middleware to enable users to transparently access data from multiple data sources without having to worry about the actual storage location of the data.
[0131] Specifically, after the overall data set is formed, the following steps may also be performed: loading the data into a target database or file, establishing a data table to maintain the integrity of the data relationship; and then establishing an index for the data table to speed up data access and query efficiency.
[0132] In practical application:
[0133] The centralized storage of all data can be considered as a physical integration method. All data can be stored in one or more large data centers or cloud storage facilities to improve the efficiency and security of data access. In order to ensure the compatibility and manageability of data, it is preferred to use the same hardware devices and storage technologies. When migrating data from its original location to a new physical storage location, operations such as data copying, compression, and encryption may be involved.
[0134] The method of establishing data mapping relationships between different data sources can be regarded as a logical integration method. Mapping data is to ensure the logical consistency and comparability of data. Before mapping, the data needs to be standardized, including the unification of data format, coding, units, etc., so as to exchange and share data between different systems.
[0135] The way of interconnecting the business logic / API interface / middleware between different application systems can be regarded as an application integration method. The business logic and data flow of different application systems can be integrated to achieve cross-application business process automation and collaborative work; the functions and data of different application systems can also be connected through API interfaces to achieve real-time data sharing and interaction; middleware technology (such as ESB, BPM, etc.) can also be used as a bridge between different application systems to achieve data transmission, conversion and routing.
[0136] Physical integration focuses on the physical storage and hardware integration of mine data, while logical integration focuses on the integration of mine data models and data structures to ensure the logical consistency and comparability of data, while application integration focuses on the integration of mine business applications to achieve cross-application business process automation and collaborative work. When carrying out specific integration, due to the complexity and diversity of mine data, one of the above integration methods can be used according to actual needs, or cross-application can be used.
[0137] Optionally, the collected multi-dimensional raw data of the mine to be tested is cleaned, specifically including:
[0138] Identify missing values in the data, use similar record-based methods or machine learning methods to predict missing values, and fill in the predicted results to ensure data integrity; in practical applications, in addition to the preferred similar record-based methods and machine learning methods, some rules or statistics can also be used to estimate missing values. Common filling methods include using the mean, median, mode or common sense in a specific field to fill missing values. In addition to the filling method, deletion and interpolation methods can also be used to handle missing values. The deletion method is suitable for situations where the number of missing values is small or the impact on the analysis task is small. By deleting records with missing values, the analyzed data can be ensured to be complete, but this method may cause information loss, especially when the missing values are not randomly occurring, deletion may introduce bias. The interpolation method is suitable for filling missing values of continuous data. It uses the information of known data points to estimate the values of unknown points. For example, linear interpolation, polynomial interpolation or more complex statistical models such as regression analysis can be used to predict missing values. Due to the complexity and diversity of mining data, the use of similar record-based methods or machine learning methods will make the processing of missing values more accurate and efficient.
[0139] Identify the outliers in the data and use the binning method to process the outliers; in practical applications, in addition to the binning method, deletion, replacement and other means can also be used to process outliers. For extreme outliers, they can be deleted directly, but deletion of outliers requires caution, and this method must be based on sufficient domain knowledge and professional judgment. For less extreme outliers, you can choose to replace them. The replacement method can use the mean, median, mode or regression. However, due to the complexity and diversity of mining data, the outliers are assigned to adjacent boxes to reduce their impact. This processing can retain part of the information of the outliers, reduce their interference with the analysis results, and improve the integrity and reliability of mining data.
[0140] Identify duplicate values in the data and deduplicate the duplicate data to avoid data redundancy and repeated calculations. In actual applications, deleting or merging can be used to deduplicate data to ensure the uniqueness of the data.
[0141] Identify the erroneous values in the data, correct the erroneous data, or convert the erroneous values into missing values, and then execute the missing value processing flow; in actual application, if the correct information of the erroneous data can be uniquely determined, the erroneous values can be corrected by supplementing the correct information, such as re-importing the original correct data, or using other reliable information sources to correct the error. The erroneous data can also be regarded as missing values and wait for subsequent processing. If the erroneous data has no effect on the analysis or can be replaced by other data, you can choose to delete the record or feature directly.
[0142] When building a model, after collecting the raw data for training the model, the process of preprocessing all the raw data also involves data cleaning steps, conversion steps, and integration steps for processing missing values, outliers, duplicate values, and error values in the mine data. Although the data preprocessing when building a model and the data preprocessing when performing predictions are similar in means, the two have different focuses: the data preprocessing process when building a model focuses more on improving data quality and standardizing data formats to ensure that the data can play the greatest role in subsequent analysis and modeling processes; while the data preprocessing process when performing predictions focuses more on adjusting or converting data according to the needs of specific tasks or algorithms to better meet the requirements of the algorithm. Preprocessing the collected data helps to improve the validity and accuracy of the data, making subsequent data analysis and processing results more reliable.
[0143] In a second aspect, an embodiment of the present application provides a mine production prediction system, including a module for implementing the aforementioned mine production prediction method.
[0144] For example, Figure 1 As shown, the mine production prediction system includes:
[0145] Data requirement matching module 10: used to match the corresponding data collection requirements in the data requirement table according to the final target of the output prediction of the mine to be tested; the data collection requirements corresponding to each final target of the output prediction are recorded in the data requirement table;
[0146] The first acquisition module 20 is used to collect multi-dimensional raw data of the mine to be tested according to the matched data acquisition requirements; wherein the data acquisition requirements include: data type, data source, data acquisition scope, data acquisition frequency, data acquisition priority and data quality; the multi-dimensional raw data includes: geological data, production data and environmental data;
[0147] The first preprocessing module 30 is used to preprocess the collected multi-dimensional raw data of the mine to be tested to form a data set to be tested;
[0148] The first feature extraction module 40 is used to extract features related to the mine production to be measured from the data set to be measured according to the feature extraction target table to form a feature set to be measured; the feature extraction target table records various types of features to be extracted related to the corresponding final target of production prediction in various data sets;
[0149] Model matching module 50: used to perform matching operations in the model library according to the final target of the output prediction of the mine to be tested and the characteristics of the feature set to be tested, so as to obtain the optimal prediction model of the output of the mine to be tested;
[0150] Yield prediction module 60: used to input the feature set to be tested into the optimal prediction model to perform yield prediction, and output the prediction result.
[0151] In some possible implementations of the third aspect, such as Figure 2 As shown, the production prediction system further includes:
[0152] Model building module 70: used to build a model library;
[0153] like Figure 3 As shown, the model building module 70 includes:
[0154] The second acquisition module 701 is used to collect all raw data related to the target mine and production;
[0155] The second preprocessing module 702 is used to preprocess all the collected raw data;
[0156] Training data preparation module 703: used to select data from the pre-processed raw data according to different final targets of production prediction of target mines, so as to form different data sets corresponding to different final targets of production prediction;
[0157] The backup model preparation module 704 is used to prepare a backup prediction model according to different final production prediction targets of the target mine and data characteristics of data sets corresponding to each final production prediction target; wherein one final production prediction target corresponds to multiple different backup prediction models;
[0158] The second feature extraction module 705 is used to extract features related to the target mine output from each data set to form a feature set corresponding to each data set;
[0159] Feature set division module 706: used to divide each feature set into a training set, a validation set and a test set;
[0160] Model training module 707: used to input each training set into each backup prediction model corresponding to each final target of output prediction for training, and use the corresponding verification set to verify each backup prediction model during the training process, and optimize each backup prediction model according to the verification result;
[0161] Model evaluation module 708: used to evaluate the performance of each trained standby machine learning model on the same training set using the corresponding test set; wherein the evaluation indicators include accuracy, recall, F1 value, AUC-ROC curve and mean square error;
[0162] The optimal model determination module 709 is used to use each backup prediction model with the best evaluation result as the optimal prediction model for achieving the corresponding final goal of production prediction;
[0163] Model library building module 710: used to store all optimal prediction models and the final target of production prediction, data set characteristics and feature set characteristics corresponding to each optimal prediction model to form a model library.
[0164] Alternatively, if Figure 4 As shown, the production prediction system further includes:
[0165] The first analysis module 80 is used to analyze the characteristics of the data sets corresponding to the final targets of the production prediction stored in the model library, and obtain the influence factors of each data in each data set in the corresponding optimal prediction model;
[0166] Data requirement table establishment module 90: used to form a data requirement table according to various influencing factors.
[0167] Alternatively, if Figure 5As shown, the production prediction system further includes:
[0168] The second analysis module 100 is used to analyze the characteristics of the data set and the characteristics of the feature set stored in the model library, and obtain the influence factor of each feature in the corresponding data set;
[0169] Feature extraction target table establishment module 110: used to form a feature extraction target table according to various influencing factors.
[0170] Alternatively, if Figure 6 As shown, the second feature extraction module 705 includes:
[0171] Statistics module 7051: used to calculate the statistics of the features contained in the data set, and select the features whose statistics are within a preset range for extraction;
[0172] Weighting module 7052: used to calculate the weights of word features in text data using a weighting method, and select features whose weight values are within a preset range for extraction;
[0173] Dimensionality reduction module 7053: used to project high-dimensional data into low-dimensional space using a dimensionality reduction method, retain the main features of the data, and extract the main features;
[0174] Deep learning module 7054: used to train the deep learning model to automatically learn the feature representation of data, use unsupervised learning methods to capture the intrinsic structure of the data, and extract the corresponding features in the data set according to the output results;
[0175] The first feature selection module 7055 is used to select features with strong correlation with the target variable using a filter-based feature selection method and extract them;
[0176] The second feature selection module 7056 is used to use a wrapper-based feature selection method to evaluate the importance of features according to the performance of the model and extract important features.
[0177] In some possible implementations of the second aspect, such as Figure 7 As shown, the first pre-processing module 30 includes:
[0178] Cleaning module 301: used to clean the collected multi-dimensional raw data of the mine to be tested;
[0179] Conversion module 302: used to convert the cleaned data so that all data have a unified format and unit;
[0180] Integration module 303: used to integrate all converted data into an overall data set by centrally storing all data, establishing data mapping relationships between different data sources, or associating business logic / API interfaces / middleware between different application systems;
[0181] Alternatively, if Figure 8 As shown, the cleaning module 301 includes:
[0182] Missing value processing module 3011: used to identify missing values in the data, predict the missing values using a method based on similar records or a machine learning method, and fill in the prediction results;
[0183] Outlier processing module 3012: used to identify outliers in the data and process the outliers using a binning method;
[0184] Duplicate value processing module 3013: used to identify duplicate values in the data and perform deduplication processing on the duplicate data;
[0185] Error value processing module 3014: used to identify error values in data, correct the error data, or convert the error values into missing values and then execute the missing value processing flow.
[0186] On the third aspect, a device is provided in an embodiment of the present application. The device can be any device that can realize mine production prediction. The device can be various terminal devices, such as: desktop computers, laptops, tablet computers, handheld devices, etc., which can be implemented through software and / or hardware.
[0187] Exemplarily, the device comprises:
[0188] Memory;
[0189] Processor; and
[0190] Computer programs;
[0191] Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the mine production prediction method as described above.
[0192] In a fourth aspect, a computer-readable storage medium is provided in an embodiment of the present application. The computer-readable storage medium may be: ROM, RAM, a disk or an optical disk, etc.
[0193] Exemplarily, a computer program is stored on the computer-readable storage medium; the computer program is executed by a processor to implement the mine production prediction method as described above.
[0194] The mine production prediction method, system and related devices provided in this application integrate multi-dimensional data sets and adopt advanced machine learning algorithms. The model can capture complex data relationships and achieve accurate prediction of mine production. Since the model is trained based on a large amount of data, it has strong generalization capabilities and can adapt to the needs of different mine types and production methods. The system can collect new data in real time, update and optimize the model, and ensure the timeliness and accuracy of the prediction results. Integrated learning models such as gradient boosting trees and random forests usually have good interpretability, and users can understand how the model makes predictions, thereby increasing trust in the prediction results.
[0195] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, C language, VHDL language, Verilog language, object-oriented programming language Java and literal scripting language JavaScript, etc.
[0196] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0197] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0198] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0199] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0200] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0201] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A mine production prediction method, characterized in that: include: According to the final goal of the production forecast of the mine to be tested, the corresponding data collection requirements are matched in the data requirement table; The data demand table records the data collection requirements corresponding to the final target of each output forecast; According to the matched data collection requirements, the multi-dimensional raw data of the mine to be tested is collected; wherein the data collection requirements include: data type, data source, data collection scope, data collection frequency, data collection priority and data quality; the multi-dimensional raw data includes: geological data, production data and environmental data; Preprocess the collected multi-dimensional raw data of the mine to be tested to form a data set to be tested; According to the feature extraction target table, features related to the mine output to be tested are extracted from the data set to be tested to form a feature set to be tested; the feature extraction target table records various types of features to be extracted related to the corresponding final target of output prediction in various data sets; According to the final goal of the output prediction of the mine to be tested and the characteristics of the feature set to be tested, a matching operation is performed in the model library to obtain the optimal prediction model for the output of the mine to be tested; The feature set to be tested is input into the optimal prediction model to make yield prediction, and the prediction results are output.
2. The mine production prediction method according to claim 1, characterized in that: The yield prediction method further includes: Establish a model library, including: Collect all raw data related to target mine production; Preprocess all the collected raw data; According to the different final production forecasting goals of the target mines, data is selected from the pre-processed raw data to form different data sets corresponding to different final production forecasting goals; Prepare backup prediction models according to different final production prediction targets of the target mines and data characteristics of data sets corresponding to the final production prediction targets; wherein one final production prediction target corresponds to multiple different backup prediction models; Extract features related to the target mine output from each data set to form a feature set corresponding to each data set, and divide each feature set into a training set, a validation set, and a test set; Input each training set into each backup prediction model corresponding to each final target of output prediction for training, and use the corresponding verification set to verify each backup prediction model during the training process, and optimize each backup prediction model according to the verification result; Use the corresponding test set to evaluate the performance of each trained backup machine learning model on the same training set; the evaluation indicators include accuracy, recall, F1 value, AUC-ROC curve and mean square error; The backup prediction models with the best evaluation results are used as the optimal prediction models for achieving the corresponding final goal of production prediction; Finally, all the optimal prediction models and the final target of production prediction, data set characteristics and feature set characteristics corresponding to each optimal prediction model are stored to form a model library.
3. The mine production prediction method according to claim 2, characterized in that: The mine output prediction method also includes: Analyze the characteristics of the data sets corresponding to the final targets of each output prediction stored in the model library, and obtain the influencing factors of each data in each data set in the corresponding optimal prediction model; According to each influencing factor, a data requirement table is formed.
4. The mine production prediction method according to claim 2, characterized in that: The mine output prediction method also includes: Analyze the characteristics of the data set and the feature set stored in the model library respectively, and obtain the influencing factors of each feature in the corresponding data set; According to each influencing factor, a feature extraction target table is formed.
5. The mine production prediction method according to claim 2, characterized in that: Perform one or more of the following steps to extract features related to the target mine production from each data set: Calculate the statistics of the features contained in the data set, and select the features whose statistics are within the preset range for extraction; Use a weighted method to calculate the weights of word features in text data, and select features whose weight values are within a preset range for extraction; Use dimensionality reduction methods to project high-dimensional data into low-dimensional space, retain the main features of the data, and extract the main features; Train the deep learning model to automatically learn the feature representation of the data, use unsupervised learning methods to capture the intrinsic structure of the data, and extract the corresponding features in the data set based on the output results; Use filter-based feature selection methods to select features that are highly correlated with the target variable and extract them; Using a wrapper-based feature selection method, the importance of features is evaluated based on the performance of the model and important features are extracted.
6. The mine production prediction method according to claim 1, characterized in that: Preprocess the collected multi-dimensional raw data of the mine to be tested, including: Clean the collected multi-dimensional raw data of the mine to be tested; Convert the cleaned data to make all data have a unified format and unit; All converted data can be integrated into an overall data set by centrally storing all data, establishing data mapping relationships between different data sources, or interconnecting the business logic / API interfaces / middleware between different application systems.
7. The mine production prediction method according to claim 6, characterized in that: Clean the collected multi-dimensional raw data of the mine to be tested, including: Identify missing values in the data, use methods based on similar records or machine learning to predict missing values, and fill in the prediction results; Identify outliers in the data and use binning to process them; Identify duplicate values in the data and deduplicate the duplicate data; Identify the error values in the data, correct the error data, or convert the error values into missing values and then perform the missing value processing flow.
8. A mine production prediction system, characterized by: The invention comprises a module for implementing the mine production prediction method according to any one of claims 1 to 7.
9. A device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the mine production prediction method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: A computer program is stored thereon; the computer program is executed by a processor to implement the mine production prediction method as described in any one of claims 1 to 7.