Mine mining data acquisition method and system based on Internet of Things

Abstract

The invention discloses a mining data acquisition method and system based on the Internet of Things, and the method comprises the steps: obtaining multi-source data streams outputted by mine Internet of Things acquisition equipment, carrying out the classification of different parameter types, and obtaining an initial data set; according to the physical characteristics of each parameter in the initial data set, an adaptive threshold segmentation algorithm is adopted to determine the collection precision range of each type of parameters, and data groups with standardized precision are obtained; acquiring sliding window features of the time sequence from the filtered data sequence, and judging an abnormal value by adopting an isolated forest algorithm to obtain a data set after abnormities are eliminated; according to the data set with the exception removed, aiming at the timestamps and the spatial positions of the multi-source data, a weighted average algorithm is adopted to carry out normalization processing, and a data matrix with consistency integration is obtained; and obtaining a dynamic change trend of each parameter from the consistency integrated data matrix, and extracting key features by adopting a principal component analysis algorithm to obtain a feature data set after dimension reduction.

Description

Technical Field

[0001] The present invention belongs to the technical field of mining data collection, and in particular relates to a mining data collection method and system based on the Internet of Things. Background Art

[0002] Mining is a critical area for ensuring energy and resource supply, and data collection is directly related to production safety, efficiency optimization, and environmental management. With the rise of the Internet of Things (IoT), IoT-based data collection methods have become a crucial path to enhancing the intelligence of mining operations. However, existing methods suffer from significant shortcomings in practical applications. Many data collection systems lack unified accuracy standards, making it difficult to directly compare and integrate data collected by different devices. Furthermore, raw data often contains noise due to environmental interference or equipment errors, compromising the reliability of analysis. These limitations hinder mining from fully leveraging the potential of the IoT, necessitating more efficient solutions.

[0003] Numerical accuracy, data consistency, and exception handling are core challenges in the data collection process. First, numerical accuracy directly determines the credibility of the data, but the characteristics of different mining parameters vary significantly, making a single collection standard difficult to apply. Second, data consistency involves integrating data from multiple sources, and the lack of effective preprocessing mechanisms can lead to data distortion. Finally, detecting and filtering abnormal data is particularly difficult in complex environments, and unprocessed outliers can mislead decision-making. Failure to properly address these technical factors results in data quality that fails to meet the requirements of intelligent analysis, which in turn hinders the optimization and safety management of mining operations.

[0004] Therefore, how to design a unified collection accuracy standard for different mining parameters and implement data filtering, normalization and anomaly detection through efficient preprocessing algorithms to ensure the accuracy and consistency of the collected values ​​has become a key issue that needs to be urgently addressed in the mining data collection method based on the Internet of Things. Summary of the Invention

[0005] In order to solve the problems existing in the existing technology, the present invention provides a mining data collection method and system based on the Internet of Things, which solves the problem of consistent integration of multi-source heterogeneous data in mines, improves data quality and availability, and provides a reliable data foundation for subsequent mine safety monitoring and production decision-making, which has important practical value.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A method for collecting mining data based on the Internet of Things, the method comprising:

[0008] Obtain multi-source data streams output by mining IoT collection equipment, classify them according to different parameter types, and obtain the initial data set;

[0009] According to the physical characteristics of each parameter in the initial data set, an adaptive threshold segmentation algorithm is used to determine the acquisition accuracy range of each type of parameter and obtain data grouping with standardized accuracy;

[0010] For the data grouping with standardized accuracy, if it is detected that the data point deviates from the preset statistical distribution range, the noise interference is filtered out by the Kalman filter algorithm to obtain the filtered data sequence;

[0011] From the filtered data sequence, the sliding window features of the time series are obtained, and the isolation forest algorithm is used to determine the outliers to obtain the data set after removing the anomalies;

[0012] Based on the data set after removing anomalies, a weighted average algorithm is used to normalize the timestamps and spatial positions of multi-source data to obtain a consistent and integrated data matrix;

[0013] The dynamic change trend of each parameter is obtained from the consistent integrated data matrix, and the principal component analysis algorithm is used to extract key features to obtain the feature data set after dimensionality reduction.

[0014] Preferably, the multi-source data stream output by the mining IoT collection equipment is obtained and classified according to different parameter types to obtain an initial data set, including:

[0015] Acquire multi-source data streams from IoT devices and use a preset acquisition frequency to obtain real-time data sequences;

[0016] The real-time data series are classified according to the parameter type, and the initial data set is determined by the classification rules.

[0017] Preferably, based on the physical characteristics of each parameter in the initial data set, an adaptive threshold segmentation algorithm is used to determine the acquisition accuracy range of each type of parameter to obtain data grouping with standardized accuracy, including:

[0018] By analyzing the initial data set, the physical characteristics of each parameter are obtained and the basis for parameter classification is determined;

[0019] Based on the parameter classification basis, the statistical analysis method is used to obtain the distribution characteristics of each parameter;

[0020] According to the distribution characteristics, the adaptive threshold segmentation algorithm is used to determine the threshold range of each parameter;

[0021] If the threshold range meets the preset conditions, the acquisition accuracy range of each parameter is generated;

[0022] If it is not satisfied, adjust the threshold and re-segment to obtain the acquisition accuracy range;

[0023] By collecting the accuracy range, the initial data set is grouped and processed to generate accuracy standardized groups;

[0024] According to the accuracy standardization grouping, the normalization method is used to obtain standardized output data.

[0025] Preferably, for the precision-standardized data grouping, if it is detected that the data point deviates from the preset statistical distribution range, the noise interference is filtered out by the Kalman filter algorithm to obtain a filtered data sequence, including:

[0026] If the data grouping is completed, the precision standardization process is performed to obtain a standardized data sequence;

[0027] Based on the standardized data sequence, a statistical distribution model is used to determine the preset range;

[0028] If the data point deviates from the preset range, the degree of deviation is determined through distribution analysis;

[0029] According to the degree of deviation, Kalman filtering is used to obtain the filtered data sequence;

[0030] By filtering the data sequence, noise interference is detected and noise distribution characteristics are generated;

[0031] According to the noise distribution characteristics, the filtering parameters are adjusted to obtain the optimized data sequence;

[0032] By optimizing the data sequence, data groups that conform to the statistical distribution are generated.

[0033] Preferably, a sliding window feature of the time series is obtained from the filtered data sequence, and an isolation forest algorithm is used to determine outliers to obtain a data set after removing outliers, including:

[0034] From the filtered data, the time series is obtained, and the feature vector is extracted by the sliding window method using the preset window size to obtain the feature set;

[0035] By using the isolation forest algorithm, anomaly scores are calculated for the feature set to identify outliers;

[0036] If the anomaly score exceeds the preset threshold, the corresponding data point is removed to obtain a preliminary cleaned data set;

[0037] From the preliminary cleaned data set, obtain the time series, use statistical methods to calculate the stationarity of the series, and determine the stability of the data;

[0038] If the sequence stationarity does not meet the standard, the sequence is adjusted through difference processing to obtain a stable time series;

[0039] According to the stable time series, the sliding window is used to re-extract the feature vector to obtain the optimized feature set;

[0040] The anomaly score is calculated again for the optimized feature set using the isolation forest algorithm to obtain the final cleaned data set.

[0041] Preferably, based on the data set after removing anomalies, a weighted average algorithm is used to perform normalization processing on the timestamps and spatial positions of the multi-source data to obtain a consistent integrated data matrix, including:

[0042] Acquire multi-source data, remove outliers, and obtain a cleaned data set;

[0043] By extracting the timestamp and spatial location from the cleaned data set, a spatiotemporal feature set is generated;

[0044] The weighted average algorithm is used to normalize the spatiotemporal feature set to obtain a standardized feature matrix;

[0045] If the dimensions of the standardized feature matrices are consistent, they are integrated into a unified data matrix;

[0046] If the dimensions are inconsistent, adjust the weights and renormalize to obtain a unified data matrix;

[0047] According to the unified data matrix, the consistency deviation is detected to obtain the data matrix after the deviation correction;

[0048] For the bias-corrected data matrix, clustering algorithm is applied to group the data to obtain the classified data set;

[0049] Extract grouping features from the categorical data set to generate the final integrated data matrix.

[0050] Preferably, the dynamic change trend of each parameter is obtained from the consistent integrated data matrix, and the principal component analysis algorithm is used to extract key features to obtain a feature data set after dimensionality reduction, including:

[0051] Obtain a consistent integrated data matrix and obtain a normalized data matrix through standardization;

[0052] If there are missing values ​​in the normalized data matrix, the mean filling method is used to obtain a complete data matrix;

[0053] Extract the dynamic change trend of each parameter from the complete data matrix and use time series analysis to obtain the trend feature set;

[0054] For the trend feature set, the principal component analysis algorithm is used to calculate the principal component loading matrix and obtain the key eigenvectors;

[0055] According to the key feature vector, the principal component whose cumulative contribution rate reaches the preset threshold is selected to obtain the dimension-reduced feature data set;

[0056] By reducing the dimension of the feature data set, cluster analysis is used to determine the data grouping structure and obtain the classification feature set;

[0057] If the inter-group distance of the classification feature set is less than the preset threshold, the principal component analysis parameters are iteratively optimized to obtain the final feature data set;

[0058] Among them, the principal component load matrix represents the contribution weight of each parameter to the principal component, which is recorded as L, where Lij represents the load of the i-th parameter on the j-th principal component;

[0059] The cumulative contribution rate represents the proportion of the selected principal component that explains the variation of the original data, which is recorded as R, where R = Σλi / Σλ, λi is the eigenvalue of the principal component;

[0060] The inter-group distance represents the difference measure between different data groups, denoted as D, and is calculated based on the Euclidean distance.

[0061] The present invention also provides a mining data acquisition system based on the Internet of Things, which is used to implement the above method. The system includes: a data acquisition module, an accuracy standardization module, a noise filtering module, an anomaly detection module, a data integration module, and a feature extraction module;

[0062] The data acquisition module is used to obtain multi-source data streams output by the mining Internet of Things acquisition equipment, classify them according to different parameter types, and obtain an initial data set;

[0063] The precision standardization module is used to determine the acquisition precision range of each parameter type based on the physical characteristics of each parameter in the initial data set using an adaptive threshold segmentation algorithm to obtain precision-standardized data groups;

[0064] The noise filtering module is used to group the data with standardized accuracy. If it is detected that the data point deviates from the preset statistical distribution range, the noise interference is filtered out by the Kalman filtering algorithm to obtain a filtered data sequence;

[0065] The anomaly detection module is used to obtain the sliding window features of the time series from the filtered data sequence, use the isolation forest algorithm to determine the outliers, and obtain the data set after eliminating the anomalies;

[0066] The data integration module is used to perform normalization processing on the timestamps and spatial positions of the multi-source data using a weighted average algorithm based on the data set after the abnormalities are eliminated, so as to obtain a consistent integrated data matrix;

[0067] The feature extraction module is used to obtain the dynamic change trend of each parameter from the consistent integrated data matrix, and use the principal component analysis algorithm to extract key features to obtain a feature data set after dimensionality reduction.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] The present invention discloses a mining data acquisition method and system based on the Internet of Things. For the complex multi-source data streams output by acquisition equipment in a mining environment, the system implements data accuracy standardization, noise filtering, and anomaly removal through algorithms such as adaptive threshold segmentation, Kalman filtering, and isolation forest anomaly detection. Then, a weighted average algorithm is used to perform spatiotemporal normalization on the multi-source data, and principal component analysis is used to extract key features, ultimately obtaining a feature data set after dimensionality reduction. The present invention solves the problem of consistent integration of multi-source heterogeneous data in mines, improves data quality and availability, and provides a reliable data foundation for subsequent mine safety monitoring and production decision-making, thus having important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 This is a flow chart of a mining data collection method based on the Internet of Things according to an embodiment of the present invention;

[0072] Figure 2 This is a process diagram of the image threshold segmentation method based on the genetic algorithm according to an embodiment of the present invention;

[0073] Figure 3 This is a structural diagram of a mining data acquisition system based on the Internet of Things according to an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0075] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0076] Example 1

[0077] like Figure 1 As shown, this embodiment provides a mining data collection method based on the Internet of Things, the method comprising:

[0078] Obtain multi-source data streams output by mining IoT collection equipment, classify them according to different parameter types, and obtain the initial data set;

[0079] According to the physical characteristics of each parameter in the initial data set, an adaptive threshold segmentation algorithm is used to determine the acquisition accuracy range of each type of parameter and obtain data grouping with standardized accuracy;

[0080] For the data grouping with standardized accuracy, if it is detected that the data point deviates from the preset statistical distribution range, the noise interference is filtered out by the Kalman filter algorithm to obtain the filtered data sequence;

[0081] From the filtered data sequence, the sliding window features of the time series are obtained, and the isolation forest algorithm is used to determine the outliers to obtain the data set after removing the anomalies;

[0082] Based on the data set after removing anomalies, a weighted average algorithm is used to normalize the timestamps and spatial positions of multi-source data to obtain a consistent and integrated data matrix;

[0083] The dynamic change trend of each parameter is obtained from the consistent integrated data matrix, and the principal component analysis algorithm is used to extract key features to obtain the feature data set after dimensionality reduction.

[0084] In this embodiment, the multi-source data streams output by the mining IoT collection equipment are obtained and classified according to different parameter types to obtain an initial data set, including:

[0085] Acquire multi-source data streams from IoT devices and use a preset acquisition frequency to obtain real-time data sequences;

[0086] The real-time data series are classified according to the parameter type, and the initial data set is determined by the classification rules.

[0087] Specifically, when IoT devices obtain multi-source data streams, they can collect mining environmental parameters such as temperature, humidity, and gas concentration through sensors.

[0088] For example, the temperature sensor collects data every 5 seconds, and the humidity sensor collects data every 10 seconds, forming a real-time data sequence. The collection frequency is preset to meet real-time monitoring requirements and ensure data timeliness. The real-time data sequence is categorized by parameter type, with temperature, humidity, and gas concentration being categorized separately. In one possible implementation, classification rules are set using a rule engine, such as setting the temperature data range to -20°C to 50°C, with out-of-range values ​​marked as abnormal, to generate the initial data set.

[0089] In this embodiment, based on the physical characteristics of each parameter in the initial data set, an adaptive threshold segmentation algorithm is used to determine the acquisition accuracy range of each type of parameter, and obtain data groups with standardized accuracy, including:

[0090] By analyzing the initial data set, the physical characteristics of each parameter are obtained and the basis for parameter classification is determined;

[0091] Based on the parameter classification basis, the statistical analysis method is used to obtain the distribution characteristics of each parameter;

[0092] According to the distribution characteristics, the adaptive threshold segmentation algorithm is used to determine the threshold range of each parameter;

[0093] If the threshold range meets the preset conditions, the acquisition accuracy range of each parameter is generated;

[0094] If it is not satisfied, adjust the threshold and re-segment to obtain the acquisition accuracy range;

[0095] By collecting the accuracy range, the initial data set is grouped and processed to generate accuracy standardized groups;

[0096] According to the accuracy standardization grouping, the normalization method is used to obtain standardized output data.

[0097] Specifically, based on the distribution characteristics, an adaptive threshold segmentation algorithm is used to determine the threshold range of each parameter, including the introduction of an automatic identification and spatiotemporal variable target background condition and the adjustment of the optimal segmentation area. This improves the quality and robustness of the segmentation algorithm and overcomes the limitations of traditional threshold methods in image segmentation. The specific implementation process is as follows:

[0098] Assume that in an image, the target brightness is higher than the background brightness.

[0099] (1) The inter-class variance method of genetic optimization threshold is used to adaptively determine the threshold on the image and segment the original image to obtain the target area and background area;

[0100] (2) Approximate each segmented region to a rectangular region, and then use the information entropy of the image as the discrimination criterion. If it meets this criterion, it means that the information content of the region is small and contains fewer gray levels. It can be considered that it is composed of only the target or background, and the segmentation is completed; otherwise, it means that the information content of the region is rich and contains more gray levels. It cannot be considered to be composed of a single target or background. The target region needs to be segmented again using the inter-class variance method of the genetic optimization threshold until all target regions meet the discrimination criterion or the segmentation result no longer changes. This is equivalent to dividing an image into several blocks and applying the maximum inter-class variance method of the genetic optimization threshold to obtain the threshold for each sub-block.

[0101] Among them, the segmentation criterion is:

[0102] Where p i is the probability of gray level i appearing in the image, iL is the maximum gray level of the image; E T =αE (α is taken as 0.85 in this paper); E is the information content of the entire image. Figure 2 shown.

[0103] It is understandable that this adaptive method can flexibly adjust according to actual data fluctuations to ensure that the threshold fits the actual scenario. The collection accuracy range is generated by the threshold range, and it is necessary to determine whether the preset conditions are met.

[0104] For example, the temperature acquisition accuracy is required to be within ±0.5 degrees Celsius. If the threshold range is too wide, it may lead to insufficient accuracy, and the range needs to be narrowed to 22 to 28 degrees Celsius.

[0105] In one possible implementation, pressure parameters may fluctuate significantly due to geological variations, so the accuracy range is adjusted to ±1000 Pascals to ensure data reliability. The acquisition accuracy range guides the grouping of the initial data set to form accuracy-standardized groups.

[0106] For example, temperature data may be divided into a high-precision group and a low-precision group, the high-precision group being used for environmental monitoring, and the low-precision group being used for trend analysis.

[0107] In this embodiment, for the precision-normalized data grouping, if it is detected that a data point deviates from a preset statistical distribution range, the noise interference is filtered out using a Kalman filter algorithm to obtain a filtered data sequence, including:

[0108] If the data grouping is completed, the precision standardization process is performed to obtain a standardized data sequence;

[0109] Based on the standardized data sequence, a statistical distribution model is used to determine the preset range;

[0110] If the data point deviates from the preset range, the degree of deviation is determined through distribution analysis;

[0111] According to the degree of deviation, Kalman filtering is used to obtain the filtered data sequence;

[0112] By filtering the data sequence, noise interference is detected and noise distribution characteristics are generated;

[0113] According to the noise distribution characteristics, the filtering parameters are adjusted to obtain the optimized data sequence;

[0114] By optimizing the data sequence, data groups that conform to the statistical distribution are generated.

[0115] In this embodiment, the sliding window features of the time series are obtained from the filtered data sequence, and the isolation forest algorithm is used to determine the outliers to obtain the data set after the outliers are eliminated, including:

[0116] From the filtered data, the time series is obtained, and the feature vector is extracted by the sliding window method using the preset window size to obtain the feature set;

[0117] By using the isolation forest algorithm, anomaly scores are calculated for the feature set to identify outliers;

[0118] If the anomaly score exceeds the preset threshold, the corresponding data point is removed to obtain a preliminary cleaned data set;

[0119] From the preliminary cleaned data set, obtain the time series, use statistical methods to calculate the stationarity of the series, and determine the stability of the data;

[0120] If the sequence stationarity does not meet the standard, the sequence is adjusted through difference processing to obtain a stable time series;

[0121] According to the stable time series, the sliding window is used to re-extract the feature vector to obtain the optimized feature set;

[0122] The anomaly score is calculated again for the optimized feature set using the isolation forest algorithm to obtain the final cleaned data set.

[0123] Specifically, the isolation forest algorithm is used to calculate anomaly scores for feature sets and identify outliers, including:

[0124] The method of constructing the isolation forest algorithm based on the improved segmentation strategy is as follows: For an N-dimensional data set, the random slope can be selected by selecting a random number for each dimension from the standard normal distribution N(0, 1) to form an N-dimensional vector The intercept can select a random number between the maximum and minimum values ​​of each dimension of the data to form an N-dimensional vector Confirmed and Then, for a given point like Then the data points Divide the data point into the left subtree; otherwise, Assign to the right subtree.

[0125] The improved isolation forest algorithm can generate anomaly scores for feature sets, but it doesn't specify an anomaly score threshold, making it impossible to identify anomalous data. Setting the anomaly score threshold too high can lead to missed anomalies, while setting it too low can misclassify normal data as anomalies. Therefore, setting a reasonable anomaly score threshold is crucial. The data represented by the anomaly score may be entirely normal data or a mixture of normal and anomaly data. When the data represented by the anomaly score contains both normal and anomaly data, the anomaly data is necessarily a minority based on the characteristics of the anomaly data. Therefore, a clustering algorithm can be used to cluster the resulting anomaly scores into two categories. If the data represented by the anomaly score contains anomalies, the number of clusters in the two categories will inevitably differ significantly, and the data contained in the cluster with the smaller number of clusters will be anomalies. If the data represented by the anomaly score contains no anomalies, the number of clusters in the two categories will inevitably be similar. In summary, the K-means++ algorithm is used to calculate the anomaly score threshold for feature sets. This method can quickly and accurately determine the anomaly score threshold, improving anomaly detection efficiency. The K-Means++ algorithm optimizes the initial cluster center selection process in the K-Means algorithm. Instead of randomly selecting initial cluster centers, it selects them based on the principle that the distance between initial cluster centers is as large as possible. The specific steps are as follows: Step 1: Randomly select a sample point as the first initial cluster center. Step 2: Calculate the distance between each sample and its nearest cluster center (the greater the distance, the greater the probability of being selected as a cluster center). A roulette wheel selection method is used to select the next cluster center. Step 3: Repeat Step 2 until K cluster centers are selected.

[0126] Specifically, this distribution feature can be used to determine the source of interference, such as device jitter or environmental fluctuations.

[0127] For example, when adjusting the filtering parameters, the covariance matrix of the Kalman filter can be optimized according to the noise distribution characteristics.

[0128] For example, if the noise variance is 0.01, the process noise covariance is appropriately increased to improve the filter's adaptability to dynamic changes. This adjusted filter better tracks the real signal and produces a smoother, optimized data sequence. This approach ensures that the data is closer to the actual physical process.

[0129] By optimizing the data sequence to generate data groups that conform to the statistical distribution, the core is to maintain the statistical characteristics of the data.

[0130] After optimizing the regrouping of the series, each group of data conforms to a normal distribution, and the variance between groups is less than 0.05. This grouping method facilitates subsequent analysis, such as anomaly detection or trend prediction.

[0131] Temperature grouping can be used to monitor equipment operating status and detect potential faults in a timely manner.

[0132] In this embodiment, based on the data set after eliminating anomalies, a weighted average algorithm is used to perform normalization processing on the timestamps and spatial positions of the multi-source data to obtain a consistent integrated data matrix, including:

[0133] Acquire multi-source data, remove outliers, and obtain a cleaned data set;

[0134] By extracting the timestamp and spatial location from the cleaned data set, a spatiotemporal feature set is generated;

[0135] The weighted average algorithm is used to normalize the spatiotemporal feature set to obtain a standardized feature matrix;

[0136] If the dimensions of the standardized feature matrices are consistent, they are integrated into a unified data matrix;

[0137] If the dimensions are inconsistent, adjust the weights and renormalize to obtain a unified data matrix;

[0138] According to the unified data matrix, the consistency deviation is detected to obtain the data matrix after the deviation correction;

[0139] For the bias-corrected data matrix, clustering algorithm is applied to group the data to obtain the classified data set;

[0140] Extract grouping features from the categorical data set to generate the final integrated data matrix.

[0141] In this embodiment, the dynamic change trend of each parameter is obtained from the consistent integrated data matrix, and the principal component analysis algorithm is used to extract key features to obtain a feature data set after dimensionality reduction, including:

[0142] Obtain a consistent integrated data matrix and obtain a normalized data matrix through standardization;

[0143] If there are missing values ​​in the normalized data matrix, the mean filling method is used to obtain a complete data matrix;

[0144] Extract the dynamic change trend of each parameter from the complete data matrix and use time series analysis to obtain the trend feature set;

[0145] For the trend feature set, the principal component analysis algorithm is used to calculate the principal component loading matrix and obtain the key eigenvectors;

[0146] According to the key feature vector, the principal component whose cumulative contribution rate reaches the preset threshold is selected to obtain the dimension-reduced feature data set;

[0147] By reducing the dimension of the feature data set, cluster analysis is used to determine the data grouping structure and obtain the classification feature set;

[0148] If the inter-group distance of the classification feature set is less than the preset threshold, the principal component analysis parameters are iteratively optimized to obtain the final feature data set;

[0149] Among them, the principal component load matrix represents the contribution weight of each parameter to the principal component, which is recorded as L, where Lij represents the load of the i-th parameter on the j-th principal component;

[0150] The cumulative contribution rate represents the proportion of the selected principal component that explains the variation of the original data, which is recorded as R, where R = Σλi / Σλ, λi is the eigenvalue of the principal component;

[0151] The inter-group distance represents the difference measure between different data groups, denoted as D, and is calculated based on the Euclidean distance.

[0152] Example 2

[0153] like Figure 3 As shown, this embodiment provides a mining data acquisition system based on the Internet of Things, which is used to implement the method described in Example 1. The system includes: a data acquisition module, an accuracy standardization module, a noise filtering module, an anomaly detection module, a data integration module, and a feature extraction module;

[0154] The data acquisition module is used to obtain the multi-source data streams output by the mining IoT acquisition equipment, classify them according to different parameter types, and obtain the initial data set;

[0155] The precision standardization module is used to determine the acquisition precision range of each parameter type based on the physical characteristics of each parameter in the initial data set and obtain precision-standardized data groups using an adaptive threshold segmentation algorithm;

[0156] The noise filtering module is used to group data with standardized accuracy. If a data point is detected to deviate from the preset statistical distribution range, the noise interference is filtered out through the Kalman filter algorithm to obtain a filtered data sequence.

[0157] The anomaly detection module is used to obtain the sliding window features of the time series from the filtered data sequence, use the isolation forest algorithm to determine the outliers, and obtain the data set after eliminating the anomalies;

[0158] The data integration module is used to normalize the timestamps and spatial positions of multi-source data using a weighted average algorithm based on the data set after removing anomalies to obtain a consistent and integrated data matrix;

[0159] The feature extraction module is used to obtain the dynamic change trend of each parameter from the consistent integrated data matrix, and use the principal component analysis algorithm to extract key features to obtain the feature data set after dimensionality reduction.

[0160] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A mining data collection method based on the Internet of Things, characterized in that: The method comprises: Obtain multi-source data streams output by mining IoT collection equipment, classify them according to different parameter types, and obtain the initial data set; According to the physical characteristics of each parameter in the initial data set, an adaptive threshold segmentation algorithm is used to determine the acquisition accuracy range of each type of parameter and obtain data grouping with standardized accuracy; For the data grouping with standardized accuracy, if it is detected that the data point deviates from the preset statistical distribution range, the noise interference is filtered out by the Kalman filter algorithm to obtain the filtered data sequence; From the filtered data sequence, the sliding window features of the time series are obtained, and the isolation forest algorithm is used to determine the outliers to obtain the data set after removing the anomalies; Based on the data set after removing anomalies, a weighted average algorithm is used to normalize the timestamps and spatial positions of multi-source data to obtain a consistent and integrated data matrix; The dynamic change trend of each parameter is obtained from the consistent integrated data matrix, and the principal component analysis algorithm is used to extract key features to obtain the feature data set after dimensionality reduction.

2. The method according to claim 1, characterized in that Obtain multi-source data streams output by mining IoT collection equipment, classify them according to different parameter types, and obtain the initial data set, including: Acquire multi-source data streams from IoT devices and use a preset acquisition frequency to obtain real-time data sequences; The real-time data series are classified according to the parameter type, and the initial data set is determined by the classification rules.

3. The method according to claim 1, characterized in that According to the physical characteristics of each parameter in the initial data set, an adaptive threshold segmentation algorithm is used to determine the acquisition accuracy range of each parameter type and obtain data groups with standardized accuracy, including: By analyzing the initial data set, the physical characteristics of each parameter are obtained and the basis for parameter classification is determined; Based on the parameter classification basis, the statistical analysis method is used to obtain the distribution characteristics of each parameter; According to the distribution characteristics, the adaptive threshold segmentation algorithm is used to determine the threshold range of each parameter; If the threshold range meets the preset conditions, the acquisition accuracy range of each parameter is generated; If it is not satisfied, adjust the threshold and re-segment to obtain the acquisition accuracy range; By collecting the accuracy range, the initial data set is grouped and processed to generate accuracy standardized groups; According to the accuracy standardization grouping, the normalization method is used to obtain standardized output data.

4. The method according to claim 1, wherein For the precision-standardized data grouping, if it is detected that the data point deviates from the preset statistical distribution range, the noise interference is filtered out through the Kalman filter algorithm to obtain the filtered data sequence, including: If the data grouping is completed, the precision standardization process is performed to obtain a standardized data sequence; Based on the standardized data sequence, a statistical distribution model is used to determine the preset range; If the data point deviates from the preset range, the degree of deviation is determined through distribution analysis; According to the degree of deviation, Kalman filtering is used to obtain the filtered data sequence; By filtering the data sequence, noise interference is detected and noise distribution characteristics are generated; According to the noise distribution characteristics, the filtering parameters are adjusted to obtain the optimized data sequence; By optimizing the data sequence, data groups that conform to the statistical distribution are generated.

5. The method according to claim 1, characterized in that From the filtered data sequence, the sliding window features of the time series are obtained, and the isolation forest algorithm is used to determine the outliers to obtain the data set after removing the anomalies, including: From the filtered data, the time series is obtained, and the feature vector is extracted by the sliding window method using the preset window size to obtain the feature set; By using the isolation forest algorithm, anomaly scores are calculated for the feature set to identify outliers; If the anomaly score exceeds the preset threshold, the corresponding data point is removed to obtain a preliminary cleaned data set; From the preliminary cleaned data set, obtain the time series, use statistical methods to calculate the stationarity of the series, and determine the stability of the data; If the sequence stationarity does not meet the standard, the sequence is adjusted through difference processing to obtain a stable time series; According to the stable time series, the sliding window is used to re-extract the feature vector to obtain the optimized feature set; The anomaly score is calculated again for the optimized feature set using the isolation forest algorithm to obtain the final cleaned data set.

6. The method according to claim 1, characterized in that Based on the data set after removing anomalies, a weighted average algorithm is used to normalize the timestamps and spatial locations of multi-source data to obtain a consistent and integrated data matrix, including: Acquire multi-source data, remove outliers, and obtain a cleaned data set; By extracting the timestamp and spatial location from the cleaned data set, a spatiotemporal feature set is generated; The weighted average algorithm is used to normalize the spatiotemporal feature set to obtain a standardized feature matrix; If the dimensions of the standardized feature matrices are consistent, they are integrated into a unified data matrix; If the dimensions are inconsistent, adjust the weights and renormalize to obtain a unified data matrix; According to the unified data matrix, the consistency deviation is detected to obtain the data matrix after the deviation correction; For the bias-corrected data matrix, clustering algorithm is applied to group the data to obtain the classified data set; Extract grouping features from the categorical data set to generate the final integrated data matrix.

7. The method according to claim 1, characterized in that From the consistent integrated data matrix, the dynamic change trend of each parameter is obtained, and the principal component analysis algorithm is used to extract key features to obtain the feature data set after dimensionality reduction, including: Obtain a consistent integrated data matrix and obtain a normalized data matrix through standardization; If there are missing values ​​in the normalized data matrix, the mean filling method is used to obtain a complete data matrix; Extract the dynamic change trend of each parameter from the complete data matrix and use time series analysis to obtain the trend feature set; For the trend feature set, the principal component analysis algorithm is used to calculate the principal component loading matrix and obtain the key eigenvectors; According to the key feature vector, the principal component whose cumulative contribution rate reaches the preset threshold is selected to obtain the dimension-reduced feature data set; By reducing the dimension of the feature data set, cluster analysis is used to determine the data grouping structure and obtain the classification feature set; If the inter-group distance of the classification feature set is less than the preset threshold, the principal component analysis parameters are iteratively optimized to obtain the final feature data set; Among them, the principal component load matrix represents the contribution weight of each parameter to the principal component, which is recorded as L, where Lij represents the load of the i-th parameter on the j-th principal component; The cumulative contribution rate represents the proportion of the selected principal component that explains the variation of the original data, which is recorded as R, where R = Σλi / Σλ, λi is the eigenvalue of the principal component; The inter-group distance represents the difference measure between different data groups, denoted as D, and is calculated based on the Euclidean distance.

8. A mining data acquisition system based on the Internet of Things, the system being used to implement the method according to any one of claims 1 to 7, characterized in that: The system includes: a data acquisition module, an accuracy standardization module, a noise filtering module, an anomaly detection module, a data integration module, and a feature extraction module; The data acquisition module is used to obtain multi-source data streams output by the mining Internet of Things acquisition equipment, classify them according to different parameter types, and obtain an initial data set; The precision standardization module is used to determine the acquisition precision range of each parameter type based on the physical characteristics of each parameter in the initial data set using an adaptive threshold segmentation algorithm to obtain precision-standardized data groups; The noise filtering module is used to group the data with standardized accuracy. If it is detected that the data point deviates from the preset statistical distribution range, the noise interference is filtered out by the Kalman filtering algorithm to obtain a filtered data sequence; The anomaly detection module is used to obtain the sliding window features of the time series from the filtered data sequence, use the isolation forest algorithm to determine the outliers, and obtain the data set after eliminating the anomalies; The data integration module is used to perform normalization processing on the timestamps and spatial positions of the multi-source data using a weighted average algorithm based on the data set after the abnormalities are eliminated, so as to obtain a consistent integrated data matrix; The feature extraction module is used to obtain the dynamic change trend of each parameter from the consistent integrated data matrix, and use the principal component analysis algorithm to extract key features to obtain a feature data set after dimensionality reduction.

Images