Multi-dimensional monitoring data processing method for hydrogeological monitoring

Abstract

The invention discloses a multi-dimensional monitoring data processing method for hydrogeological monitoring, which comprises the following steps: acquiring multi-source heterogeneous monitoring data, and processing the multi-source heterogeneous monitoring data by adopting a time sequence alignment algorithm to obtain a data set with a unified time scale; performing data cleaning on the data set with the unified time scale to generate a cleaned data set; dynamically monitoring real-time data by adopting a data stream processing framework according to the cleaned data set to obtain a processing result; performing data fusion on the processing result to generate a fused data set; according to the fused data set, carrying out data compression and transmission by adopting a low-power-consumption transmission protocol; calculating the fused data set through a unified data model to generate a data analysis report; and according to the data analysis report, a model training and verification method is adopted to optimize the prediction model, and a final model is obtained.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and more specifically, to a method for processing multi-dimensional monitoring data for hydrogeological monitoring. Background Art

[0002] In the fields of hydrogeological exploration and environmental protection, the acquisition and recording of data are the basis for environmental assessment and monitoring. However, in actual operation, there is a unique technical problem: how to ensure the real-time and consistency of multi-dimensional monitoring data under complex geological and hydrological conditions. Specifically, environmental monitoring in the pre-construction, construction, and post-construction stages of hydraulic engineering involves multiple dimensions such as hydrology, ecology, and water quality. These data sources are scattered and have different collection frequencies. For example, hydrological data may be obtained through groundwater level monitoring equipment, ecological data depends on vegetation coverage and biodiversity surveys, and water quality data requires water sampling and laboratory analysis.

[0003] This method of collecting multi-source heterogeneous data is prone to problems such as inconsistent data timestamps, large differences in accuracy, and data loss. At the same time, dynamic changes during the engineering construction process, such as the disturbance of the groundwater level by construction activities or the short-term impact on the surrounding ecology, require the monitoring system to be able to capture these changes in real time. However, existing monitoring equipment is often limited by factors such as data transmission bandwidth, equipment power supply stability, and environmental interference, making it difficult to achieve high-frequency and high-precision continuous monitoring. In addition, the integration of multi-dimensional data requires the establishment of a unified data model. However, due to the complex non-linear relationships between different monitoring indicators, it is often difficult to balance the calculation efficiency and accuracy of the model. These problems together constitute the core technical contradiction in the development of a real-time environmental impact assessment system for hydraulic engineering. Summary of the Invention

[0004] The purpose of the present invention is to solve the above problems and provide a method for processing multi-dimensional monitoring data for hydrogeological monitoring, providing reliable data support and analysis tools for hydraulic environment monitoring, mineral exploration, and environmental protection.

[0005] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0006] A method for processing multi-dimensional monitoring data for hydrogeological monitoring, comprising the following steps:

[0007] a. Obtain multi-source heterogeneous monitoring data, and process the multi-source heterogeneous monitoring data using a time series alignment algorithm to obtain a dataset with a unified time scale;

[0008] b. Clean the dataset with the unified time scale to generate a cleaned dataset;

[0009] c Based on the cleaned dataset, use a data stream processing framework to dynamically monitor real-time data and obtain a processing result;

[0010] d Perform data fusion on the processing result to generate a fused dataset;

[0011] e According to the fused dataset, use a low-power transmission protocol for data compression and transmission;

[0012] f Calculate the fused dataset through a unified data model to generate a data analysis report;

[0013] g According to the data analysis report, use model training and validation methods to optimize the prediction model and obtain the final model.

[0014] Further, in step a, the time series alignment algorithm is used to process the multi-source heterogeneous monitoring data, including the following steps:

[0015] For the time series characteristics of different data sources, use interpolation or resampling methods to correct the data and generate a dataset with a unified time scale;

[0016] According to the dataset with the unified time scale, use a clustering algorithm to classify the multi-dimensional monitoring data and obtain potential patterns;

[0017] According to the potential patterns, use a regression algorithm to predict the time series and obtain the monitoring data values at future time points.

[0018] Further, the specific steps for data cleaning of the dataset with the unified time scale in step b are as follows:

[0019] For the case where the data missing rate exceeds the preset threshold, use neighboring data interpolation or a pre-trained machine learning model to supplement the missing values;

[0020]

[0021] MR represents the data missing rate, Mi represents the number of missing data at the i-th time point, N represents the total data volume, and n represents the time series length; this formula is used to calculate the missing rate of the dataset;

[0022] According to the supplemented monitoring dataset, use interpolation or resampling methods to align the time series and generate a monitoring dataset with a unified time scale;

[0023] According to the monitoring dataset with the unified time scale, use a clustering analysis method to extract potential pattern features;

[0024] According to the potential pattern features, use a regression algorithm to predict the monitoring data values at future time points.

[0025] Further, the specific steps of dynamically monitoring the real-time data using the data stream processing framework in step c are as follows:

[0026] Judge the processing delay amount of the real-time data stream according to the preset time window. If the processing delay amount exceeds the preset time window, trigger the priority scheduling mechanism;

[0027] Identify and extract key data through the priority scheduling mechanism, and preferentially allocate computing resources to process key data;

[0028] Use a dynamic monitor to track the status and delay of the real-time data stream, and dynamically update the trigger conditions of the priority scheduling mechanism;

[0029] Dynamically adjust the resource allocation strategy for data processing according to the classification result of the real-time data stream and the execution situation of the priority scheduling mechanism.

[0030] Further, the specific steps of data fusion for the processing result in step d are as follows:

[0031] Process the original data through the timestamp alignment algorithm to generate a dataset with aligned timestamps;

[0032] Clean the dataset with aligned timestamps to remove noise and outliers;

[0033] For data with non-linear relationships in multi-dimensional data, use the kernel function mapping method for processing to generate the mapped data;

[0034]

[0035] represents the non-linear metric index, n represents the number of samples, α represents the weight coefficient, K represents the kernel function, and x represents the input data vector; this formula is used to measure the non-linearity of the data;

[0036] Use the principal component analysis method to perform dimensionality reduction processing on the mapped data and extract the main features;

[0037] Use the data integration algorithm to integrate the dataset after retaining the features to generate the fused dataset.

[0038] Further, the specific steps of data compression and transmission using the low-power transmission protocol in step e are as follows:

[0039] Judge whether the data volume of the original monitoring data exceeds the preset threshold. If it exceeds, start the compression algorithm;

[0040] Use the compression algorithm to perform dimensionality reduction processing on the original monitoring data to generate the compressed data packet;

[0041] Transmit the compressed data packets in batches to the server by means of segmented transmission;

[0042] Dynamically adjust the parameters of the low-power transmission protocol according to the power supply stability monitoring data;

[0043] Use a decompression algorithm to decompress and verify the received compressed data packets to generate the original monitoring data.

[0044] Further, the specific steps of calculating the fused data set through the unified data model in step f are as follows:

[0045] Obtain multi-dimensional data from multiple data sources;

[0046] Fuse the multi-dimensional data according to the preset unified data model to generate a fused data set;

[0047] Preprocess the fused data set to remove noise data and redundant data;

[0048] Judge whether the complexity of the unified data model exceeds a preset threshold. If it exceeds, use a distributed computing framework for calculation to generate a calculation result;

[0049] Generate a data analysis report according to the calculation result.

[0050] The beneficial effects of the present invention are:

[0051] 1. The present invention is applicable to data acquisition and recording in the fields of hydraulic environment, prospecting and environmental protection. For multi-source heterogeneous data, the present invention first performs time series alignment to ensure that the data has a unified time scale. Subsequently, data cleaning techniques are used to solve the problems of precision difference and data missing. When necessary, interpolation or machine learning models are used to supplement the missing values. To cope with the characteristics of dynamic changes, the present invention constructs a real-time data stream processing framework and sets a priority scheduling mechanism to ensure the timely processing of key data. Through the data fusion algorithm, the present invention integrates multi-dimensional data and uses dimensionality reduction techniques to retain the main features when necessary. Considering the bandwidth limitation and power supply stability problems in practical applications, the present invention adopts data compression technology and low-power transmission protocol. Finally, the present invention establishes a unified data model, improves the prediction accuracy through model training and verification, and provides reliable data support and analysis tools for hydraulic environment monitoring, mineral exploration and environmental protection. Description of the Drawings

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0053] Figure 1 This is the flowchart of the multi-dimensional monitoring data processing of the present invention. Detailed implementation manners

[0054] In order to enable those skilled in the art of this technology to better understand the technical solutions in the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.

[0055] A multi-dimensional monitoring data processing method for hydrogeological monitoring includes the following steps:

[0056] a Obtain multi-source heterogeneous monitoring data, and process the multi-source heterogeneous monitoring data using a time series alignment algorithm to obtain a dataset with a unified time scale;

[0057] b Clean the dataset with the unified time scale to generate a cleaned dataset;

[0058] c According to the characteristics of dynamic changes, construct a real-time data stream processing framework. According to the cleaned dataset, use the data stream processing framework to dynamically monitor the real-time data to obtain a processing result. If the data processing delay exceeds a preset time window, trigger a priority scheduling mechanism to ensure that critical data is processed first;

[0059] d Perform data fusion on the processing result to generate a fused dataset; through a data fusion algorithm, integrate the multi-dimensional data with aligned timestamps and cleaned. If there is a non-linear relationship between the data, use the principal component analysis or kernel function mapping method to reduce the data dimension and retain the main features.

[0060] e According to the fused dataset, use a low-power transmission protocol for data compression and transmission; for the problems of bandwidth limitation and power supply stability, use data compression technology and a low-power transmission protocol. If the data volume exceeds a preset threshold, start the compression algorithm to reduce the amount of transmitted data.

[0061] f Calculate the fused dataset through a unified data model to generate a data analysis report. Input the fused multi-dimensional data into the model. If the model calculation complexity exceeds the preset threshold, adopt a distributed computing framework to improve the calculation efficiency;

[0062] g Optimize the prediction model using model training and validation methods according to the data analysis report to obtain the final model.

[0063] In step a, the time series alignment algorithm is used to process the multi-source heterogeneous monitoring data, including the following steps: Obtain multi-source heterogeneous data sources from the multi-dimensional monitoring system. For the time series characteristics of different data sources, use the time series alignment algorithm to process the data. If the data time scales are inconsistent, correct them through interpolation or resampling methods to generate a dataset with a unified time scale; According to the dataset with the unified time scale, use a clustering algorithm to classify the multi-dimensional monitoring data and obtain the potential patterns in the data; According to the potential patterns, use a regression algorithm to predict the time series and obtain the monitoring data values at future time points.

[0064] The specific steps for data cleaning of the dataset with the unified time scale in step b are as follows: For the case where the data missing rate exceeds the preset threshold, use neighboring data interpolation or a pre-trained machine learning model to supplement the missing values;

[0065]

[0066] MR represents the data missing rate, M_i represents the number of missing data at the i-th time point, N represents the total data volume, and n represents the time series length; This formula is used to calculate the missing rate of the dataset;

[0067] If the neighboring data interpolation operation cannot meet the preset accuracy requirements, use a pre-trained machine learning model to supplement the missing data to obtain the supplemented monitoring dataset;

[0068]

[0069] ε represents the interpolation accuracy, yi represents the true value, yi hat represents the interpolation estimate value, and n represents the number of samples; This formula is used to evaluate the accuracy of the interpolation result;

[0070] According to the supplemented monitoring dataset, use interpolation or resampling methods to align the time series to generate a monitoring dataset with a unified time scale; According to the monitoring dataset with the unified time scale, use a clustering analysis method to extract the potential pattern features; According to the potential pattern features, use a regression algorithm to predict the monitoring data values at future time points.

[0071] Specifically, the integration of multi-source heterogeneous monitoring data is crucial for building a high-quality analysis model. When the missing rate exceeds the preset threshold of 20%, data repair is required. For short-term missing data, linear interpolation can be used. For example, if the values of a certain pressure sensor at 10:00 and 10:02 are 5 MPa and 7 MPa respectively, the value at 10:01 can be estimated as 6 MPa. For long-term missing data or complex working conditions, a pre-trained neural network model can be used for data supplementation. The model learns the correlation between variables in historical data to achieve high-precision prediction.

[0072] Time series alignment is a key step in achieving multi-source data fusion. The sampling frequencies of different monitoring devices are often inconsistent. For example, the temperature is sampled once per minute, while the pressure is sampled once per second. By using resampling methods, all data can be unified to the same time scale for subsequent analysis. Cluster analysis helps to discover potential patterns in the data. Based on the identified operating modes, a regression model can be established to predict the change trends of key parameters.

[0073] The specific steps for dynamically monitoring real-time data using a data stream processing framework in step c are as follows: Determine the processing delay of the real-time data stream according to a preset time window. If the processing delay exceeds the preset time window, trigger a priority scheduling mechanism; Identify and extract key data through the priority scheduling mechanism, and preferentially allocate computing resources to process the key data; Continuously track the status and delay of the real-time data stream using a dynamic monitor, and dynamically update the trigger conditions of the priority scheduling mechanism; Dynamically adjust the resource allocation strategy for data processing according to the classification result of the real-time data stream and the execution situation of the priority scheduling mechanism; Process the key data through the data stream processing framework, obtain the processing result and store it in the target database; Optimize the trigger conditions and resource allocation strategy of the priority scheduling mechanism according to the processing result and the status of subsequent data streams.

[0074] Specifically, the real-time data stream processing framework plays a key role in industrial production. By classifying and dynamically monitoring data, real-time control can be achieved. When the system detects that the processing delay of temperature data exceeds the preset five-second time window, it immediately triggers the priority scheduling mechanism. The dynamic monitor realizes intelligent allocation by tracking the processing status of data streams in each process. When it is observed that data processing frequently experiences delays during a certain period, the system automatically reduces the trigger threshold and conducts scheduling in advance. After the processing result shows that the delay is effectively controlled, the scheduling strategy is gradually optimized to improve the resource utilization efficiency.

[0075] Furthermore, the specific steps for data fusion of the processing result in step d are as follows:

[0076] Process the original data through a timestamp alignment algorithm to generate a dataset after timestamp alignment; perform data cleaning on the dataset after timestamp alignment to remove noise and outliers; for data with non-linear relationships in multi-dimensional data, use a kernel function mapping method for processing to generate the mapped data;

[0077]

[0078] λ represents the non-linear metric index, n represents the number of samples, α represents the weight coefficient, K represents the kernel function, and x represents the input data vector; this formula is used to measure the non-linearity of the data;

[0079] Use the principal component analysis method to perform dimensionality reduction on the mapped data to obtain a dataset after dimensionality reduction; extract the main features of the dataset after dimensionality reduction to obtain a dataset after feature retention; use a data integration algorithm to integrate the dataset after feature retention to obtain an integrated dataset; generate a data fusion result based on the integrated dataset.

[0080] Specifically, timestamp alignment is of great significance in industrial field sensor data processing. The acquisition times of parameters such as temperature, pressure, and flow at multiple measurement points are not completely synchronized, and these data need to be aligned according to time. Through the timestamp alignment algorithm, temperature data with a sampling interval of 0.5 seconds can be unified with pressure data with a sampling interval of 1 second.

[0081] The data cleaning stage focuses on dealing with outliers and noise. Through the kernel function mapping method, non-linear relationships can be transformed into linear relationships in a high-dimensional space. When the non-linear metric index shows that the correlation coefficient between temperature and conversion rate is lower than 0.3, it indicates the existence of obvious non-linear characteristics, and a radial basis kernel function needs to be used for mapping processing.

[0082] When dealing with data from hundreds of measurement points, dimensionality reduction needs to be performed through the principal component analysis method. Practice has shown that taking the first five principal components can retain 85% of the information volume, greatly reducing the data processing burden. The data integration algorithm realizes the effective fusion of multi-source data by establishing an association model between parameters. The model first aligns data with different time scales, removes abnormal fluctuations, and then uses a kernel function to process the non-linear relationship between energy consumption and output, and finally obtains an accurate energy-saving potential assessment result.

[0083] The specific steps of using the low-power transmission protocol for data compression and transmission in step e are as follows: establish a data connection with the water conservancy and environmental protection monitoring equipment to obtain the original monitoring data; determine whether the data volume of the original monitoring data exceeds a preset threshold, and if it exceeds the threshold, start the compression algorithm; use the compression algorithm to reduce the dimension of the original monitoring data to obtain a compressed data packet; when the current network bandwidth is limited, use a segmented transmission method to transmit the compressed data packet to the server in batches; obtain the power supply stability monitoring data of the current power supply equipment, and dynamically adjust the parameters of the low-power transmission protocol according to the power supply stability monitoring data; wherein, when the power supply of the power supply equipment is unstable, lower the transmission frequency parameters of the low-power transmission protocol; the server receives the compressed data packet, and uses the decompression algorithm corresponding to the compression algorithm to perform decompression verification to obtain the original monitoring data; perform integrity verification on the original monitoring data, and store the verified original monitoring data in the environmental protection database to form a water conservancy and environmental protection monitoring data record.

[0084] Water conservancy and environmental prospecting environmental monitoring faces complex environments and data transmission challenges. The use of low-power transmission protocols can meet long-term monitoring needs. Taking groundwater monitoring in mining areas as an example, a connection is established with water quality sensors through Bluetooth low energy or LoRa protocols, and parameters such as dissolved oxygen and turbidity are collected once an hour. When data from multiple monitoring points are aggregated, the amount of data collected at a single time can reach ten megabytes, exceeding the preset five-megabyte threshold. At this time, compression processing needs to be started. In view of the characteristics of water quality monitoring data, a compression algorithm combining differential encoding and run-length encoding is used to reduce the amount of data to 40% of the original data. Data packets are transmitted in segments, and the size of each data packet is sixty-four kilobytes. The sending interval is dynamically adjusted according to the current network bandwidth status to ensure transmission stability.

[0085] Power supply stability directly affects data transmission reliability. In remote mining areas, the solar power supply system is greatly affected by the weather. When the battery voltage is detected to be lower than 11.5 volts, the data collection frequency is automatically adjusted from once an hour to once every two hours, and the transmission frequency is reduced accordingly to ensure that the monitoring equipment continues to work.

[0086] The server uses the corresponding decompression algorithm to restore the original data. Taking dissolved oxygen data as an example, after decompression, the data integrity check is performed to confirm the continuity of the data packet and the rationality of the value. The verification rules include value range verification and time series continuity check to ensure that the dissolved oxygen value is within the range of zero to twenty milligrams per liter and that the sampling time interval meets the preset requirements. In dust monitoring in open-pit mines, particle concentration data at multiple measuring points are transmitted via narrowband Internet of Things. Due to the drastic changes in dust concentration, a sliding average compression algorithm is used to take an average value for every ten data points, reducing the amount of data while retaining key trends.

[0087] When a power supply voltage fluctuation is detected, the system automatically extends the sampling interval to prioritize ensuring the monitoring continuity of key areas. The soil heavy metal content monitoring adopts a periodic sampling mode and adjusts the sampling frequency in combination with real-time meteorological data. During rainfall, the system automatically increases the sampling frequency and focuses on the impact of rainwater scouring on heavy metal migration.

[0088] Data transmission adopts a time-division strategy, giving priority to transmitting exceeded-standard data and sending regular data opportunistically, which not only ensures the timely delivery of alarm information but also avoids transmission congestion. Through this intelligent scheduling scheme, the monitoring equipment works more than 330 days per year on average, providing reliable data support for the environmental governance of the mining area.

[0089] The specific steps of calculating the fused data set through the unified data model in step f are as follows: Obtain multi-dimensional data from multiple data sources; According to the preset unified data model, fuse the multi-dimensional data to obtain a fused data set; Preprocess the fused data set to obtain a preprocessed fused data set, where the preprocessing includes removing noise data and redundant data; Input the preprocessed fused data set into the unified data model; Calculate the complexity of the unified data model to obtain the model complexity; Determine whether the model complexity exceeds a preset complexity threshold; If the model complexity exceeds the preset complexity threshold, use a distributed computing framework to perform distributed computing on the unified data model to obtain a calculation result; Generate a data analysis report according to the calculation result; Store the data analysis report.

[0090] The hydrogeology, environment and engineering geology (hydrogeology, environment and engineering geology, HEEG) mining area environmental monitoring involves multiple data sources such as water quality, atmosphere, and soil, and each data source contains multi-dimensional data such as temperature, humidity, and concentration. It is necessary to establish a unified data processing model. Taking the open-pit mining area environmental monitoring as an example, a multi-source data acquisition network is formed by establishing water quality monitoring stations, atmosphere monitoring stations, and soil monitoring stations. The water quality monitoring station collects parameters such as dissolved oxygen, pH value, and turbidity, the atmosphere monitoring station collects data such as particulate matter concentration and harmful gas content, and the soil monitoring station collects indicators such as heavy metal content and organic matter content.

[0091] According to the characteristics of mining area environmental monitoring, a unified data model including three sub-modules of water quality, atmosphere, and soil is established to achieve data fusion. Taking heavy metal pollution monitoring as an example, data such as soil heavy metal content, surface water heavy metal content, and air heavy metal particulate matter concentration are correlated and analyzed to form a complete heavy metal migration law. By removing abnormally fluctuating monitoring values and duplicate sampling data, the data quality is improved.

[0092] The model complexity is mainly reflected in two aspects: data dimension and computational volume. Taking the monitoring of a certain open-pit coal mine area as an example, twenty monitoring stations are arranged, and thirty indicators are collected at each station per hour, with the daily data volume reaching millions of records. When the model complexity exceeds the preset threshold, a distributed computing framework is adopted for data processing. The data is sliced according to the time dimension and the space dimension and distributed to multiple computing nodes for parallel processing. Taking the evaluation of the treatment effect of acidic wastewater as an example, a unified data model needs to process multi-dimensional data such as the pH value of surface water, heavy metal content, and rainfall.

[0093] Through distributed computing, analyze the operation effect of the wastewater treatment facility and generate a water quality compliance analysis report. The report includes the change trend of key indicators, the statistics of exceeding standards, and the evaluation of treatment efficiency, etc., and is stored in a distributed file system for easy query and sharing. For dust pollution control, a multi-dimensional data model including meteorological conditions, production conditions, and dust concentration is established. By analyzing the dust diffusion law under different weather conditions, evaluate the effect of dust suppression measures. When strong wind weather occurs, the system automatically increases the sampling frequency and focuses on the change of dust concentration in sensitive areas. The distributed computing framework can process the real-time data of multiple monitoring points at the same time, detect abnormal situations in time and generate warning information. Through the long-term accumulated monitoring data, analyze the seasonal change characteristics of dust pollution and provide a basis for optimizing pollution prevention measures.

[0094] In step g, through model training and verification, if the model prediction error exceeds the preset threshold, then adjust the model parameters or adopt an ensemble learning method to improve the prediction accuracy.

[0095] Obtain a model training data set, and use at least one machine learning algorithm to generate an initial model; verify the initial model to obtain a verification error; judge whether the verification error exceeds the preset threshold. If it exceeds, then adjust the parameters of the initial model and retrain according to the adjusted parameters to obtain an optimized model; if it does not exceed, then use the initial model as a candidate final model; if the verification error of the model after adjusting the parameters still exceeds the preset threshold, then for the training data set, adopt an ensemble learning method to obtain multiple training sub-models; fuse the multiple training sub-models to obtain an ensemble model; verify the ensemble model to obtain its corresponding model accuracy, and use the ensemble model as a candidate final model; select the model with the highest model accuracy from the candidate final models as the final model and save the final model.

[0096] Machine learning model training first requires obtaining a high-quality training data set. Taking water quality monitoring and early warning as an example, collect historical monitoring data including indicators such as dissolved oxygen, pH value, and heavy metal content, and at the same time mark whether there is an over-standard situation.

[0097] Select the random forest algorithm to construct the initial model. This algorithm can handle multi-dimensional feature data and has good anti-noise ability.

[0098] Evaluate the model performance through cross-validation and calculate the error between the prediction result and the actual annotation value. In the prediction of mine dust pollution, the validation error of the initial model may exceed the preset threshold. At this time, it is necessary to adjust the model parameters, such as the number of decision trees, the maximum depth of the tree, etc.

[0099] For the prediction of dust diffusion, the model parameters can be optimized according to the historical data under different weather conditions to improve the prediction accuracy.

[0100] When a single model is difficult to achieve the expected effect, an ensemble learning method is used to improve the model performance. Taking the treatment of heavy metal pollution as an example, sub-models such as support vector machines and neural networks are constructed respectively, and each sub-model is trained for different pollutant characteristics. The prediction results of multiple sub-models are fused through weighted voting to obtain an ensemble model.

[0101] In the assessment of soil pollution, different sub-models focus on characteristics such as heavy metal content and organic matter content respectively. The fused model can comprehensively consider various pollution factors. When validating the ensemble model, the focus is on evaluating its prediction accuracy under different seasons and weather conditions. Taking the treatment of acidic wastewater as an example, verify the early warning accuracy of the model under normal working conditions and emergencies. Finally, select the solution with the highest model accuracy as the production deployment model.

[0102] In practical applications, a pollution early warning model based on ensemble learning is adopted in an environmental monitoring system of a certain mining area, with an accuracy rate reaching 95%, which is 15% higher than that of a single model. This model can predict the risk of pollutant exceeding the standard according to real-time monitoring data and provide decision-making support for the adjustment of treatment measures. When the model is saved, it includes information such as training parameters and data preprocessing methods, which is convenient for subsequent maintenance and update. By continuously accumulating new monitoring data, regularly evaluating the model performance and optimizing it, the early warning effect is ensured.

[0103] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by terms such as "left", "right", "up", "down", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0104] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "installed", "connected", and "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, a direct connection, or an indirect connection through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

Claims

1. A multi-dimensional monitoring data processing method for hydrogeological monitoring, characterized in that, It includes the following steps: a Obtain multi-source heterogeneous monitoring data, and process the multi-source heterogeneous monitoring data using a time series alignment algorithm to obtain a dataset with a unified time scale; b Clean the dataset with the unified time scale to generate a cleaned dataset; c According to the cleaned dataset, use a data stream processing framework to dynamically monitor real-time data to obtain a processing result; d Perform data fusion on the processing result to generate a fused dataset; e According to the fused dataset, use a low-power transmission protocol for data compression and transmission; f Calculate the fused dataset through a unified data model to generate a data analysis report; g According to the data analysis report, optimize the prediction model using model training and verification methods to obtain a final model.

2. A multi-dimensional monitoring data processing method for hydrogeological monitoring according to claim 1, characterized in that, In step a, when using the time series alignment algorithm to process the multi-source heterogeneous monitoring data, it includes the following steps: For the time series characteristics of different data sources, use interpolation or resampling methods to correct the data to generate a dataset with a unified time scale; According to the dataset with the unified time scale, use a clustering algorithm to classify multi-dimensional monitoring data to obtain potential patterns; According to the potential patterns, use a regression algorithm to predict the time series to obtain the monitoring data values at future time points.

3. A multi-dimensional monitoring data processing method for hydrogeological monitoring according to claim 1, characterized in that, The specific steps for cleaning the dataset with the unified time scale in step b are as follows: For the case where the data missing rate exceeds a preset threshold, use neighboring data interpolation or a pre-trained machine learning model to supplement the missing values; MR represents the data missing rate, Mi represents the number of missing data at the i-th time point, N represents the total data volume, and n represents the time series length; this formula is used to calculate the missing rate of the dataset; According to the supplemented monitoring dataset, use interpolation or resampling methods to align the time series to generate a monitoring dataset with a unified time scale; According to the monitoring dataset with a unified time scale, use a clustering analysis method to extract potential pattern features; According to the potential pattern features, use a regression algorithm to predict the monitoring data values at future time points.

4. A multi-dimensional monitoring data processing method for hydrogeological monitoring according to claim 1, characterized in that, The specific steps for dynamically monitoring real-time data using a data stream processing framework in step c are as follows: Judge the processing delay of the real-time data stream according to a preset time window. If the processing delay exceeds the preset time window, trigger a priority scheduling mechanism; Identify and extract key data through the priority scheduling mechanism, and preferentially allocate computing resources to process key data; Use a dynamic monitor to track the status and delay of the real-time data stream, and dynamically update the trigger conditions of the priority scheduling mechanism; Dynamically adjust the resource allocation strategy for data processing according to the classification result of the real-time data stream and the execution situation of the priority scheduling mechanism.

5. A multi-dimensional monitoring data processing method for hydrogeological monitoring according to claim 1, characterized in that, The specific steps for performing data fusion on the processing result in step d are as follows: Process the original data through a timestamp alignment algorithm to generate a dataset with aligned timestamps; Clean the dataset with aligned timestamps to remove noise and outliers; For data with non-linear relationships in multi-dimensional data, use a kernel function mapping method for processing to generate mapped data; λ represents the non - linear metric index, n represents the number of samples, α represents the weight coefficient, K represents the kernel function, and x represents the input data vector; this formula is used to measure the non - linear degree of the data; The principal component analysis method is used to perform dimensionality reduction on the mapped data and extract the main features; The data integration algorithm is used to integrate the data set after feature retention to generate a fused data set.

6. A multi-dimensional monitoring data processing method for hydrogeological monitoring according to claim 1, characterized in that, The specific steps of data compression and transmission using the low - power transmission protocol in step e are as follows: Judge whether the data volume of the original monitoring data exceeds the preset threshold. If it exceeds, start the compression algorithm; Use the compression algorithm to perform dimensionality reduction on the original monitoring data to generate a compressed data packet; Use the segmented transmission method to transmit the compressed data packet to the server in batches; Dynamically adjust the parameters of the low - power transmission protocol according to the power supply stability monitoring data; Use the decompression algorithm to decompress and verify the received compressed data packet to generate the original monitoring data.

7. A multi-dimensional monitoring data processing method for hydrogeological monitoring according to claim 1, characterized in that, The specific steps of calculating the fused data set through the unified data model in step f are as follows: Obtain multi - dimensional data from multiple data sources; Fuse the multi - dimensional data according to the preset unified data model to generate a fused data set; Pre - process the fused data set to remove noise data and redundant data; Judge whether the complexity of the unified data model exceeds the preset threshold. If it exceeds, use the distributed computing framework for calculation to generate a calculation result; Generate a data analysis report according to the calculation result.

Images