Monitoring data quality control method based on deep learning and self-built rules

Abstract

The invention discloses a monitoring data quality control method based on deep learning and a self-built rule. According to the method, the reliability and the practicability of the natural resource monitoring data are remarkably improved by combining flexible rule configuration and an advanced time sequence analysis technology. Firstly, the method supports a user to self-define multi-dimensional quality inspection rules according to actual scene requirements, the multi-dimensional quality inspection rules comprise abnormal values, threshold values, sudden change values, time continuity, logicality and the like, and the limitation of traditional fixed rules is broken through. By introducing time logic check, the system can effectively identify abnormalities caused by seasonal or periodic changes in the data, such as reasonable fluctuation ranges of vegetation indexes in different seasons, thereby more accurately capturing time features of the data. Besides, the multi-device consistency comparison function realizes cross verification of cross-device data, device faults or local environment anomalies can be found in time, and logic consistency and overall credibility of multi-source data are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of data quality control, and specifically is a monitoring data quality control method based on deep learning and self-built rules. Background Art

[0002] Natural resource monitoring data refers to information collected through long-term, systematic, and dynamic observation and recording of the quantity, quality, distribution, utilization, and ecological environment of natural resources, such as land, water, forests, minerals, and oceans, using technologies such as remote sensing, geographic information systems (GIS), ground surveys, and sensor networks. These data serve as a crucial foundation for national and local natural resource management, environmental protection, planning and decision-making, and disaster early warning. To ensure the reliability, accuracy, and timeliness of these data, rigorous quality control is essential. This includes developing unified data collection standards and specifications, implementing advanced calibration and verification techniques, establishing multi-source data fusion mechanisms, conducting regular field verification and sampling inspections, and developing a data quality assessment system. Quality control permeates the entire data collection, processing, analysis, and application process, aiming to promptly identify and correct deviations, ensure that monitoring results accurately reflect the current status and changing trends of natural resources, and provide solid data support for sustainable development and the development of an ecological civilization. Quality control of natural resource monitoring data is a crucial component of natural resource monitoring.

[0003] However, traditional manual quality control methods are no longer able to meet the needs. The existing technology has the following problems: 1. Fixed quality inspection rules lack flexibility. Data quality requirements may vary in different monitoring scenarios. 2. Lack of comparison between data from different devices. 3. Seasonality and cyclicality of time series. Natural resource monitoring data often exhibit significant seasonal and cyclical characteristics. 4. Existing simple interpolation methods (such as linear interpolation) assume that data changes linearly within the missing interval when processing missing data, but in reality, natural resource data is often nonlinear. For example, in an atmospheric pollution event, the change in pollutant concentration may be exponential, and linear interpolation will result in a large deviation between the corrected data and the actual data. Summary of the Invention

[0004] The purpose of the present invention is to provide a monitoring data quality control method based on deep learning and self-built rules in order to solve the above-mentioned problems.

[0005] The technical solution adopted by the present invention is as follows: a monitoring data quality control method based on deep learning and self-built rules, characterized in that the method comprises the following steps: a) establishing a natural resource monitoring data quality control database, the database including a monitoring data table, a quality inspection rule table, and a quality inspection log table; b) Import natural resource monitoring data, and support direct import and export of Excel files; c) Perform multi-dimensional quality checks based on self-built quality inspection rules, including: outlier check; boundary value check; mutation value check; time continuity check; time logic check; and multi-device consistency comparison check.

[0006] d) Use deep learning modules to perform data prediction and repair based on deep learning models; e) Generate quality inspection report.

[0007] In a preferred embodiment, in step c), the quality inspection rule establishment and inspection method for abnormal values, limit values, and mutation values ​​include: a) Calculate outliers (Z-score method), upper and lower limits (percentile threshold method), data change rate, etc. to assist in determining quality inspection rules; b) Set quality inspection rules for abnormal values, limit values, and mutation values; c) determining whether the data point exceeds the threshold range; d) Mark or delete data points that exceed the threshold.

[0008] In a preferred embodiment, in step c), Outlier detection is determined based on the specific device requirements and expert experience. Outliers are determined using the Z-score method, as follows.

[0009] .

[0010] in, is the data mean, σ is the standard deviation, To monitor the data value; the “3σ” principle is used to test the data. When the Z-score is greater than the set threshold (3.0), x is determined to be an outlier.

[0011] The limit value check uses the percentile threshold method to determine the upper and lower limits of the limit value. The principle is: after removing outliers from the original data, add or subtract 1 standard deviation based on the 99.9% (upper limit) or 0.1% (lower limit) profile to form dynamic upper and lower limit thresholds.

[0012] The mutation value check determines the rules by calculating the rate of change between adjacent moments. The principle is as follows.

[0013] .

[0014] in It is the monitoring data at time t. When the change rate exceeds the set maximum allowable change rate, it is determined to be a mutation value.

[0015] Multi-device consistency comparison check: Users can ensure the consistency of data with internal logical relationships between different devices by defining the maximum difference allowed between different data on different devices. This is important for discovering equipment failures, data transmission errors, or local anomalies of natural resources.

[0016] In a preferred embodiment, in step c), the method for establishing and checking time continuity quality inspection rules includes: a) Set the monitoring frequency of the data according to the equipment, in minutes; b) Determine whether there are missing values ​​in the time series of data points; c) Mark the locations of missing data or simply interpolate to fill them.

[0017] In a preferred embodiment, in step c), the method for establishing and checking the quality inspection rules of the time logic includes: a) Set rules for monitoring data changes over time; b) Check whether the data value is within the expected range for the time period; c) Mark the data that does not conform to the rules.

[0018] In a preferred embodiment, in step c), the quality inspection rule establishment and inspection method for multi-device consistency comparison includes: a) Set up associated device groups; b) define the maximum permissible difference; c) Compare the differences in monitoring values ​​of devices within the group; d) Mark data that exceeds the allowed range.

[0019] In a preferred embodiment, in step d), data prediction and repair based on the deep learning model are performed using the following steps: a) Data preprocessing, including outlier processing and moving average smoothing; b) Build a time series forecasting model; c) Train the model and perform monitoring data prediction or repair; d) The prediction accuracy was evaluated using RMSE and Pearson correlation coefficient.

[0020] In a preferred embodiment, in step d), a deep learning module is used to repair missing data and predict future data. Different types of monitoring data often have different temporal characteristics. Traditional simple interpolation and mean / median replacement methods fail to reflect the true nature of the data. The deep learning module uses an LSTM layer to construct a time series prediction model. LSTM is an algorithm that improves the recurrent neural network (RNN) model. It incorporates input gates, forget gates, output gates, and storage units into the RNN network structure, thereby improving the performance of RNNs. This method effectively overcomes short-term dependencies in RNNs, enabling more efficient information extraction from long time series data. Furthermore, the LSTM model offers the advantage of dynamically processing historical data, enabling storage and memorization of historical data. Compared to traditional neural network models, LSTM maintains the advantages of RNNs in parameter transfer. Its hidden layer also enables parameter sharing through inter-node connections and can continuously calculate and update based on input information. Figure 9 is the network structure diagram of LSTM, where For the input gate, For hidden doors, is the output gate, and the mathematical expressions of the three gates are as follows.

[0021]

[0022] Where, is the output of the previous time step; is the input information; σ is the sigmoid function; W is the coefficient matrix of each gate; By output gate and cell status Get, unit status By input gate With hidden door The calculation method is as follows.

[0023]

[0024] in Indicates the updated value of the unit state, and finally the output gate and cell status Calculate the output of the next layer .

[0025] .

[0026] In a preferred embodiment, in step d), the user learns a large amount of historical data by setting indicators such as sequence length, prediction steps, moving average window size, outlier threshold, and number of training rounds, and obtains the characteristics of the monitoring data changing over time, so as to accurately simulate missing data or predict future data. The present invention also provides RSME and Pearson correlation coefficient indicators to evaluate the accuracy of deep learning predicted data, and the principle is as follows.

[0027]

[0028] Among them, is the actual value, is the predicted value, and n is the number of observed values; the closer the value of RMSE is to 0, the higher the prediction accuracy.

[0029]

[0030] Among them, and are the i-th observed values of the two variables respectively, and are the means of the two variables respectively, n is the number of observed values, and r is the Pearson correlation coefficient. The value range of r is: [-1, 1]; when 0 < r ≤ 1, there is a positive correlation between the independent variable and the dependent variable; when -1 < r ≤ 0, there is a negative correlation between the independent variable and the dependent variable; when |r| ≥ 0.8, it is considered highly correlated; when 0.5 ≤ |r| < 0.8, it is considered moderately correlated; when 0.3 ≤ |r| < 0.5, it is considered weakly correlated; when |r| ≤ 0.3, the correlation between the independent variable and the dependent variable is considered very weak or irrelevant.

[0031] In a preferred embodiment, in step e), the content of the quality inspection report includes: a) Statistical information of basic data; b) Summary of quality inspection results; c) Export of quality inspection report.

[0032] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention are: 1. In the present invention, by combining flexible rule configuration with advanced time series analysis technology, the reliability and practicality of natural resource monitoring data are significantly improved. First, the method supports users to customize multi-dimensional quality inspection rules according to actual scenario requirements, including rules such as outliers, boundary values, mutation values, time continuity and logic, breaking through the limitations of traditional fixed rules. By introducing time logic checks, the system can effectively identify anomalies in the data caused by seasonal or cyclical changes, such as the reasonable fluctuation range of vegetation index in different seasons, thereby more accurately capturing the temporal characteristics of the data. In addition, the multi-device consistency comparison function realizes cross-device data cross-validation, which can promptly detect equipment failures or local environmental anomalies, and ensure the logical consistency and overall credibility of multi-source data.

[0033] 2. In the present invention, a deep learning model is applied to solve the problem of data repair and prediction. By utilizing the dynamic modeling ability of the LSTM network for long time series data, the system can learn the complex nonlinear characteristics of historical data, such as the exponential change trend of pollutant concentration, which significantly improves the accuracy of missing data repair and future predictions. Compared with traditional linear interpolation or mean replacement methods, the deep learning model can better fit the real data distribution and reduce the repair error. At the same time, the system supports quantitative evaluation of prediction effects through indicators such as RMSE and Pearson correlation coefficient, providing a scientific verification basis for data quality. Ultimately, this method forms a complete closed loop from data import, quality inspection, repair optimization to report export through the synergy of flexible rules and intelligent algorithms, providing an efficient, accurate and scalable solution for comprehensive natural resource monitoring sites. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a system architecture diagram of the present invention; Figure 2 This is a flow chart of data quality control in the present invention; Figure 3 This is a schematic diagram of the main interface of the quality control tool in the present invention; Figure 4 This is a schematic diagram of the interface for setting time-continuous property inspection rules in the present invention; Figure 5 This is a schematic diagram of the interface for setting quality inspection rules for outliers and mutation values ​​in the present invention; Figure 6 This is a schematic diagram of the interface for setting the threshold value quality inspection rules of the present invention; Figure 7 This is a schematic diagram of the interface for setting time logic quality inspection rules in the present invention; Figure 8 This is a schematic diagram of the interface for setting quality inspection rules for multi-device consistency comparison in the present invention; Figure 9This is the LSTM network structure diagram in the present invention; Figure 10 This is a schematic diagram of the deep learning module interface in the present invention; Figure 11 This is a schematic diagram of the quality inspection report interface of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Example

[0036] Reference Figure 1-11 , A monitoring data quality control method based on deep learning and self-built rules, the method comprising the following steps: a) establishing a natural resource monitoring data quality control database, the database including a monitoring data table, a quality inspection rule table, and a quality inspection log table; b) Import natural resource monitoring data, and support direct import and export of Excel files; c) Perform multi-dimensional quality checks based on self-built quality inspection rules, including: outlier check; boundary value check; mutation value check; time continuity check; time logic check; and multi-device consistency comparison check.

[0037] d) Use deep learning modules to perform data prediction and repair based on deep learning models; e) Generate quality inspection report.

[0038] In step c), the establishment and inspection method of quality inspection rules for abnormal values, limit values, and mutation values ​​include: a) Calculate outliers (Z-score method), upper and lower limits (percentile threshold method), data change rate, etc. to assist in determining quality inspection rules; b) Set quality inspection rules for abnormal values, limit values, and mutation values; c) determining whether the data point exceeds the threshold range; d) Mark or delete data points that exceed the threshold.

[0039] In the step c), Outlier detection is determined based on the specific device requirements and expert experience. Outliers are determined using the Z-score method, as follows.

[0040] .

[0041] in, is the data mean, σ is the standard deviation, To monitor the data value; the "3σ" principle is used to test the data. When the Z-score is greater than the set threshold (3.0), x is determined to be an abnormal value; The limit value check uses the percentile threshold method to determine the upper and lower limits of the limit value. The principle is: after removing outliers from the original data, add or subtract 1 standard deviation based on the 99.9% (upper limit) or 0.1% (lower limit) profile to form dynamic upper and lower limit thresholds.

[0042] The mutation value check determines the rules by calculating the rate of change between adjacent moments. The principle is as follows.

[0043] .

[0044] in It is the monitoring data at time t. When the change rate exceeds the set maximum allowable change rate, it is determined to be a mutation value.

[0045] The limit value check uses the percentile threshold method to determine the upper and lower limits of the limit value. The principle is: after removing outliers from the original data, add or subtract 1 standard deviation based on the 99.9% (upper limit) or 0.1% (lower limit) profile to form dynamic upper and lower limit thresholds.

[0046] The mutation value check determines the rules by calculating the rate of change between adjacent moments. The principle is as follows.

[0047] .

[0048] in It is the monitoring data at time t. When the change rate exceeds the set maximum allowable change rate, it is determined to be a mutation value.

[0049] Multi-device consistency comparison check: Users can ensure the consistency of data with internal logical relationships between different devices by defining the maximum difference allowed between different data on different devices. This is important for discovering equipment failures, data transmission errors, or local anomalies of natural resources.

[0050] In step c), the establishment and inspection method of the quality inspection rules for time continuity include: a) Set the monitoring frequency of the data according to the equipment, in minutes; b) Determine whether there are missing values ​​in the time series of data points; c) Mark the locations of missing data or simply interpolate to fill them.

[0051] In step c), the establishment and inspection method of the quality inspection rules of the time logic include: a) Set rules for monitoring data changes over time; b) Check whether the data value is within the expected range for the time period; c) Mark the data that does not conform to the rules.

[0052] In step c), the quality inspection rule establishment and inspection method for multi-device consistency comparison include: a) Set up associated device groups; b) define the maximum permissible difference; c) Compare the differences in monitoring values ​​of devices within the group; d) Mark data that exceeds the allowed range.

[0053] In step d), data prediction and repair based on the deep learning model are performed using the following steps: a) Data preprocessing, including outlier processing and moving average smoothing; b) Build a time series forecasting model; c) Train the model and perform monitoring data prediction or repair; d) The prediction accuracy was evaluated using RMSE and Pearson correlation coefficient.

[0054] In step d), the deep learning module is used to repair missing data and predict future data. Different types of monitoring data often have different temporal characteristics. Traditional methods such as simple interpolation and mean / median replacement for data repair fail to accurately reflect the data's true state. The deep learning module uses an LSTM layer to construct a time series prediction model. LSTM is an algorithm that improves the recurrent neural network (RNN) model. It incorporates input gates, forget gates, output gates, and storage units into the RNN network structure, thereby improving the performance of RNNs. This method effectively overcomes short-term dependencies in RNNs, enabling more efficient information extraction from long-term time series data. Furthermore, the LSTM model offers the advantage of dynamically processing historical data, enabling storage and memorization of historical data. Compared to traditional neural network models, LSTM maintains the advantages of RNNs in parameter transfer. Its hidden layer also shares parameters through inter-node connections and can continuously calculate and update based on input information. Figure 9 is the network structure diagram of LSTM, where For the input gate, For hidden doors, is the output gate, and the mathematical expressions of the three gates are as follows.

[0055]

[0056] Where, is the output of the previous time step; is the input information; σ is the sigmoid function; W is the coefficient matrix of each gate; is obtained by the output gate and the cell state The cell state is obtained by the input gate and the hidden gate The calculation is as follows.

[0057]

[0058] where represents the cell state update value, and finally the output of the next layer is calculated by the output gate and the cell state .

[0059] .

[0060] In step d), the user learns a large amount of historical data by setting indicators such as sequence length, prediction steps, moving average window size, outlier threshold, and number of training rounds to obtain the characteristics of the monitoring data changing over time, so as to accurately simulate missing data or predict future data. The present invention also provides RSME and Pearson correlation coefficient indicators to evaluate the accuracy of deep learning prediction data, and the principle is as follows.

[0061]

[0062] where, is the actual value, is the predicted value, and n is the number of observed values; the closer the value of RMSE is to 0, the higher the prediction accuracy.

[0063]

[0064] where, and are the i-th observed values of two variables respectively, and are the means of two variables respectively, n is the number of observed values, and r is the Pearson correlation coefficient. The value range of r is: [-1, 1]; when 0 < r ≤ 1, there is a positive correlation between the independent variable and the dependent variable; when -1 < r ≤ 0, there is a negative correlation between the independent variable and the dependent variable; when |r| ≥ 0.8, it is considered highly correlated; when 0.5 ≤ |r| < 0.8, it is considered moderately correlated; when 0.3 ≤ |r| < 0.5, it is considered lowly correlated; when |r| ≤ 0.3, the correlation between the independent variable and the dependent variable is considered very weak or considered uncorrelated.

[0065] In step e), the contents of the quality inspection report generated include: a) Statistics of basic data information; b) Summary of quality inspection results; c) Export quality inspection report.

[0066] From the above we can know: In the present invention, by combining flexible rule configuration with advanced time series analysis technology, the reliability and practicality of natural resource monitoring data are significantly improved. First, the method supports users to customize multi-dimensional quality inspection rules according to actual scenario requirements, including rules such as outliers, boundary values, mutation values, time continuity and logic, breaking through the limitations of traditional fixed rules. By introducing time logic checks, the system can effectively identify anomalies in the data caused by seasonal or cyclical changes, such as the reasonable fluctuation range of vegetation index in different seasons, thereby more accurately capturing the temporal characteristics of the data. In addition, the multi-device consistency comparison function realizes cross-device data cross-validation, which can promptly detect equipment failures or local environmental anomalies, and ensure the logical consistency and overall credibility of multi-source data.

[0067] In the present invention, a deep learning model is applied to solve the problem of data repair and prediction. By utilizing the dynamic modeling ability of the LSTM network for long time series data, the system can learn the complex nonlinear characteristics of historical data, such as the exponential change trend of pollutant concentration, which significantly improves the accuracy of missing data repair and future predictions. Compared with traditional linear interpolation or mean replacement methods, the deep learning model can better fit the real data distribution and reduce the repair error. At the same time, the system supports quantitative evaluation of the prediction effect through indicators such as RMSE and Pearson correlation coefficient, providing a scientific verification basis for data quality. Ultimately, this method forms a complete closed loop from data import, quality inspection, repair optimization to report export through the synergy of flexible rules and intelligent algorithms, providing an efficient, accurate and scalable solution for comprehensive natural resource monitoring sites.

[0068] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0069] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A monitoring data quality control method based on deep learning and self-built rules, characterized by: The method comprises the following steps: a) establishing a natural resource monitoring data quality control database, the database including a monitoring data table, a quality inspection rule table, and a quality inspection log table; b) Import natural resource monitoring data, and support direct import and export of Excel files; c) Perform multi-dimensional quality checks based on self-built quality inspection rules, including: outlier check; boundary value check; mutation value check; time continuity check; time logic check; and multi-device consistency comparison check; d) Use deep learning modules to perform data prediction and repair based on deep learning models; e) Generate quality inspection report.

2. The monitoring data quality control method based on deep learning and self-built rules according to claim 1, characterized in that: In step c), the establishment and inspection method of quality inspection rules for abnormal values, limit values, and mutation values ​​include: a) Calculate outliers (Z-score method), upper and lower limits (percentile threshold method), data change rate, etc. to assist in determining quality inspection rules; b) Set quality inspection rules for abnormal values, limit values, and mutation values; c) determining whether the data point exceeds the threshold range; d) Mark or delete data points that exceed the threshold.

3. The monitoring data quality control method based on deep learning and self-built rules according to claim 1, characterized in that: In the step c), Outlier checks are determined based on the specific device requirements and expert experience. Outliers are determined using the Z-score, as follows: 。 in, is the data mean, σ is the standard deviation, To monitor the data value; the "3σ" principle is used to test the data. When the Z-score is greater than the set threshold (3.0), x is determined to be an abnormal value; The limit value check uses the percentile threshold method to determine the upper and lower limits of the limit value. The principle is: after removing outliers from the original data, add or subtract 1 standard deviation based on the 99.9% (upper limit) or 0.1% (lower limit) profile to form dynamic upper and lower limit thresholds; The mutation value check determines the rules between them by calculating the rate of change between adjacent moments. The principle is as follows: 。 in is the monitoring data at time t. When the change rate exceeds the set maximum allowable change rate, it is determined to be a sudden change value; Multi-device consistency comparison check: Users can ensure the consistency of data with internal logical relationships between different devices by defining the maximum difference allowed between different data on different devices. This is important for discovering equipment failures, data transmission errors, or local anomalies of natural resources.

4. The monitoring data quality control method based on deep learning and self-built rules according to claim 1, characterized in that: In step c), the establishment and inspection method of the quality inspection rules for time continuity include: a) Set the monitoring frequency of the data according to the equipment, in minutes; b) Determine whether there are missing values ​​in the time series of data points; c) Mark the locations of missing data or simply interpolate to fill them.

5. The monitoring data quality control method based on deep learning and self-built rules according to claim 1, characterized in that: In step c), the quality inspection rule establishment and inspection method of time logic includes: a) Set rules for monitoring data changes over time; b) Check whether the data value is within the expected range for the time period; c) Mark the data that does not conform to the rules.

6. The monitoring data quality control method based on deep learning and self-built rules according to claim 1, characterized in that: In step c), the quality inspection rule establishment and inspection method for multi-device consistency comparison include: a) Set up associated device groups; b) define the maximum permissible difference; c) Compare the differences in monitoring values ​​of devices within the group; d) Mark data that exceeds the allowed range.

7. The monitoring data quality control method based on deep learning and self-built rules according to claim 1, characterized in that: In step d), data prediction and repair based on the deep learning model are performed using the following steps: a) Data preprocessing, including outlier processing and moving average smoothing; b) Build a time series forecasting model; c) Train the model and perform monitoring data prediction or repair; d) The prediction accuracy was evaluated using RMSE and Pearson correlation coefficient.

8. The monitoring data quality control method based on deep learning and self-built rules according to claim 1, characterized in that: In the step d), the deep learning module is used to repair missing data and predict future data; different types of monitoring data often have different time characteristics, and traditional simple interpolation and mean / median replacement repair data methods are difficult to reflect the actual situation of the data; the deep learning module uses the LSTM layer to construct a time series prediction model. LSTM is an algorithm that improves the recurrent neural network (RNN) model. In the structure of the RNN network, input gates, forget gates, output gates, storage units and other functions are added to the RNN network structure, thereby improving the performance of the recurrent neural network; this method can well overcome the short-term correlation in the recurrent neural network, so that it can more efficiently extract information from long time series data; at the same time, the LSTM model also has the advantage of dynamically processing historical data, and can realize the storage and memory of historical data; compared with the traditional neural network model, LSTM maintains the advantages of recurrent neural networks in parameter transfer, and its hidden layer can also realize parameter sharing through the connection between nodes, and can continuously calculate and update according to the input information; Figure 9 is a network structure diagram of LSTM, where For the input gate, For hidden doors, is the output gate, and the mathematical expressions of the three gates are as follows; Where, is the output of the previous time step; is input information; σ is the sigmoid function; W is the coefficient matrix of each gate; By output gate and cell status Get, unit status By input gate With hidden door The calculation is as follows; in Indicates the updated value of the unit state, and finally the output gate and cell status Calculate the output of the next layer ; 。 9. The monitoring data quality control method based on deep learning and self-built rules according to claim 1, characterized in that: In step d), the user sets indicators such as sequence length, number of prediction steps, moving average window size, outlier threshold, and number of training rounds to learn from a large amount of historical data and obtain the characteristics of monitoring data changing over time, so as to accurately simulate missing data or predict future data. The present invention also provides RSME and Pearson correlation coefficient indicators to evaluate the accuracy of deep learning prediction data. The principle is as follows: in, is the actual value, is the predicted value, n is the number of observations; the closer the RMSE value is to 0, the higher the prediction accuracy; wherein, and are respectively the i-th observed values of two variables, and are respectively the means of the two variables, n is the number of observed values, and r is the Pearson correlation coefficient; the value range of r is: [-1, 1]; when 0 < r ≤ 1, there is a positive correlation between the independent variable and the dependent variable; when -1 < r ≤ 0, there is a negative correlation between the independent variable and the dependent variable; when |r| ≥ 0.8, it is considered highly correlated; when 0.5 ≤ |r| < 0.8, it is considered moderately correlated; when 0.3 ≤ |r| < 0.5, it is considered lowly correlated; when |r| ≤ 0.3, the correlation between the independent variable and the dependent variable is considered very weak or considered uncorrelated.

10. The monitoring data quality control method based on deep learning and self-built rules according to claim 1, characterized in that: In step e), the contents of the quality inspection report generated include: a) Statistics of basic data information; b) Summary of quality inspection results; c) Export quality inspection report.

Images