Data optimization and early warning method and system based on recurrent neural network

Abstract

The invention provides a data optimization and early warning method and system based on a recurrent neural network, and relates to the technical field of data processing, and the method comprises the steps: obtaining historical measurement data and reference data; performing normalization operation on the historical measurement data to obtain an input vector; constructing a data optimization model based on a recurrent neural network according to a time relationship between historical measurement data; inputting the input vector into the data optimization model, and smoothing abnormal data in the historical measurement data; constructing a data early warning model; inputting the smoothed historical measurement data into a data early warning model in the form of an input vector, and outputting a final prediction value of the data early warning model; calibrating the historical measurement data and the final prediction value according to the reference data and the final prediction value; and judging whether the calibration result is greater than the calibration threshold value of the corresponding calibration category, if so, performing early warning, otherwise, continuing monitoring. The precision and accuracy of the measuring equipment can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data optimization and early warning method and system based on a recurrent neural network. Background Art

[0002] Recurrent Neural Network (RNN) is a neural network architecture that is mainly used to process time series data or data with time dependencies. Data optimization refers to processing, converting or modifying raw data through various methods to improve data quality or its effectiveness in model training. Data optimization and early warning based on recurrent neural networks is a technical method that uses recurrent neural networks (RNN) to process and optimize data, and combines early warning mechanisms to timely identify anomalies and potential risks, issue warnings in advance, or take corresponding measures.

[0003] The data optimization and early warning method based on recurrent neural network (RNN) can effectively process time series data, capture the time dependencies in the data, and improve the accuracy and reliability of the data. It is of great significance to improve the accuracy and reliability of agricultural environmental monitoring, and can improve the safety and efficiency of the monitoring system in complex and dynamic environments.

[0004] In the long-term monitoring process, traditional data processing methods are difficult to effectively capture its dynamic changes, resulting in insufficient data calibration and prediction accuracy. Data optimization and calibration mostly use simple filtering algorithms or linear regression methods. These methods are difficult to handle complex time series data characteristics, especially in the case of data anomalies and nonlinear drift. In addition, traditional methods lack the ability to predict future data and cannot realize the real-time warning function of predicted values. In general, the existing data processing methods have many technical bottlenecks in practical applications, resulting in insufficient data calibration and prediction accuracy, unable to realize the real-time warning function of predicted values, and difficult to meet the requirements of detection accuracy and environmental monitoring safety capabilities. Summary of the invention

[0005] In order to solve the technical problems that existing data processing methods have many technical bottlenecks in practical applications, resulting in insufficient data calibration and prediction accuracy, inability to realize the real-time warning function of the predicted value, and difficulty in meeting the requirements of detection accuracy and environmental monitoring safety capabilities, the present invention provides a data optimization and early warning method and system based on a recurrent neural network.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect:

[0008] The embodiment of the present invention provides a data optimization and early warning method based on a recurrent neural network, comprising:

[0009] S1: Obtain historical measurement data and reference data;

[0010] S2: Normalize the historical measurement data to obtain an input vector;

[0011] S3: According to the time relationship between historical measurement data, a data optimization model based on recurrent neural network is constructed;

[0012] S4: Input the input vector to the data optimization model to smooth the abnormal data in the historical measurement data;

[0013] S5: constructing a data early warning model, wherein the data early warning model includes a recurrent neural network module and a feature enhancement structure module, and the feature enhancement structure module includes a dropout layer, a first linear layer, a second linear layer, and a fully connected layer;

[0014] S6: Input the smoothed historical measurement data into the data early warning model in the form of an input vector, and output the final prediction value of the data early warning model;

[0015] S7: Calibrate the historical measurement data and the final prediction value according to the reference data and the final prediction value;

[0016] S8: Determine whether the calibration result is greater than the calibration threshold of the corresponding calibration category; if so, issue an early warning; otherwise, continue monitoring.

[0017] Second aspect:

[0018] An embodiment of the present invention provides a data optimization and early warning system based on a recurrent neural network, comprising:

[0019] processor;

[0020] A memory having computer-readable instructions stored thereon, which, when executed by a processor, implements the data optimization and early warning method based on a recurrent neural network as in the first aspect.

[0021] The third aspect:

[0022] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the data optimization and early warning method based on a recurrent neural network as in the first aspect is implemented.

[0023] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0024] In an embodiment of the present invention, by acquiring historical measurement data and reference data, and performing normalization operations on the historical measurement data, an input vector is generated, and according to the time relationship between the historical measurement data, a data optimization model based on a recurrent neural network is constructed. The data optimization model can process the dependency of time series data, and model the time series of historical data to capture the rules therein, thereby providing a basis for data optimization. Then, the input vector is input into the data optimization model, the abnormal data in the historical measurement data is smoothed, and a data early warning model is constructed to further enhance the prediction and abnormality detection capabilities of the model. Finally, the smoothed historical measurement data is input into the data early warning model in the form of an input vector, the final prediction value of the data early warning model is output, and the historical measurement data and the final prediction value are calibrated, thereby starting the early warning mechanism, reducing the error of low-cost sensors, making the prediction results more credible, improving data calibration and prediction accuracy, and realizing the real-time early warning function of the prediction value, meeting the requirements of detection accuracy and environmental monitoring safety capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic diagram of a flow chart of a data optimization and early warning method based on a recurrent neural network provided in an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of the structure of a data optimization and early warning system based on a recurrent neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0029] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0030] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0031] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0032] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0033] Reference Manual Attached Figure 1 , showing a flow chart of a data optimization and early warning method based on a recurrent neural network provided in an embodiment of the present invention.

[0034] The embodiment of the present invention provides a data optimization and early warning method based on a recurrent neural network, which can be implemented by a data optimization and early warning device based on a recurrent neural network, and the data optimization and early warning device based on a recurrent neural network can be a terminal or a server. The processing flow of the data optimization and early warning method based on a recurrent neural network may include the following steps:

[0035] S1: Obtain historical measurement data and reference data.

[0036] Among them, historical measurement data refers to data about specific environments or phenomena collected by sensors or other devices over a period of time in the past, and reference data refers to data used as a comparison standard, usually from high-precision sensors or other reliable sources, as the basis for model optimization, calibration or verification.

[0037] Specifically, the historical measurement data are collected by low-cost soil EC sensors, and the reference data are collected by high-precision soil EC sensors.

[0038] Specifically, low-cost soil EC sensors are usually less accurate and easily affected by environmental factors (such as temperature, humidity, soil type, etc.), and may have measurement errors and data drift problems. High-precision soil EC sensors are more accurate, can accurately measure the electrical conductivity of the soil, and are less affected by environmental factors. They are usually equipped with calibration functions to reduce errors that may occur in long-term use. Low-cost soil EC sensors are suitable for large-scale, low-budget soil monitoring tasks. Although they have lower accuracy, they are very practical in large-scale data collection and cost-sensitive situations. High-precision soil EC sensors are suitable for scenarios with higher accuracy requirements. They can provide accurate and reliable soil conductivity data and are suitable for applications that require high precision such as precision agriculture and scientific research, but they are more expensive.

[0039] It should be noted that obtaining historical measurement data and reference data lays the foundation for the entire data optimization and early warning process. Historical measurement data provides the model with actual data from the past to help capture the regularity of time series. Reference data, as a high-precision standard, can effectively improve the accuracy of the model and reduce the impact of low-cost sensor measurement errors.

[0040] In a possible implementation manner, after S1, the step further includes: preprocessing the historical measurement data.

[0041] In a possible implementation manner, the preprocessing specifically includes: time alignment processing and simple outlier removal processing.

[0042] Among them, the time alignment process is to determine the data time interval according to the data sampling rate, and the missing data is supplemented by cubic spline difference. The simple outlier removal process is to exclude the data at each time point from the available data if it is zero or exceeds 95% of the sensor measurement threshold.

[0043] It should be noted that through time alignment and simple outlier removal processing, the consistency, integrity and reliability of the data can be guaranteed, which provides higher quality data input for subsequent model training, thereby improving the accuracy, stability and predictive ability of the model.

[0044] S2: Normalize the historical measurement data to obtain an input vector.

[0045] Normalization is to scale the data to a fixed range, usually from 0 to 1. This operation can eliminate the magnitude differences of different eigenvalues, ensuring that the data has the same importance during processing, making subsequent analysis or model training more efficient and stable. The input vector refers to the conversion of normalized data into a form that the model can process, usually in the form of a vector or matrix, which contains all the feature data that the model needs to process.

[0046] It should be noted that by normalizing the historical measurement data, data with different features can be compared on a unified scale, thus eliminating the impact of different dimensions between features. This not only improves the efficiency of data processing, but also helps improve the training performance of the model, because the normalized data is easier to adapt to various machine learning models, avoiding the excessive impact of certain features on model learning.

[0047] S3: According to the temporal relationship between historical measurement data, a data optimization model based on recurrent neural network is constructed.

[0048] Among them, the recurrent neural network is a neural network specifically used to process time series data. Unlike traditional feedforward neural networks, RNN has memory ability and can pass the output of the previous moment to the current moment, thereby capturing the time dependency in time series data. The data optimization model is a model that corrects and optimizes data by learning the patterns and rules of historical data.

[0049] It should be noted that by building a data optimization model based on a recurrent neural network (RNN) according to the temporal relationship of historical measurement data, the temporal dependencies in the data can be captured. RNN can automatically learn and understand the regularity of data changes over time, and is particularly suitable for processing time series data. By establishing such an optimization model, noise and outliers in the data can be effectively removed, and historical data can be smoothed, making the data more stable and accurate, and enhancing the applicability of the data to subsequent models.

[0050] S4: Input the input vector to the data optimization model to smooth the abnormal data in the historical measurement data.

[0051] Among them, abnormal data refers to values ​​that deviate from normal data patterns. These values ​​are usually caused by sensor failure, environmental interference, or data collection errors. Smoothing is a technique to eliminate anomalies and noise by reducing data fluctuations. Data is usually smoothed by averaging or other mathematical methods to improve its stability and predictability.

[0052] It should be noted that by inputting the normalized input vector into the data optimization model and using the model to smooth outliers in the historical measurement data, the data quality can be significantly improved.

[0053] In a possible implementation, S4 specifically includes:

[0054] S401: Segment the input vector according to the time window size.

[0055] S402: Divide the segmented input vector into a training set and a test set according to a preset ratio.

[0056] Among them, those skilled in the art can set the size of the preset ratio according to actual conditions. In the present invention, the preset ratio is specifically: 7:3.

[0057] S403: Input the training set into the data optimization model for training until the loss function value of the data optimization model is less than a preset function value.

[0058] Among them, those skilled in the art can set the size of the preset function value according to actual conditions, and the present invention is not limited here.

[0059] In a possible implementation, the loss function of the data optimization model is specifically: a root mean square error loss function.

[0060] S404: Use the trained data optimization model to smooth abnormal data in the historical measurement data.

[0061] It should be noted that by utilizing the ability of data optimization models, abnormal data and noise are effectively suppressed, and the data is smoothed to be more stable, reducing interference in model training. Such optimized data not only reduces the impact of abnormal fluctuations on predictions, but also helps subsequent model training to be more accurate and efficient, and improves the stability and reliability of prediction results.

[0062] In a possible implementation, S404 specifically includes:

[0063] S4041: Input the training set and the test set into the trained data optimization model respectively, and output the training prediction value and the test prediction value respectively.

[0064] S4042: Perform denormalization operations on the training prediction value and the test prediction value respectively.

[0065] S4043: Concatenate the training prediction value and the test prediction value after the denormalization operation to obtain a concatenated prediction value.

[0066] S4044: Calculate the abnormal threshold value based on the relationship between the splicing prediction value and the input vector:

[0067] ;

[0068] ;

[0069] Among them, T represents the abnormal threshold, E represents the average value of the error, and e i represents the error of the ith data point, , N represents the total number of data points, e represents the error, represents the predicted value, and X represents the input vector.

[0070] S4045: Smoothing the abnormal data in the historical measurement data according to the abnormal threshold.

[0071] It should be noted that the data optimization model can be constructed based on a recurrent neural network according to the time correlation between sensor measurement data, discover the law of data change, set the outlier threshold, and smooth the abnormal data that exceeds the outlier threshold to achieve anomaly detection and data smoothing.

[0072] S5: Construct a data early warning model, wherein the data early warning model includes a recurrent neural network module and a feature enhancement structure module, and the feature enhancement structure module includes a dropout layer, a first linear layer, a second linear layer, and a fully connected layer.

[0073] Among them, the data warning model is a model that identifies potential anomalies or risks by analyzing historical data and current data, and the recurrent neural network (RNN) module is a deep learning structure for processing time series data. The feature enhancement structure module is used to enhance the feature learning ability of the network. It usually includes a dropout layer (to prevent overfitting), as well as multiple linear layers and fully connected layers (to enhance feature expression).

[0074] Among them, the Dropout layer is a regularization technology used to prevent the neural network from overfitting. The first linear layer, the second linear layer and the fully connected layer are common neural network structures in the model. The linear layer realizes the linear transformation of the input data through weighted summation, while the fully connected layer further enhances the learning ability of the model through more nonlinear transformations, helping the model to make better predictions.

[0075] It should be noted that a powerful data early warning model is constructed by combining recurrent neural network (RNN) and feature enhancement structure module. RNN can effectively capture the time dependency of time series data, so that the model can process and predict complex data with time characteristics. The feature enhancement structure module enhances the model's learning ability of data features through dropout layers and multi-layer fully connected layers, and effectively prevents overfitting.

[0076] S6: Input the smoothed historical measurement data into the data early warning model in the form of an input vector, and output the final prediction value of the data early warning model.

[0077] Among them, the final prediction value is the result output by the data early warning model, which represents the model's prediction of future data. By inputting the smoothed data into the data early warning model and using the recurrent neural network (RNN) module for further prediction, the model can better understand the patterns and changing trends in time series data. RNN can capture the time dependency of historical data and improve the accuracy and reliability of predictions. At the same time, using smoothed data as input reduces the impact of data noise and outliers and improves the stability of model predictions.

[0078] In a possible implementation, S6 specifically includes:

[0079] S601: Input the smoothed historical measurement data into the data early warning model in the form of an input vector, and calculate the current implicit state through the recurrent neural network module:

[0080] ;

[0081] Among them, h t represents the hidden state at time t, tanh represents the tanh activation function, W h and W x Both represent weight matrices, h t-1 represents the implicit state at time t-1, x t represents the input vector at time t, and b represents the bias term.

[0082] It should be noted that the hidden state is the mechanism for storing and transmitting information in RNN. At each time step, RNN calculates a new hidden state based on the current input and the hidden state of the previous moment, which represents the network's memory of historical information. The hidden state helps the network capture long-term dependencies in time series.

[0083] S602: Output prediction results according to the current implicit state:

[0084] ;

[0085] Among them, y t represents the output at time t, W y represents the weight matrix of the output layer, b y Represents the bias term of the output layer.

[0086] S603: Input the prediction result to the feature enhancement structure module until the loss function value of the data early warning model is less than the preset loss function value, and output the final prediction value of the data early warning model:

[0087] ;

[0088] Among them, y outrepresents the final prediction value, W1 represents the coefficient matrix of the first fully connected layer, W2 represents the coefficient matrix of the second fully connected layer, W3 represents the coefficient matrix of the third fully connected layer, z1 represents the linear output of the first layer, z2 represents the linear output of the second layer, z3 represents the linear output of the third layer, b1 represents the bias term of the first layer, b2 represents the bias term of the second layer, b3 represents the bias term of the second layer, and ReLU represents the ReLU activation function.

[0089] It should be noted that those skilled in the art can set the size of the preset loss function value according to actual needs, and the present invention is not limited here.

[0090] It should be noted that through this structure, the early warning model can not only improve the prediction accuracy, but also provide more reliable early warning information in various dynamically changing scenarios, identify potential anomalies or risks in advance, and improve the system's response speed and decision-making efficiency.

[0091] In a possible implementation, the loss function of the data early warning model is specifically:

[0092] ;

[0093] Among them, loss second represents the loss function, α represents the weight coefficient controlling the mean square error loss function, loss MSE represents the mean square error loss function, β represents the weight coefficient of the control shape loss function, loss shape represents the shape loss function.

[0094] It should be noted that in practical applications, the weight of the shape loss function is greater than the mean square error loss function in order to better adjust the weight of the model's optimization objectives, thereby paying more attention to the accuracy of shape and time distortion. This makes the model sensitive to shape features during the learning process while taking into account the overall fitting effect of the mean square error.

[0095] S7: Calibrate the historical measurement data and the final prediction value based on the reference data and the final prediction value.

[0096] Among them, calibration is the process of comparing the model's prediction results with high-precision reference data, adjusting and correcting the model output. Through calibration, the model can reduce measurement errors and deviations to ensure that the prediction results are closer to the actual values.

[0097] It should be noted that the accuracy and reliability of the prediction can be significantly improved by calibrating the reference data and the final prediction value. The calibration process ensures that the output of the low-cost sensor and prediction model is consistent with the high-precision reference data, reducing the error caused by sensor error or model bias. This process makes the entire data warning system more stable and reliable, providing a more accurate basis for subsequent risk identification and decision-making.

[0098] S8: Determine whether the calibration result is greater than the calibration threshold of the corresponding calibration category. If so, issue an early warning. Otherwise, continue monitoring.

[0099] The calibration result is the output value obtained after the calibration process, which represents the adjusted data prediction result. When the calibration result exceeds the preset threshold, the system triggers an alarm mechanism to remind users of possible risks or abnormal situations. Monitoring means that the system continues to observe and track changes in data until an abnormality is found and an alarm is triggered.

[0100] Among them, those skilled in the art can set the size of the calibration threshold according to actual conditions, and the present invention is not limited here.

[0101] The calibration categories specifically include two calibration categories: historical measurement data and final prediction value.

[0102] It should be noted that whether to trigger an early warning or continue monitoring is determined based on the calibration results and calibration thresholds. This process ensures the real-time and accuracy of data processing, effectively reduces false alarms, and avoids overreaction. When the calibration result exceeds the threshold, the system will trigger an early warning in time to help users identify potential risks or anomalies in advance and take measures to avoid further development of the problem. When the data is within the normal range, the system continues to monitor to avoid unnecessary interference. This mechanism improves the flexibility and efficiency of the system and ensures timely and accurate responses in various situations.

[0103] In the present invention, an optimization and early warning model of non-stationary time series is constructed through a recurrent neural network method to provide abnormal situation warnings, improve the accuracy and precision of the measured values ​​of low-cost soil EC sensors, and extend their service life.

[0104] At the same time, the present invention does not affect the process of sensor data collection. It optimizes the obtained measurement values ​​at the software algorithm level, takes the measurement values ​​of high-precision sensors as true values, removes abnormal values, smoothes data, and obtains calibration values ​​and future prediction values. It is simple and easy to operate, and improves the detection accuracy and environmental monitoring safety capabilities of low-cost soil EC sensors.

[0105] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0106] In an embodiment of the present invention, by acquiring historical measurement data and reference data, and performing normalization operations on the historical measurement data, an input vector is generated, and according to the time relationship between the historical measurement data, a data optimization model based on a recurrent neural network is constructed. The data optimization model can process the dependency of time series data, and model the time series of historical data to capture the rules therein, thereby providing a basis for data optimization. Then, the input vector is input into the data optimization model, the abnormal data in the historical measurement data is smoothed, and a data early warning model is constructed to further enhance the prediction and abnormality detection capabilities of the model. Finally, the smoothed historical measurement data is input into the data early warning model in the form of an input vector, the final prediction value of the data early warning model is output, and the historical measurement data and the final prediction value are calibrated, thereby starting the early warning mechanism, reducing the error of low-cost sensors, making the prediction results more credible, improving data calibration and prediction accuracy, and realizing the real-time early warning function of the prediction value, meeting the requirements of detection accuracy and environmental monitoring safety capabilities.

[0107] Reference Manual Attached Figure 2 , showing a structural schematic diagram of a data optimization and early warning system based on a recurrent neural network provided by the present invention.

[0108] The present invention further provides a data optimization and early warning system 20 based on a recurrent neural network, which is applied to the above-mentioned data optimization and early warning method based on a recurrent neural network, comprising:

[0109] Processor 201.

[0110] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201, the data optimization and early warning method based on a recurrent neural network as in the method embodiment is implemented.

[0111] The data optimization and early warning system 20 based on recurrent neural network provided by the present invention can execute the above-mentioned data optimization and early warning method based on recurrent neural network, and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0112] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0113] In an embodiment of the present invention, by acquiring historical measurement data and reference data, and performing normalization operations on the historical measurement data, an input vector is generated, and according to the time relationship between the historical measurement data, a data optimization model based on a recurrent neural network is constructed. The data optimization model can process the dependency of time series data, and model the time series of historical data to capture the rules therein, thereby providing a basis for data optimization. Then, the input vector is input into the data optimization model, the abnormal data in the historical measurement data is smoothed, and a data early warning model is constructed to further enhance the prediction and abnormality detection capabilities of the model. Finally, the smoothed historical measurement data is input into the data early warning model in the form of an input vector, the final prediction value of the data early warning model is output, and the historical measurement data and the final prediction value are calibrated, thereby starting the early warning mechanism, reducing the error of low-cost sensors, making the prediction results more credible, improving data calibration and prediction accuracy, and realizing the real-time early warning function of the prediction value, meeting the requirements of detection accuracy and environmental monitoring safety capabilities.

[0114] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0115] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0116] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0117] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0118] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0119] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0120] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0121] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0122] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0125] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0126] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, a data optimization and early warning method based on a recurrent neural network as in the method embodiment is implemented.

[0127] A computer-readable storage medium provided by the present invention can implement the steps and effects of the data optimization and early warning method based on a recurrent neural network in the above-mentioned method embodiment. To avoid repetition, the present invention will not go into details.

[0128] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0129] In an embodiment of the present invention, by acquiring historical measurement data and reference data, and performing normalization operations on the historical measurement data, an input vector is generated, and according to the time relationship between the historical measurement data, a data optimization model based on a recurrent neural network is constructed. The data optimization model can process the dependency of time series data, and model the time series of historical data to capture the rules therein, thereby providing a basis for data optimization. Then, the input vector is input into the data optimization model, the abnormal data in the historical measurement data is smoothed, and a data early warning model is constructed to further enhance the prediction and abnormality detection capabilities of the model. Finally, the smoothed historical measurement data is input into the data early warning model in the form of an input vector, the final prediction value of the data early warning model is output, and the historical measurement data and the final prediction value are calibrated, thereby starting the early warning mechanism, reducing the error of low-cost sensors, making the prediction results more credible, improving data calibration and prediction accuracy, and realizing the real-time early warning function of the prediction value, meeting the requirements of detection accuracy and environmental monitoring safety capabilities.

[0130] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0131] There are a few points to note:

[0132] (1) The drawings of the embodiments of the present invention only involve structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0133] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0134] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0135] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A data optimization and early warning method based on recurrent neural network, characterized in that: include: S1: Obtain historical measurement data and reference data; S2: performing a normalization operation on the historical measurement data to obtain an input vector; S3: constructing a data optimization model based on a recurrent neural network according to the time relationship between the historical measurement data; S4: inputting the input vector into the data optimization model to smooth the abnormal data in the historical measurement data; S5: constructing a data early warning model, wherein the data early warning model comprises a recurrent neural network module and a feature enhancement structure module, and the feature enhancement structure module comprises a dropout layer, a first linear layer, a second linear layer, and a fully connected layer; S6: inputting the smoothed historical measurement data into the data early warning model in the form of an input vector, and outputting a final prediction value of the data early warning model; S7: Calibrate the historical measurement data and the final prediction value according to the reference data and the final prediction value; S8: Determine whether the calibration result is greater than the calibration threshold of the corresponding calibration category; if so, issue an early warning; otherwise, continue monitoring.

2. The data optimization and early warning method based on recurrent neural network according to claim 1 is characterized in that: After S1, the step further includes: preprocessing the historical measurement data.

3. The data optimization and early warning method based on recurrent neural network according to claim 2 is characterized in that: The preprocessing specifically includes: time alignment processing and simple outlier removal processing.

4. The data optimization and early warning method based on recurrent neural network according to claim 1 is characterized in that: The S4 specifically includes: S401: Segment the input vector according to the time window size; S402: Divide the segmented input vector into a training set and a test set according to a preset ratio; S403: Inputting the training set into the data optimization model for training until the loss function value of the data optimization model is less than a preset function value; S404: Smoothing abnormal data in the historical measurement data using the trained data optimization model.

5. The data optimization and early warning method based on recurrent neural network according to claim 4 is characterized in that: The loss function of the data optimization model is specifically: root mean square error loss function.

6. The data optimization and early warning method based on recurrent neural network according to claim 4 is characterized in that: The S404 specifically includes: S4041: inputting the training set and the test set into the trained data optimization model respectively, and outputting the training prediction value and the test prediction value respectively; S4042: performing a denormalization operation on the training prediction value and the test prediction value respectively; S4043: concatenating the training prediction value and the test prediction value after the denormalization operation to obtain a concatenated prediction value; S4044: Calculate an abnormal threshold according to the relationship between the splicing prediction value and the input vector: ； ； Among them, T represents the abnormal threshold, E represents the average value of the error, and e i represents the error of the ith data point, , N represents the total number of data points, e represents the error, represents the predicted value, X represents the input vector; S4045: Smoothing the abnormal data in the historical measurement data according to the abnormal threshold.

7. The data optimization and early warning method based on recurrent neural network according to claim 1 is characterized in that: The S6 specifically includes: S601: Input the smoothed historical measurement data into the data early warning model in the form of an input vector, and calculate the current implicit state through the recurrent neural network module: ； Among them, h t represents the hidden state at time t, tanh represents the tanh activation function, W h and W x Both represent weight matrices, h t-1 represents the implicit state at time t-1, x t represents the input vector at time t, and b represents the bias term; S602: Output prediction results according to the current implicit state: ； Among them, y t represents the output at time t, W y represents the weight matrix of the output layer, b y Represents the bias term of the output layer; S603: Input the prediction result to the feature enhancement structure module until the loss function value of the data early warning model is less than the preset loss function value, and output the final prediction value of the data early warning model: ； Among them, y out represents the final prediction value, W1 represents the coefficient matrix of the first fully connected layer, W2 represents the coefficient matrix of the second fully connected layer, W3 represents the coefficient matrix of the third fully connected layer, z1 represents the linear output of the first layer, z2 represents the linear output of the second layer, z3 represents the linear output of the third layer, b1 represents the bias term of the first layer, b2 represents the bias term of the second layer, b3 represents the bias term of the second layer, and ReLU represents the ReLU activation function.

8. The data optimization and early warning method based on recurrent neural network according to claim 7 is characterized in that: The loss function of the data early warning model is specifically: ； Among them, loss second represents the loss function, α represents the weight coefficient controlling the mean square error loss function, loss MSE represents the mean square error loss function, β represents the weight coefficient of the control shape loss function, loss shape represents the shape loss function.

9. A data optimization and early warning system based on recurrent neural network, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the data optimization and early warning method based on a recurrent neural network as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data optimization and early warning method based on a recurrent neural network as described in any one of claims 1 to 8 is implemented.

Images