Rule engine construction method, electronic device and storage medium

Abstract

The present invention discloses a rule engine construction method, electronic device, and storage medium. The method includes obtaining hydrological data, organizing the hydrological data, adding the organized hydrological data to a rule engine form, and forming rule conditions in the rule engine form; associating the monitoring data table with the rule conditions in the rule engine form according to the element names in the monitoring data table and the rule engine form; obtaining the associated rule conditions, element names, and monitoring data table according to the association relationship between the monitoring data table and the rule conditions in the rule engine form, and generating a rule engine script; setting detection tasks and detection parameters for the monitoring data table; extracting data from the monitoring data table according to the detection parameters, and detecting and processing the extracted data using the rule engine script; and generating a data detection and processing report after the detection and processing are completed. The present invention ensures that correct data is not accidentally deleted.

Description

Technical Field

[0001] The present invention belongs to hydrological monitoring data processing technology, and in particular relates to a rule engine construction method for discovering singular values ​​of hydrological monitoring data, an electronic device and a storage medium. Background Art

[0002] Hydrological monitoring data mainly comes from the real-time monitoring of rivers, lakes, reservoirs, channels and groundwater by the hydrological department. Specific monitoring data include real-time rainfall, water level, water temperature, flow, evaporation, etc.

[0003] Due to equipment damage, communication anomalies and external environmental interference, monitoring data generally has abnormal conditions such as incomplete data and many strange data. Specifically, the monitoring data is missing, the monitoring data is not within the detection range, and the monitoring data has a large mutation.

[0004] The existing methods for handling abnormalities are mainly:

[0005] Method 1: Regularly delete singular values:

[0006] (a) The administrator screens the data regularly on a monthly or annual basis and deletes outliers based on the actual local hydrological conditions;

[0007] (b) The administrator screens the data during data compilation and removes outliers based on hydrological trends;

[0008] Method 2: Use software tools to screen singular values:

[0009] (a) Use traditional ETL tools to configure outlier removal rules, including maximum and minimum values;

[0010] (b) Use the business system’s screening criteria to find out the singular values ​​and delete them.

[0011] The existing processing methods have the following problems:

[0012] (a) Deleting singular values ​​by year or month is not very real-time;

[0013] (b) Abnormal data is not processed in a timely manner, causing the business system to add abnormal data to the calculation queue, resulting in incorrect decision-making;

[0014] (c) It requires a deep understanding of hydrology; otherwise, improper operation may lead to the accidental deletion of correct data;

[0015] (d) The use of traditional ETL tools does not fully consider interference factors such as seasonality, geographical environment, equipment loss, network delay, etc., resulting in poor data processing results and easy deletion of correct data.

[0016] Evolving from inference engines, rule engines are components embedded within applications. They separate business decisions from application code and author them using predefined semantic modules. Rule engines offer advantages such as multi-input support, extensive dynamic expressions, and exceptional speed. Users can customize a series of rule conditions in JSON or XML format, such as value matching, value comparison, and data filtering. By parsing and executing these rules, unified validation logic is provided for large amounts of rapidly changing data.

[0017] Because hydrological data must be reliable, consistent, and representative, conventional rule engine technology encounters numerous difficulties when processing monitoring data, ultimately failing to meet the requirements for hydrological analysis and calculations. This is primarily reflected in the following aspects:

[0018] (a) The rule conditions in conventional rule engines lack data from hydrological stations at all levels, such as reservoir data, river data, water level data, and flow data. Without basic data support, the rule conditions set are prone to eliminating correct data, resulting in accidental deletions and failing to meet the reliability requirements of hydrological data.

[0019] (b) Due to factors such as seasonality and geographical environment, there are curve fluctuations. To ensure data consistency, it is necessary to dynamically compare and calculate the data of the current period with the previous period, including the amplitude and rate of change. However, conventional rule engines can only compare and calculate with fixed values, which cannot meet such calculation requirements and cannot guarantee data consistency.

[0020] (c) Hydrological monitoring data is continuous time series data. However, due to factors such as equipment loss and network delays, the monitoring data may be missing measurements or delayed. When using conventional rule engines, the missing data is not taken into account, resulting in large differences in the calculation results and ultimately causing the correct data to be eliminated. Summary of the Invention

[0021] The purpose of the present invention is to provide a rule engine construction method, electronic device and storage medium for discovering singular values ​​of hydrological monitoring data, so as to solve the problem that the existing technology is prone to accidental deletion of correct data and cannot meet the reliability and consistency requirements of data.

[0022] The present invention solves the above technical problems through the following technical solutions: a method for constructing a rule engine for discovering singular values ​​of hydrological monitoring data, comprising the following steps:

[0023] Acquiring hydrological data, including monitoring sensor parameter information, hydrological protocol element information, and basic monitoring station information;

[0024] Arrange the hydrological data, add the arranged hydrological data to a rule engine form, and form rule conditions in the rule engine form;

[0025] Associating the monitoring data table to be detected with the rule conditions in the rule engine form according to the element names in the monitoring data table to be detected and the rule engine form;

[0026] Obtaining associated rule conditions, element names, and monitoring data tables based on the association relationship between the monitoring data tables and the rule conditions in the rule engine form, and generating a rule engine script;

[0027] Setting the detection tasks and detection parameters of the monitoring data table;

[0028] Extracting data from the monitoring data table according to the set detection parameters, and detecting and processing the extracted data using the rule engine script;

[0029] After the detection and processing are completed, a data detection and processing report is generated.

[0030] Furthermore, the monitoring sensor parameter information includes the monitoring sensor name, sampling period, maximum value of the range, minimum value of the range, and data amplitude;

[0031] The hydrological protocol element information includes element name, identifier, data definition and unit;

[0032] The basic information of the measuring station includes the station name, measuring station number and hydrological characteristic value.

[0033] Furthermore, when the sorted hydrological data is added to the rule engine form, if the range in the monitoring sensor parameter information overlaps with the data definition in the hydrological specification element information, the hydrological characteristic value planned during the construction of the measuring station shall prevail.

[0034] Furthermore, the specific generation process of the rule engine script is: using the element name as the engine execution precondition, and using the maximum value of the range, the minimum value of the range, the data variation, whether it can be empty, and whether it can be zero as the engine constraint conditions to generate the rule engine script.

[0035] Furthermore, the specific process of the detection and processing is as follows:

[0036] If the extracted data meets the corresponding rule conditions, the data is a normal value, otherwise it is a singular value;

[0037] The singular values ​​are marked, removed and alarmed; the alarm notification includes the station code, element name, data monitoring time, specific data value, alarm reason and detection time.

[0038] Furthermore, the data detection processing report includes the table name, total number of processing, normal rate and error rate.

[0039] Based on the same inventive concept, the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor runs the computer program, it executes the steps of the rule engine construction method for discovering singular values ​​of hydrological monitoring data as described above.

[0040] Based on the same inventive concept, the present invention also provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the rule engine construction method for discovering singular values ​​of hydrological monitoring data as described above are executed.

[0041] Beneficial effects

[0042] Compared with the prior art, the advantages of the present invention are:

[0043] The present invention provides a rule engine construction method, electronic device, and storage medium for discovering singular values ​​in hydrological monitoring data. The method is based on data such as hydrological sensor parameters, hydrological protocol elements, and basic information about measuring stations. It combines the "three properties" requirements of hydrological data and the characteristics of time series data, while fully considering factors such as seasonality, geographical environment, equipment loss, and network delay. After comprehensive calculation, a rule engine script is dynamically generated, thereby efficiently and in real time performing rationality checks on hydrological monitoring data, ultimately discovering singular values ​​in the hydrological monitoring data.

[0044] The present invention uses data such as hydrological sensor parameters, hydrological protocol elements, and basic information of measuring stations as the basis for rule engine condition processing, thereby ensuring the reliability of hydrological data; dynamically generates rule conditions based on data before and after the monitoring period, which is more suitable for hydrological calculations and analysis, solves the impact of factors such as seasonality and geographical environment on rule conditions, and ensures the consistency of hydrological data; fully considers the continuity and temporal characteristics of monitoring data, and performs weighted calculation processing for data missing, data delay, and other situations during rule engine calculation, effectively solving the problem of rule engine calculation errors after data gaps due to equipment loss or network delays, and ensuring that correct data is not deleted by mistake; real-time data report output informs administrators of the measuring station data status, enhancing the measuring station operation and maintenance capabilities.

[0045] The correct data detected by the method of the present invention can better meet the "three properties" requirements of hydrological data and provide more accurate data support for monitoring and early warning of hydrological departments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 This is a flowchart of rule engine construction in an embodiment of the present invention;

[0048] Figure 2 1 is a flow chart of singular value detection in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following is a clear and complete description of the technical solutions of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0050] The following specific embodiments are used to describe the technical solution of the present application in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0051] The method for constructing a rule engine to discover singular values ​​in hydrological monitoring data provided by the embodiment of the present invention specifically includes two parts, one of which is the generation of a rule engine script, and the other is the detection and processing using the generated rule engine script. Figure 1 and 2 As shown, the rule engine construction method specifically includes the following steps:

[0052] Step 1: Collect or obtain hydrological data.

[0053] In this embodiment, hydrological data includes monitoring sensor parameter information, hydrological protocol element information, and basic station information. Monitoring sensor parameter information includes the monitoring sensor name, sampling period, maximum range value, minimum range value, and data amplitude; hydrological protocol element information includes element name, identifier, data definition, and unit; and basic station information includes station name, station number, and hydrological characteristic values.

[0054] For example, the monitoring sensor parameter information is shown in Table 1, the hydrological protocol element information is shown in Table 2, and the basic information of the monitoring station is shown in Table 3.

[0055] Table 1 Sensor parameters

[0056]

[0057]

[0058] Table 2 Hydrological specification elements

[0059] Serial number Feature name Identifier Data Definition unit 1 Precipitation in every 5-minute period within 1 hour DRP Maximum value 25.4 mm 2 Corresponding water level within 5 minutes within 1 hour DRZ Maximum value 655.34 rice 3 pH value pH Maximum value 14 / 4 Soil moisture content at 10CM M10 N(4,1) percentage 5 Soil moisture content at 20CM M20 N(4,1) percentage 6 Instantaneous water temperature C N(3,1) degrees Celsius 7 Daily evaporation ED N(5,1) mm 8 humidity MST N(4,1) percentage 9 Instantaneous river water level Z N(7,3) rice 10 Turbidity TURB N(3) Spend

[0060] Table 3 Water level and storage capacity characteristics of reservoirs in Shenzhen Futian District

[0061]

[0062] Step 2: Dynamically generate rule conditions: organize the collected or acquired hydrological data, add the organized hydrological data to the rule engine form, and form rule conditions in the rule engine form.

[0063] When adding the sorted hydrological data to the rule engine form, if the range in the monitoring sensor parameter information overlaps with the data definition in the hydrological specification element information, the hydrological characteristic value planned during the construction of the measuring station will prevail, ultimately providing a processing basis for the reliability of the hydrological data. For example, generate a five-minute rainfall rule condition:

[0064]

[0065] For example, the information collected in Tables 1, 2, and 3 is integrated and added to the rule engine form, where the element name field is filled with the hydrological protocol identifier, the station code corresponds to the station code in the basic information of the station, the maximum range value and the minimum range value are filled with the sensor parameters or hydrological characteristic values, the sampling period corresponds to the sampling period of the sensor, and the data amplitude is filled in if it exists, as shown in Table 4.

[0066] Note: If the sensor parameters overlap with the defined intervals of the hydrological characteristic values, the actual hydrological characteristic values ​​planned during the construction of the measuring station shall prevail.

[0067] Table 4 Rule engine form parameters

[0068]

[0069]

[0070] Step 3: Associate rule conditions: According to the monitoring data table to be detected and the element name in the rule engine form, associate the monitoring data table with the rule conditions in the rule engine form.

[0071] For example, if the element name field in the rainfall table to be detected is DRP, and the element name in the rule condition setting is also DRP, then an association can be made. One monitoring data table is bound to at least one rule condition. If the monitoring data table has multiple different element name fields, multiple rule conditions can be bound. For example, the association relationship between the rainfall table and the rainfall rule condition is as follows:

[0072]

[0073] The association form at least includes the table name, field name, field type, and whether to monitor, as shown in Table 5 and Table 6;

[0074] Table 5. Association of rainfall table rules

[0075]

[0076] Table 6 Reservoir water level table rule association

[0077]

[0078] Step 4: Generate rule engine script: Based on the association between the monitoring data table and the rule conditions in the rule engine form, the associated rule conditions, element names, and monitoring data tables are obtained. The element name is used as the engine execution precondition, and the maximum value of the range, the minimum value of the range, the data variation, whether it can be empty, and whether it can be zero are used as engine constraints. Finally, the rule engine script is generated. The rule engine script is as follows:

[0079]

[0080] Step 5: Set the detection task and detection parameters of the monitoring data table. The detection parameters include the task name, monitoring data table name and running frequency.

[0081] Taking the rainfall meter detection task as an example, the specific tasks are:

[0082]

[0083] For example, a detection task of a rainfall meter is set, as shown in Table 7, and its configuration includes at least a task name, a monitoring data table, and an operating frequency.

[0084] Table 7 Add detection tasks

[0085] Serial number name Monitoring data table Operating frequency Start time 1 Rainfall detection task Rainfall monitoring data table 2 minutes 2021-01-01 00:00:00 2 Reservoir water level detection task Reservoir water level monitoring data table 2 minutes 2021-01-01 00:00:00

[0086] The rainfall monitoring data table in Table 8 and the reservoir water level monitoring data table in Table 9 were tested and processed respectively.

[0087] Table 8 Rainfall meter monitoring data

[0088] STCD TM DRP 80000022 2022-04-11 11:00:00 0.0 80000022 2022-04-11 10:55:00 0.0 80000022 2022-04-11 10:50:00 0.0 80000022 2022-04-11 10:45:00 0.0 80000022 2022-04-11 10:40:00 0.0 80000022 2022-04-11 10:35:00 1.5 80000022 2022-04-11 10:30:00 0.0 80000022 2022-04-11 10:25:00 50 80000022 2022-04-11 10:20:00 0.0 80000022 2022-04-11 10:15:00 0.0 80000022 2022-04-11 10:10:00 0.0 80000022 2022-04-11 10:05:00 1.5 80000022 2022-04-11 10:00:00 0.0 80000022 2022-04-11 09:55:00 0.0 80000022 2022-04-11 09:50:00 0.0 80000022 2022-04-11 09:45:00 0.0 80000022 2022-04-11 09:40:00 0.0 80000022 2022-04-11 09:35:00 0.0 80000022 2022-04-11 09:30:00 0.0

[0089] Table 9 Reservoir water level monitoring data

[0090] STCD TM RZ W 80000010 2022-04-11 00:00:00 38.703 22.112 80000010 2022-04-10 23:55:00 38.703 22.112 80000010 2022-04-10 23:50:00 38.693 22.066 80000010 2022-04-10 23:45:00 38.703 22.112 80000010 2022-04-10 23:40:00 38.703 22.112 80000010 2022-04-10 23:35:00 38.703 22.112 80000010 2022-04-10 23:30:00 50.00 22.112 80000010 2022-04-10 23:25:00 38.703 22.112 80000010 2022-04-10 23:20:00 38.703 22.112 80000010 2022-04-10 23:15:00 38.703 22.112 80000010 2022-04-10 23:10:00 38.703 22.112 80000010 2022-04-10 23:05:00 38.703 22.112 80000010 2022-04-10 23:00:00 40.123 22.112 80000010 2022-04-10 22:55:00 38.703 22.112 80000010 2022-04-10 22:50:00 / 22.112 80000010 2022-04-10 22:45:00 38.703 22.112 80000010 2022-04-10 22:40:00 38.693 22.066 80000010 2022-04-10 22:35:00 38.693 49.013 80000010 2022-04-10 22:30:00 38.703 22.112 80000010 2022-04-10 22:25:00 38.693 22.066

[0091] Step 6: Rationality analysis: Extract data from the monitoring data table according to the set operating frequency, and use the rule engine script (as shown in Table 5, rainfall association rule condition 1) to detect and process the extracted data; if the extracted data meets the corresponding rule conditions, the data is a normal value, otherwise it is a singular value; the singular value is marked, removed, and an alarm notification is processed; the alarm notification includes the station code, feature name, data monitoring time, data specific value, alarm reason, and detection time.

[0092] For example, the integrity and rationality of the monitoring data of markers 8 and 9 are analyzed according to the set operating frequency. The data that meets the rule conditions are normal values, and the data that does not meet the rule conditions are singular values. At the same time, the singular values ​​are marked, eliminated and alarmed. The alarm information includes the station code (i.e., the measurement station code), element name, data time, monitoring value, alarm reason, and detection time, as shown in Table 10.

[0093] Table 10 Alarm exception table

[0094]

[0095] Step 7: After the detection and processing are completed, a data detection and processing report is generated based on the detection and processing results, and the monitoring data status is promptly reported to the operation and maintenance personnel.

[0096] In this embodiment, the data detection processing report includes the table name, the total number of processing, the normal rate and the error rate, as shown in Table 11.

[0097] Table 11 Data detection and processing report

[0098]

[0099] The above disclosure is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or modifications within the technical scope disclosed in the present invention, and they should all be covered by the scope of protection of the present invention.

Claims

1. A rule engine construction method for discovering singular values ​​in hydrological monitoring data, characterized in that: The following steps are involved: Acquire hydrological data before and after the monitoring period, the hydrological data including monitoring sensor parameter information, hydrological protocol element information, and basic station information. The monitoring sensor parameter information includes the monitoring sensor name, sampling period, maximum range value, minimum range value, and data amplitude. The hydrological protocol element information includes the element name, identifier, data definition, and unit. The basic station information includes the station name, station number, and hydrological characteristic value. Arrange the hydrological data, add the arranged hydrological data to a rule engine form, and form rule conditions in the rule engine form; Associating the monitoring data table to be detected with the rule conditions in the rule engine form according to the element names in the monitoring data table to be detected and the rule engine form; Obtaining associated rule conditions, element names, and monitoring data tables based on the association relationship between the monitoring data tables and the rule conditions in the rule engine form, and generating a rule engine script; Setting the detection tasks and detection parameters of the monitoring data table; Extracting data from the monitoring data table according to the set detection parameters, and detecting and processing the extracted data using the rule engine script; Generate a data detection and processing report after the detection and processing are completed; Among them, the specific generation process of the rule engine script is: using the element name as the engine execution precondition, and using the maximum value of the range, the minimum value of the range, the data variation, whether it can be empty, and whether it can be zero as the engine constraint conditions to generate the rule engine script.

2. The rule engine construction method according to claim 1, characterized in that: When adding the sorted hydrological data to the rule engine form, if the range in the monitoring sensor parameter information overlaps with the data definition in the hydrological protocol element information, the hydrological characteristic value planned during the construction of the measuring station shall prevail.

3. The rule engine construction method according to claim 1 or 2, characterized in that: The specific process of the detection and processing is as follows: If the extracted data meets the corresponding rule conditions, the data is a normal value, otherwise it is a singular value; The singular values ​​are marked, removed and alarmed; the alarm notification includes the station code, element name, data monitoring time, specific data value, alarm reason and detection time.

4. The rule engine construction method according to claim 1, characterized in that: The data detection processing report includes the table name, total number of processing, normal rate and error rate.

5. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor runs the computer program, the processor executes the steps of the rule engine construction method for discovering singular values ​​of hydrological monitoring data as described in any one of claims 1 to 4.

6. A computer-readable storage medium, wherein the computer-readable storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer program is stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the rule engine construction method for discovering singular values ​​of hydrological monitoring data described in any one of claims 1 to 4 are executed.

Images