Rain condition monitoring data intelligent diagnosis method based on spatial relationship

Abstract

The invention discloses a rain condition monitoring data intelligent diagnosis method based on a spatial relationship, and the method comprises the steps: firstly screening a plurality of adjacent unmarked rainfall collectors through the management of a rainfall monitoring collector pool, and reading the data; carrying out rainfall monitoring data preprocessing and abnormal preliminary judgment, including length detection of discontinuous same rainfall data and deviation judgment of 0-value data, and labeling abnormal data; for the collector data in different time periods, analyzing the total rainfall capacity on a time sequence, and setting a threshold value for rainfall event division; a dynamic time warping algorithm is utilized to calculate the data difference degree of different collectors in different rainfall events, and a dynamic threshold value is set according to the difference degree and the spatial relation of the collectors to mark the states of the collectors; and all samples in the collector pool are marked through loop iteration of the process. The method effectively improves the accuracy and reliability of rainfall monitoring data, and is especially suitable for a rainfall monitoring and early warning system in a reservoir area.

Description

Technical Field

[0001] The invention relates to the technical field of intelligent processing and abnormal diagnosis of rainfall monitoring data, and in particular to an intelligent diagnosis method for rainfall monitoring data based on spatial relations. Background Art

[0002] It is well known that reservoirs are extremely important water conservancy infrastructure, and their safe and stable operation is an important guarantee for the national economy and people's livelihood. Accurate rainfall monitoring data is an important basis for predicting reservoir water level changes and dispatching flood control and drainage. In modern meteorological monitoring, accurate rainfall data is of great significance to disaster prevention and mitigation, water resources management, and agriculture. In the prior art, the analysis of reservoirs and other rainfall monitoring systems usually relies on data from a single rain collector to evaluate rainfall conditions. However, due to hardware failures, environmental interference or transmission problems, the data from a single collector may have errors or deviations, making it difficult to fully and accurately reflect the actual situation.

[0003] In order to achieve safety prediction of important infrastructure such as reservoirs, it is particularly important to develop a method for processing and diagnosing abnormalities that can integrate data from multiple rain gauge collectors. This method can not only screen out collectors with abnormal data and eliminate error data, but also combine data from multiple collectors for comprehensive analysis, thereby improving the overall quality of rainfall data and the accuracy of analysis results, ensuring the stable operation of reservoirs and the reliability of rainfall monitoring systems. Summary of the invention

[0004] The present invention provides an intelligent diagnosis method for rainfall monitoring data based on spatial relationships, aiming to improve the accuracy and reliability of rainfall monitoring data. The method comprehensively analyzes the data from multiple adjacent rain gauges, firstly screens out the collectors that may have errors and marks the abnormal data. Then, the system identifies the start and end time of the rainfall event through time series analysis, extracts key features, and synchronizes and integrates the data of multiple collectors. Subsequently, the dynamic time warping algorithm is used to calculate the difference between the collectors, and the dynamic threshold is set according to the spatial relationship of the collectors to judge the working status of each collector. Finally, based on the statistical characteristics of the difference, the system automatically triggers an alarm for the abnormal collector and generates a detailed report for further analysis. The method effectively integrates data screening, feature extraction, difference calculation and abnormal diagnosis technology, significantly improves the quality of rainfall monitoring data, and is particularly suitable for rainfall monitoring and early warning systems with high requirements for accuracy and reliability.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] A spatial relationship-based intelligent diagnosis method for rainfall monitoring data comprises the following steps:

[0007] Step S1: Read and record rainfall data and location information of multiple rain collectors, save them in a database, extract the rainfall value at each moment, and build a collector pool;

[0008] Step S2: randomly select a single unmarked target collector from the collector pool, and select one year's data of N unmarked collectors within L kilometers of the target collector;

[0009] Step S3: performing data preprocessing, including length detection of discontinuous identical rainfall data and deviation judgment of zero-value data;

[0010] Step S4: merging rainfall data from different collectors at different times in chronological order, and summarizing rainfall according to the time series;

[0011] Step S5: Identify and set thresholds to divide the start and end time of rainfall events, and calculate the total rainfall and duration of each rainfall event;

[0012] Step S6: using a dynamic time warping algorithm to calculate the data differences between different collectors in different rainfall events;

[0013] Step S7: Calculate the sum of differences between each collector and other collectors, and set a dynamic threshold according to the spatial relationship with other collectors. Collectors whose sum of differences exceeds the threshold are marked as abnormal, otherwise they are marked as normal;

[0014] Step S8: The loop is iterated until all samples in the collector pool are marked.

[0015] The present invention has significant beneficial effects. Through efficient data preprocessing, feature extraction, rainfall event recognition, and the application of dynamic time warping algorithms, the data differences between collectors are accurately calculated, and thresholds are dynamically set based on the difference and the spatial relationship between the collectors for anomaly detection, effectively improving the ability of rainfall data processing and anomaly detection. This method not only significantly improves the reliability of data analysis, but also enhances the practicality of rainfall monitoring and prediction. It is particularly suitable for safety monitoring fields such as reservoir rainfall monitoring, flood control and drainage, and ensures the efficiency and stability of the data processing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1It is a flow chart of an intelligent diagnosis method for rainfall monitoring data based on spatial relationship disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be further described in detail below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention but not all. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] Combination Figure 1 As shown, the present invention provides an intelligent diagnosis method for rainfall monitoring data based on spatial relationship, comprising:

[0020] Step S1: Read and record rainfall data and location information of multiple rain collectors, save them in a database, extract the rainfall value at each moment, and build a collector pool.

[0021] S11. Preliminarily filter out the information of collectors whose rainfall data span is one year or more, and build a collector pool. Let S be the set of all rainfall collectors, T i Indicates the time when the collector starts collecting data, t current is the current time. It is expressed as:

[0022] P={i∈S|t current -T i ≥12 months} ​​(1)

[0023] Among them, P is the set of collectors that meet the conditions;

[0024] Step S2: Randomly select a single unmarked target collector from the collector pool, and select the monitoring data of N unmarked collectors within 5km of the target collector within one year.

[0025] S21, select a target collector and randomly select an unmarked target collector from the collector pool P. If it exists, proceed to step S22; if it does not exist, terminate the program;

[0026] i target ~Uniform(P) (2)

[0027] S22, select N unmarked collectors within 5km of the target collector and express it as:

[0028] P 5km ={j∈P|d(i target ,j)≤5km} (3)

[0029] P N ={j 1 , j 2 , ..., j N |j k ∈P 5km , k = 1, 2, ..., N} (4)

[0030] where i target is the target collector, d(i, j) represents the straight-line distance between collector i and collector j;

[0031] S23. Obtain data within one year as:

[0032] Data(j k )={R(j k , t)|T start ≤T≤T current}, k = 1, 2, ..., N + 1 (5)

[0033] Where T start Indicates the starting time point. For each collector extracting [T start , T current ] data within the time period;

[0034] Step S3: Perform data preprocessing, including length detection of discontinuous identical rainfall data and deviation judgment of zero-value data. Annotate the detected abnormal data.

[0035] S31, in the data preprocessing stage, the empty value records in the collector data are deleted, and the number of abnormal mark items M in steps S32 and S33 is recorded;

[0036] S32, length detection of non-continuous identical rainfall data, after removing the null values ​​and 0 values ​​in the current data set, calculate the proportion of the longest continuous equal non-empty substring length and compare it with the threshold ρ 子串 For comparison, if the proportion exceeds ρ 子串 The current data is discarded and marked as abnormal, and other collectors within 5km are selected for judgment, where:

[0037]

[0038] S33, the deviation judgment of 0 value data is expressed as follows: if the number of records with 0 values ​​in the collector exceeds the threshold value ρ 子串 It is abnormal and marked;

[0039]

[0040] S34, if N+1-M>3, then continue to step S4, otherwise if N=0, it means that there are only i in the current collector.target If it cannot be determined, a manual labeling is prompted and step S21 is continued. If N≠0, the two collectors are not labeled and are put back into the collector pool;

[0041] Step S4: The rainfall data from different collectors at different times are merged in chronological order, and the rainfall is summarized according to the time series.

[0042] S41, merging the data of multiple collectors into one data set by time and summarizing the data by time:

[0043]

[0044] Where T = {t 1 , t 2 , ..., t m} represents a discrete time series, where t 1 , t 2 , ..., t m It is different time points, covering the observation data of all collectors within a certain time range;

[0045] Step S5: Identify and set thresholds to divide the start and end time of a rainfall event, and calculate the total rainfall and duration of each rainfall event.

[0046] S51, determining the rainfall start time and end time;

[0047] The start time of the kth rainfall event and end time Respectively expressed as:

[0048]

[0049] S52. Calculate the total rainfall and duration of each rainfall event and express them as follows:

[0050]

[0051] Step S6: Using the dynamic time warping (DTW) algorithm, the difference of rainfall data between different collectors is calculated. The specific steps include constructing a distance matrix, calculating the shortest path by dynamic programming, and calculating the DTW distance.

[0052] S61, construct a distance matrix, for each collector A and B, calculate the difference of their rainfall data sequences in the same rainfall event k. Construct a distance matrix D of size m×m (k) , where each element D (k) (i, j) represents time point t i and t j The difference in rainfall between:

[0053]

[0054] Since the time points of rainfall events are the same, here t i =t j The corresponding rainfall data were directly compared;

[0055] S62, dynamic programming to calculate the shortest path;

[0056] Using dynamic programming algorithm, calculate the matrix D (k) The shortest path from the upper left corner (1, 1) to the lower right corner (m, m). The cumulative distance of the path ψ (k) The recursive formula for (i, j) is:

[0057]

[0058] S63, calculating the DTW distance;

[0059] For the kth rainfall event, the DTW distance Defined as the cumulative distance ψ of the shortest path (k) (m, m) divided by the total number of path points m:

[0060]

[0061] Step S7: Calculate the sum of differences between each collector and other collectors, and dynamically adjust the threshold according to the spatial relationship with other collectors. Collectors whose sum of differences exceeds the dynamic threshold are marked as abnormal, otherwise they are marked as normal.

[0062] S71, calculating the sum of differences;

[0063] For the N+1-M collectors selected in step S4, for each collector i, calculate the sum of the differences between it and the other NM collectors in all rainfall events S i And the sum of distances L i , where the sum of differences is expressed as:

[0064]

[0065] The sum of distances is expressed as:

[0066]

[0067] Where l(i, j) represents the distance between collector i and collector j, and the average value of the sum of distances is expressed as:

[0068]

[0069] S72, calculate the average value μ of the sum of the differences of all collectors S and standard deviation σ S , calculated as follows:

[0070]

[0071] S73. For each collector i, if the sum of its differences S i If the calculated dynamic threshold is exceeded, the collector is marked as abnormal;

[0072] Dynamic threshold ρ i It is expressed as the sum of the distances of collector i and the mean of the sum of the distances of all collectors multiplied by the standard deviation σ of the sum of the differences between collectors S , which is expressed as follows:

[0073]

[0074] Anomalous={i|S i >ρ i} (twenty two)

[0075] Step S8: When an abnormal collector is detected, the system will automatically trigger a warning message and generate a detailed report for further analysis and processing by the user. At the same time, the system will loop step S21 until all samples in the collector pool are marked.

[0076] The present invention discloses an intelligent diagnosis method for rainfall monitoring data based on spatial relations. The method first reads rainfall data from multiple adjacent rain gauges and performs data preprocessing, including length detection of non-continuous identical rainfall data and deviation judgment of zero-value data, so as to preliminarily mark and accumulate abnormal data to ensure the integrity and consistency of the data. Then, feature engineering processing is performed on the data of each collector, and useful features are extracted by cumulative rainfall and rainfall difference calculation. On this basis, the rainfall data of different collectors at different times are merged in chronological order to generate a comprehensive data set, and by analyzing the time series data, the start and end time of the rainfall event are identified and divided, and the total rainfall and duration of each rainfall event are calculated.

[0077] In addition, this method uses the dynamic time warping (DTW) algorithm to calculate the difference in rainfall data from different collectors in the same rainfall event. By calculating the sum of the standard deviations of the differences between each collector and other collectors, the threshold is dynamically adjusted according to its spatial relationship with other collectors, and abnormal collectors are detected based on this threshold, automatically triggering warning messages. This method integrates multiple steps such as data preprocessing, feature engineering, rainfall event identification, difference calculation, anomaly detection and fault diagnosis, which significantly improves the accuracy and reliability of data processing. It is particularly suitable for high-precision rainfall data analysis and early warning systems in the fields of reservoir rainfall monitoring, flood control and drainage.

[0078] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. An intelligent diagnosis method for rainfall monitoring data based on spatial relations, characterized in that: The steps include: Step S1: Read and record rainfall data and location information of multiple rain collectors, save them in a database, extract the rainfall value at each moment, and build a collector pool; Step S2: randomly select a single unmarked target collector from the collector pool, and select one year's data of N unmarked collectors within L kilometers of the target collector; Step S3: performing data preprocessing, including length detection of discontinuous identical rainfall data and deviation judgment of zero-value data; Step S4: merging rainfall data from different collectors at different times in chronological order, and summarizing rainfall according to the time series; Step S5: Identify and set thresholds to divide the start and end time of rainfall events, and calculate the total rainfall and duration of each rainfall event; Step S6: using a dynamic time warping algorithm to calculate the data differences between different collectors in different rainfall events; Step S7: Calculate the sum of differences between each collector and other collectors, and set a dynamic threshold according to the spatial relationship with other collectors. Collectors whose sum of differences exceeds the threshold are marked as abnormal, otherwise they are marked as normal; Step S8: The loop is iterated until all samples in the collector pool are marked.

Images