A method and medium for removing outliers from time series data of glacial lake area based on Gaussian process regression

Abstract

The present application relates to a kind of ice lake area time series data outlier rejection method and medium based on Gaussian process regression.The ice lake area time series data extracted by automatic mapping tool is modeled by Gaussian process regression;Through the Gaussian process regression model established, the confidence interval of each data record in the ice lake area time series data is calculated, and is classified as upper limit anomaly, normal and lower limit anomaly;Ice lake breach event is identified, and the lower limit anomaly caused by ice lake breach event is reclassified as normal.This method can accurately eliminate the abnormal values in the ice lake area time series data extracted by automatic mapping tool caused by random interference such as cloud snow mountain shadow, and provides technical means for accurate monitoring of ice lake area.

Description

Technical Field

[0001] This invention belongs to the field of glacial lake disaster monitoring, specifically relating to a method for outlier removal from time-series glacial lake area data based on Gaussian process regression. This method can accurately remove outliers caused by random interference such as clouds, snow, and mountain shadows from time-series glacial lake area data extracted by automated mapping tools, providing a technical means for accurate monitoring of glacial lake area and improving the monitoring capability of glacial lake outburst disasters. Background Technology

[0002] Glacial lakes are formed by the melting of high-altitude glaciers or the accumulation of glacial dams. With global warming, the area of ​​glacial lakes is generally expanding while their stability is decreasing. Glacial lake outburst floods (GLOFs) release massive amounts of water, triggering secondary disasters such as floods and mudslides, posing a serious threat to downstream populations and infrastructure. Because glacial lakes are mostly located at high altitudes, on-site monitoring is difficult, making remote sensing time-series data the primary means of dynamic monitoring. However, remote sensing data is often affected by cloud cover, shadows, and noise, leading to false anomalies in the time-series data of glacial lake area. Traditional methods for detecting glacial lake outliers often rely on thresholds or machine learning models to identify outliers, but these methods struggle to distinguish between genuine outbursts and the effects of remote sensing interference, resulting in a still relatively high false alarm rate. Therefore, a new method is urgently needed to improve the accuracy and robustness of glacial lake area outlier detection based on automatically extracted remote sensing images by combining statistical prediction and physical constraints. Summary of the Invention

[0003] The technical problem to be solved by this application is to provide a method and medium for outlier removal in time series data of glacial lake area based on Gaussian process regression, which has the characteristics of full automation and streamlined process to improve data quality.

[0004] In a first aspect, one embodiment provides a method for outlier removal from time-series data of glacial lake area based on Gaussian process regression, characterized by comprising:

[0005] An initial sequence of glacial lake area was constructed by sorting multiple observation samples consisting of observation time, observation area and observation error according to observation time.

[0006] A Gaussian process regression model was constructed and trained based on the initial sequence of glacial lake area.

[0007] The trained Gaussian process regression model was used to calculate the predicted area and standard deviation of each observed sample in the initial sequence of glacial lake area, and the confidence interval was calculated in combination with the observation error.

[0008] Compare the observed area and confidence interval of each observed sample, and mark the observed samples as upper bound abnormal, normal, and lower bound abnormal;

[0009] Identify glacial lake outburst events and mark observation samples with lower bound anomalies caused by glacial lake outburst events as normal;

[0010] By removing all observation samples marked as upper or lower bound anomalies from the initial sequence of glacial lake area, an accurate time series dataset of glacial lake area is obtained.

[0011] In one embodiment, the construction and training of the Gaussian process regression model based on the initial sequence of glacial lake area specifically involves:

[0012] The radial basis functions constructed using the squared exponential kernel serve as the kernel function for the Gaussian process regression model.

[0013]

[0014] In the formula Represents a pair of observation samples. Let K be the kernel function for this observation pair. Indicates the signal variance. Indicates length measurement;

[0015] Using the observation time difference of the observed sample pairs as the radial distance input, the selected kernel function is used, and a noise term is added to construct the covariance matrix of the Gaussian process regression model:

[0016]

[0017] in These are the indexes of the two samples in the observed sample pair. For the first The observation time for each observation sample For the first The observation time for each observation sample for Square array for Unit matrix of order, This represents the number of centrally observed samples in the time-series dataset of glacial lake area. Noise term;

[0018] The optimal hyperparameters are obtained by optimizing the training model by maximizing the log-marginal likelihood, where the log-marginal likelihood is specifically:

[0019]

[0020] in, Let be a column vector consisting of the observation times of each observation sample. Let be the column vector formed by the observed areas of each observed sample. To give input Under the condition of, the output matrix is ​​observed The conditional probability, Represents conditional probability The natural logarithm, This is the inverse of the aforementioned covariance matrix. Let be the determinant of the aforementioned covariance matrix.

[0021] In one embodiment, the step of using a trained Gaussian process regression model to calculate the predicted area and standard deviation of each observed sample in the initial sequence of glacial lake area, and calculating the confidence interval in conjunction with observation errors, includes:

[0022] Calculate the predicted area value for each observed sample;

[0023] Calculate the predicted standard deviation for each observed sample;

[0024] The total uncertainty for each observation sample is calculated using the following formula:

[0025]

[0026] in, Indicates the observed sample Total uncertainty, Indicates the standard deviation of the forecast. Indicates the observed sample The observation error;

[0027] The confidence interval for each observed sample is calculated using the following formula:

[0028]

[0029] in, Indicates the observed sample The confidence interval, For observation samples The predicted area value, Indicates the observed sample Total uncertainty, This refers to the tolerance for errors.

[0030] In one embodiment, the identification of glacial lake outburst events involves marking abnormal lower bound observation samples caused by glacial lake outburst events as normal, specifically as follows:

[0031] Multiple observation samples within the historical window before the observation time of the current lower bound anomaly observation sample and the future window thereafter are selected as observation windows. The historical window and the future window are as follows:

[0032]

[0033]

[0034] in, Indicates the first The observed area of ​​each observed sample. This indicates the historical window of the observed sample. This indicates the future window of the observed sample. and The length of the window;

[0035] Perform post-collapse area drop constraint detection within the observation window, specifically by determining whether the following inequality holds:

[0036]

[0037] in, For observation samples The median of the observed area within the historical window. For the corresponding standard deviation, This represents the preset threshold for the sudden drop in post-collapse area;

[0038] Perform a low-level constraint check on the post-collapse area within the observation window, specifically by determining whether the following inequality holds:

[0039]

[0040] in, For observation samples The median of the observed area within the historical window. For observation samples The median of the observed area within the future window. The pre-defined threshold for detecting the low-level constraint of the post-collapse area is described.

[0041] Perform a gradual change in post-collapse area constraint check within the observation window, specifically determining whether the following inequality holds:

[0042]

[0043] in, Indicates the observed sample The slope of the linear regression equation fitting the observed area within the future window. This represents the preset threshold for detecting gradually varying post-collapse area constraints.

[0044] Perform post-collapse area stability constraint checks within the observation window, specifically by determining whether the following inequality holds:

[0045]

[0046] in, Represented as observation sample The standard deviation of the observed area in the future window, This represents the preset post-collapse area stability constraint threshold;

[0047] Summarizing the four discrimination constraints, when all inequalities are true, the current lower bound abnormal observation sample is marked as normal.

[0048] In a second aspect, one embodiment provides a computer-readable storage medium storing a program that can be loaded by a processor and executed as the outlier detection method described in any of the above embodiments.

[0049] The beneficial effects of this invention are:

[0050] By integrating the statistical characteristics of Gaussian process modeling with four constraints on the dynamic changes following glacial lake outbursts, this method effectively distinguishes between genuine outburst events and pseudo-outliers caused by disturbances such as cloud and snow shadows. This fully automated workflow can improve the quality of area time-series data generated by automated glacial lake mapping tools. Attached Figure Description

[0051] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating a method for removing outliers from time-series data of glacial lake area based on Gaussian process regression, as proposed in this application.

[0053] Figure 2 This is a schematic diagram of the method and process for constructing and training the Gaussian process regression model in this application;

[0054] Figure 3 This is a schematic diagram of the method used in this application to initially divide each observation sample in the initial sequence of glacial lake area into upper bound anomalies, normal values, and lower bound anomalies based on the aforementioned confidence interval;

[0055] Figure 4 This application establishes a flowchart of a method for determining four criteria based on the dynamic characteristics of glacial lakes and the natural evolutionary patterns after a glacial breach.

[0056] Note: Figure 3 The labels are: L100 is the lower bound of the confidence interval for each observed sample, L200 is the observed area for each observed sample, L300 is the upper bound of the confidence interval for each observed sample, L400 is the fitted curve of the upper bound of the confidence interval for each observed sample, L500 is the fitted curve of the observed area for each observed sample, and L600 is the fitted curve of the lower bound of the confidence interval for each observed sample. Detailed Implementation

[0057] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0058] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0059] The serial numbers assigned to components in this article, such as "first" and "second", are used only to distinguish the objects being described and have no sequential or technical meaning.

[0060] To facilitate the explanation of the inventive concept of this application, the source of the data on the area of ​​the ice lake is briefly explained below.

[0061] Glacial lake area is the primary indicator for monitoring glacial lake hazards. Obtaining accurate time-series data on glacial lake area is one of the hot topics in related hazard research. Typically, automated mapping tools based on deep learning or computer vision are used to segment and extract data from multiple remote sensing images from different dates, thereby obtaining glacial lake area data for multiple periods, which are then sorted to form time-series data.

[0062] Glacial lake data automatically extracted from remote sensing images using mapping tools such as deep learning is susceptible to random interference from clouds, snow-covered mountain shadows, and other factors, resulting in significant outliers that severely impact data reliability. Effectively removing these outliers to improve data reliability and credibility is a pressing issue that needs to be addressed.

[0063] The applicant's research also revealed that the dramatic reduction in glacial lake area caused by glacial outburst events exhibits a pattern in the data sequence highly similar to outliers caused by interference from cloud and snow-covered mountain shadows. Accurately identifying whether the sudden drop in glacial lake area stems from random interference or a genuine outburst event is also a question of practical value.

[0064] In view of this, this application provides an outlier detection method for time series data of glacial lake area that integrates Gaussian process regression. On the one hand, the time series dataset of glacial lake area is fitted by Gaussian process modeling to obtain confidence intervals for preliminary classification. On the other hand, the lower bound outliers are constrained and verified to obtain the final classification result.

[0065] One embodiment of this application provides a method for detecting anomalies in glacial lake area that integrates Gaussian process modeling and glacial lake evolution feature verification. Please refer to [link / reference]. Figure 1 ,include:

[0066] Step S10: Sort multiple ice lake area observation samples, which consist of observation time, observation area and observation error, according to the observation time to construct an initial sequence of ice lake area.

[0067] The area of ​​glacial lakes is the primary indicator in the field of glacial lake disaster monitoring.

[0068] Various methods exist for extracting the area of ​​glacial lakes, including spectral thresholding, supervised / unsupervised classification, and deep learning-based approaches. The core of automatic glacial lake area extraction lies in accurately segmenting the glacial and non-glacial lake regions; method selection requires a balance between accuracy, efficiency, and data availability. Currently, methods employing deep learning models (such as U-Net) combined with water body indices and topographic constraints are the primary means of automated extraction and are widely used in dynamic monitoring of glacial lakes.

[0069] In one embodiment, the deep learning model U-Net is used to segment multiple images of the study area, calculate the area of ​​the segmented ice lake and the mapping error, and sort these data according to the image capture time.

[0070] To facilitate the use of statistical terminology, this application uses "observed area" to refer to the area of ​​the ice lake obtained by the mapping tool, "observation error" to refer to the mapping error of the mapping tool itself, and "observed sample" to refer to a single data record obtained from a single mapping.

[0071] To eliminate differences in time span, the image capture time is usually normalized to obtain the observation time of the observation sample.

[0072] In one embodiment, the earliest image capture time is defined as 0, and the latest image capture time is defined as 1. The calculation formula for the normalized mapping of each image capture time is as follows:

[0073]

[0074] in, For the first The observation time for image normalization Indicates the earliest time the image was captured. Indicates the first Each image shooting time, This indicates the latest time the image was captured, and n represents the number of images.

[0075] In the applicant's actual work, a trained U-Net model was used to segment 58 satellite images of a glacial lake on the Qinghai-Tibet Plateau. The resulting time-series dataset of the glacial lake area is shown in Table 1 below. The observation time... Obtained by normalizing the image capture time. The earliest image capture time in this table. The latest time the footage was taken was October 14, 1988. The date is November 26, 2024. The record for observation sample number 2 was taken on October 22, 1991, and its observation time was calculated to be 0.0837 using the normalization formula.

[0076] surface In one embodiment, the obtained initial time-series dataset of glacial lake area

[0077]

[0078] Note: All area-related data (such as observed area, observation error, area prediction, etc.) mentioned in the embodiments used to illustrate the invention in this application are in square kilometers (km). 2 ).

[0079] In financial forecasting and scientific experiments, Gaussian process regression is commonly used to predict the research subjects. Its main steps include selecting the kernel function, constructing the covariance matrix, and training and optimization. The applicant notes that this method is also suitable for research on changes in glacial lake area.

[0080] Step S20: Construct and train a Gaussian process regression model based on the initial sequence of ice lake area.

[0081] In Gaussian process regression (GPR), there are various kernel functions. In practice, the applicant found that the radial basis function (RBF, i.e., the radial kernel function that depends only on the distance between sample points) is better at characterizing the similarity between various record samples of the ice lake area time series than other kernel functions (such as linear kernel functions or polynomial kernel functions).

[0082] Elements of the covariance matrix in Gaussian process regression Indicates the first The and the first The correlation of each observed sample. After determining to use the radial basis function as the kernel function, the elements of the covariance matrix. This will depend on the radial distance between observed sample points. There are various methods for calculating the radial distance between sample points, such as using one dimension or multiple dimensions of the sample as the basis for calculation. The applicant found that for time series data of glacial lake area, using observation time as the single dimension of radial distance yields the best results.

[0083] The applicant also found that using the maximum log marginal likelihood method to train a Gaussian process regression model is better able to fit the relevant hyperparameters required for time series data on glacial lake area compared to other training methods, such as empirical Bayesian methods and Markov chain Monte Carlo (MCMC) methods.

[0084] In one embodiment, for each time series dataset of glacial lake area, a Gaussian process regression model is constructed and optimized, and the model is applied to fit the glacial lake area. Please refer to [reference needed]. Figure 2 ,include:

[0085] Step S201: Construct a Gaussian process regression model, using radial basis functions as the kernel function.

[0086] Radial basis functions (such as the squared exponential kernel and the Matérn kernel) are a class of kernel functions that depend on the distance between input points. Its core function is to quantify the similarity between any two input samples: the closer the samples are, the larger the kernel function value (the higher the correlation), and vice versa.

[0087] In one embodiment, the radial basis function constructed with a squared exponential kernel serves as the kernel function for the Gaussian process regression model:

[0088]

[0089] in, Represents a pair of observation samples. Indicates the signal variance. This represents the length scale. The signal variance in this kernel function... and length scale These are hyperparameters, and their optimal values ​​need to be found through training.

[0090] Step S202: Using the observation time difference of the observed sample pairs as the radial distance input, the selected kernel function is used, and a noise term is superimposed to construct the covariance matrix of the Gaussian process regression model.

[0091] The covariance matrix is ​​a matrix( (where is the number of observation samples), and the element value is the correlation between any two observation samples. After selecting the radial basis function as the kernel function, the correlation between two observation samples depends on the radial distance between the pair of observation samples.

[0092] In one embodiment, the observation time difference of the observed sample pair is used as the radial distance to calculate the correlation between the observed samples, and correlation noise is superimposed:

[0093]

[0094] in These are the indexes of the two samples in the observed sample pair. For the first The observation time for each observation sample For the first The observation time for each observation sample for Square array for Unit matrix of order, This represents the number of centrally observed samples in the time-series dataset of glacial lake area. The noise term is used as a hyperparameter of the model. The optimal value needs to be found through training.

[0095] Step S203: Optimize the training model to obtain the optimal hyperparameters by maximizing the logarithmic marginal likelihood.

[0096] Marginal likelihood in Gaussian process regression, given the model hyperparameters, is... In this context, the probability density of the observed data is obtained. The hyperparameter that yields the maximum probability density is considered the optimal parameter. To simplify calculations and avoid numerical underflow, the logarithmic marginal likelihood is typically used in the computation, and its mathematical expression is:

[0097]

[0098] in, Let be a column vector consisting of the observation times of each observation sample. Let be the column vector formed by the observed areas of each observed sample. To give input Under these conditions, the output data was observed. The conditional probability, Represents conditional probability The natural logarithm, This is the inverse of the aforementioned covariance matrix. Let be the determinant of the aforementioned covariance matrix.

[0099] Maximizing the logarithmic marginal likelihood refers to maximizing the logarithmic marginal likelihood by selecting appropriate hyperparameters. Since this process cannot be solved analytically directly, in practice, numerical optimization algorithms such as L-BFGS or the conjugate gradient method are often used to find the maximum value.

[0100] In one embodiment, step S203 uses the L-BFGS optimization method to obtain the optimal hyperparameters for the observed sample data in Table 1. .

[0101] Step S30: Use the trained model to calculate the predicted area and standard deviation of each observed sample in the initial sequence of glacial lake area, and calculate the confidence interval in combination with the observation error;

[0102] After obtaining the hyperparameters of the Gaussian process regression model through the above step S20, they can be substituted into the calculation formula of each element of the covariance matrix to obtain the specific value of the correlation between the observed samples.

[0103] Step S301: Calculate the predicted area value for each observed sample. The calculation formula is as follows:

[0104]

[0105] in, Indicates the first The predicted area of ​​the glacial lake for each observed sample is given by the formula. For the first The correlation matrix between each observed sample and all samples. Let it be its transpose matrix. This is the inverse of the aforementioned covariance matrix. This is a column vector consisting of the observed areas of each observed sample.

[0106] Step S302: Calculate the predicted standard deviation for each observed sample. The calculation formula is as follows:

[0107]

[0108] in, The standard deviation of the forecast is represented by the formula. For the first In the embodiments of this application, the autocorrelation value of an observed sample is calculated as follows: Since the kernel function is constructed based on radial basis functions, and the radial distance between a sample and itself is 0, referring to the previously defined kernel function formula, at this time… The value is the signal variance. .

[0109] Step S303: Calculate the total uncertainty of the ice lake area for each observation sample, using the following formula:

[0110]

[0111] in, Indicates the observed sample The corresponding total uncertainty, Indicates the observed sample The corresponding observation error, This represents the predicted standard deviation calculated in step S302.

[0112] In practice, the applicant found that step S303, which uses Euclidean distance to combine observation error and prediction standard deviation, has better results than using Manhattan distance.

[0113] Step S304: Calculate the confidence interval for the ice lake area of ​​each observation sample. The calculation formula is as follows:

[0114]

[0115] in, Indicates the first Confidence intervals for the area of ​​ice lakes for each observation sample. This is the predicted area of ​​the glacial lake calculated in step S301 for the observed sample. The total uncertainty of the observed sample calculated in step S303 is... This refers to the tolerance for errors.

[0116] In one embodiment, step S203 establishes a Gaussian process regression model (hyperparameter) based on the observed data in Table 1. The model was then used to calculate the predicted area, standard deviation, total uncertainty, and confidence interval for each observation sample in Table 1, as shown in Table 2. List, List, column sum As shown in the column. It is configured as 1.4.

[0117] surface In one embodiment, the area prediction, prediction standard deviation, total uncertainty, and confidence interval are calculated using a trained Gaussian process regression model.

[0118]

[0119] Step S40: Compare the observation area and confidence interval of each observation sample, and mark the observation samples as upper bound anomaly, normal and lower bound anomaly;

[0120] When the observed area is greater than the upper bound of the confidence interval, the observed sample is marked as an upper bound anomaly;

[0121] When the observed area is less than the lower bound of the confidence interval, the observed sample is marked as a lower bound anomaly;

[0122] When the observed area is within the confidence interval (including the endpoints), the observed sample is marked as normal.

[0123] In one embodiment, the classification label column after step S40 is shown in Table 3.

[0124] surface A table showing the correspondence between observed area, confidence interval, and classification label in one embodiment.

[0125]

[0126] Step S50: Identify glacial lake outburst events and mark the lower bound anomalous observation samples caused by glacial lake outburst events as normal.

[0127] Gaussian process regression analysis can effectively eliminate anomalous observations caused by random disturbances such as clouds, cloud shadows, and mountain shadows in automated glacial lake mapping tools. However, the sudden drop in area caused by glacial lake outburst events is often judged as anomaly by the model. To obtain more accurate time-series data on glacial lake area, it is necessary to identify glacial lake outburst events and mark the resulting area drop as a normal confidence value.

[0128] There are many ways to identify glacial lake outburst events. It can be done through on-site investigation or by relying on supplementary information such as news reports.

[0129] The applicant discovered in practice that, based on the dynamic characteristics of glacial lakes and the natural evolutionary patterns after a glacial lake outburst, establishing four discriminant constraints can effectively identify whether a sudden drop in area originates from a glacial lake outburst event or random disturbance. These four discriminant constraints are:

[0130] Constraints on the sudden drop in post-collapse area within the observation window;

[0131] Low-level constraint on the post-collapse area within the observation window;

[0132] The constraint of gradually varying post-collapse area within the observation window;

[0133] Stability constraint on the post-collapse area within the observation window.

[0134] In one embodiment, step S50 includes the following sub-steps:

[0135] Step S501: Select multiple observation samples within the historical window before the current lower bound anomaly sample observation time and the future window after the current lower bound as the observation window. The historical window and the future window are as follows:

[0136]

[0137]

[0138] in, Indicates the first The observed area of ​​each observed sample. This indicates the historical window of the observed sample. This indicates the future window of the observed sample. and Window length

[0139] In one embodiment, the length 'a' of the history window is configured to be 5, and the length 'b' of the future window is configured to be 4.

[0140] Taking observation #54 in Table 1 as an example, its historical window includes observation samples #49, #50, #51, #52, and #53, with corresponding observation areas of 1.4572, 1.1169, 1.4416, 1.4211, and 1.4013, respectively. Its future window includes observation samples #54, #55, #56, #57, and #58, with corresponding observation areas of 0.8064, 0.8752, 0.8158, 0.8356, and 0.8462, respectively.

[0141] Step S502: Perform post-collapse area drop constraint detection within the observation window, specifically determining whether the following inequality holds:

[0142]

[0143] in, For observation samples The median of the observed area within the historical window. For the corresponding standard deviation, This represents the preset threshold for the sudden drop in post-collapse area;

[0144] In one embodiment, It is configured as 3.

[0145] Taking observation sample #54 in Table 1 as an example, the median observed area in its historical window is 1.4211, and the standard deviation is 0.141. The inequality holds. Observation sample #54 passed the post-collapse area drop constraint test.

[0146] Step S503: Perform a post-collapse area low-level constraint detection within the observation window, specifically by determining whether the following inequality holds:

[0147]

[0148] in, For observation samples The median of the observed area within the historical window. For the sample The median of the observed area within the future window. The preset low-level constraint detection threshold for the post-collapse area is described.

[0149] In one embodiment, It is configured to 0.1.

[0150] Taking observation sample #54 in Table 1 as an example, its median observed area in the historical window is 1.4211, and its median observed area in the future window is 0.8356. The inequality holds. Observation sample #54 passed the continuous low-level constraint test.

[0151] Step S504: Perform post-collapse area gradual change constraint detection within the observation window, specifically determining whether the following inequality holds:

[0152]

[0153] in, Indicates the observed sample The slope of the linear regression equation fitting the observed area within the future window. This represents the preset threshold for detecting gradually varying post-collapse area constraints.

[0154] In one embodiment, It is configured to 0.03.

[0155] Taking observation sample #54 as an example, the slope of the equation is obtained by fitting the observed area in its future window through linear regression. It is 0.004, 0.004 The inequality holds. Observation sample #54 passed the post-collapse area gradual change constraint test.

[0156] Step S505: Perform post-collapse area stability constraint detection within the observation window, specifically by determining whether the following inequality holds:

[0157]

[0158] in, Represented as observation sample The standard deviation of the observed area in the future window, This represents the preset post-collapse area stability constraint threshold;

[0159] In one embodiment, It is configured to 0.1.

[0160] Taking observation sample #54 in Table 1 as an example, the observation areas in the future window are 0.8064, 0.8752, 0.8158, 0.8356, and 0.8462, with a corresponding standard deviation of 0.027, which is greater than the threshold of 0.1, so the inequality holds.

[0161] Step S506: Summarize the four discrimination constraints. When all inequalities are true, re-mark the current observation sample as normal.

[0162] Taking observation #54 in Table 1 as an example, it was classified as a lower bound anomaly by the Gaussian process regression model. In the post-flood area sudden drop constraint detection, post-flood area low-level constraint detection, post-flood area gradual change constraint, and post-flood area stability constraint detection, all four inequalities were valid, indicating that the area decrease of this observation sample was caused by the glacial lake outburst event. Given that the area decrease in this observation did not originate from random disturbance, observation sample #54 was re-marked as normal, as shown in Table 4.

[0163] surface In one embodiment, lower bound anomaly observation samples caused by glacial lake outburst events are marked as normal.

[0164]

[0165] Step S60: Remove all observation samples marked as upper or lower bound anomalies from the initial glacial lake area time series dataset to obtain the accurate glacial lake area time series dataset.

[0166] In the embodiments shown in Table 4, it can be seen that #50 is a lower bound anomaly and #48 is an upper bound anomaly, which are removed. The final glacial lake sequence is shown in Table 5. Compared with the initial glacial lake area sequence, this sequence has cleaned up anomalies caused by interference from clouds, snow, and mountain shadows, thus improving its reliability and providing a solid foundation for monitoring glacial lake area.

[0167] surface One embodiment of the ice lake area sequence after removing outliers

[0168]

[0169] One embodiment of this application provides a computer-readable storage medium storing a program, the stored program including the ice lake area anomaly removal method in any of the above embodiments, which can be loaded by a processor and processed.

[0170] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the above functions are implemented by executing the program with a computer. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be implemented. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash drive, or portable hard drive, and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be implemented.

[0171] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A method for outlier removal in time-series data of glacial lake area based on Gaussian process regression, characterized in that, include: An initial sequence of glacial lake area was constructed by sorting multiple observation samples consisting of observation time, observation area and observation error according to observation time. A Gaussian process regression model was constructed and trained based on the initial sequence of glacial lake area. The trained Gaussian process regression model is used to calculate the area prediction and standard deviation of each observed sample in the initial sequence, and the confidence interval is calculated in combination with the observation error. Compare the observed area and confidence interval of each observed sample, and mark the observed samples as upper bound abnormal, normal, and lower bound abnormal; Based on the dynamic characteristics of glacial lakes, an observation window is constructed for observation samples marked as lower bound anomalies. Within this observation window, four constraint checks are performed: sudden drop in post-breach area, low post-breach area, gradual change in post-breach area, and stable post-breach area. When the observation sample of the lower bound anomaly simultaneously meets all four constraint conditions, it is determined to be caused by a breach event and remarked as normal. Specifically: Perform post-collapse area drop constraint detection within the observation window, specifically by determining whether the following inequality holds: in, The median of the observed area within the historical window of observed sample i. For the corresponding standard deviation, θ d This represents the preset threshold for the sudden drop in post-collapse area; Perform a low-level constraint check on the post-collapse area within the observation window, specifically by determining whether the following inequality holds: in, The median of the observed area within the historical window of observed sample i. θ represents the median of the observed area within a future window of observed sample i. l The pre-defined threshold for detecting the low-level constraint of the post-collapse area is described. Perform a gradual change in post-collapse area constraint check within the observation window, specifically determining whether the following inequality holds: a i <θ s Where, α i θ represents the slope of the linear regression equation that represents the area of ​​the observed sample i within a future window. s This represents the preset threshold for detecting gradually varying post-collapse area constraints. Perform post-collapse area stability constraint checks within the observation window, specifically by determining whether the following inequality holds: in, Let θ represent the standard deviation of the observed area within a future window of observed sample i. b This represents the preset post-collapse area stability constraint threshold; By removing all records marked as upper or lower bound anomalies from the initial sequence of glacial lake areas, an accurate time series dataset of glacial lake areas is obtained.

2. The outlier removal method for time-series data of glacial lake area as described in claim 1, characterized in that, The aforementioned construction and training of a Gaussian process regression model based on the initial sequence of glacial lake area is as follows: The radial basis functions constructed using the squared exponential kernel serve as the kernel function for the Gaussian process regression model. In the formula, x, x ' Let k(x,x) represent a pair of observed samples. ' ) is the kernel function for this observation pair. ι represents the signal variance, and ι represents the length scale. Using the observation time difference of the observed sample pairs as the radial distance input, the selected kernel function is used, and a noise term is added to construct the covariance matrix of the Gaussian process regression model: Where i,j are the indices of the two observation samples in the observation sample pair, x i Let x be the observation time of the i-th observation sample. j Let K be the observation time of the j-th observation sample. y Let I be an N-order square matrix, and let N be the number of centrally observed samples in the time series dataset of glacial lake area. Noise term; The optimal hyperparameters are obtained by optimizing the training model by maximizing the log-marginal likelihood, where the log-marginal likelihood is specifically: Where X = [x1,...,x] n ] T The input matrix consists of the observation times of each observation sample, Y = [y1,...,y2]. n ] T Let p(Y|X) be the output matrix composed of the observed areas of each observed sample, p(Y|X) be the conditional probability of observing the output data Y given the input X, and logp(Y|X) be the natural logarithm of the conditional probability p(Y|X). Let |K be the inverse of the aforementioned covariance matrix. y | represents the determinant of the aforementioned covariance matrix.

3. The method for outlier removal in time-series data of glacial lake area as described in claim 1, characterized in that, The method of using a trained Gaussian process regression model to calculate the predicted area and standard deviation of each observed sample in the initial sequence, and calculating the confidence interval in conjunction with the observation error, includes: Calculate the predicted area value for each observed sample; Calculate the predicted standard deviation for each observed sample; The total uncertainty for each observation sample is calculated using the following formula: Among them, U i σ(x) represents the total uncertainty of observed sample i. i ) represents the prediction standard deviation, ε i This represents the observation error of observed sample i; The formula for calculating the confidence interval range for each observation sample is: Among them, CI i This represents the confidence interval for observed sample i. U is the predicted area of ​​observed sample i. i γ represents the total uncertainty of observed sample i, and γ is the error tolerance.

4. The method for removing outliers from time-series data of glacial lake area as described in claim 1, characterized in that, The specific method for constructing the corresponding observation window for observation samples that have been marked as lower bound anomalies is as follows: Multiple observation samples within the historical window before the observation time of the current lower bound anomaly observation sample and the future window thereafter are selected as observation windows. The historical window and the future window are as follows: Among them, y i Let represent the observed area of ​​the i-th observed sample. This indicates the historical window of the observed sample. This represents the future window for the observed sample, where a and b are the window lengths.

5. A computer-readable storage medium, characterized in that, The medium stores a program that can be loaded by a processor and executed as described in any one of claims 1 to 4, the method for removing outliers from time-series data of glacial lake area.

Images