Expert knowledge-based extraction method for massive three-dimensional deformation monitoring data of dam and high slope

Abstract

The embodiment of the invention discloses a dam and high slope mass three-dimensional deformation monitoring data extraction method based on expert knowledge, and the method comprises the steps: carrying out the sampling of the three-dimensional deformation data of a dam and a high slope according to a certain time interval, dividing the three-dimensional deformation data into refined time periods with different characteristics, and carrying out the noise reduction; different indexes are adopted to measure deformation changes of different time segments, a comprehensive abnormal score of polynomial weighting is calculated, and a deformation mode is judged; and for an abnormal type deformation mode and a stable type deformation mode, subcategories are refined, and representative values of the Japanese deformation quantity are selected in a simplified mode according to the subcategories. Through segmented noise reduction driven by expert rules, customized noise reduction is carried out according to characteristics of different deformation stages; and an isolated forest anomaly detection and comprehensive anomaly scoring method is introduced, so that the automation level and accuracy of deformation mode judgment are greatly improved, and wide applicability of the method in different climate, terrain and hydrological environments is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of deformation monitoring and measurement, specifically to a method for extracting massive three-dimensional deformation monitoring data of dams and high slopes based on expert knowledge. This method can be widely applied to long-term safety monitoring, hazard warning and health assessment of major infrastructure such as ultra-high dams and deep-excavated slopes, providing key technical support for intelligent operation and maintenance of infrastructure. Background Technology

[0002] As a core infrastructure for water resource management, flood control, and power generation, the structural health of dams directly impacts the safety of millions of people downstream and regional economic development. Traditional deformation monitoring relies primarily on point-contact methods such as total stations and leveling, which suffer from bottlenecks such as low monitoring frequency, limited spatial coverage, and inability to operate in adverse weather conditions. In recent years, Synthetic Aperture Radar Interferometry (InSAR) has gradually become an important tool for dam deformation monitoring due to its advantages of all-weather operation, high precision, and area monitoring. By continuously acquiring millimeter-level precision deformation time-series data, InSAR can not only capture the dynamic response of dams under the coupled effects of multiple factors such as temperature, water level, and geology, but also provide crucial data support for structural safety early warning. However, facing massive amounts of high-dimensional radar monitoring data, how to achieve the leap from "data accumulation" to "knowledge discovery" remains a pressing scientific challenge in the field of engineering safety.

[0003] The existing technologies commonly used for extracting and analyzing dam and high slope deformation data include the following: 1. Short-cycle analysis: Real-time capture of sudden deformations Short-cycle analysis (hourly / daily) is suitable for emergency monitoring scenarios such as construction periods, extreme weather (e.g., rainstorms / floods), and earthquakes. This method, through sliding window statistics (e.g., hourly averages) and abrupt change detection (e.g., the CUSUM algorithm), can capture sudden deformations in real time, providing timely data support for emergency response. Sliding window statistics quickly identify abnormal fluctuations in the data by calculating the mean and standard deviation within a short time window. Abrupt change detection algorithms (e.g., cumulative sum control charts, CUSUM) accurately detect abrupt changes in deformation by monitoring cumulative changes in the data. The advantage of short-cycle analysis lies in its rapid response capability, enabling the detection of abnormal changes in deformation within a short period, thereby supporting timely decision-making and action. For example, during rainstorms or floods, this method can monitor the deformation of dams in real time, promptly identify potential safety hazards, and provide a scientific basis for emergency management and decision-making.

[0004] 2. Medium-term analysis: Separating the influence of environmental factors Medium-term cycle analysis (weekly / monthly) is suitable for seasonal water level changes and periodic loads (such as water storage / discharge). See also Figure 1By utilizing Fourier transform to extract periodic components and STL (Seasonal-Trend Decomposition) to effectively separate the influence of environmental factors and clearly identify the long-term trend of deformation, this method can effectively separate the influence of environmental factors and clearly identify the long-term trend of deformation. Fourier transform extracts the main periodic components by converting time series data to the frequency domain, helping to identify deformation caused by seasonal water level changes or other periodic loads. STL decomposition further separates the influence of environmental factors by decomposing the time series into seasonal, trend, and residual components. The advantage of this method is that it can filter out short-term fluctuations and focus on the periodic patterns of deformation, providing important evidence for understanding the behavior of dams under different environmental conditions. For example, during seasonal water level changes, medium-term analysis can identify deformation patterns caused by water level changes, helping to assess the long-term stability of the dam.

[0005] Long-term analysis: assessing performance degradation throughout the entire lifecycle Long-term analysis (years / multi-year periods) primarily targets slow processes such as material aging and foundation settlement. See also... Figure 2 By fitting long-term trends using linear regression and employing the Mann-Kendall test to assess trend significance, a comprehensive evaluation of dam performance degradation throughout its entire lifespan can be achieved. Linear regression identifies long-term deformation trends by fitting time-series data, helping to predict the dam's future performance. The Mann-Kendall test is used to detect the significance of the trend, verifying whether the deformation trend is statistically significant. The advantage of this method lies in its long-term perspective, enabling the prediction of dam performance changes and providing scientific decision support for maintenance and management. For example, through long-term analysis, the deformation trend of the dam due to material aging or foundation settlement can be assessed, allowing for the development of reasonable maintenance plans and the extension of the dam's service life.

[0006] However, existing technologies still have various shortcomings: short-cycle monitoring is limited by the superposition effect of sensor sampling frequency and environmental noise, and in extreme scenarios such as rainstorms / earthquakes, high-frequency interference may be confused with the actual deformation; the universality of environmental factor decoupling models in medium-cycle analysis is insufficient; and the adaptability of existing Fourier transforms and STL decompositions to non-steady-state periodic loads (such as sudden flood discharges) needs to be verified. Long-cycle performance evaluation relies on the completeness of historical data, but most dams lack continuous monitoring data for more than 30 years, leading to the extrapolation risk of the Mann-Kendall test in ultra-long-term trend prediction. In addition, a multi-cycle collaborative analysis framework has not yet been established, and the data coupling mechanism and spatiotemporal correlation model of short-, medium-, and long-cycle periods still need further research.

[0007] Therefore, how to optimize three-dimensional deformation monitoring data, achieve accurate separation of high-frequency interference in extreme scenarios, and ensure the reliability of monitoring data has become a technical problem that needs to be solved by existing technologies. Summary of the Invention

[0008] In view of this, embodiments of the present invention provide a simplified extraction method for massive three-dimensional deformation monitoring data based on expert knowledge, which can be widely applied to long-term safety monitoring, hazard warning and health assessment of major infrastructure such as ultra-high dams and deep excavated slopes, providing key technical support for intelligent operation and maintenance of infrastructure.

[0009] A method for extracting massive three-dimensional deformation monitoring data of dams and high slopes based on expert knowledge includes: Three-dimensional deformation data of the dam and high side dam are sampled at certain time intervals to obtain a three-dimensional deformation data sequence X ( ...), based on prior knowledge and combined with the geographical environment of the dam, we divide the time periods into refined time periods with different characteristics, and denoise the three-dimensional deformation data of the time periods respectively; Based on the filtered and denoised segmented three-dimensional deformation data, different indicators are used to measure the deformation changes of different segments according to prior knowledge for different time periods. The isolated forest method is combined to identify abrupt anomalies in the three-dimensional deformation data. Using the above indicators and anomaly scores, and combining them with weight values, a polynomial-weighted comprehensive anomaly score is calculated. The deformation pattern is determined using the comprehensive anomaly score, thereby identifying the daily deformation pattern of the dam and high slope. For abnormal deformation patterns and stationary deformation patterns, the subcategories are refined, and representative values ​​of the daily deformation variables are simplified based on the subcategories.

[0010] Optionally, the sampling of three-dimensional deformation data of the dam and high side dam at certain time intervals, based on prior knowledge and combined with the geographical environment of the dam, divides the data into refined time periods with different characteristics, specifically including: Based on prior knowledge of three-dimensional deformation monitoring of dams and slopes, and according to the trend and periodic changes in deformation, the daily data is divided into three typical stages: the nighttime stable period (X_stable), the water level rise and fall period (X_wlevel), and the temperature sensitive period (X_tsense). The nighttime stable period (X_stable) is a time period in which temperature and water level are relatively stable, with small overall fluctuations and no obvious trend. The water level rise and fall period (X_wlevel) is a time period in which the dam water level gradually rises or falls under the physical effects of tides and other forces, showing a clear trend. The temperature sensitive period (X_tsense) is a time period in which daily temperature changes drastically and has a clear periodicity.

[0011] Optionally, the denoising of the three-dimensional deformation data for the time period includes: For each stable nighttime period, the 3D deformation data is processed using a moving average filter, where the window size is an odd number. The window unit is the deformation monitoring acquisition interval. Supplement both ends of the sequence data of the three-dimensional deformation data Points, for each 3D deformation data Calculate its total before and after Calculate the arithmetic mean of the points and output the smoothed result. As shown in formula (1): Formula (1); Three-dimensional deformation data for each period of water level rise and fall. Savitzky-Golay filtering is used, and the filter window size is taken as... To cover typical timescales of water level changes, the order of the polynomial fit is... Constructing a matrix Each row corresponds to the position of a point within the filter window, the first row... Line number Listed as Calculate the filter coefficient vector ,in Symmetrically fill m points at both ends of the three-dimensional deformation data sequence, and for each three-dimensional deformation data... Performing linear combinations, we obtain As shown in formula (2): Formula (2) For the three-dimensional deformation data X_tsense of each temperature-sensitive period, a Fourier low-pass filter is applied, and a frequency domain window is defined. ,set up exist Set all others to zero, retain low-frequency filtering, and process the three-dimensional deformation data within the segment. Perform a discrete Fourier transform to obtain ,structure Unwanted frequency components are removed, and an inverse discrete Fourier transform is performed; the real part is the filtering result. .

[0012] Optionally, based on the filtered and denoised segmented three-dimensional deformation data, different indices are used to measure the deformation changes of different segments according to prior knowledge for different time periods, specifically including: For sampling interval Observational data after filtering and noise reduction ,in Construct the time vector Implement decentralization , ; For the denoised 3D deformation data during the nighttime stabilization period Using the standard deviation of fluctuation As an indicator reflecting the strength of fluctuations under stable conditions, the average value of the nighttime stable period data is... Then the standard deviation of the fluctuation Represented as: Formula (3) For the noise-reduced three-dimensional deformation data during the rise and fall of water levels Using linear trend slope with goodness of fit To reflect the strength and significance of the trend, the slope of the linear trend. Represented as: , Formula (4) Goodness of fit Represented as Formula (5) The total sum of squares is The regression sum of squares is The predicted value is The intercept is expressed as ; For the denoised three-dimensional deformation data during the temperature-sensitive period and the rise and fall of water levels Using decentralized receipts, perform discrete Fourier transform;

[0013] N represents the number of data points during the temperature-sensitive period, calculated by dividing the duration of the temperature-sensitive period by the sampling interval, where the sampling frequency is... Calculate the corresponding period of the temperature-sensitive period. frequency via frequency point Calculate periodic amplitude and spectral significance This reflects the amplitude of the main period and the prominence of that period relative to other frequencies. Periodic amplitude Represented as: , Spectral significance Represented as: , ; The method of combining isolated forests to identify abrupt outliers in 3D deformation data specifically includes: For the morphological sequence that has been filtered and denoised Given the number of trees in the forest Subsampling size of each tree Maximum tree depth Referring to the average height of a binary search tree, a normalization constant is defined as follows: ,

[0014] in Let be the Euler constant, for each Calculate abnormal scores ,in show Easily isolated, highly likely to be abnormal. Difficult to isolate, tends to be normal. Using the average outlier score This is used to represent the anomaly index of the overall data.

[0015] Optionally, the step of using the above indicators and anomaly scores, combined with weight values, to calculate a polynomial-weighted comprehensive anomaly score, and using the comprehensive anomaly score to determine the deformation pattern, specifically involves: The comprehensive anomaly score of the daily shape variable data is calculated using formula (6). Formula (6) in, For standardized scores, weights The threshold for the comprehensive anomaly score is defined and set by experts based on experience. When the value is greater than or equal to the threshold, it belongs to an abnormal deformation mode. When the value is less than the threshold, it belongs to a stable deformation mode.

[0016] Optionally, the subcategories for abnormal deformation modes and stable deformation modes are further refined, including: for abnormal deformation modes, subdividing them into trend abnormal, periodic abnormal and sudden abnormal; and for stable deformation modes, subdividing them into absolute stable, trend stable and periodic stable.

[0017] Optionally, the step of simplifying the representative value of the daily shape variable according to the sub-category specifically involves: For abnormal trends, the extreme value method of trend direction is used. If the trend is positive, i.e., an upward trend, the representative value is the maximum value of the whole day: If the trend is negative, i.e., a downward trend, the value represents the minimum value of the day: ,in This represents a set of deformation monitoring data for a whole day after segmented noise reduction; For periodic anomalies, the maximum period amplitude method is used to determine the pair of points with the maximum period peak-valley range (maximum amplitude) based on Fourier analysis: Or choose the one with the largest absolute amplitude, among which This represents the peak value of the most typical cycle. This represents the trough value of the most typical cycle; For anomalous mutations, the isolated forest anomaly score extreme value method is used to select the point with the highest absolute value of the anomaly score from the anomalous data points detected by the isolated forest. For absolutely stationary data, a robust median method is used to remove outliers from isolated forests. ,in This represents the set of normal data remaining after filtering out abnormal data points; For trend-stable data, a robust trend-fitting method is used. First, a linear fit is applied to the entire day's data, and then the midpoint (i.e., the midpoint of the time) on the trend-fitting line is calculated. ,in The midpoint of the day. and The slope and intercept of the trend-fitted line; For periodically stationary types, the typical period amplitude center method is used, and Fourier analysis is employed to determine the principal period amplitude. That is, the midpoint of the most typical cyclical peak and trough values ​​of a day, where This represents the peak value of the most typical cycle. This represents the trough value of the most typical cycle.

[0018] Optionally, the three-dimensional deformation data may be millimeter-wave radar data, LiDAR scanning data, continuous GNSS monitoring data, three-dimensional photogrammetric reconstruction data, ground synthetic aperture radar (GB-SAR) monitoring data, or three-dimensional deformation data collected by a smart sensor network (IoT device).

[0019] The present invention further discloses an electronic device, comprising: The processor and memory are connected via a bus. The memory is adapted to store instructions or programs executable by the processor. The processor executes the instructions stored in the memory to perform the above-mentioned method for extracting massive three-dimensional deformation monitoring data of dams and high slopes based on expert knowledge.

[0020] The present invention has the following advantages: 1. A piecewise denoising approach driven by expert rules is employed, using moving average, Savitzky-Golay filtering, and Fourier low-pass filtering respectively to customize denoising for features at different deformation stages. Compared to traditional uniform filtering methods, this approach better preserves key feature information. 2. This invention introduces an isolated forest anomaly detection and comprehensive anomaly scoring method to replace the traditional pattern judgment based on human experience, which greatly improves the automation level and accuracy of deformation pattern judgment, reduces human subjective error, and is especially suitable for engineering scenarios that require large-scale and rapid screening of anomalies.

[0021] 3. This invention allows for adaptive adjustment of time period division criteria (such as time intervals for nighttime stability periods, water level rise and fall periods, and temperature-sensitive periods) and specific parameters of each filter (such as moving average window size, polynomial order, and frequency domain cutoff frequency) according to actual monitoring needs. This flexible parameter configuration ensures the invention's broad applicability under different climates, terrains, and hydrological environments, further enhancing the technical solution's flexibility and practical application effectiveness. Attached Figure Description

[0022] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which: Figure 1 This is a schematic diagram of the Fourier transform principle of existing technology; Figure 2 This is a flowchart of the existing linear regression fitting trend analysis process; Figure 3 This is a flowchart of a method for extracting massive three-dimensional deformation monitoring data of dams and high slopes based on expert knowledge, according to a specific embodiment of the present invention. Figure 4 This is a principle analysis diagram of the method for extracting massive three-dimensional deformation monitoring data of dams and high slopes based on expert knowledge according to a specific embodiment of the present invention; Figure 5 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0023] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0024] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0025] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0026] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0027] The main features of this invention are: introducing expert prior knowledge and understanding of the geographical environment to perform fine segmentation and noise reduction on deformation data; then calculating simplified deformation indicators based on time series analysis; integrating the isolated forest method to identify anomalies and achieve deformation pattern determination; finally, further subdividing deformation patterns based on expert prior characteristics and selectively choosing representative values ​​to achieve simplified data extraction and improve the accuracy and effectiveness of monitoring data processing as a whole.

[0028] This method effectively addresses the challenges of large data volumes, information redundancy, and low feature extraction efficiency in the current field of dam and high slope monitoring. It avoids the blind calculations of traditional algorithms in complex engineering scenarios, facilitating engineers' rapid risk assessment. This invention can be widely applied to long-term safety monitoring, hazard warning, and health assessment of major infrastructure such as ultra-high dams and deep-excavated slopes, providing key technical support for intelligent operation and maintenance of infrastructure.

[0029] For details, see Figure 1 , Figure 2 The flowchart and principle analysis diagram of the method for extracting massive three-dimensional deformation monitoring data of dams and high slopes based on expert knowledge according to specific embodiments of the present invention are shown respectively.

[0030] Specifically, it includes: Deformation monitoring data segmentation and noise reduction steps S110: Three-dimensional deformation data of the dam and high side dam are sampled at certain time intervals to obtain a three-dimensional deformation data sequence X ( ...), based on prior knowledge, such as experts' prior knowledge of dam and high slope deformation, combined with the geographical environment of the dam, we divide the data into refined time periods with different characteristics, and then perform noise reduction on the three-dimensional deformation data of the time periods respectively.

[0031] Segmented noise reduction can reduce the impact of filtering methods on different features such as periodic and trend data, thereby reducing the amount of data with minimal information loss.

[0032] The process of sampling three-dimensional deformation data of the dam and high side dam at certain time intervals, and dividing the data into refined time periods with different characteristics based on prior knowledge and the geographical environment of the dam, specifically includes: Based on prior knowledge of dam and slope three-dimensional deformation monitoring, such as expert experience, and according to the trend and periodic changes in deformation, daily data is divided into three typical stages: the nighttime stable period (X_stable), the water level rise and fall period (X_wlevel), and the temperature sensitive period (X_tsense). The nighttime stable period (X_stable) is a time when temperature and water level are relatively stable, with small overall fluctuations and no obvious trend, typically concentrated at night. The water level rise and fall period (X_wlevel) is a time when the dam water level gradually rises or falls under the influence of tides and other physical forces, exhibiting a clear trend. The temperature sensitive period (X_tsense) is a time when daily temperature changes drastically and exhibits a clear periodicity.

[0033] In an optional embodiment, the nighttime stable period X_stable is 220:00-6:00, the water level rise and fall period X_wlevel is 6:00-12:00, and the temperature sensitive period X_tsense is 12:00-18:00.

[0034] Furthermore, the denoising of the three-dimensional deformation data for the time period includes: For each nighttime stable period, the three-dimensional deformation data Considering its relatively small fluctuations, a moving average filter is used to smooth out minor random fluctuations in the deformed data, highlighting the overall stable trend. In this moving average filter, the window size is an odd number. The window unit is the deformation monitoring acquisition interval. Supplement both ends of the sequence data of the three-dimensional deformation data Points, for example, can be filled with mirror images or repeated boundary values ​​to prevent the window from going out of bounds, for each 3D deformation data point. Calculate its total before and after Calculate the arithmetic mean of the points and output the smoothed result. As shown in formula (1): Formula (1).

[0035] Three-dimensional deformation data for each period of water level rise and fall. Savitzky-Golay filtering (a filtering method based on local polynomial fitting) is employed to smooth noise while preserving trends and higher-order features as much as possible. The filter window size is selected. To cover typical timescales of water level changes, the order of the polynomial fit is... Constructing a matrix Each row corresponds to the position of a point within the filter window, the first row... Line number Listed as Calculate the filter coefficient vector ,in Symmetrical padding is applied to both ends of the sequence, i.e., in three-dimensional deformation data. The sequence is symmetrically padded with m points at both ends, for example, mirror filling, and for each 3D deformation data Performing linear combinations, we obtain As shown in formula (2): Formula (2) For the three-dimensional deformation data X_tsense of each temperature-sensitive period, a Fourier low-pass filter is used to retain the diurnal periodic component, remove high-frequency random noise, and define a frequency domain window. ,set up exist Set all others to zero, retain low-frequency filtering, and process the three-dimensional deformation data within the segment. Perform a discrete Fourier transform to obtain ,structure Unwanted frequency components are removed, and an inverse discrete Fourier transform is performed; the real part is the filtering result. .

[0036] Deformation mode determination step S120 based on time series analysis: Based on the filtered and denoised segmented three-dimensional deformation data , and Based on prior knowledge for different time periods, different indicators are used to measure deformation changes in different segments. The isolated forest method is combined to identify abrupt anomalies in the three-dimensional deformation data. Using the above indicators and anomaly scores, and combining them with weight values, a polynomial-weighted comprehensive anomaly score is calculated. The deformation pattern is determined using the comprehensive anomaly score, thereby identifying the daily deformation pattern of the dam and high slope.

[0037] The three-dimensional deformation data segmented after filtering and denoising. , and For prior knowledge at different time periods, different indicators are used to measure deformation changes in different segments, specifically including: For sampling interval Observational data after filtering and noise reduction ,in Construct the time vector Implement decentralization , ; For the denoised 3D deformation data during the nighttime stabilization period Using the standard deviation of fluctuation As an indicator reflecting the strength of fluctuations under stable conditions, the average value of the nighttime stable period data is... Then the standard deviation of the fluctuation Represented as: Formula (3) For the noise-reduced three-dimensional deformation data during the rise and fall of water levels Using linear trend slope with goodness of fit To reflect the strength and significance of the trend, the slope of the linear trend. Represented as: , Formula (4) Goodness of fit Represented as Formula (5) The total sum of squares is The regression sum of squares is The predicted value is The intercept is expressed as ; For the denoised three-dimensional deformation data during the temperature-sensitive period and the rise and fall of water levels Using decentralized receipts, perform discrete Fourier transform;

[0038] N represents the number of data points during the temperature-sensitive period, calculated by dividing the duration of the temperature-sensitive period by the sampling interval, where the sampling frequency is... Calculate the corresponding period of the temperature-sensitive period. frequency via frequency point Calculate periodic amplitude and spectral significance This reflects the amplitude of the main period and the prominence of that period relative to other frequencies. Periodic amplitude Represented as: It reflects the intensity of temperature deformation; the larger the amplitude, the more significant the effect of temperature on deformation.

[0039] Spectral significance Represented as: , This quantifies the prominence of the main cycle relative to other frequencies.

[0040] For example, if Exceeding the threshold and Significant, classified as a periodic anomaly.

[0041] In addition to considering the trend changes and periodic fluctuations of the deformation variables in the fine segmentation, it is still necessary to consider the impact of outliers on the overall data stationarity. Therefore, the isolated forest method is used for unsupervised anomaly detection on the filtered data.

[0042] The method of combining isolated forests to identify abrupt outliers in 3D deformation data specifically includes: For the morphological sequence that has been filtered and denoised Given the number of trees in the forest Subsampling size of each tree Maximum tree depth Referring to the average height of a binary search tree, a normalization constant is defined as follows: ,

[0043] in Let be the Euler constant, for each Calculate abnormal scores ,in show Easily isolated, highly likely to be abnormal. Difficult to isolate, tends to be normal.

[0044] Using the average outlier score This is used to represent the anomaly index of the overall data.

[0045] For example, the Isolation Forest algorithm can be:

[0046] The process involves using the aforementioned indicators and anomaly scores, combined with weight values, to calculate a polynomial-weighted comprehensive anomaly score. This comprehensive anomaly score is then used to determine the deformation pattern. Specifically: The comprehensive anomaly score of the daily shape variable data is calculated using formula (6). Formula (6) in, Standardized scores can be calculated using historical data of the relevant indicators, with weights... The threshold for the comprehensive anomaly score is defined and set by experts based on experience. When the value is greater than or equal to the threshold, it belongs to an abnormal deformation mode. When the value is less than the threshold, it belongs to a stable deformation mode.

[0047] Simplified selection of deformation variables for multi-deformation modes, S130: For abnormal deformation patterns and stationary deformation patterns, the subcategories are refined, and representative values ​​of the daily deformation variables are simplified based on the subcategories.

[0048] The abnormal deformation patterns and the stable deformation patterns are further subdivided into subcategories, including: for abnormal deformation patterns, trend abnormality, periodic abnormality and sudden abnormality; for stable deformation patterns, absolute stability, trend stability and periodic stability.

[0049] See Table 1 for a classification of deformation patterns.

[0050] Table 1 Classification of Deformation Modes

[0051] Based on the aforementioned subcategories, the representative values ​​for the simplified selection of the daily-shaped variables are as follows: For abnormal trends, the extreme value method of trend direction is used. If the trend is positive, i.e., an upward trend, the representative value is the maximum value of the whole day: If the trend is negative, i.e., a downward trend, the value represents the minimum value of the day: ,in This represents a set of deformation monitoring data for a whole day after segmented noise reduction.

[0052] For periodic anomalies, the maximum period amplitude method is used to determine the pair of points with the maximum period peak-valley range (maximum amplitude) based on Fourier analysis: Or choose the one with the largest absolute amplitude, among which This represents the peak value of the most typical cycle. This represents the trough value of the most typical cycle.

[0053] For mutation anomalies, the isolated forest anomaly score extreme value method is used to select the point with the highest absolute value of the anomaly score from the anomaly data points detected by the isolated forest.

[0054] For absolutely stationary data, a robust median method is used to remove outliers from isolated forests. ,in express.

[0055] For trend-stable data, a robust trend-fitting method is used. First, a linear fit is applied to the entire day's data, and the midpoint (i.e., the midpoint of the time interval) on the trend-fitting line is calculated: ,in The midpoint of the day. and The slope and intercept of the trend-fitted line.

[0056] For periodically stationary types, the typical period amplitude center method is used. The principal period amplitude is determined using Fourier analysis: That is, the midpoint of the most typical cyclical peak and trough values ​​of a day, where This represents the peak value of the most typical cycle. This represents the trough value of the most typical cycle.

[0057] In this invention, the three-dimensional deformation data is millimeter-wave radar data, but this invention is not limited to this. LiDAR scanning data, continuous GNSS monitoring data, three-dimensional photogrammetric reconstruction data, ground synthetic aperture radar (GB-SAR) monitoring data, and three-dimensional deformation data collected by smart sensor networks (IoT devices) can all be used as data inputs for this invention.

[0058] Example: This embodiment takes a dam slope in Fujian Province, China as the research object, and installs a millimeter-wave radar and one observation point device. Considering the consistency of the three-dimensional deformation changes in all three directions, the x-axis deformation is used as an example to demonstrate the detailed implementation process and results:

[0059] Segmented noise reduction of deformation monitoring data S110 S111. Based on expert experience in three-dimensional deformation monitoring of dams and slopes, and according to the trend and periodic changes in deformation, the daily data is divided into three typical stages: the nighttime stable period. Water level rise and fall period Temperature sensitive period Assume the nighttime stable period is from 22:00 to 6:00, the water level rise and fall period is from 6:00 to 12:00, and the temperature sensitive period is from 12:00 to 18:00.

[0060] S112. Different filtering methods are used to denoise the deformation data based on the typical time periods defined by expert knowledge. The following shows some filtering results for the nighttime stable period and the water level rise / fall period:

[0061] Deformation mode determination based on time series analysis S120 S121 - For deformation monitoring data that has undergone segmented noise reduction, calculate the simplified deformation index for refined segmentation.

[0062] S121-1 and S121-2. For nighttime stable period data, the standard deviation of volatility is used as an indicator to reflect the strength of volatility under stable conditions. The mean of the stable period data is... Then the standard deviation of the fluctuation The value of 0.1481 indicates that the fluctuation range of the deformation is relatively small during the stable period.

[0063] S121-3. For periods of water level rise and fall, the slope of the linear trend and the goodness of fit are used to reflect the trend strength and significance. Linear trend slope The value is close to 0 (-2.49e-15), indicating that the deformation change during this period is very small, with almost no significant trend. Furthermore, the intercept can be expressed as... The predicted value is The total variation is The regression sum of squares is Then the goodness of fit The value of 0.9837 indicates that the trend change was very significant during this period, and most of the data can be well fitted by linear regression.

[0064] S121-4. For the temperature-sensitive period, Fourier transform was used, and the period amplitude and spectral significance were employed to reflect the amplitude of the main period and the prominence of the period relative to other frequencies. The calculated period amplitude was 0.9943, indicating that the amplitude of the main period was relatively large, and the spectral significance was 84.45, indicating that the prominence of the main period relative to other frequencies was very significant.

[0065] S121-5. In addition to considering the trend changes and periodic fluctuations of deformation variables in fine-grained segmentation, it is still necessary to consider the impact of outliers on the overall data stationarity. Therefore, the isolated forest method is used for unsupervised anomaly detection on the filtered data. The outlier threshold is set to 0.7, and the anomaly results from the isolated forest screening are as follows:

[0066] The average outlier score was 0.4856, and a total of 9.85% of the observation records were identified as outliers.

[0067] S122. The comprehensive anomaly score for diurnal variation data calculated using polynomials can be expressed as:

[0068] in For standardized scores, weights ( The scores were defined by experts based on experience as 0.2, 0.2, 0.2, and 0.4, and a binary anomaly threshold of 0.7 was given. The calculated comprehensive anomaly score was 0.7688, which is considered an anomaly.

[0069] Simplified selection of deformation variables for multi-deformation modes S130 S131. For the identified deformation patterns, further detailed categorization is performed based on expert knowledge. Abnormal deformation patterns can be subdivided into trend abnormalities, periodic abnormalities, and sudden change abnormalities; stationary deformation patterns can be subdivided into absolutely stationary, trend-stationary, and periodic-stationary patterns. Based on expert analysis of the contribution of factors to the overall abnormality score, the A×P of the data for this day is extremely high, with a significant period, indicating a periodic abnormality, which may present a temperature sensitivity issue.

[0070] S132. For periodic anomalies, the maximum period amplitude method is used. After excluding outliers, the pair of points with the maximum period peak-to-valley range (maximum amplitude) is determined using Fourier analysis: That is, the peak value is 0.1623.

[0071] This invention is not only applicable to three-dimensional deformation monitoring of dams and high slopes, but can also be extended to fields such as tunnels, subways, mines, and urban surface subsidence. Especially in subway and tunnel construction, continuous monitoring of surface subsidence and retaining structure displacement data is crucial for ensuring project safety. The refined segmented noise reduction and pattern determination method of this invention can effectively improve the utilization rate of monitoring data and anomaly identification capabilities. Simultaneously, in large-scale surface subsidence monitoring in mining areas, due to the massive data volume and complex variation patterns, the expert knowledge-driven data simplification strategy provided by this invention can significantly reduce the complexity of data processing and improve the timeliness of early warning. Furthermore, the method of this invention can also be applied to the processing of large-scale InSAR deformation data in cities to achieve efficient data filtering and pattern recognition, contributing to the safety management of urban infrastructure.

[0072] The present invention has the following advantages: 1. A piecewise denoising approach driven by expert rules is employed, using moving average, Savitzky-Golay filtering, and Fourier low-pass filtering respectively to customize denoising for features at different deformation stages. Compared to traditional uniform filtering methods, this approach better preserves key feature information. 2. This invention introduces an isolated forest anomaly detection and comprehensive anomaly scoring method to replace the traditional pattern judgment based on human experience, which greatly improves the automation level and accuracy of deformation pattern judgment, reduces human subjective error, and is especially suitable for engineering scenarios that require large-scale and rapid screening of anomalies.

[0073] 3. This invention allows for adaptive adjustment of time period division criteria (such as time intervals for nighttime stability periods, water level rise and fall periods, and temperature-sensitive periods) and specific parameters of each filter (such as moving average window size, polynomial order, and frequency domain cutoff frequency) according to actual monitoring needs. This flexible parameter configuration ensures the invention's broad applicability under different climates, terrains, and hydrological environments, further enhancing the technical solution's flexibility and practical application effectiveness.

[0074] This invention features high scalability in its methodological structure, significant breadth and portability in its application fields, and high efficiency and flexibility in its technical implementation. It can play an important role in various complex geological environments and diverse monitoring needs, providing strong technical support for infrastructure safety monitoring, disaster early warning and risk management, and has good industrialization prospects and promotion value.

[0075] Embodiments of the present invention further disclose an electronic device, such as... Figure 5 As shown, Figure 5 The illustrated electronic device is a general address lookup device, comprising a general computer hardware architecture, including at least a processor 91 and a memory 92. The processor 91 and memory 92 are connected via a bus 93. The memory 92 is adapted to store instructions or programs executable by the processor 91. The processor 91 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 91 executes the instructions stored in the memory 92, thereby performing the method for extracting massive three-dimensional deformation monitoring data of dams and high slopes based on expert knowledge, as described in the embodiments of the present invention.

[0076] Processor 91 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, processor 91 executes instructions stored in memory 92 to perform the method flow described above in this embodiment of the invention, thereby processing data and controlling other devices. Bus 93 connects the aforementioned components together, and connects these components to display controller 94, display device, and input / output (I / O) device 95. Input / output (I / O) device 95 can be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, input / output device 95 is connected to the system via input / output (I / O) controller 96.

[0077] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] This application is described with reference to flowchart illustrations of methods, apparatus (devices), and computer program products according to embodiments of this application. It should be understood that each step in the flowchart can be implemented by computer program instructions.

[0079] These computer program instructions may be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means, the implementation process of which is described in the instruction means. Figure 1 The function specified in one or more processes.

[0080] These computer program instructions may also be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, produce instructions for implementing processes. Figure 1 A device for a function specified in one or more processes.

[0081] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program for use by a computer to execute some or all of the above-described method embodiments.

[0082] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program specifying the relevant hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0083] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for extracting massive three-dimensional deformation monitoring data of dams and high slopes based on expert knowledge, characterized in that, Includes the following steps: Three-dimensional deformation data of the dam and high side dam are sampled at certain time intervals to obtain a three-dimensional deformation data sequence X ( ...), based on prior knowledge and combined with the geographical environment of the dam, we divide the time periods into refined time periods with different characteristics, and denoise the three-dimensional deformation data of the time periods respectively; Based on the filtered and denoised segmented three-dimensional deformation data, different indicators are used to measure the deformation changes of different segments according to prior knowledge for different time periods. The isolated forest method is combined to identify abrupt anomalies in the three-dimensional deformation data. Using the above indicators and anomaly scores, and combining them with weight values, a polynomial-weighted comprehensive anomaly score is calculated. The deformation pattern is determined using the comprehensive anomaly score, thereby identifying the daily deformation pattern of the dam and high slope. For abnormal deformation patterns and stationary deformation patterns, the subcategories are refined, and representative values ​​of the daily deformation variables are simplified based on the subcategories.

2. The extraction method according to claim 1, characterized in that: The process involves sampling three-dimensional deformation data of the dam and high side dam at regular time intervals. Based on prior knowledge and the geographical environment of the dam, refined time periods with different characteristics are defined, specifically including: Based on prior knowledge of three-dimensional deformation monitoring of dams and slopes, and according to the trend and periodic changes in deformation, the daily data is divided into three typical stages: the nighttime stable period (X_stable), the water level rise and fall period (X_wlevel), and the temperature sensitive period (X_tsense). The nighttime stable period (X_stable) is a time period in which temperature and water level are relatively stable, with small overall fluctuations and no obvious trend. The water level rise and fall period (X_wlevel) is a time period in which the dam water level gradually rises or falls under the physical effects of tides and other forces, showing a clear trend. The temperature sensitive period (X_tsense) is a time period in which daily temperature changes drastically and has a clear periodicity.

3. The extraction method according to claim 2, characterized in that: The noise reduction of the three-dimensional deformation data for the time period includes: For each stable nighttime period of 3D deformation data, a moving average filter is applied, wherein the window size in the moving average filter is an odd number. The window unit is the deformation monitoring acquisition interval. Supplement both ends of the sequence data of the three-dimensional deformation data Points, for each 3D deformation data Calculate its total before and after Calculate the arithmetic mean of the points and output the smoothed result. As shown in formula (1): Official (1); Three-dimensional deformation data for each period of water level rise and fall. Savitzky-Golay filtering is used, and the filter window size is taken as... To cover typical timescales of water level changes, the order of the polynomial fit is... Constructing a matrix Each row corresponds to the position of a point within the filter window, the first row... Line 1 Listed as Calculate the filter coefficient vector ,in Symmetrically fill m points at both ends of the three-dimensional deformation data sequence, and for each three-dimensional deformation data... Performing a linear combination yields... As shown in formula (2): Official (2) For the three-dimensional deformation data X_tsense of each temperature-sensitive period, a Fourier low-pass filter is applied, and a frequency domain window is defined. ,set up exist Set all others to zero, retain low-frequency filtering, and process the three-dimensional deformation data within the segment. Perform a discrete Fourier transform to obtain ,structure Unwanted frequency components are removed, and an inverse discrete Fourier transform is performed; the real part is the filtering result. .

4. The extraction method according to claim 3, characterized in that: Based on the filtered and denoised segmented 3D deformation data, different metrics are used to measure the deformation changes of different segments according to prior knowledge for different time periods. Specifically, this includes: For sampling interval Observational data after filtering and noise reduction ,in Construct the time vector Implement decentralization , ; For the denoised 3D deformation data during the nighttime stabilization period Using the standard deviation of fluctuation As an indicator reflecting the strength of fluctuations under stable conditions, the average value of the nighttime stable period data is... Then the standard deviation of the fluctuation Represented as: Official (3) For the noise-reduced three-dimensional deformation data during the rise and fall of water levels Using linear trend slope with goodness of fit To reflect the strength and significance of the trend, the slope of the linear trend. Represented as: , Official (4) Goodness of fit Represented as Official (5) The total sum of squares is The regression sum of squares is The predicted value is The intercept is expressed as ; For the denoised three-dimensional deformation data during the temperature-sensitive period and the rise and fall of water levels Using decentralized receipts, perform discrete Fourier transform; ； N represents the number of data points during the temperature-sensitive period, calculated by dividing the duration of the temperature-sensitive period by the sampling interval, where the sampling frequency is... Calculate the corresponding period of the temperature-sensitive period. frequency via frequency point Calculate periodic amplitude and spectral significance This reflects the amplitude of the main period and the prominence of that period relative to other frequencies. Periodic amplitude Represented as: , Spectral significance Represented as: , ; The method of combining isolated forests to identify abrupt outliers in 3D deformation data specifically includes: For the morphological sequence that has been filtered and denoised Given the number of trees in the forest Subsampling size of each tree Maximum tree depth Referring to the average height of a binary search tree, a normalization constant is defined as follows: ， ； in Let be the Euler constant, for each Calculate abnormal scores ,in show Easily isolated, highly likely to be abnormal. Difficult to isolate, tends to be normal. Using the average outlier score This is used to represent the anomaly index of the overall data.

5. The extraction method according to claim 4, characterized in that: The process involves using the aforementioned indicators and anomaly scores, combined with weight values, to calculate a polynomial-weighted comprehensive anomaly score. This comprehensive anomaly score is then used to determine the deformation pattern. Specifically: The comprehensive anomaly score of the daily shape variable data is calculated using formula (6). Official (6) in, For standardized scores, weights The threshold for the comprehensive anomaly score is defined and set by experts based on experience. When the value is greater than or equal to the threshold, it belongs to an abnormal deformation mode. When the value is less than the threshold, it belongs to a stable deformation mode.

6. The extraction method according to claim 5, characterized in that: The abnormal deformation patterns and the stable deformation patterns are further subdivided into subcategories, including: for abnormal deformation patterns, trend abnormality, periodic abnormality and sudden abnormality; for stable deformation patterns, absolute stability, trend stability and periodic stability.

7. The extraction method according to claim 6, characterized in that: The process of simplifying the selection of representative values ​​for daily shape variables based on the subcategories is as follows: For abnormal trends, the extreme value method of trend direction is used. If the trend is positive, i.e., an upward trend, the representative value is the maximum value of the whole day: If the trend is negative, i.e., a downward trend, the value represents the minimum value of the day: ,in This represents a set of deformation monitoring data for a whole day after segmented noise reduction; For periodic anomalies, the maximum period amplitude method is used to determine the pair of points with the maximum period peak-valley range (maximum amplitude) based on Fourier analysis: Or choose the one with the largest absolute amplitude, among which This represents the peak value of the most typical cycle. This represents the trough value of the most typical cycle; For anomalous mutations, the isolated forest anomaly score extreme value method is used to select the point with the highest absolute value of the anomaly score from the anomalous data points detected by the isolated forest. For absolutely stationary data, a robust median method is used to remove outliers from isolated forests. ,in This represents the set of normal data remaining after filtering out abnormal data points; For trend-stable data, a robust trend-fitting method is used. First, a linear fit is applied to the entire day's data, and then the midpoint (i.e., the midpoint of the time) on the trend-fitting line is calculated. ,in The midpoint of the day. and The slope and intercept of the trend-fitted line; For periodically stationary types, the typical period amplitude center method is used, and Fourier analysis is employed to determine the principal period amplitude. That is, the midpoint of the most typical cyclical peak and trough values ​​of a day, where This represents the peak value of the most typical cycle. This represents the trough value of the most typical cycle.

8. The extraction method according to any one of claims 1-7, characterized in that: The three-dimensional deformation data can be millimeter-wave radar data, LiDAR scanning data, continuous GNSS monitoring data, three-dimensional photogrammetric reconstruction data, ground synthetic aperture radar (GB-SAR) monitoring data, or three-dimensional deformation data collected by smart sensor networks (IoT devices).

9. An electronic device, characterized in that, include: A processor and a memory are connected via a bus. The memory is adapted to store processor-executable instructions or programs. The processor executes the instructions stored in the memory to perform the method for extracting massive three-dimensional deformation monitoring data of dams and high slopes based on expert knowledge, as described in any one of claims 1-8.

Images