Hydraulic engineering monitoring data processing method and system based on cloud computing

Abstract

The invention discloses a water conservancy project monitoring data processing method and system based on cloud computing, and relates to the technical field of data processing, and the method comprises the steps: carrying out the preprocessing of monitoring data, obtaining a target data segment, analyzing the target data segment, obtaining a time correlation feature and a space correlation feature, forming a correlation relation based on the two features, and setting a constraint condition. And calculating a dependency weight of the association relationship according to the constraint condition, and complementing the target data segment through the dependency weight to obtain complete monitoring data. According to the method, the time correlation characteristics and the space correlation characteristics are extracted and subjected to fusion analysis, and data completion is realized by utilizing timeliness constraint, relevance constraint, consistency constraint and contribution degree dynamic weight allocation, so that the accuracy and reasonability of data completion are improved; accurate data basis is provided for evaluating the real-time working condition of the water conservancy project, so that managers can master the operation condition of the project in time, and a solid foundation is laid for scientific decisions such as project scheduling and maintenance plan making.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a water conservancy project monitoring data processing method and system based on cloud computing. Background Art

[0002] The monitoring process of water conservancy projects generates a vast amount of monitoring data, which is crucial for understanding the project's operational status and ensuring its safety. However, due to sensor failures, transmission interruptions, and interference from harsh environments, monitoring data is often missing, compromising the accuracy and reliability of monitoring results. With the continuous expansion of water conservancy projects and the advancement of monitoring technology, the volume of monitoring data has increased dramatically. Traditional data processing methods, faced with massive amounts of monitoring data, suffer from low processing efficiency and difficulty achieving real-time processing. Cloud computing, with its powerful computing power, vast storage capacity, and efficient distributed processing capabilities, can provide strong support for processing massive amounts of monitoring data. Applying cloud computing technology to water conservancy project monitoring data processing holds promise for resolving these challenges.

[0003] At present, a data analysis and processing method and system based on water conservancy projects is disclosed in a Chinese invention with publication number CN119647783A. The invention analyzes the water conservancy project data to be processed in the database system through a data management terminal, locates the position of abnormal data, analyzes the abnormal type, and focuses on locating and judging the type of missing data to determine whether it can be supplemented. If it cannot be supplemented, it will be deleted to avoid the missing data from interfering with the subsequent water conservancy project power generation equipment operating status analysis. Although the supplemented water conservancy project data is sorted by time continuity, the analysis of water conservancy project data is more convenient and quick, but the related technology does not perform a fusion analysis of water conservancy project data from different dimensions of time and space, lacks the globality of data analysis, and does not achieve data completion by setting constraints and dynamically allocating weights through contribution. There are certain limitations. Summary of the Invention

[0004] The technical problem solved by the present invention is that the relevant technologies do not conduct fusion analysis of water conservancy project data from different dimensions of time and space, lack the globality of data analysis, and do not achieve data completion by setting constraints and dynamically allocating weights through contribution, which has certain limitations.

[0005] To solve the above technical problems, the present invention provides the following technical solution: a water conservancy project monitoring data processing method based on cloud computing, comprising the following steps: Step S1, preprocessing the received monitoring data to obtain the target data segment; In step S2, the target data segment is analyzed to obtain time correlation features and space correlation features, and a correlation relationship is formed based on the time correlation features and the space correlation features; In step S3, a constraint condition is set, a dependent weight of the correlation relationship is calculated according to the constraint condition, the target data segment is completed through the dependent weight, and complete monitoring data is obtained.

[0006] As a preferred scheme of the water conservancy project monitoring data processing method based on cloud computing, the pre-processing of the monitoring data includes removing outliers and removing duplicate values, and the monitoring data includes dam displacement data, reservoir water level data, rainfall data and monitoring device working voltage; The removing outliers is used to remove monitoring data that is outside a preset reasonable range; The removing duplicate values is used to remove multiple same monitoring data collected by the same monitoring device at the same time point, and one of the same monitoring data is retained; The pre-processed monitoring data is marked for effectiveness to obtain effective monitoring data, the effective monitoring data is compared with a theoretical monitoring sequence, the proportion of effective monitoring data in each period is counted, and a period whose proportion does not reach a preset completeness threshold is marked as a target data segment, the target data segment includes a missing time node, a missing time interval and a missing monitoring point; The theoretical monitoring sequence is theoretical monitoring data collected according to a preset collection frequency.

[0007] As a preferred scheme of the water conservancy project monitoring data processing method based on cloud computing, the time correlation features include a numerical change rate of M continuous effective monitoring data before the target data segment, a periodic fluctuation feature and an initial change trend of K continuous effective monitoring data after the target data segment, where M and K are preset positive integers; The numerical change rate is used to extract numerical values {V1, V2,..., V M} and time stamps {T M1 ,T M2 ,...,T MM} corresponding to M continuous effective monitoring data before the target data segment, to calculate the numerical difference and the time stamp difference between two adjacent effective monitoring data in the M continuous effective monitoring data before the target data segment, to obtain a time interval, to take the ratio of the numerical difference and the time interval as the instantaneous change rate between the two adjacent data, and to take the average of M-1 instantaneous change rates calculated for the M continuous effective monitoring data as the numerical change rate of the data before the target data segment. The periodic fluctuation feature is determined by calling monitoring data of a historical period, generating a trend curve of the monitoring data of the historical period in time sequence, locating a historical time interval in which the monitoring data of the historical period presents periodic rising and falling by calculating coincidence degrees of fluctuations of the monitoring data of the historical period in the historical time interval, screening a historical time interval with the highest impact on data fluctuation as a periodic law if there are multiple historical time intervals with coincidence degrees meeting a standard, determining a length of a fluctuation period, extracting a peak value, a valley value and a fluctuation amplitude of the monitoring data of the historical period in each fluctuation period based on the fluctuation period, and taking the length of the period, the peak value, the valley value and the fluctuation amplitude as a periodic fluctuation feature of data before a target data segment. The time range of the monitoring data of the historical period needs to cover at least one complete fluctuation period. The initial change trend is used to extract values {U1, U2,..., U K} and time stamps {T K1 ,T K2 ,...,T KK} corresponding to K continuous effective monitoring data after the target data segment, establish a plane rectangular coordinate system with the time stamp as the horizontal axis and the value as the vertical axis, construct a regression straight line through the distribution law of the effective monitoring data points, determine the initial change trend according to the slope of the regression straight line, and obtain: If the slope is positive, the initial change trend is rising. If the slope is negative, the initial change trend is falling. If the slope is close to 0, the initial change trend is stable.

[0008] As a preferred scheme of the water conservancy project monitoring data processing method based on cloud computing, the spatial correlation feature includes a value difference range and a data distribution gradient feature of adjacent monitoring points of other monitoring points in the same region as the target data segment. The value difference range of the adjacent monitoring points is obtained by calculating the difference between the monitoring data values of each pair of adjacent monitoring points, taking the absolute value to obtain the absolute difference, and statistically analyzing the absolute difference obtained by calculating the value difference of all adjacent monitoring points to screen the minimum value and the maximum value to form an interval range. The data distribution gradient feature is obtained by obtaining three-dimensional structure coordinates of each monitoring point in the same region, calculating the spatial straight line distance between two adjacent monitoring points as a coordinate difference, and obtaining the value change amount in a unit distance by dividing the coordinate difference by the corresponding spatial straight line distance, which is the data distribution gradient feature in the same region.

[0009] As a preferred scheme of the cloud computing-based water conservancy project monitoring data processing method, the forming of the correlation relationship based on the time correlation feature and the space correlation feature specifically comprises: Based on the time stamps of the previous M continuous effective monitoring data {T M1 , T M2 , ..., T MM} and the time stamps of the following K continuous effective monitoring data {T K1 , T K2 , ..., T KK} of the target data segment, a continuous time sequence is generated, and the continuous time sequence is divided into time periods according to a preset time granularity, so that each time period corresponds to a unique time stamp; The three-dimensional structure coordinates of each monitoring point involved in the space correlation feature are obtained, the monitoring points are sorted according to the three-dimensional structure coordinates, a space grid is formed, and each monitoring point in the space grid is identified to correspond to a unique three-dimensional structure coordinate position; By mapping the unique time stamp and the unique three-dimensional structure coordinate position, the continuous time sequence after the time period division is dimensionally aligned with the space grid to form a one-to-one correlation relationship corresponding to the target data segment, and the correlation relationship includes monitoring values, time stamp information, monitoring points, and value change trends.

[0010] As a preferred scheme of the cloud computing-based water conservancy project monitoring data processing method, the constraint condition comprises a timeliness constraint, a correlation constraint, and a consistency constraint; The timeliness constraint is used to judge the time interval threshold of the correlation relationship and the target data segment in the time dimension, and if the difference between the time stamp of the correlation relationship and the time stamp of the target data segment exceeds the time interval threshold, the timeliness weight of the correlation relationship is reduced; The time interval threshold is a maximum effective time difference value determined based on the missing time interval and the period length of the periodic fluctuation feature; The correlation constraint is used to judge the matching degree of the correlation relationship and the target data segment in the parameter attribute, and the correlation weight is positively correlated with the matching degree, and the parameter attribute includes dam displacement, reservoir water level, rainfall, and monitoring device working voltage; The consistency constraint is used to judge whether the correlation relationship and the historical data change trend of the monitoring area where the target data segment is located are consistent in the value change trend, and if there is a trend conflict, the consistency weight of the correlation relationship is reduced.

[0011] As a preferred scheme of the cloud computing-based water conservancy project monitoring data processing method, the constraint condition is used to calculate the dependent weight of the correlation relationship, specifically: A weight coefficient is set for the constraint condition, and a calculation formula of the weight coefficient is: W i =Cont i / (Sum cont123 ) (i=1, 2, 3); Wherein, W represents the weight coefficient of the i-th constraint condition, Cont represents the contribution degree of the i-th constraint condition, Sum cont123 represents the total contribution degree of the constraint condition, and the sum of the weight coefficients of all constraint conditions is 1; The score of the association relationship under each constraint condition is calculated, the score under each constraint condition is multiplied by the corresponding weight coefficient, and then summed to obtain the dependency weight of the association relationship.

[0012] As a preferred scheme of the water conservancy project monitoring data processing method based on cloud computing, wherein: the calculation of the score of the association relationship under each constraint condition includes timeliness score calculation, association score calculation and consistency score calculation; The timeliness score calculation sets a time interval threshold of the association relationship and the target data segment, calculates the actual time interval between the timestamp of the association relationship and the timestamp of the target data segment, and the smaller the actual time interval, the higher the timeliness score; The association score calculation analyzes the parameter attributes of the association relationship and the target data segment, compares the matching degree of the two through a preset parameter attribute matching comparison table, and the higher the matching degree, the higher the association score; The consistency score calculation compares the numerical change trend with the periodic fluctuation characteristic trend, calculates the trend coincidence degree of the two, and the trend coincidence degree is the consistency score, and the more consistent the trend, the higher the consistency score; Wherein, the score range is between 0 and 1.

[0013] As a preferred scheme of the water conservancy project monitoring data processing method based on cloud computing, wherein: according to the dependency weight, a data completion operation is performed on the target data segment, specifically: The association relationship is sorted according to the dependency weight from high to low, the time association feature and the space association feature corresponding to the top N association relationships in the dependency weight are extracted, and a missing value judgment calculation is performed, the missing value judgment calculation includes: If the missing value of the target data segment is a periodic physical quantity, the average value of the values at the same period positions before and after the missing time interval is taken; If the missing value of the target data segment is a physical quantity showing a trend change, the incremental value of the missing time interval is calculated according to the numerical change rate; Generate the missing value of the target data segment, and fill in the missing position of the target data segment to obtain the completed complete monitoring data, wherein N is a positive integer set according to the missing length of the target data segment. The water conservancy project monitoring data processing system based on cloud computing comprises a preprocessing module, an analysis module and a completion module. The preprocessing module is used for preprocessing the received monitoring data to obtain a target data segment. The analysis module is used for analyzing the target data segment to obtain time correlation characteristics and space correlation characteristics, and forming a correlation relationship based on the time correlation characteristics and the space correlation characteristics. The completion module is used for setting a constraint condition, calculating a dependent weight of the correlation relationship according to the constraint condition, and performing a data completion operation on the target data segment according to the dependent weight to obtain the completed complete monitoring data.

[0014] The beneficial effects of the present application are: by extracting the time correlation characteristics and space correlation characteristics of the monitoring data, and based on the fusion analysis of the time and space dimensions, the global control of the monitoring data analysis is realized, and the internal law of the monitoring data can be more comprehensively captured. By setting the timeliness, correlation and consistency constraint conditions, and based on the contribution degree, the weight coefficients of each constraint condition are dynamically allocated to calculate the dependent weight of the correlation relationship, and then the differentiated data completion is performed according to the high-weight correlation characteristics, which effectively improves the accuracy and rationality of the data completion. The output complete monitoring data can clearly reflect the actual situation and change trend of the key indicators such as dam displacement and reservoir water level, and provide reliable support for real-time working condition evaluation, safety risk early warning and engineering scheduling, maintenance plan development and other scientific decision-making of water conservancy projects, which helps to improve the fine and intelligent level of water conservancy project management, ensures the safe operation of the project and fully plays the benefits. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The basic flowchart of the water conservancy project monitoring data processing method based on cloud computing provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0016] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments.

[0017] Embodiment 1, refer to Figure 1 For an embodiment of the present application, a water conservancy project monitoring data processing method based on cloud computing is provided, comprising the following steps: Step S1, preprocessing the received monitoring data to obtain a target data segment; In step S2, the target data segment is analyzed to obtain time correlation features and space correlation features, and a correlation relationship is formed based on the time correlation features and the space correlation features; In step S3, a constraint condition is set, a dependent weight of the correlation relationship is calculated according to the constraint condition, the target data segment is completed through the dependent weight, and complete monitoring data is obtained.

[0018] The preprocessing of the monitoring data includes removing abnormal values and removing duplicate values, and the monitoring data includes dam displacement data, reservoir water level data, rainfall data and monitoring device working voltage; The removing of abnormal values is used to remove monitoring data that exceeds a preset reasonable range; The removing of duplicate values is used to remove multiple same monitoring data collected by the same monitoring device at the same time point, and one of the same monitoring data is retained; The effective monitoring data is obtained by marking the validity of the preprocessed monitoring data, and the effective monitoring data is compared with a theoretical monitoring sequence, the proportion of the effective monitoring data in each period is counted, and the period whose proportion does not reach a preset complete threshold is marked as a target data segment. The target data segment includes missing time nodes, missing time intervals and missing monitoring points. The theoretical monitoring sequence is a theoretical monitoring data collected according to a preset collection frequency.

[0019] In specific embodiments, by removing abnormal values and duplicate values, for example, removing dam displacement abnormal values that exceed the device range and deleting 3 same water level data collected by a certain monitoring device at the same time point and retaining only 1, the monitoring data is ensured to have no basic problems, and then the target data segment is marked by comparing with the theoretical monitoring sequence, for example, the effective data proportion of a certain period is only 80% and does not reach the complete threshold of 90%, which is marked as a target data segment containing missing data. The data missing area is accurately positioned, the processing object is clear for subsequent analysis and completion, and the data quality is ensured.

[0020] The preset complete threshold refers to an effective data proportion standard preset for judging the completeness of the monitoring data period, which is used to determine whether the monitoring data of a certain period needs to be processed due to too much data missing.

[0021] The time correlation features include a numerical change rate of M continuous effective monitoring data before the target data segment, a periodic fluctuation feature and an initial change trend of K continuous effective monitoring data after the target data segment, wherein M and K are preset positive integers; The numerical change rate is used to extract the numerical values {V1, V2,..., V M} and time stamps {T M1 ,T M2 ,...,T MM}, calculate the time interval by the numerical difference and timestamp difference between two adjacent valid monitoring data in the M consecutive valid monitoring data before the target data segment, take the ratio of the numerical difference to the time interval as the instantaneous change rate between the two adjacent data, and take the average of the M-1 instantaneous change rates calculated for the M consecutive valid monitoring data as the numerical change rate of the data before the target data segment; The periodic fluctuation characteristics are obtained by retrieving the monitoring data of the historical period, generating a trend curve for the monitoring data of the historical period according to the time series, and locating the historical time intervals in which the monitoring data of the historical period show periodic rise and fall by calculating the overlap of the fluctuations of the monitoring data of the historical period within the historical time interval. If there are multiple historical time intervals with a satisfactory overlap, the historical time interval with the greatest impact on the data fluctuation is selected as the periodic law, and the duration of the fluctuation cycle is determined. Based on the fluctuation cycle, the peak value, valley value and fluctuation amplitude of the monitoring data of the historical period within each fluctuation cycle are extracted, and the cycle duration, peak value, valley value and fluctuation amplitude are used as the periodic fluctuation characteristics of the data before the target data segment. The time range of the monitoring data in the historical period must cover at least one complete fluctuation cycle; The initial change trend is used to extract the values ​​corresponding to the K consecutive valid monitoring data after the target data segment {U1, U2, ..., U K} and timestamp {T K1 ,T K2 ,...,T KK}, with the timestamp as the horizontal axis and the value as the vertical axis, a plane rectangular coordinate system is established. The regression line is constructed by effectively monitoring the distribution law of data points. The initial change trend is determined according to the slope of the regression line, and the following is obtained: If the slope is positive, the initial trend of change is upward; If the slope is negative, the initial trend of change is downward; If the slope is close to 0, the initial trend is stable.

[0022] In a specific embodiment, the core role of the time correlation feature is to provide key support for the time dimension of the entire data processing process. The numerical change rate provides a dynamic reference for the timeliness judgment of the correlation relationship in the invention by quantifying the overall change rate before the target data segment, ensuring that the time continuity of the data change can be accurately reflected when calculating the dependency weight. The periodic fluctuation feature reveals the historical periodic law and provides a benchmark basis for the subsequent data completion of the missing value of the periodic physical quantity, so that the completed missing value fits the historical fluctuation. The initial change trend provides a trend basis for the dynamic adjustment of the correlation relationship in the invention by predicting the short-term direction after the target data segment, ensuring the adaptability of the correlation relationship over time. The three work together to comprehensively characterize the temporal dynamic law of the target data segment, directly supporting the core links of correlation relationship construction, dependency weight calculation and data completion.

[0023] The spatial correlation feature includes a numerical difference range of adjacent monitoring points of other monitoring points in the same region as the target data segment and a data distribution gradient feature; The numerical difference range of adjacent monitoring points is obtained by calculating the difference between the monitoring data values of each pair of adjacent monitoring points, taking the absolute value to obtain the absolute difference, and statistically analyzing the absolute difference calculated for all adjacent monitoring points to filter out the minimum value and the maximum value to form an interval range; The data distribution gradient feature is obtained by obtaining the three-dimensional structure coordinates of each monitoring point in the same region, calculating the spatial straight-line distance between two adjacent monitoring points as the coordinate difference, and dividing the coordinate difference by the corresponding spatial straight-line distance to obtain the numerical change amount per unit distance, which is the data distribution gradient feature in the same region.

[0024] In a specific embodiment, taking a dam body monitoring region as an example, three adjacent monitoring points A, B and C are provided in the region, and the three-dimensional structure coordinates are A(0, 0, 0), B(10, 0, 0) and C(20, 0, 0) (unit: m), respectively. The dam body displacement data monitored at a certain period of time are A=2.0 mm, B=2.2 mm and C=2.5 mm, respectively. When calculating the numerical difference range of adjacent monitoring points, the absolute difference between A and B is |2.2-2.0|=0.2 mm, and the absolute difference between B and C is |2.5-2.2|=0.3 mm. After statistics, the minimum value is 0.2 mm and the maximum value is 0.3 mm, and the numerical difference range is [0.2 mm, 0.3 mm], which can reflect the spatial consistency of the monitoring data in the same region. When calculating the data distribution gradient feature, the spatial straight-line distance between A and B is 10 m, and the unit distance change amount is 0.2 mm / 10 m=0.02 mm / m. The unit distance change amount between B and C is 0.3 mm / 10 m=0.03 mm / m. The gradient feature clearly shows that the dam body displacement increases by 0.02-0.03 mm / m from A to C. When data is completed, if the B point data is missing, the reasonable value of the B point can be calculated based on the gradient feature of the A and C points, which provides spatial dimension support for data completion based on the data correlation relationship, and enhances the scientificity and reliability of monitoring data processing.

[0025] The correlation relationship based on the time correlation feature and the spatial correlation feature specifically includes: The time stamps of the M consecutive effective monitoring data before the target data segment are {T M1 ,T M2 ,...,T MM} and the time stamps of the K consecutive effective monitoring data after the target data segment are {T K1 ,T K2 ,...,T KKgenerate a continuous time sequence, divide the continuous time sequence into time periods according to a preset time granularity, and make each time period correspond to a unique timestamp; The method comprises the following steps: obtaining three-dimensional structure coordinates of each monitoring point involved in the spatial correlation feature, sorting the monitoring points according to the three-dimensional structure coordinates, forming a spatial grid, and marking each monitoring point in the spatial grid with a unique three-dimensional structure coordinate position; By mapping the unique timestamp and the unique three-dimensional structure coordinate position, the continuous time sequence after the time period division is dimensionally aligned with the spatial grid to form a one-to-one correspondence relationship with the target data segment, which includes monitoring values, timestamp information, monitoring points, and value change trends.

[0026] In specific embodiments, the correlation relationship is formed based on the time correlation feature and the spatial correlation feature. A continuous time sequence is generated by using the timestamps before and after the target data segment, and the time periods are divided according to a preset granularity. For example, the target data segment is the reservoir water level data from 2024-08-01 10:00 to 10:30. The first M=3 timestamps {9:30, 9:45, 9:59} and the last K=2 timestamps {10:35, 10:50} are used to generate a continuous sequence, which is divided into time periods of 9:30-9:45, 9:45-10:00, etc. according to a 15-minute granularity, and each time period corresponds to a unique timestamp. The three-dimensional coordinates of the monitoring points, such as dam monitoring points A (10, 20, 5) and B (15, 25, 5), are sorted to form a spatial grid, and each point corresponds to a unique coordinate position. Then, by mapping the unique timestamp and the coordinate position, such as the coordinate (10, 20, 5) of point A in the time period 9:30-9:45, the time sequence and the spatial grid are dimensionally aligned, and finally the correlation relationship including monitoring values, timestamp information, monitoring points, and value change trends is formed, such as the water level of point A at 9:30-9:45 is 10.2m, and the water level of point A rises by 0.1m every 15 minutes. This process can accurately establish the corresponding relationship of data in time and space dimensions, provide a structured correlation basis for subsequent weight calculation and data completion, ensure that the data correlation is consistent with the time sequence logic and the spatial distribution rule, and improve the accuracy and interpretability of the correlation relationship.

[0027] The constraint conditions include timeliness constraints, correlation constraints, and consistency constraints; The timeliness constraint is used to judge the time interval threshold of the correlation relationship and the target data segment in the time dimension. If the difference between the timestamp of the correlation relationship and the timestamp of the target data segment exceeds the time interval threshold, the timeliness weight of the correlation relationship is reduced. The time interval threshold is the maximum effective time difference determined based on the missing time interval and the periodic fluctuation feature; The correlation constraint is used to judge the matching degree of the correlation relationship on the parameter attribute and the target data segment, the correlation weight is positively correlated with the matching degree, and the parameter attribute includes dam displacement, reservoir water level, rainfall and monitoring equipment working voltage; The consistency constraint is used to judge whether the correlation relationship is consistent with the historical data trend of the monitoring area where the target data segment is located in the numerical change trend, and if there is a trend conflict, the consistency weight of the correlation relationship is reduced.

[0028] In specific embodiments, the time interval threshold is determined based on the missing time interval and the periodic fluctuation characteristics of the period length, and the time effectiveness of the correlation relationship is ensured by reducing the time effectiveness weight of the correlation relationship that exceeds the threshold. The correlation weight is dynamically adjusted according to the matching degree of the parameter attribute such as dam displacement and reservoir water level through the correlation constraint, and the higher the matching degree, the greater the weight to strengthen the correlation adaptability of the parameter attribute. The consistency constraint checks the consistency of the correlation relationship and the historical data trend of the area where the target data segment is located, and the consistency weight of the correlation relationship with trend conflict is reduced to ensure the coordination of the numerical change rule. The three types of constraints work together to accurately select the correlation relationship that is superior in time effectiveness, parameter adaptability and trend consistency to the target data segment, provide a scientific constraint basis for weight calculation, and further improve the accuracy and reliability of subsequent data completion. Ensure that the processing logic of the cloud computing platform for water conservancy engineering monitoring data is more in line with the rules and needs of the actual monitoring scene.

[0029] The dependent weight of the correlation relationship is calculated according to the constraint condition, specifically: The weight coefficient of the constraint condition is set, and the weight coefficient calculation formula is: W i =Cont i / (Sum cont123 )(i=1,2,3); Wherein, Wᵢ represents the weight coefficient of the i-th constraint condition, Contᵢ represents the contribution degree of the i-th constraint condition, Sum cont123 represents the total contribution degree of the constraint condition, and the sum of the weight coefficients of all constraint conditions is 1; The score of the correlation relationship under each constraint condition is calculated, and the sum of the score under each constraint condition and the corresponding weight coefficient is multiplied to obtain the dependent weight of the correlation relationship.

[0030] The score of the correlation relationship under each constraint condition includes time effectiveness score calculation, correlation score calculation and consistency score calculation; The time interval threshold of the correlation relationship and the target data segment is set in the time effectiveness score calculation, the actual time interval between the time stamp of the correlation relationship and the time stamp of the target data segment is calculated, and the smaller the actual time interval, the higher the time effectiveness score; The correlation score calculation analyzes the parameter attributes of the correlation relationship and the target data segment, and compares the matching degree of the two through a pre-set parameter attribute matching comparison table. The higher the matching degree, the higher the correlation score. The consistency score calculation compares the numerical change trend with the periodic fluctuation characteristic trend, calculates the trend coincidence degree of the two, and the coincidence degree is the consistency score. The more consistent the trend, the higher the consistency score. The score range is between 0 and 1.

[0031] In specific embodiments, by setting a weight coefficient formula and combining the time effectiveness, correlation, and consistency three types of score calculations, the dependence degree of different correlation relationships on the target data segment can be scientifically quantified. The weight coefficient is determined by the contribution ratio of each constraint condition, for example, the contribution degrees of time effectiveness, correlation, and consistency are 4, 3, and 3 respectively, and the total contribution degree is 10. Therefore, the corresponding weight coefficients are 0.4, 0.3, and 0.3 respectively. The dependence weight is obtained by weighting and summing the three types of scores and the corresponding weight coefficients. The dependence weight calculation reflects the relative importance of each constraint condition and the matching degree of the correlation relationship and the target data segment. This calculation method can accurately select the correlation relationship that has the greatest impact on the target data segment. For example, the time effectiveness score of a certain correlation relationship is 0.9, the correlation score is 0.8, and the consistency score is 0.7. According to the above weight coefficient calculation, the dependence weight is 0.9x0.4+0.8x0.3+0.7x0.3=0.81. This provides a reliable basis for subsequent data completion, anomaly detection, and other operations, ensuring that the cloud computing processing efficiency and result accuracy can be improved through correlation relationships when processing massive monitoring data. At the same time, through standardized score range and clear calculation logic, the dependence weight has interpretability and comparability, enhancing the scientificity and stability of the entire data processing method.

[0032] According to the dependence weight, a data completion operation is performed on the target data segment, specifically: The correlation relationships are sorted in descending order of dependence weight, and the time correlation features and spatial correlation features corresponding to the top N correlation relationships in terms of dependence weight are extracted for missing value judgment calculation. The missing value judgment calculation includes: If the missing value of the target data segment is a periodic physical quantity, the average value of the values at the same position before and after the missing time interval is taken; If the missing value of the target data segment is a physical quantity showing a trend change, the incremental value of the missing time interval is calculated according to the numerical change rate; The missing values of the target data segment are generated and filled into the missing positions of the target data segment to obtain the completed monitoring data.

[0033] In specific embodiments, the data completion operation on the target data segment according to the dependency weight is performed by sorting and extracting the time and space correlation characteristics of the top N correlation relationships according to the dependency weight, and combining the physical quantity type of the missing value to perform targeted calculation. For example, for missing values of reservoir water level with periodicity, the average water level of the same period corresponding to the time interval before and after the missing interval is taken. For missing values of dam displacement showing a trend change, the cumulative displacement increment of the missing time interval is calculated according to the numerical change rate. The missing values that conform to the data law can be accurately generated and filled into the target data segment, which not only ensures the rationality of the missing values by using the strong correlation of the correlation relationship, but also improves the completion accuracy by distinguishing the calculation logic of periodic and trend physical quantities, effectively solving the problem of incomplete monitoring data caused by equipment failure, transmission interruption, etc. It provides complete and reliable monitoring data basis for subsequent feature extraction, correlation relationship construction and water conservancy engineering working condition analysis, and at the same time, the N value is dynamically set according to the missing length to balance the completion efficiency and accuracy, ensuring that the processing results of the cloud computing platform for water conservancy engineering monitoring data are more in line with the actual engineering requirements.

[0034] In an embodiment 2, a water conservancy engineering monitoring data processing system based on cloud computing includes a preprocessing module, an analysis module and a completion module. The preprocessing module is used for preprocessing the received monitoring data to obtain a target data segment. The analysis module is used for analyzing the target data segment to obtain time correlation characteristics and space correlation characteristics, and forming correlation relationships based on the time correlation characteristics and the space correlation characteristics. The completion module is used for setting constraint conditions, calculating dependency weights of the correlation relationships according to the constraint conditions, completing the target data segment according to the dependency weights, and obtaining complete monitoring data after completion.

[0035] The method preprocesses the received monitoring data to obtain a target data segment, provides regular basic data for subsequent analysis, analyzes the target data segment to obtain time correlation characteristics and space correlation characteristics and forms correlation relationships, builds the internal relationship between data, sets timeliness, correlation and consistency constraint conditions, calculates the dependency weights of the correlation relationships according to the constraint conditions, completes the target data segment according to the dependency weights, generates and fills in the missing values to obtain complete monitoring data, and finally achieves the effect of significantly improving the completeness, accuracy and reliability of the monitoring data, so that the output complete monitoring data can more accurately support water conservancy engineering working condition evaluation, safety warning and scientific decision-making.

[0036] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (or computer- readable storage media) having computer-usable program code embodied in the medium. The medium can be any available medium or combination thereof that is accessible by a general purpose or special purpose computer. By way of example, such computer-usable storage media can include a volatile memory, a non-volatile memory, a hard disk drive, a solid state drive, a magnetic diskette, an optical disk (e.g., a compact disk (CD) or a digital versatile disk (DVD)), or a floppy disk, all of which are tangible of computer- readable media. The computer-usable program code, when executed, can enable a general purpose computer, special purpose computer, or other programmable data processing apparatus to perform a method, process, or procedure as described in the above embodiments. The computer-usable program code can also be loaded onto a computer and / or other programmable instruction execution devices to cause a series of operations to be performed on the computer and / or other Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks

[0037] It should be noted that the above-mentioned embodiments are only used to illustrate but not to limit the technical solutions of the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A water conservancy project monitoring data processing method based on cloud computing, characterized in that: The following steps are involved: Step S1, preprocessing the received monitoring data to obtain the target data segment; Step S2, analyzing the target data segment to obtain a time correlation feature and a space correlation feature, and forming a correlation relationship based on the time correlation feature and the space correlation feature; Step S3: Set constraints, calculate the dependency weights of the association relationships based on the constraints, and complete the target data segments using the dependency weights to obtain complete monitoring data.

2. The method for processing water conservancy project monitoring data based on cloud computing according to claim 1, wherein: Preprocessing the monitoring data includes removing outliers and removing duplicate values, wherein the monitoring data includes dam displacement data, reservoir water level data, rainfall data, and monitoring equipment operating voltage; The outlier removal is used to remove monitoring data that exceeds a preset reasonable range; The duplicate value removal is used to remove multiple identical monitoring data collected by the same monitoring device at the same time point, and retain one identical monitoring data; Marking the preprocessed monitoring data for validity to obtain valid monitoring data, comparing the valid monitoring data with the theoretical monitoring sequence, and counting the proportion of valid monitoring data in each time period. Marking the time period whose proportion does not reach a preset complete threshold as a target data segment, wherein the target data segment includes missing time nodes, missing time intervals, and missing monitoring points; The theoretical monitoring sequence is theoretical monitoring data collected according to a preset collection frequency.

3. The method for processing water conservancy project monitoring data based on cloud computing according to claim 2, wherein: The time-related features include the numerical change rate and periodic fluctuation characteristics of M consecutive valid monitoring data before the target data segment and the initial change trend of K consecutive valid monitoring data after the target data segment, where M and K are preset positive integers; The value change rate is used to extract the values ​​{V1, V2, ..., V M } and timestamp {T M1 ,T M2 ,...,T MM }, calculate the time interval between the numerical difference and the timestamp difference of two adjacent valid monitoring data in the M consecutive valid monitoring data before the target data segment, use the ratio of the numerical difference to the time interval as the instantaneous change rate between the two adjacent data, and average the M-1 instantaneous change rates calculated for the M consecutive valid monitoring data as the numerical change rate of the data before the target data segment; The periodic fluctuation feature is obtained by retrieving the monitoring data of the historical period, generating a trend curve for the monitoring data of the historical period in a time series, and locating the historical time intervals in which the monitoring data of the historical period presents a periodic rise and fall by calculating the overlap of the fluctuations of the monitoring data of the historical period within the historical time interval. If there are multiple historical time intervals with a satisfactory overlap, the historical time interval with the greatest impact on the data fluctuation is selected as the periodic law, and the duration of the fluctuation cycle is determined. Based on the fluctuation cycle, the peak value, valley value and fluctuation amplitude of the monitoring data of the historical period within each fluctuation cycle are extracted, and the cycle duration, peak value, valley value and fluctuation amplitude are used as the periodic fluctuation feature of the data before the target data segment. The time range of the monitoring data of the historical period must cover at least one complete fluctuation cycle; The initial change trend is used to extract the values ​​{U1, U2, ..., U K } and timestamp {T K1 ,T K2 ,...,T KK }, with the timestamp as the horizontal axis and the value as the vertical axis, a plane rectangular coordinate system is established. The regression line is constructed by effectively monitoring the distribution law of data points. The initial change trend is determined according to the slope of the regression line, and the following is obtained: If the slope is positive, the initial trend of change is upward; If the slope is negative, the initial trend is downward; If the slope is close to 0, the initial change trend is stable.

4. The method for processing water conservancy project monitoring data based on cloud computing according to claim 3, wherein: The spatial correlation features include the numerical difference range and data distribution gradient features of adjacent monitoring points of other monitoring points in the same area as the target data segment; The numerical difference range of the adjacent monitoring points is obtained by calculating the difference between the monitoring data values ​​of each pair of adjacent monitoring points and taking the absolute value to obtain the absolute difference, performing statistical analysis on the absolute difference calculated for the numerical difference of all adjacent monitoring points, and screening out the minimum and maximum values ​​to form the interval range; The data distribution gradient feature is obtained by obtaining the three-dimensional structural coordinates of each monitoring point in the same area, calculating the spatial straight-line distance between two adjacent monitoring points as the coordinate difference, and dividing the coordinate difference by the corresponding spatial straight-line distance to obtain the numerical change within the unit distance, which is the data distribution gradient feature in the same area.

5. The water conservancy project monitoring data processing method based on cloud computing according to claim 4, characterized in that: Forming an association relationship based on the temporal association feature and the spatial association feature specifically includes: Based on the timestamps of M consecutive valid monitoring data before the target data segment {T M1 ,T M2 ,...,T MM } and the timestamps of the following K consecutive valid monitoring data {T K1 ,T K2 ,...,T KK }, generating a continuous time series, dividing the continuous time series into time periods according to a preset time granularity, so that each time period corresponds to a unique timestamp; Obtaining the three-dimensional structural coordinates of each monitoring point involved in the spatial correlation feature, sorting the monitoring points according to the three-dimensional structural coordinates to form a spatial grid, wherein each monitoring point identifier in the spatial grid corresponds to a unique three-dimensional structural coordinate position; By mapping the unique timestamp with the unique three-dimensional structure coordinate position, the continuous time series after the division of time periods is dimensionally aligned with the spatial grid to form a one-to-one association relationship with the target data segment, and the association relationship includes monitoring values, timestamp information, monitoring points and value change trends.

6. The water conservancy project monitoring data processing method based on cloud computing according to claim 5, characterized in that: The set constraints include timeliness constraints, relevance constraints and consistency constraints; The timeliness constraint is used to determine the time interval threshold between the association relationship and the target data segment in the time dimension. If the difference between the timestamp of the association relationship and the timestamp of the target data segment exceeds the time interval threshold, the timeliness weight of the association relationship is reduced; The time interval threshold is a maximum valid time difference determined based on the missing time interval and the period duration of the periodic fluctuation feature; The association constraint is used to determine the matching degree between the association relationship and the target data segment in terms of parameter attributes, and the association weight is positively correlated with the matching degree. The parameter attributes include dam displacement, reservoir water level, rainfall, and monitoring equipment operating voltage. The consistency constraint is used to determine whether the numerical change trend of the association relationship is consistent with the historical data change trend of the monitoring area where the target data segment is located. If there is a trend conflict, it means that the consistency weight of the association relationship is reduced.

7. The water conservancy project monitoring data processing method based on cloud computing according to claim 6, characterized in that: The dependency weight of the association relationship is calculated according to the constraint conditions, specifically: A weight coefficient is set for the constraint condition, and the weight coefficient calculation formula is: W i =Cont i / (Sum cont123 )(i=1,2,3); Among them, Wᵢ represents the weight coefficient of the i-th constraint condition, Contᵢ represents the contribution of the i-th constraint condition, Sum cont123 Represents the total contribution of the constraints, and the sum of the weight coefficients of all constraints is 1; The score of the association relationship under each constraint condition is calculated, and the score under each constraint condition is multiplied by the corresponding weight coefficient and then summed to obtain the dependency weight of the association relationship.

8. The water conservancy project monitoring data processing method based on cloud computing according to claim 7, characterized in that: The calculation of the score of the association relationship under each constraint condition includes timeliness score calculation, relevance score calculation and consistency score calculation; The timeliness score calculation sets a time interval threshold between the association relationship and the target data segment, and calculates the actual time interval between the timestamp of the association relationship and the timestamp of the target data segment. The smaller the actual time interval, the higher the timeliness score. The correlation score calculation analyzes the correlation relationship and the parameter attributes of the target data segment, and compares the matching degree between the two through a preset parameter attribute matching comparison table. The higher the matching degree, the higher the correlation score. The consistency score calculation compares the numerical change trend with the periodic fluctuation characteristic trend, and calculates the trend consistency between the two. The consistency score is the consistency score. The more consistent the trend, the higher the consistency score. The score range of calculating the association relationship under each constraint condition is between 0 and 1.

9. The water conservancy project monitoring data processing method based on cloud computing according to claim 8, characterized in that: Performing a data completion operation on the target data segment according to the dependency weight is specifically as follows: The association relationships are sorted from high to low according to the dependency weights, and the temporal association features and spatial association features corresponding to the top N association relationships with the dependency weights are extracted to perform missing value judgment calculation. The missing value judgment calculation includes: If the missing value of the target data segment is a periodic physical quantity, then the average of the values ​​of the corresponding positions in the same period before and after the missing time interval is taken; If the missing value of the target data segment is a physical quantity that changes in a trend, the incremental value of the missing time interval is calculated according to the value change rate; Generate the missing value of the target data segment and fill it into the missing position of the target data segment to obtain the completed complete monitoring data, where N is a positive integer set according to the missing length of the target data segment.

10. The water conservancy project monitoring data processing system based on cloud computing is characterized by: Includes preprocessing module, analysis module and completion module; The preprocessing module is used to preprocess the received monitoring data to obtain the target data segment; The analysis module is used to analyze the target data segment to obtain a time correlation feature and a space correlation feature, and form a correlation relationship based on the time correlation feature and the space correlation feature; The completion module is used to set constraint conditions, calculate the dependency weight of the association relationship according to the constraint conditions, perform data completion operations on the target data segment according to the dependency weight, and obtain complete monitoring data after completion.

Images