Dam safety monitoring data validity identification method and system and electronic equipment

Abstract

The invention provides a dam safety monitoring data validity identification method and system and electronic equipment, and relates to the technical field of industrial internet platforms and hydraulic engineering monitoring. Comprising the steps of S1, constructing a multi-dimensional validity recognition model which comprises logic discrimination, statistical discrimination and a clustering analysis model, S2, obtaining to-be-tested measuring point data, inputting the recognition model for preliminary discrimination, carrying out measuring point importance grading and multi-physical quantity joint discrimination, and obtaining grading and corresponding discrimination results; s3, continuously tracking and monitoring the key measuring points which are higher than a preset importance threshold for a preset time length, S4, carrying out routing inspection when a real value is abnormal, and carrying out differentiation treatment based on a routing inspection result, and S5, carrying out manual regular recheck: regularly extracting a judgment result for manual check; s6, according to the validity identification result of the measured value, marking a corresponding measuring point state for the monitored measuring point, and adopting a corresponding disposal mode; the data identification accuracy and reliability are improved, and the monitoring efficiency and the response speed are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial internet platform and water conservancy engineering monitoring, in particular to a dam safety monitoring data validity identification method, system and electronic equipment. BACKGROUND

[0002] In the technical field of industrial internet platform and water conservancy engineering safety monitoring, a large amount of monitoring data will be generated during the operation of the dam. These data are the core basis for evaluating the safety state of the dam and guiding the maintenance of the project. However, due to factors such as monitoring instrument failure, system error, and environmental interference, a large amount of invalid data is often mixed in the massive monitoring data. If such data is directly used for dam safety analysis, it is easy to lead to decision deviation, which poses a potential risk to the safe operation of the dam. Therefore, a reliable monitoring data validity identification means is urgently needed.

[0003] In the prior art, the dam monitoring data validity identification method has significant defects and cannot meet the actual monitoring needs. The specific problems are as follows: 1. Single discrimination model, poor adaptability: traditional methods generally use a single discrimination model for data validity identification, which cannot adapt to the differences in characteristics of different monitoring projects. For example, a model suitable for data with obvious statistical rules cannot accurately identify data with outliers, gross errors, or abnormal physical meaning, resulting in a high overall misjudgment rate and failing to effectively cover multiple types of invalid data identification scenarios. 2. Rigidity of invalid data handling process, low efficiency: for invalid data, traditional methods use a fixed and unified retest process, lacking a hierarchical disposal mechanism based on the importance of the measuring point. On the one hand, for core risk measuring points such as dam body anti-seepage body displacement and near-dam bank slope seepage flow, there is no priority disposal channel, which may delay the risk response. On the other hand, for general measuring points such as dam body surface temperature and secondary area seepage pressure, excessive intervention results in waste of labor costs and low overall disposal efficiency. 3. Lack of continuous tracking of measuring point state, incomplete management: traditional methods do not systematically define and dynamically track the state of the measuring point, which cannot reflect the whole life cycle evolution process of the measuring point from "normal- abnormal-disposal-recovery", making it difficult to realize the whole life cycle management of data quality. 4. Insufficient intelligence and standardization: traditional methods rely more on manual operation and lack a unified identification process framework and data processing standard, making it difficult to integrate with the industrial internet platform and realize full-process digitization. Not only does this increase the cost of system development and maintenance, but it also makes it difficult to meet the needs of the development of dam safety monitoring towards intelligence and automation.

[0004] Therefore, an intelligent identification method that integrates multiple discrimination methods, has hierarchical disposal capability, and can realize dynamic optimization and engineering maintenance linkage is urgently needed to improve the quality of monitoring data and the intelligence level of the monitoring system, and to ensure the safe operation of the dam. SUMMARY

[0005] The main purpose of the present application is to provide a dam safety monitoring data validity identification method, system and electronic equipment, which solves the technical problems of poor adaptability, high misjudgment rate, low data processing efficiency of traditional single discrimination model, lack of continuous tracking of measuring point state and inability to realize whole life cycle management.

[0006] To solve the above technical problems, the technical solution adopted by the present application is: a dam safety monitoring data validity identification method, comprising the following steps: S1: Construct a multi-dimensional validity identification model: the identification model includes a logical discrimination, a statistical discrimination and a clustering analysis model, and the model is adapted to the measuring point or physical quantity according to the data time sequence characteristics; S2: Obtain the data of the measuring point to be measured, input the identification model for preliminary discrimination, filter the valid data, and then perform secondary discrimination on the invalid data, obtain the current unqualified measuring point, and perform measuring point importance classification and multi-physical quantity joint discrimination to obtain the classification and the corresponding discrimination result; S3: Continuously track and monitor the key measuring point higher than the preset importance threshold for a preset time length, and if an anomaly occurs, perform on-site comparison, and according to the comparison result, distinguish whether it is an instrument fault anomaly or a real value anomaly; S4: When it is a real value anomaly, perform an inspection, and based on the inspection result, perform differential treatment, confirm the defect, give a repair suggestion and include it in the defect management, and if no anomaly is found, supplement the monitoring part or project to the inspection plan to realize the linkage management of data anomaly and engineering maintenance; S5: Artificial periodic review: periodically extract the discrimination result for artificial checking, check the model method according to the review result, automatically trigger model re-matching for the inapplicable model, form a self-evolution closed loop of "discrimination-review-optimization"; S6: According to the validity identification result of the measured value, mark the corresponding measuring point state of the monitoring measuring point, and adopt the corresponding treatment method, wherein the validity identification result includes the comprehensive determination result of the step-by-step discrimination, verification and review of the measured value validity obtained by S1-S5.

[0007] In the preferred scheme, the measuring point importance classification and multi-physical quantity joint discrimination are performed according to the preset importance classification, wherein: The general measuring point starts the multi-measuring point multi-physical quantity judgment; The measuring point without greater risk is included in the list of key attention measuring points and the monitoring frequency is increased; The measuring point with greater risk sends an instruction to the station side for on-site inspection.

[0008] In the preferred scheme, in S6, each time the monitoring data inspection is started, the branch process corresponding to the measuring point state is disposed; The measuring point state includes: Normal: The measured value is identified by the mathematical model; Recheck: The measured value is identified by the mathematical model; Attention: The recheck value of the measured point is still unqualified, but it is determined that there is no major risk through multi-measured point and multi-physical quantity identification; Fault: The measured value cannot pass the validity check continuously, and it is determined that the instrument is faulty after on-site comparison; Verification: The measured value cannot pass the validity check continuously, but it is determined that the measured value is the true value after on-site comparison.

[0009] In the preferred embodiment, in S1, the logical identification method model is specifically: according to the monitoring instrument range, monitoring accuracy, and physical meaning of monitoring data, the validity of the measured value is judged, and the data exceeding the instrument range, sign error, and exceeding the human set value are all invalid data; The statistical identification method model is specifically: taking the monitoring physical quantity under similar working conditions as sample data, using historical monitoring data to establish a multiple regression equation and a confidence interval for each measured point, and when the difference between the monitoring effect and the measured value exceeds 6 times the standard deviation, it is invalid data; The cluster analysis method model is specifically: using cluster analysis model, the outlier data is determined as gross error; The model is matched with the measured point or the physical quantity, that is, according to the time sequence characteristics of the data, the identification model suitable for different physical quantities and different measured points is matched.

[0010] In the preferred embodiment, in S2, after obtaining the to-be-measured data, data preprocessing is performed, specifically: First, according to the data format difference of different monitoring instruments, format standardization processing is performed, including: data structure unification, unit and dimension calibration, and abnormal format cleaning; Then, for missing values, a hierarchical filling strategy is adopted: for short-term missing data, interpolation method is used for filling, for long-term missing data, correlation model is used for filling; for super-long-term missing data, recheck is performed; And according to the sampling frequency of different instruments, a reference sampling frequency is preset; According to the multi-dimensional validity identification model, the corresponding identification model is allocated for each measured point data, and preliminary judgment is performed; Through the results of preliminary judgment, the effective data is identified as valid measured value and exits the process, and the invalid data enters the monitoring data recheck and secondary verification.

[0011] In the preferred embodiment, the measured point importance grading and multi-physical quantity joint identification specifically includes: Measurement point type and importance discrimination: According to the part and item to which the measurement point belongs, the measurement point is divided into important measurement points and general measurement points, and the specific division method is shown in the following table; Importance grading judgment: important measurement points directly issue instructions to the station side for on-site inspection and comparison; general measurement points start multiple measurement point and multiple physical quantity evaluation; General measurement point risk disposal: measurement points with no greater risk are included in the list of key attention measurement points and the monitoring frequency is increased, and measurement points with greater risk issue instructions to the station side for on-site inspection.

[0012] In a preferred embodiment, the artificial periodic review includes: using stratified sampling, wherein the stratification is based on the state of the measurement point, the discrimination type or the importance; The periodic extraction of discrimination results for artificial verification is performed from the dimensions of data authenticity, model applicability, process compliance, etc., and the model is automatically triggered for re-matching according to the review results, forming a self-evolution closed loop of "discrimination-review-optimization".

[0013] In a preferred embodiment, the discrimination methods and indicators from the dimensions of data authenticity, model applicability, process compliance, etc. are as follows: Data authenticity: check the original data and preprocessing records, check whether there are errors, missing values and filling biases; compare the historical data trend of the same measurement point to determine whether the abnormality conforms to the physical law; correlate the on-site inspection records to verify whether the current state is consistent with the actual instrument state or dam body condition; Model applicability: check the misjudgment rate of the model, and if the misjudgment rate exceeds the preset misjudgment rate in the extracted samples, it is determined as not applicable; check the misjudgment trend of the model, i.e. whether the misjudgment rate of the same model for the same type of measurement point shows an upward trend in the last three reviews; Process compliance: including checking the measurement point state conversion record, the instruction pushing time or whether the gross error data is completely archived.

[0014] In a second aspect, a dam safety monitoring data effectiveness identification system is provided, comprising: A model construction module for constructing a multi-dimensional effectiveness identification model: the identification model includes a logic discrimination, a statistical discrimination and a clustering analysis model, and the model is adapted to the measurement point or physical quantity according to the data time sequence characteristics; A measurement point discrimination module for obtaining measurement point data, inputting the identification model for preliminary discrimination, filtering valid data, and performing secondary discrimination on invalid data to obtain current unqualified measurement points, and performing measurement point importance grading and multi-physical quantity joint discrimination to obtain grading and corresponding discrimination results; A monitoring comparison module for continuously tracking and monitoring key measurement points higher than a preset importance threshold for a preset time, and performing on-site comparison if an abnormality is found, and distinguishing between instrument fault abnormality and real value abnormality according to the comparison results; When it is a real value anomaly, inspection is carried out, and differential treatment is executed based on the inspection result, a repair suggestion is given and is incorporated into defect management if a defect is confirmed, and the monitoring part or item is supplemented to the inspection plan if no anomaly is found, thereby realizing linkage management of data anomaly and engineering maintenance; The artificial review module is used for artificial periodic review: periodic extraction of the discrimination result is carried out to be artificially checked, the model method is checked according to the review result, the model is automatically triggered to be re-matched if it is not applicable, and a self-evolution closed loop of "discrimination-review-optimization" is formed. The identification result module is used for marking the monitoring measuring point according to the validity identification result of the measured value, and adopting the corresponding treatment mode, and the validity identification result includes the comprehensive determination result of step-by-step discrimination, verification and review of the validity of the measured value.

[0015] In a third aspect, an electronic device is provided, including a memory and a processor; The memory is used for storing a computer program; The processor is used for implementing the dam safety monitoring data validity identification method of claim 1 when executing the computer program, wherein the computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0016] The dam safety monitoring data validity identification method provided by the application, through S1: constructing a multi-dimensional validity identification model: including a logic discrimination, a statistical discrimination and a clustering analysis model, S2: obtaining the data of the measuring point to be measured, inputting the identification model to perform preliminary discrimination, performing measuring point importance classification and multi-physical quantity joint discrimination, obtaining the classification and the corresponding discrimination result; S3: continuously tracking and monitoring the key measuring point higher than the preset importance threshold for a preset time length, S4: when it is a real value anomaly, inspection is carried out, and differential treatment is executed based on the inspection result, S5: artificial periodic review: periodic extraction of the discrimination result is carried out to be artificially checked; S6: according to the validity identification result of the measured value, the monitoring measuring point is marked with the corresponding measuring point state, and the corresponding treatment mode is adopted; the data identification accuracy and reliability are improved, and the monitoring efficiency and response speed are improved.

[0017] The application has the following beneficial effects: 1. Improve data recognition accuracy and reliability: integrate logical discrimination, statistical discrimination and clustering analysis, etc. Various mathematical models form a multi-level discrimination system, effectively identify invalid data such as exceeding instrument range, sign error, statistical anomaly and outlier gross error, avoid misjudgment risk of single model, greatly improve data recognition accuracy. According to the importance of measuring point and the time sequence characteristics of physical quantity, the discrimination model is dynamically matched, and through the joint analysis of multiple measuring points and multiple physical quantities, the instrument failure and real data anomaly are effectively distinguished, the misjudgment caused by single measuring point data fluctuation is reduced, especially suitable for complex monitoring scene of dam multi working condition and multi physical field coupling.

[0018] 2. Improve monitoring efficiency and response speed: grade measuring points according to importance, avoid "one size fits all" disposal, reduce unnecessary on-site intervention through correlation analysis for general measuring points, improve monitoring disposal efficiency and reduce labor cost. From data preliminary screening, retest verification to hierarchical disposal, on-site comparison, the whole process realizes "model discrimination-automatic shunt-instruction pushing" closed loop management. Compared with traditional manual inspection mode, the abnormal response time is shortened from hours to minutes, ensuring timely risk disposal.

[0019] 3. Build self-evolution system to realize long-term accurate monitoring: through regular manual verification of data effectiveness discrimination results, automatically label and push messages for inapplicable data, and re-match algorithms to form a self-evolution closed loop of "discrimination-recheck-optimization". The state of measuring point is divided into "normal, retest, attention, fault and verification" five categories, combined with inspection result feedback (such as crack and water seepage defect repair), realize seamless linkage of monitoring data anomaly and engineering maintenance, ensure two-way tracking of data anomaly and engineering hidden danger, and improve the systematicness of dam safety management.

[0020] 4. Improve the level of intelligence and standardization: unify the definition of measuring point state and disposal rules, provide a standard framework for the automation programming and modular development of monitoring system, which can be quickly integrated into dam safety monitoring system to realize the digitization of data processing, anomaly warning and maintenance scheduling, and reduce system development and maintenance cost. Continuous high-frequency monitoring of key measuring points and on-site instrument comparison can accurately identify the nature of data anomaly, avoid missing or false alarm of safety hidden danger caused by data misjudgment, and provide more reliable decision basis for dam safety operation. BRIEF DESCRIPTION OF DRAWINGS

[0021] The application will be further described below in combination with the drawings and examples: Figure 1 It is a flow chart of a dam safety monitoring data effectiveness identification method; Figure 2 It is a method for constructing multi-dimensional discrimination model; Figure 3 It is a flow chart of manual recheck optimization; Figure 4 For the point state transition mechanism; Figure 5 Is the data comparison chart before and after the validity identification. DETAILED DESCRIPTION

[0022] Embodiment 1 As Figures 1-5 shown, a dam safety monitoring data validity identification method, comprising the following steps: S1: Construct a multi-dimensional validity identification model: the identification model includes a logic discrimination, a statistical discrimination and a clustering analysis model, and the model is adapted to the measuring point or physical quantity according to the data time sequence characteristics.

[0023] S2: Obtain the data of the measuring point to be tested, input the identification model for preliminary discrimination, filter the valid data, and then perform secondary discrimination on the invalid data, obtain the current unqualified measuring point, and perform measuring point importance classification and multi-physical quantity joint discrimination to obtain the classification and the corresponding discrimination result.

[0024] S3: Continuously track and monitor the key measuring point higher than the preset importance threshold for a preset time period, and if an anomaly occurs, perform on-site comparison, and according to the comparison result, distinguish whether it is an instrument fault anomaly or a real value anomaly.

[0025] S4: When it is a real value anomaly, perform an inspection, and based on the inspection result, perform differential treatment, confirm the defect, give a repair suggestion, and include it in the defect management, and if no anomaly is found, supplement the monitoring site or project to the inspection plan, to realize the linkage management of data anomaly and engineering maintenance.

[0026] S5: Artificial periodic review: periodically extract the discrimination result for artificial checking, check the model method according to the review result, automatically trigger the model to re-match if it is not applicable, and form a self-evolution closed loop of “discrimination-review-optimization”.

[0027] S6: According to the validity identification result of the measured value, mark the corresponding measuring point state of the monitoring measuring point, and adopt the corresponding treatment method, the validity identification result includes the gradual discrimination, verification and review of the validity of the measured value after the whole process of S1-S5, and the final comprehensive judgment conclusion of “whether the measured value is valid, invalid reason (such as instrument failure / real dam anomaly / need to retest)”.

[0028] In this embodiment, by constructing a multi-dimensional effectiveness identification model, obtaining the data of the to-be-tested measuring points, inputting the identification model for preliminary discrimination, filtering the effective data, and then performing secondary discrimination on the invalid data, obtaining the current unqualified measuring points, and performing measuring point importance classification and multi-physical quantity joint discrimination, obtaining the classification and corresponding discrimination results, and performing preset duration continuous tracking monitoring on the key measuring points, performing differentiated treatment, periodically extracting the discrimination results for manual verification, and finally marking the monitoring measuring points with the corresponding measuring point state according to the effectiveness identification result of the value, and adopting the corresponding treatment method, through multi-measuring point and multi-physical quantity joint analysis, effectively distinguishing instrument failure and real data anomaly, reducing the misjudgment caused by single measuring point data fluctuation, improving the data recognition accuracy and reliability, improving the monitoring efficiency and response speed, realizing the whole-process digitization of data processing, abnormal early warning and maintenance scheduling, and reducing the system development and maintenance cost.

[0029] In the preferred scheme, the measuring point importance classification and multi-physical quantity joint discrimination are performed according to the preset importance classification, wherein: The general measuring points start the multi-measuring point and multi-physical quantity evaluation.

[0030] The measuring points with no great risk are listed in the list of key attention measuring points and the monitoring frequency is increased.

[0031] The measuring points with great risk issue instructions to the station side for on-site inspection.

[0032] In this embodiment, the resource utilization rate is improved and the false intervention and missed detection risk is reduced through the risk-oriented hierarchical treatment.

[0033] In the preferred scheme, in step S6, the branch process corresponding to the measuring point state is disposed each time the monitoring data inspection is started. The measuring point state includes: Normal: The measuring point whose value is discriminated by the mathematical model.

[0034] Recheck: The measuring point whose recheck value is still unqualified but is discriminated by the multi-measuring point and multi-physical quantity as having no great risk.

[0035] Attention: The measuring point whose recheck value is still unqualified but is discriminated by the multi-measuring point and multi-physical quantity as having no great risk.

[0036] Fault: The value of the measuring point cannot pass the effectiveness check continuously, and the instrument failure is determined after on-site comparison.

[0037] Verification: The value of the measuring point cannot pass the effectiveness check continuously, but the value is determined to be a real value after on-site comparison.

[0038] Each time the monitoring data check is started, for the measuring points in the normal state, the data analysis process is entered from the beginning, for the measuring points in the verification state, the measured values are directly made to enter the abnormality identification link, and for the measuring points in other states, the respective branch processes are handled.

[0039] In this embodiment, the five types of measuring point states of normal, retest, attention, fault, and verification are clearly defined, the standardized state definition is performed, the process management is simplified, the accurate shunting is achieved, the treatment pertinence is improved, and the data processing efficiency is improved.

[0040] In the preferred scheme, in step S1, the logical discrimination model is specifically: according to the monitoring instrument range, monitoring accuracy, and physical meaning of monitoring data, the validity of the measured value is judged, and the data exceeding the instrument range, sign error, and exceeding the human set value are invalid data.

[0041] The statistical discrimination model is specifically: taking the monitoring physical quantity under similar working conditions as sample data, a multivariate regression equation and a confidence interval are established for each measuring point by using historical monitoring data, and the difference between the monitoring effect quantity and the measured value exceeding 6 times the standard deviation is invalid data.

[0042] The cluster analysis model is specifically: using a cluster analysis model, the outlier data is determined as a gross error.

[0043] The model is matched with the measuring point or the physical quantity, that is, according to the time sequence characteristics of the data, different physical quantities and different measuring points are matched with the identification model suitable for them.

[0044] In this embodiment, the three types of models are complementary and cover multiple types of invalid data, and the adaptive model is matched for different physical quantities and measuring points according to the time sequence characteristics of the data, thereby solving the problem that the traditional fixed model is difficult to cope with the complex scene of dam multi-working condition and multi-physical field coupling, and further improving the identification accuracy.

[0045] In the preferred scheme, in step S2, after the to-be-measured data is obtained, data preprocessing is performed, specifically: First, the format standardization processing is performed for the data format differences of different monitoring instruments, including: data structure unification, unit and dimension calibration, and abnormal format cleaning.

[0046] Then, the hierarchical filling strategy is adopted for the missing values: the interpolation method is used to fill the short-term missing data, the correlation model is used to fill the long-term missing data, and the retest is performed for the ultra-long-term missing data.

[0047] And the reference sampling frequency is preset according to the sampling frequency of different instruments.

[0048] According to the multi-dimensional validity identification model, the corresponding discrimination model is allocated to the data of each measuring point for preliminary judgment. Through the preliminary judgment result, the effective data is identified as valid measurement value and exits the process, and the data judged as invalid enters the monitoring data retest and secondary verification.

[0049] In this embodiment, after data preprocessing, the data quality is guaranteed, the effective data is directly filtered after preliminary discrimination, only the invalid data is subjected to secondary verification, redundant calculation is reduced; the reference frequency is preset according to the sampling frequency of the instrument, the data collection rhythm is ensured to be uniform, and the processing efficiency is improved.

[0050] In the preferred scheme, the measurement point importance grading and multi-physical quantity joint discrimination specifically comprises: Measurement point type and importance discrimination: according to the part and project to which the measurement point belongs, the measurement point is divided into important measurement point and general measurement point; the specific division method is shown in Table 1.

[0051] Importance grading determination: the important measurement point directly issues an instruction to the station side for on-site inspection comparison; the general measurement point starts multi-measurement point multi-physical quantity judgment; General measurement point risk disposal: the measurement point without greater risk is listed in the key attention measurement point list and the monitoring frequency is increased, and the measurement point with greater risk issues an instruction to the station side for on-site inspection.

[0052] Table 1 Measurement point division table

[0053] In the preferred scheme, artificial periodic review includes: using stratified sampling, which is stratified according to measurement point state, discrimination type or importance; Periodically, the discrimination results are manually checked, that is, they are discriminated from the dimensions of data authenticity, model applicability, process compliance, etc., and the model method that is not applicable according to the review result automatically triggers model re-matching, forming a self-evolution closed loop of “discrimination-review-optimization”.

[0054] In this embodiment, stratified sampling is performed according to the measurement point state, discrimination type and importance, avoiding the high cost of full-volume review, while ensuring that the review sample covers key scenarios and guarantees the representativeness of the review result; multi-dimensional checking automatically triggers re-matching of the model that is not applicable, solving the problem of lack of dynamic optimization mechanism in traditional methods and decline in model adaptability after long-term use, and improving the long-term identification accuracy of the system.

[0055] In the preferred scheme, the discrimination method and index from the dimensions of data authenticity, model applicability, process compliance, etc. are as follows: Data authenticity: check the original data and preprocessing records, check whether there are formula errors, missing value filling deviations; compare the historical data trend of the same measurement point to determine whether the anomaly conforms to the physical law; correlate the on-site inspection records to verify whether the “fault” or “verification” state is consistent with the actual instrument state or dam body condition; Model applicability: The model's misclassification rate is checked. If the misclassification rate exceeds the preset rate in the sampled data, the model is deemed unapplicable. The model's misclassification trend is checked, i.e., whether the misclassification rate of the same model for the same type of measurement points shows an upward trend in three consecutive reviews. Process compliance: This includes checking whether the measurement point status transition records, instruction push time, or gross error data are completely archived.

[0056] This embodiment improves the reliability of judgment through multi-dimensional verification and clarifies the verification indicators for each dimension, such as preset false judgment rate threshold and three consecutive trend checks, providing accurate basis for model re-matching and process adjustment.

[0057] like Figure 5 As shown in the figure, the comparison curves before and after the validity identification of dam safety monitoring data intuitively present the filtering effect on gross errors in the monitoring data. The specific analysis is as follows: Coordinate axes and data dimensions: The horizontal axis represents the time dimension, with a time range from April 1st to July 1st, recording a 3-month monitoring period; the vertical axis represents the measurement dimension, with the unit being millimeters (mm), and the measurement range being between -1.20mm and 0.00mm, reflecting the numerical changes of monitored physical quantities (such as dam displacement).

[0058] Data curve types: The two key curves represent "data (including gross errors)" and "data (excluding gross errors)". The "data (including gross errors)" curve has obvious abnormal fluctuations (marked with "×"). Such fluctuations do not conform to the changing trend of normal monitoring data and belong to invalid gross errors caused by instrument failure, environmental interference, etc. The "data (excluding gross errors)" curve is smooth and continuous without abnormal jumps. It is a valid data curve after being processed by the multi-dimensional validity identification model of this invention (logical discrimination, statistical discrimination, cluster analysis, etc.) to remove invalid gross errors.

[0059] Key Comparison Results: Through a direct comparison of the two curves, this embodiment accurately identifies and filters out gross errors in the monitoring data, making the processed effective data more consistent with the physical laws of the actual operation of the dam, and providing more reliable data support for subsequent dam safety status assessment and engineering maintenance decisions.

[0060] In this embodiment, when a model is deemed unsuitable, the system uses three months of historical data from that measurement point as a test set to evaluate the accuracy and misclassification rate of the new model. The new model must outperform the original model. A five-day real-time discrimination test is then conducted on the new model to ensure consistency between its analysis results and the multi-physical quantity correlation analysis results. After the new model passes validation, the system automatically replaces the original model and records the reason for replacement, the new model parameters, and test indicators. Within one month of replacement, the system focuses on the discrimination results of that measurement point and includes them in the next round of manual review, forming a closed loop of "optimization – validation – re-optimization".

[0061] The specific ratio measurement method is as follows: Comparison of the same type of instrument: at the same measurement point, the calibrated standby instrument is used to collect data synchronously, and the deviation of the measured value of the original instrument from the measured value of the standby instrument is compared.

[0062] Manual auxiliary measurement: manual measurement method is used as a reference for key physical quantities to verify the accuracy of the data of the original instrument. Instrument state inspection: on-site inspection of the physical state of the original instrument, including: whether the sensor is damaged, whether the wiring is loose, whether the waterproof / dustproof device is failed; whether the sampling frequency is abnormal, whether there is electromagnetic interference or signal transmission failure.

[0063] Comparison of historical data: compare the historical data of the same period at the measurement point to determine whether the current abnormal value is outside the historical reasonable fluctuation range. If it is within the historical extreme value range and consistent with the trend, it may be a true value; if the jump amplitude is far beyond the historical range, it is inclined to be instrument failure.

[0064] Example 2 The embodiment provides a dam safety monitoring data validity identification system, comprising: A model construction module is configured to construct a multi-dimensional validity identification model: the identification model includes a logic discrimination, a statistical discrimination and a clustering analysis model, and the model is adapted to the measurement point or physical quantity according to the data time sequence characteristics.

[0065] A measurement point discrimination module is configured to obtain measurement point data to be measured, input the identification model for preliminary discrimination, filter valid data, and then perform secondary discrimination on invalid data, obtain current unqualified measurement points, and perform measurement point importance classification and multi-physical quantity joint discrimination to obtain classification and corresponding discrimination results.

[0066] A monitoring comparison module is configured to continuously track and monitor key measurement points higher than a preset importance threshold for a preset time length, perform on-site comparison if an anomaly occurs, and distinguish between instrument failure anomaly or true value anomaly according to the comparison result.

[0067] When it is a true value anomaly, a patrol is performed, and based on the patrol result, a differentiated treatment is performed, a repair suggestion is given and is incorporated into defect management if a defect is confirmed, and the monitoring part or project is supplemented to the patrol plan if no anomaly is found, so as to realize the linkage management of data anomaly and engineering maintenance.

[0068] An artificial review module is configured to periodically review artificially: periodically extract the discrimination results for artificial verification, check the model method according to the review result, automatically trigger model re-matching if it is not applicable, and form a self-evolution closed loop of "discrimination-review-optimization".

[0069] The identification result module is configured to mark the monitoring measuring point with a corresponding measuring point state according to the validity identification result of the measuring value, and to adopt a corresponding treatment mode.

[0070] Embodiment 3 The embodiment of the present application provides an electronic device, comprising a memory and a processor; The memory is configured to store a computer program.

[0071] The processor is configured to implement the dam safety monitoring data validity identification method in embodiment 1 when executing the computer program. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0072] For example, the memory can include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out memory (FIFO), and / or first-in-last-out memory (FILO), etc.; the processor can be, but is not limited to, a microprocessor of STM32F105 series, an ARM (Advanced RISC Machines) processor, an X86 architecture processor, or a processor integrated with NPU (neural-network processing units).

[0073] The above-mentioned embodiments are only preferred technical solutions of the present application, and should not be regarded as limitations of the present application. The protection scope of the present application should be based on the technical solutions recited in the claims, and the equivalent replacement solutions of the technical features recited in the claims are within the protection scope. That is, equivalent replacement improvements within this range are also within the protection scope of the present application.

Claims

1. A method for identifying the validity of dam safety monitoring data, characterized in that, Includes the following steps: S1: Construct a multi-dimensional validity identification model: The identification model includes logical discrimination, statistical discrimination and cluster analysis model, and completes the adaptation of the model to the measurement points or physical quantities according to the time series characteristics of the data; S2: Obtain the data of the measurement point to be measured, input it into the recognition model for preliminary judgment, filter the valid data, perform secondary judgment on the invalid data, obtain the current unqualified measurement point, and perform measurement point importance classification and multi-physical quantity joint judgment to obtain the classification and corresponding judgment results; S3: Continuously track and monitor key measurement points that exceed the preset importance threshold for a preset duration. If an abnormality is found, conduct on-site comparison and distinguish between instrument malfunction and abnormal true value based on the comparison results. S4: When the true value is abnormal, conduct inspections and implement differentiated handling based on the inspection results. If a defect is confirmed, provide repair suggestions and include them in defect management. If no abnormality is found, add the monitored parts or items to the inspection plan to achieve linkage management between data anomalies and engineering maintenance. S5: Periodic manual review: The discrimination results are periodically extracted for manual verification. The model method is checked based on the review results. If it is not applicable, the model is automatically re-matched, forming a self-evolving closed loop of "discrimination-review-optimization". S6: Based on the validity identification result of the measured value, mark the corresponding measuring point status of the monitoring measuring point and adopt the corresponding handling method. The validity identification result includes the comprehensive judgment result of the step-by-step discrimination, verification and review of the validity of the measured value obtained in S1-S5.

2. The method for identifying the validity of dam safety monitoring data according to claim 1, characterized in that, The importance classification of measurement points and the joint discrimination of multiple physical quantities are based on a preset importance classification, wherein: General measurement point start-up for multi-measurement point and multi-physical quantity evaluation; Monitoring points with no significant risk are included in the list of key monitoring points and their monitoring frequency is increased; For measurement points with higher risks, instructions are sent to the plant side for on-site inspection.

3. The method for identifying the validity of dam safety monitoring data according to claim 1, characterized in that, In S6, each time the monitoring data check is initiated, the branch process corresponding to the status of the measuring point is handled. The status of the measuring point includes: Normal: Measurement points whose values ​​are determined through mathematical models; Retest: Measurement points that failed the mathematical model's judgment and are awaiting retest results; Note: The retested values ​​at the measuring points are still unqualified, but after multi-point and multi-physical quantity analysis, no major risks are identified. Fault: The measured value consistently failed the validity check, and the instrument was determined to be faulty after on-site comparison testing; Verification: The measured values ​​consistently failed the validity check, but after on-site comparison, the measured values ​​were determined to be true values.

4. The method for identifying the validity of dam safety monitoring data according to claim 1, characterized in that, In S1, the logical discrimination model is specifically as follows: the validity of the measured value is judged based on the range of the monitoring instrument, the monitoring accuracy, and the physical meaning of the monitoring data. Data that exceeds the instrument range, has an incorrect sign, or exceeds the manually set value is invalid data. The statistical discrimination model is as follows: using the monitored physical quantities under similar working conditions as sample data, and using historical monitoring data to establish a multiple regression equation and confidence interval for each measuring point, the data is considered invalid when the difference between the monitored effect size and the measured value exceeds 6 times the standard deviation. The cluster analysis model specifically involves using a cluster analysis model to identify outlier data as gross errors. Model matching with measurement points or physical quantities refers to the identification model that is adapted to different physical quantities and measurement points based on the temporal characteristics of the data.

5. The method for identifying the validity of dam safety monitoring data according to claim 1, characterized in that, In step S2, after acquiring the data to be tested, data preprocessing is performed, specifically as follows: First, to address the differences in data formats among different monitoring instruments, format standardization is performed, including: data structure unification, unit and dimension calibration, and abnormal format cleaning. Furthermore, a tiered imputation strategy is adopted for missing values: interpolation is used to impute short-term missing data, association models are used to impute long-term missing data, and retesting is performed for very long-term missing data. And preset a reference sampling frequency for the sampling frequency of different instruments; Based on the multidimensional validity identification model, a corresponding discrimination model is assigned to each measurement point data for preliminary judgment; Based on the preliminary assessment, valid data are identified as valid measurements and exit the process, while invalid data enters the monitoring data retesting and secondary verification.

6. The method for identifying the validity of dam safety monitoring data according to claim 1, characterized in that, The classification of the importance of measurement points and the joint discrimination of multiple physical quantities specifically include: Measurement point type and importance determination: Measurement points are divided into important measurement points and general measurement points according to their location and project. The specific division method is shown in the table below. Importance classification determination: Important measurement points directly issue instructions to the plant side for on-site inspection and comparison. General measurement points initiate multi-measurement and multi-physical quantity evaluation. General monitoring point risk management: Monitoring points without significant risks are included in the list of key monitoring points and the monitoring frequency is increased; monitoring points with significant risks are instructed to conduct on-site inspections at the plant.

7. The method for identifying the validity of dam safety monitoring data according to claim 1, characterized in that, The manual periodic review includes: using stratified sampling, wherein stratification is based on the shape of the measurement points, the type of discrimination, or the importance; The periodic sampling and manual verification of the judgment results involves judging from dimensions such as data authenticity, model applicability, and process compliance. Based on the inapplicable model methods in the verification results, the model is automatically re-matched, forming a self-evolving closed loop of "judgment-verification-optimization".

8. The method for identifying the validity of dam safety monitoring data according to claim 7, characterized in that, The following are the methods and indicators for judging data authenticity, model applicability, and process compliance: Data authenticity: Verify the raw data against the preprocessed records to check for errors in format and deviations in filling missing values; compare the trend of historical data at the same measurement point to determine whether anomalies conform to physical laws; Link the on-site inspection records to verify whether the current status is consistent with the actual instrument status or dam condition; Model applicability: The model's misclassification rate is checked. If the misclassification rate exceeds the preset rate in the sampled data, the model is deemed unapplicable. The model's misclassification trend is checked, i.e., whether the misclassification rate of the same model for the same type of measurement points shows an upward trend in three consecutive reviews. Process compliance: This includes checking whether the measurement point status transition records, instruction push time, or gross error data are completely archived.

9. A system for identifying the validity of dam safety monitoring data, characterized in that, include: The model building module is used to build a multidimensional validity identification model: the identification model includes logical discrimination, statistical discrimination and cluster analysis model, and completes the adaptation of the model to the measurement points or physical quantities according to the time series characteristics of the data; The measurement point discrimination module is used to acquire the measurement point data to be measured, input the recognition model for preliminary discrimination, filter the valid data, perform secondary discrimination on the invalid data, acquire the current unqualified measurement points, and perform measurement point importance classification and multi-physical quantity joint discrimination to obtain the classification and corresponding discrimination results. The monitoring and comparison module is used to continuously track and monitor key measurement points that are above the preset importance threshold for a preset duration. If an abnormality is found, an on-site comparison test is performed, and the test results are used to distinguish between instrument malfunction and abnormal actual value. When the true value is abnormal, an inspection is carried out, and differentiated handling is performed based on the inspection results. If a defect is confirmed, a repair suggestion is given and included in the defect management. If no abnormality is found, the monitored parts or items are added to the inspection plan to achieve linkage management between data anomalies and engineering maintenance. The manual review module is used for periodic manual review: the discrimination results are periodically extracted for manual verification, and the model method is checked based on the review results. If it is not applicable, the model is automatically re-matched, forming a self-evolving closed loop of "discrimination-review-optimization". The identification result module is used to mark the status of the monitoring point according to the validity identification result of the measured value, and to adopt the corresponding handling method. The validity identification result includes the comprehensive judgment result of step-by-step discrimination, verification and review of the validity of the measured value obtained in S1-S5.

10. An electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to implement the method for identifying the validity of dam safety monitoring data according to claim 1 when executing a computer program, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

Images