Water level cleaning method, device and electronic equipment for multi-layer calibration

Abstract

The present invention discloses a water level cleaning method, device and electronic device for multi-layer verification, including: obtaining water level data to build a queue and training an isolation forest model; using the trained isolation forest model to detect the real-time collected water level data, if the detection is normal, it is directly saved to the queue, if the detection is abnormal, it is marked as the first abnormal data point; with the first abnormal data point as the center, a predetermined number of water level data before and after the first abnormal data point are obtained to form an abnormal window data set; fitting the abnormal window data set, if the fitting degree is greater than a first threshold, the data of the abnormal window data set starting from the first abnormal data point is saved to the queue, and the isolation forest model is retrained; otherwise, the latest data in the normal water level preceding the water level data of the first abnormal data point is used to replace the data of the abnormal window data set, and saved to the queue. The method adopted in this application cleans the water level and provides a good data basis for the decision-making of relevant responsible persons.

Description

Technical Field

[0001] The present application relates to the field of data cleaning technology, and in particular to a water level cleaning method, device and electronic equipment for multi-layer verification. Background Art

[0002] In recent years, non-contact radar water level meters, with their strong anti-interference capabilities and high environmental adaptability, have been widely used for water level monitoring in mountainous areas, providing data support for hydrological analysis and flash flood warnings. However, the complex topography and climate characteristics of mountainous areas pose multiple challenges to radar water level monitoring. For example, the high sediment content and floating debris prevalent in mountain rivers exacerbate the dynamic distortion of radar wave reflection signals; dense vegetation can distort the propagation path of radar signals. These factors combine to cause the raw data collected by radar water level meters to be mixed with some abnormal jumps.

[0003] The presence of jump data not only reduces the accuracy of water level monitoring but can also interfere with subsequent data analysis and hydrological forecasting. For example, in flash flood warning systems, the presence of jump data can lead to misjudgments or omissions, thus impacting disaster prevention and mitigation decisions. Therefore, identifying and removing jump data from the raw data of radar water level gauges to extract accurate and stable water level information has become a pressing challenge. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a multi-layer calibration water level cleaning method, device and electronic equipment to solve the technical problem in related technologies that the accuracy of radar water level meter monitoring is affected by factors such as the complex environment in mountainous areas, resulting in inaccurate water level monitoring data, which in turn affects the relevant responsible persons in disaster prevention and mitigation decision-making.

[0005] According to a first aspect of an embodiment of the present application, a water level cleaning method for multi-layer calibration is provided, comprising:

[0006] S1: Obtaining raw water level data within a specified sampling interval within a specified time, using the raw water level data as a training data set to train an isolation forest model;

[0007] S2: real-time collection of water level data;

[0008] S3: Use the trained isolation forest model to detect the real-time collected water level data. If the detection result is normal, it is directly saved to the queue. If the detection result is abnormal, it is marked as the first abnormal data point and S4 is executed;

[0009] S4: Taking the first abnormal data point as the center, obtaining a predetermined amount of water level data before and after the first abnormal data point to form an abnormal window data set;

[0010] S5: Fitting the abnormal window data set, and judging the fitting effect by the fitting degree. If the fitting degree is greater than a first threshold, it indicates that the abnormal window data set conforms to the normal physical trend of the water level, and a predetermined number of data of the abnormal window data set starting from the first abnormal data point position is saved to a queue, and S6 is executed; otherwise, it indicates that the data starting from the first abnormal data point position in the abnormal window data set is abnormal data, and S7 is executed;

[0011] S6: Taking the latest data in the queue as the end point, obtain a specified amount of original water level data from the queue, retrain the isolation forest model, and execute S2 after the training is completed;

[0012] S7: Use the latest data in the normal water level preceding the first abnormal data point water level data to replace the abnormal data in the abnormal window data set, and save the data to the queue.

[0013] According to a second aspect of an embodiment of the present application, a water level cleaning device for multi-layer calibration is provided, comprising:

[0014] A model building module is used to obtain raw water level data at a specified sampling interval within a specified time, and use the raw water level data as a training data set to train an isolation forest model;

[0015] Data acquisition module, used to collect water level data in real time;

[0016] The first detection module is used to use the trained isolation forest model to detect the real-time collected water level data. If the detection result is normal, it is directly saved to the queue. If the detection result is abnormal, it is marked as the first abnormal data point and the dataset construction module is executed;

[0017] a data set construction module, configured to obtain a predetermined amount of water level data before and after the first abnormal data point, with the first abnormal data point as the center, to form an abnormal window data set;

[0018] a second detection module, configured to fit the abnormal window data set, and determine the fitting effect by the degree of fit; if the degree of fit is greater than a first threshold, indicating that the abnormal window data set conforms to the normal physical trend of the water level, a predetermined number of data of the abnormal window data set starting from the first abnormal data point position is saved to a queue, and a model updating module is executed; otherwise, indicating that the data starting from the first abnormal data point position in the abnormal window data set is abnormal data, and a data correction module is executed;

[0019] A model updating module is used to obtain a specified amount of raw water level data from the queue with the latest data in the queue as the end point, retrain the isolation forest model, and execute the data acquisition module after the training is completed;

[0020] The data correction module is used to replace the abnormal data in the abnormal window data set with the latest data in the normal water level preceding the water level data of the first abnormal data point, and save the data to the queue.

[0021] According to a third aspect of the embodiments of the present application, there is provided an electronic device, including:

[0022] one or more processors;

[0023] a memory for storing one or more programs;

[0024] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect.

[0025] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0026] The technical solutions provided by the embodiments of the present application may have the following beneficial effects:

[0027] It can be seen from the above embodiments that the present application establishes a sample set by obtaining the water level collected by the water level meter, constructs an isolation forest model after pre-processing the sample set, detects the water level data based on the isolation forest model, and after detecting the data anomaly, obtains the data trend by fitting the abnormal window data set, and further judges whether the water level data is abnormal jump data. Finally, the isolation forest model is updated and the abnormal jump data is corrected, overcoming the technical problems of frequent jumps and inaccurate data of the water level collected by the radar water level meter due to factors such as the mountainous environment, thereby achieving the improvement of the accuracy of water level monitoring data in mountainous areas, and providing accurate and reliable data support for flood control personnel or flood control equipment.

[0028] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0030] Figure 1 The present invention is a flow chart showing a water level cleaning method for multi-layer verification according to an exemplary embodiment.

[0031] Figure 2 This is a comparison of the abnormal data site water level before and after processing according to an exemplary embodiment. Figure 1 .

[0032] Figure 3 This is a comparison of the abnormal data site water level before and after processing according to an exemplary embodiment. Figure 2 .

[0033] Figure 4 This is a comparison of the abnormal data site water level before and after processing according to an exemplary embodiment. Figure 3 .

[0034] Figure 5 1 is a comparison diagram of normal data site water level before and after processing according to an exemplary embodiment.

[0035] Figure 6 The figure is a block diagram of a water level cleaning device for multi-layer verification according to an exemplary embodiment.

[0036] Figure 7 The figure is a schematic structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0037] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0038] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0039] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0040] Figure 1 FIG. 1 is a flow chart showing a water level cleaning method for multi-layer calibration according to an exemplary embodiment. Figure 1As shown, the method may include the following steps:

[0041] S1: Obtaining raw water level data within a specified sampling interval within a specified time, using the raw water level data as a training dataset, and training an isolation forest model. This step includes the following sub-steps:

[0042] S11: acquiring raw water level data of a specified sampling interval within a specified time, and saving the raw water level data into a queue in the order of acquisition time;

[0043] Specifically, the original water level data of the specified sampling interval within the specified time is obtained, and the original water level data is saved in a queue in the order of collection time. The queue can only store a specified number of data in a first-in-first-out mode. When the queue is full, the earliest data will be removed.

[0044] The main purpose of this step is to obtain and store the raw water level data and provide a sample set for subsequent training samples. In theory, the larger the sample set, the better. Since the measurement cycle of the radar water level meter is generally 5-8 seconds, to ensure timeliness and the number of sample sets per unit time, 8 seconds is recommended.

[0045] S12: using the original water level data in the queue as training data set samples to train an isolation forest model;

[0046] Specifically, the original water level data in the queue is used as a training data set sample, and binomial fitting is performed on the training data set sample. If the fitting degree is greater than a first threshold, it indicates that the data trend is normal; otherwise, it indicates that the data trend is abnormal, that is, the data is in a jumping or unchanged state. The minimum value in the training data set sample is obtained, the training data set sample is formatted as the minimum value, and the training data set sample is used to train the isolation forest model.

[0047] The main purpose of this step is to pre-process the training data set samples, and then train the isolation forest model, ensure the accuracy of the model, and provide a benchmark for the subsequent detection of abnormal data. Since the initial sample value may be in a state of rising water level or abnormal jump, if it is in an abnormal jump state, it may contain too much abnormal data, which will affect the accuracy of the model. Therefore, by performing binomial fitting on the training data set samples, the accuracy of the training data set samples is judged. Provide an accurate data set for the training of the isolation forest model. The size of the training data set sample is 100, which is about 15 minutes of data. In theory, the larger the amount of data, the more accurate the model. After a large number of simulation tests, considering timeliness and accuracy, the sample size is determined to be 100. The queue can be consistent with the sample size, or slightly larger.

[0048] S2: real-time collection of water level data;

[0049] Specifically, water level data is collected in real time according to a specified sampling interval, and the collected water level data is not saved in a queue.

[0050] The main purpose of this step is to collect water level data in real time and provide a data source for subsequent detection of the water level data.

[0051] S3: Use the trained isolation forest model to detect the real-time collected water level data. If the detection result is normal, it is directly saved to the queue. If the detection result is abnormal, it is marked as the first abnormal data point and S4 is executed;

[0052] Specifically, the trained isolation forest model is used to detect and mark the real-time collected water level data. If the real-time collected water level data is marked as normal, it indicates that the real-time collected water level data is within the normal range and is saved to the queue. If the real-time collected water level data is marked as abnormal, the real-time collected water level data is marked as the first abnormal data point, and all subsequent real-time collected data will no longer be marked using the trained isolation forest model.

[0053] The main purpose of this step is to perform a first-level test on the real-time water level data to determine whether the water level is normal. This is primarily done by using the isolation forest model to determine the degree of outliers at each point, i.e., whether the water level has jumped. This first-level test screens out normal and abnormal points. If abnormal, the second-level test proceeds to provide a more accurate data foundation for subsequent judgments.

[0054] S4: Taking the first abnormal data point as the center, obtaining a predetermined amount of water level data before and after the first abnormal data point to form an abnormal window data set;

[0055] Specifically, an abnormal window data set is constructed, the size of the window is centered on the first abnormal data point, a predetermined number of data in front of the center in the queue and a predetermined number of water level data collected in real time are obtained and stored in the abnormal window data set, and the water level data collected in real time starting from the first abnormal data point is stored in the abnormal window data set.

[0056] The main purpose of this step is to construct an abnormal window data set to provide a data basis for the subsequent second-level detection. The size of the abnormal window data set is 21, that is, with the first abnormal data point as the center, the first 10 and the last 10, according to the sampling period of 8s, about 1 minute and a half before and 1 minute and a half after the data. For the flash flood warning scenario, due to the timeliness requirement, if the data accumulation time is too long, it may affect the timeliness of the flash flood. If the time is too short, it may cause the data to remain unchanged. Therefore, the data volume of 1 minute and a half before and after is taken to take into account both real-time and data changes. For non-flash flood scenarios, since the water level changes more slowly, the size of the abnormal window data set can be appropriately increased.

[0057] S5: Fitting the abnormal window data set, judging the data condition by the degree of fit; if the degree of fit is greater than a first threshold, indicating that the abnormal window data set conforms to the normal physical trend of the water level, saving a predetermined number of data in the abnormal window data set starting from the first abnormal data point position to a queue, and executing S6; otherwise, indicating that the data in the abnormal window data set starting from the first abnormal data point position is abnormal data, executing S7;

[0058] Specifically, a quadratic function model is constructed based on formula (1), the residual of each data point is calculated based on formula (2), and the residual sum of squares of all data points is calculated based on the residual of each data point, and the fitting degree of the abnormal window data set is calculated based on formula (3). If the fitting degree is greater than or equal to a first threshold, it indicates that the abnormal window data set conforms to the normal physical trend of the water level, and the data trend of the abnormal window data set is rising or falling. The data of the abnormal window data set starting from the first abnormal data point position is stored in the queue, and S6 is executed; otherwise, if the fitting degree is less than the first threshold, it indicates that the abnormal window data set does not conform to the normal physical trend of the water level, and S7 is executed.

[0059] Formula (1) is as follows:

[0060] ;

[0061] Formula (2) is as follows:

[0062] ;

[0063] Formula (3) is as follows:

[0064] ;

[0065] In the above formula, y represents the water level value after fitting, x represents the data sequence index number, and a, b, and c represent the binomial coefficients; represents the residual, Indicates the true water level value. represents the water level value after fitting; represents the degree of fit, The average water level representing the true water level value.

[0066] The main purpose of this step is to determine whether the abnormal window data set is normal. Since the normal physical trend of water level is generally an upward or downward curve, a smooth curve of the data is fitted by binomial fitting, thereby calculating the data fit. If the fit is closer to 1, it means that the data fluctuation is small, the data is in a state of rising or falling curves, and the data conforms to the normal trend. Otherwise, it indicates that the data fluctuation is large, the data jitters up and down or does not change, indicating that the data does not conform to the normal trend. This step serves as a supplementary check for the first-level detection. Since the first-level test may have misjudgments, the abnormal window data set is combined with binomial fitting to perform data trend analysis to further determine whether the water level data is abnormal jump data, which can more accurately analyze the data quality. The results of the analysis can be used as a mark for whether the subsequent isolation forest model is updated.

[0067] S6: Taking the latest data in the queue as the end point, obtain a specified amount of original water level data from the queue, retrain the isolation forest model, and execute S2 after the training is completed;

[0068] Specifically, taking the first abnormal data point as the starting point, all data in the abnormal window data set are saved to a queue, taking the latest data in the queue as the end point, obtaining a specified amount of original water level data from the queue as a training data set, and retraining the isolation forest model to obtain a retrained isolation forest model.

[0069] The main purpose of this step is to retrain the Isolation Forest model. Since the water level data trend has begun to change, using the original Isolation Forest model could result in each piece of real-time water level data being identified as abnormal, leading to frequent second-level detection. Therefore, after the second-level detection passes, the Isolation Forest model needs to be retrained to align with the actual data trend and ensure model detection accuracy.

[0070] S7: Use the latest data in the normal water level preceding the first abnormal data point water level data to replace the abnormal data in the abnormal window data set, and save it to the queue; this step includes the following sub-steps:

[0071] Specifically, the preceding normal water level of the water level data of the first abnormal data point in the abnormal window data set is obtained, and the preceding normal water level is used to replace all abnormal data starting from the first abnormal data point in the abnormal window data set, and is saved in a queue.

[0072] The main purpose of this step is to correct the abnormal data, and the corrected data is the latest data of the normal water level preceding the water level data of the first abnormal data point.

[0073] Figure 2 、 Figure 3 and Figure 4 This is a comparison chart of abnormal water level data before and after processing, where Figure 2 The water level frequently fluctuates up and down due to the weeds under the radar water level gauge. Figure 3 The water level occasionally jumps due to tree branches blocking the water. Figure 4 The water level keeps jumping because of the weeds under the radar water level gauge. Figure 5 This is a comparison chart of normal water level data before and after processing. It can be seen that after being processed by this method, the cleaned water level data is consistent with the trend of the actual observed water level data, and a good processing effect has been achieved for both abnormal and normal water level data. It should be noted that Figure 2-Figure 5 The water level data were converted using the 85 elevation datum.

[0074] It can be seen from the above embodiments that the present application establishes a sample set by obtaining the water level collected by the water level meter, constructs an isolation forest model after pre-processing the sample set, detects the water level data based on the isolation forest model, and after detecting the data anomaly, obtains the data trend by fitting the abnormal window data set, and further judges whether the water level data is abnormal jump data. Finally, the isolation forest model is updated and the abnormal jump data is corrected, overcoming the technical problems of frequent jumps and inaccurate data of the water level collected by the radar water level meter due to factors such as the mountainous environment, thereby achieving the improvement of the accuracy of water level monitoring data in mountainous areas, and providing accurate and reliable data support for flood control personnel or flood control equipment.

[0075] Corresponding to the aforementioned embodiment of the water level cleaning method for multi-layer calibration, the present application also provides an embodiment of a water level cleaning device for multi-layer calibration.

[0076] Figure 6 This is a block diagram of a water level cleaning device for multi-layer verification according to an exemplary embodiment. Figure 6 , the device comprises:

[0077] Model building module 1 is used to obtain raw water level data at a specified sampling interval within a specified time, and use the raw water level data as a training data set to train an isolation forest model;

[0078] Data acquisition module 2, used for collecting water level data in real time;

[0079] The first detection module 3 is used to use the trained isolation forest model to detect the real-time collected water level data. If the detection result is normal, it is directly saved to the queue. If the detection result is abnormal, it is marked as the first abnormal data point and the data set construction module is executed;

[0080] The data set construction module 4 is configured to obtain a predetermined amount of water level data before and after the first abnormal data point, with the first abnormal data point as the center, to form an abnormal window data set;

[0081] A second detection module 5 is configured to perform binomial fitting on the abnormal window data set, and determine the fitting effect by the degree of fit. If the degree of fit is greater than a first threshold, it indicates that the abnormal window data set conforms to the normal physical trend of the water level, and a predetermined number of data in the abnormal window data set starting from the first abnormal data point position is saved to a queue, and a model updating module is executed; otherwise, it indicates that the data in the abnormal window data set starting from the first abnormal data point position is abnormal data, and a data correction module is executed;

[0082] Model updating module 6 is used to obtain a specified amount of original water level data from the queue with the latest data in the queue as the end point, retrain the isolation forest model, and execute the data acquisition module after the training is completed;

[0083] The data correction module 7 is configured to replace the abnormal data in the abnormal window data set with the latest data in the normal water level preceding the water level data of the first abnormal data point, and save the data in the queue.

[0084] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0085] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0086] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the water level cleaning method for multi-layer verification as described above. Figure 7 As shown in the figure, a hardware structure diagram of a water level cleaning device for multi-layer calibration provided by an embodiment of the present invention is provided in any device with data processing capability, except Figure 7 In addition to the processor, memory, DMA controller, disk, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0087] Accordingly, the present application also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the water level cleaning method for multi-layer verification as described above. The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0088] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims.

[0089] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A water level cleaning method for multi-layer calibration, characterized in that: include: S1: Obtaining raw water level data within a specified sampling interval within a specified time, using the raw water level data as a training data set to train an isolation forest model; S2: real-time collection of water level data; S3: Use the trained isolation forest model to detect the real-time collected water level data. If the detection result is normal, it is directly saved to the queue. If the detection result is abnormal, it is marked as the first abnormal data point and S4 is executed; S4: Taking the first abnormal data point as the center, obtaining a predetermined amount of water level data before and after the first abnormal data point to form an abnormal window data set; S5: Fitting the abnormal window data set, and judging the fitting effect by the fitting degree. If the fitting degree is greater than a first threshold, it indicates that the abnormal window data set conforms to the normal physical trend of the water level, and a predetermined number of data of the abnormal window data set starting from the first abnormal data point position is saved to a queue, and S6 is executed; otherwise, it indicates that the data starting from the first abnormal data point position in the abnormal window data set is abnormal data, and S7 is executed; S6: Taking the latest data in the queue as the end point, obtain a specified amount of original water level data from the queue, retrain the isolation forest model, and execute S2 after the training is completed; S7: Use the latest data in the normal water level preceding the water level data of the first abnormal data point to replace the abnormal data in the abnormal window data set, and save the data to the queue.

2. The water level cleaning method for multi-layer calibration according to claim 1, characterized in that: Obtaining raw water level data within a specified sampling interval within a specified time, using the raw water level data as a training data set, and training an isolation forest model, including: Get the raw water level data of the specified sampling interval within the specified time, and save the raw water level data into a queue in the order of acquisition time. The queue can only store a specified number of data in a first-in-first-out mode. When the queue is full, the oldest data will be removed. The original water level data in the queue is used as a training data set sample, and the training data set sample is fitted. If the fitting degree is greater than a first threshold, it indicates that the data trend is normal; otherwise, it indicates that the data trend is abnormal, that is, the data is in a jittering or unchanged state. The minimum value of the training data set sample is obtained, the training data set sample is formatted as the minimum value, and the training data set sample is used to train the isolation forest model.

3. The water level cleaning method for multi-layer calibration according to claim 1, characterized in that: Use the trained isolation forest model to detect the real-time water level data. If the test result is normal, it is directly saved to the queue. If the test result is abnormal, it is marked as the first abnormal data point, including: The trained isolation forest model is used to detect and mark the real-time collected water level data. If the real-time collected water level data is marked as normal, it indicates that the real-time collected water level data is within the normal range and is saved to the queue. If the real-time collected water level data is marked as abnormal, the real-time collected water level data is marked as the first abnormal data point, and all subsequent real-time collected data will no longer be marked using the trained isolation forest model.

4. The water level cleaning method for multi-layer calibration according to claim 1, characterized in that: Taking the first abnormal data point as the center, a predetermined amount of water level data before and after the first abnormal data point is obtained to form an abnormal window data set, including: An abnormal window data set is constructed, where the size of the window is centered on the first abnormal data point, and a predetermined number of data preceding the center in the queue and a predetermined number of water level data collected in real time are obtained and stored in the abnormal window data set.

5. The water level cleaning method for multi-layer calibration according to claim 1, characterized in that: Fitting the abnormal window data set, and judging the fitting effect by the fitting degree. If the fitting degree is greater than a first threshold, it indicates that the abnormal window data set conforms to the normal physical trend of the water level. A predetermined number of data of the abnormal window data set starting from the first abnormal data point position is saved in a queue, and S6 is executed. Otherwise, it indicates that the data starting from the first abnormal data point position in the abnormal window data set is abnormal data, and S7 is executed, including: A quadratic function model is constructed based on formula (1), the residual of each data point is calculated based on formula (2), and the residual sum of squares of all data points is calculated based on the residual of each data point, and the fitting degree of the abnormal window data set is calculated based on formula (3). If the fitting degree is greater than or equal to a first threshold, it indicates that the abnormal window data set conforms to the normal physical trend of the water level, and the data trend of the abnormal window data set is rising or falling. The data of the abnormal window data set starting from the position of the first abnormal data point is stored in the queue, and S6 is executed; otherwise, if the fitting degree is less than the first threshold, it indicates that the abnormal window data set does not conform to the normal physical trend of the water level, and S7 is executed. Formula (1) is as follows: ； Formula (2) is as follows: ； Formula (3) is as follows: ； In the above formula, y represents the water level value after fitting, x represents the data sequence index number, and a, b, and c represent the binomial coefficients; represents the residual, Indicates the true water level value. represents the water level value after fitting; represents the degree of fit, The average water level representing the true water level value.

6. The water level cleaning method for multi-layer calibration according to claim 1, characterized in that: Taking the latest data in the queue as the end point, obtaining a specified amount of raw water level data from the queue, and retraining the isolation forest model, including: Taking the first abnormal data point as the starting point, all data in the abnormal window data set are saved to a queue, and taking the latest data in the queue as the end point, a specified number of original water level data are obtained from the queue as a training data set, and the isolation forest model is retrained to obtain a retrained isolation forest model.

7. The water level cleaning method for multi-layer calibration according to claim 1, characterized in that: Replacing the abnormal data in the abnormal window data set with the latest data in the preceding normal water level of the first abnormal data point water level data and saving the data to the queue includes: Obtain the preceding normal water level of the water level data of the first abnormal data point in the abnormal window data set, use the preceding normal water level to replace all abnormal data starting from the first abnormal data point in the abnormal window data set, and save it to the queue.

8. A water level cleaning device for multi-layer calibration, characterized in that: include: A model building module is used to obtain raw water level data at a specified sampling interval within a specified time, and use the raw water level data as a training data set to train an isolation forest model; Data acquisition module, used to collect water level data in real time; The first detection module is used to use the trained isolation forest model to detect the real-time collected water level data. If the detection result is normal, it is directly saved to the queue. If the detection result is abnormal, it is marked as the first abnormal data point and the dataset construction module is executed; a data set construction module, configured to obtain a predetermined amount of water level data before and after the first abnormal data point, with the first abnormal data point as the center, to form an abnormal window data set; a second detection module, configured to fit the abnormal window data set, and determine the fitting effect by the degree of fit; if the degree of fit is greater than a first threshold, it indicates that the abnormal window data set conforms to the normal physical trend of the water level, and a predetermined number of data of the abnormal window data set starting from the first abnormal data point position is saved to a queue, and a model updating module is executed; otherwise, it indicates that the data of the abnormal window data set starting from the first abnormal data point position is abnormal data, and a data correction module is executed; A model updating module is used to obtain a specified amount of raw water level data from the queue with the latest data in the queue as the end point, retrain the isolation forest model, and execute the data acquisition module after the training is completed; The data correction module is used to replace the abnormal data in the abnormal window data set with the latest data in the normal water level preceding the water level data of the first abnormal data point, and save the data to the queue.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images