Geological disaster monitoring data quality evaluation method and device, medium and product
Through improved discrete Fourier transform and Butterworth low-pass filter, geological disaster monitoring data is processed, and the evaluation index is calculated using data quality evaluation formulas, the problem of the monitoring data quality cannot be evaluated quickly and effectively, and the accuracy and stability of data quality evaluation are improved.
Patent Information
- Application Number
- CN202411510792.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2025-05-27
AI Technical Summary
In the existing geological disaster monitoring and early warning technology, the evaluation and analysis of monitoring data quality is ignored, resulting in the inability to quickly and effectively evaluate the data quality, affecting the accuracy of early warning and forecast.
By obtaining the original time domain data for geological disaster monitoring, using improved discrete Fourier transform and Butterworth low-pass filter, the data is converted and filtered, and then the monitoring data quality evaluation index is calculated through the data quality evaluation formula to determine the data quality evaluation results.
It realizes rapid and accurate evaluation of the quality of geological disaster monitoring data, improves the accuracy and stability of the quality evaluation of monitoring data, and provides technicians with better data decision-making support.
Smart Images
Figure CN120047289A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of geological disaster monitoring and early warning, and particularly to a method, device, medium and product for evaluating the quality of geological disaster monitoring data. Background Art
[0002] In the existing geological disaster monitoring and early warning, based on the massive monitoring data collected by professional monitoring equipment, through methods such as threshold early warning and model early warning, the stability evaluation and early warning of geological disasters can be realized. Therefore, the accuracy of the stability evaluation and early warning results is not only closely related to the accuracy of the threshold and the model, but also directly affected by the quality of the monitoring data.
[0003] Currently, the research in the field of geological disaster monitoring and early warning mainly focuses on improving the accuracy of threshold and model early warning, often neglecting the quality evaluation and analysis of monitoring data. Therefore, how to achieve a reasonable and accurate evaluation of the quality of geological disaster monitoring data has become an urgent problem in the industry. Summary of the Invention
[0004] The purpose of this application is to provide a method, device, medium and product for evaluating the quality of geological disaster monitoring data, which can improve the accuracy and stability of the quality evaluation of geological disaster monitoring data.
[0005] To achieve the above purpose, this application provides the following solutions:
[0006] In the first aspect, this application provides a method for evaluating the quality of geological disaster monitoring data, including:
[0007] Obtain the original time-domain data of geological disaster monitoring;
[0008] Use the improved discrete Fourier transform formula to convert the original time-domain data of geological disaster monitoring into original frequency-domain data;
[0009] Filter the original frequency-domain data using a Butterworth low-pass filter to obtain the filtered frequency-domain data;
[0010] Use the improved inverse discrete Fourier transform formula to convert the filtered frequency-domain data into filtered time-domain data;
[0011] According to the difference value between the original time-domain data and the filtered time-domain data, use the data quality evaluation formula to determine the monitoring data quality evaluation index;
[0012] According to the monitoring data quality evaluation index, determine the quality evaluation result of geological disaster monitoring data.
[0013] Optionally, the improved discrete Fourier transform formula is:
[0014]
[0015] Among them, X(k) is the original frequency-domain data, n is the time-domain data sampling point, N is the total amount of monitoring data, x(n) is the original time-domain data of geological disaster monitoring, i is the imaginary unit, k is the frequency-domain data sampling point, and e is the natural base.
[0016] Optionally, the improved inverse discrete Fourier transform formula is:
[0017]
[0018] Among them, x 1 (n) is the filtered time-domain data of the improved inverse discrete Fourier transform, k c is the cut-off frequency of the Butterworth low-pass filter, and m is the order of the Butterworth low-pass filter.
[0019] Optionally, the difference value between the original time-domain data and the filtered time-domain data is:
[0020]
[0021] Among them, y(n) is the difference value between the original time-domain data and the filtered time-domain data, is the filtered time-domain data.
[0022] Optionally, the data quality evaluation formula is:
[0023]
[0024] Among them, z is the monitoring data quality evaluation index, Q 0.75 [y(n)] is the corresponding value of the first 75% points within the difference value, Q 0.25 [y(n)] is the corresponding value of the first 25% points within the difference value, is the average value of the difference value, Q 0.95 [y(n)] is the corresponding value of the first 95% points within the difference value, Q 0.05 [y(n)] is the corresponding value of the first 5% points within the difference value, std[y(n)] is the standard deviation of the difference value, and a is the standard error value of the monitoring data.
[0025] Optionally, according to the monitoring data quality evaluation index, determine the geological disaster monitoring data quality evaluation result, including:
[0026] When the monitoring data quality evaluation index is greater than or equal to the first threshold and less than the second threshold, determine that the geological disaster monitoring data quality evaluation result is low; the first threshold is less than the second threshold;
[0027] When the quality evaluation index of the monitoring data is greater than or equal to the second threshold and less than the third threshold, it is determined that the quality evaluation result of the geological disaster monitoring data is medium; the second threshold is less than the third threshold.
[0028] When the quality evaluation index of the monitoring data is greater than or equal to the third threshold and less than the fourth threshold, it is determined that the quality evaluation result of the geological disaster monitoring data is high; the third threshold is less than the fourth threshold.
[0029] Optionally, the first threshold is 0; the second threshold is 0.6; the third threshold is 0.9; the fourth threshold is 1.
[0030] In a second aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the above-mentioned geological disaster monitoring data quality evaluation method.
[0031] In a third aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the above-mentioned geological disaster monitoring data quality evaluation method.
[0032] In a fourth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the above-mentioned geological disaster monitoring data quality evaluation method.
[0033] According to the specific embodiments provided by the present application, the following technical effects are disclosed in the present application:
[0034] The present application provides a method, device, medium and product for evaluating the quality of geological disaster monitoring data, which acquires the original time-domain data of geological disaster monitoring collected by a sensor; converts the original time-domain data into original frequency-domain data by using an improved discrete Fourier transform method; filters the original frequency-domain data by using a Butterworth low-pass filter to obtain filtered frequency-domain data; converts the filtered frequency-domain data into filtered time-domain data by using an improved inverse discrete Fourier transform method; and calculates the quality evaluation result of the geological disaster monitoring data by using a data quality evaluation formula according to the difference value distribution between the original time-domain data and the filtered time-domain data. The present invention solves the problem that the quality of geological disaster monitoring data cannot be quickly and effectively evaluated for a long time, and relying on the basic parameters of the sensor and the real-time monitoring data, it can realize the rapid and accurate evaluation of the quality of geological disaster sensor monitoring data, and provide better data decision support for technicians to quickly analyze the reliability of monitoring early warning results and the stability of geological disasters. Description of the Drawings
[0035] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.
[0036] Figure 1 It is a flowchart of a method for evaluating the quality of geological disaster monitoring data provided by an embodiment of the present application.
[0037] Figure 2 It is a curve graph of the original time-domain data for geological disaster monitoring provided by an embodiment of the present application.
[0038] Figure 3 It is a curve graph of the original frequency-domain data after improved discrete Fourier transform provided by an embodiment of the present application.
[0039] Figure 4 It is a curve graph of the time-domain data after filtering by the improved inverse discrete Fourier transform provided by an embodiment of the present application.
[0040] Figure 5 It is a curve graph of the original time-domain data and the time-domain data after filtering provided by an embodiment of the present application.
[0041] Figure 6 It is a schematic structural diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners
[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.
[0043] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the drawings and specific implementation manners.
[0044] In an exemplary embodiment, as Figure 1 shown, a method for evaluating the quality of geological disaster monitoring data is provided, including:
[0045] Step 101: Obtain the original time-domain data of geological disaster monitoring.
[0046] Step 102: Use the improved discrete Fourier transform formula to convert the original time-domain data of geological disaster monitoring into original frequency-domain data.
[0047] Step 103: Filter the original frequency-domain data using a Butterworth low-pass filter to obtain the filtered frequency-domain data.
[0048] Step 104: Use the improved inverse discrete Fourier transform formula to convert the filtered frequency-domain data into filtered time-domain data.
[0049] Step 105: Determine the monitoring data quality evaluation index using the data quality evaluation formula based on the difference value between the original time-domain data and the filtered time-domain data.
[0050] Step 106: Determine the geological disaster monitoring data quality evaluation result based on the monitoring data quality evaluation index.
[0051] The improved discrete Fourier transform formula is:
[0052]
[0053] where X(k) is the original frequency-domain data, n is the time-domain data sampling point, N is the total amount of monitoring data, x(n) is the original time-domain data of geological disaster monitoring, i is the imaginary unit, k is the frequency-domain data sampling point, and e is the natural base.
[0054] The improved inverse discrete Fourier transform formula is:
[0055]
[0056] where x 1 (n) is the filtered time-domain data of the improved inverse discrete Fourier transform, k c is the cut-off frequency of the Butterworth low-pass filter, and m is the order of the Butterworth low-pass filter.
[0057] The difference value between the original time-domain data and the filtered time-domain data is:
[0058]
[0059] where y(n) is the difference value between the original time-domain data and the filtered time-domain data, is the filtered time-domain data.
[0060] The data quality evaluation formula is:
[0061]
[0062] where z is the monitoring data quality evaluation index, Q 0.75 [y(n)] is the corresponding value of the first 75% points within the difference value, Q 0.25 [y(n)] is the corresponding value of the first 25% points within the difference value, is the average value of the difference value, Q 0.95[y(n)] is the corresponding value of the first 95% point within the difference value, Q 0.05 [y(n)] is the corresponding value of the first 5% point within the difference value, std[y(n)] is the standard deviation of the difference value, a is the standard error value of the monitoring data, and a is determined by the parameters of the monitoring instrument.
[0063] According to the monitoring data quality evaluation index, determine the quality evaluation result of the geological disaster monitoring data, including:
[0064] When the monitoring data quality evaluation index is greater than or equal to the first threshold and less than the second threshold, determine that the quality evaluation result of the geological disaster monitoring data is low. The first threshold is less than the second threshold.
[0065] When the monitoring data quality evaluation index is greater than or equal to the second threshold and less than the third threshold, determine that the quality evaluation result of the geological disaster monitoring data is medium. The second threshold is less than the third threshold.
[0066] When the monitoring data quality evaluation index is greater than or equal to the third threshold and less than the fourth threshold, determine that the quality evaluation result of the geological disaster monitoring data is high. The third threshold is less than the fourth threshold.
[0067] The first threshold is 0. The second threshold is 0.6. The third threshold is 0.9. The fourth threshold is 1. When 0 ≤ z < 0.6, the quality evaluation result of the geological disaster monitoring data is low, and the reliability of the monitoring early warning and forecasting result is low. When 0.6 ≤ z < 0.9, the quality evaluation result of the geological disaster monitoring data is medium, and the reliability of the monitoring early warning and forecasting result is average. When 0.9 ≤ z ≤ 1, the quality evaluation result of the geological disaster monitoring data is high, and the reliability of the monitoring early warning and forecasting result is high.
[0068] In an exemplary embodiment, taking the GNSS displacement monitoring data of a certain landslide disaster as an example, a method for evaluating the quality of geological disaster monitoring data includes:
[0069] S1: Obtain the original time-domain data within a certain interval of the GNSS displacement monitoring, and draw a curve as Figure 2 shown, and there are obvious up and down fluctuations in the monitoring curve.
[0070] S2: Adopt an improved discrete Fourier transform method to convert the original time-domain data x(n) into the original frequency-domain data X(k).
[0071] The improved discrete Fourier transform method is:
[0072]
[0073] Among them, N is the total amount of monitoring data, which is 1000 here; n is the time-domain data sampling point; k is the frequency-domain data sampling point; i is the imaginary unit.
[0074] The original frequency-domain data contains amplitude and phase information, and the plotted curve is as Figure 3 shown.
[0075] S3: Use a Butterworth low-pass filter to filter the original frequency-domain data to obtain the filtered frequency-domain data;
[0076] S4: Use an improved inverse discrete Fourier transform method to convert the filtered frequency-domain data into the filtered time-domain data x 1 (n).
[0077] The improved inverse discrete Fourier transform method is:
[0078]
[0079] Among them: k c is the cut-off frequency of the Butterworth low-pass filter, which is 200 here; m is the order of the Butterworth low-pass filter, which is 10 here.
[0080] The filtered time-domain data x 1 (n) is plotted as shown in Figure 4 shown; The curve of the original time-domain data x(n) and the curve of the filtered time-domain data x 1 (n) are compared as shown in Figure 5 shown.
[0081] S5: According to the distribution of the difference value y(n) between the original time-domain data x(n) and the filtered time-domain data x 1 (n); Use the data quality evaluation formula to calculate the geological disaster monitoring data quality evaluation result.
[0082] The calculation method of the difference value is:
[0083]
[0084] The data quality evaluation formula is:
[0085]
[0086] Among them, Q 0.05 [y(n)] is the corresponding value at the 5% point from the end of the difference value, which is -1.1036 here; Q 0.25 [y(n)] is the corresponding value at the 25% point from the end of the difference value, which is -0.3738 here; Q 0.75 [y(n)] is the corresponding value at the 25% point from the beginning of the difference value, which is 0.3561 here; Q 0.95[y(n)] is the corresponding value at the top 5% point within the difference value, which is 1.0537 here; is the average value of the difference value, which is 0.0024 here; std[y(n)] is the standard deviation of the difference value, which is 0.6998 here; a is the standard error value of the monitoring data, which is 1 here.
[0087] The calculated result of the GNSS displacement monitoring data quality evaluation index z is 0.8092. Therefore, the quality evaluation result of this geological disaster monitoring data is medium, and the reliability of the monitoring early warning and forecasting results is average.
[0088] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for evaluating the quality of geological disaster monitoring data.
[0089] Those skilled in the art can understand that Figure 6 the structure shown in
[0090] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.
[0091] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.
[0092] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.
[0093] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.
[0094] The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.
[0095] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.
[0096] In this text, specific examples are used to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. At the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.
Claims
1. A method for evaluating the quality of geological disaster monitoring data, characterized in that: include: Obtain original time domain data for geological disaster monitoring; The original time domain data of geological disaster monitoring is converted into original frequency domain data by using the improved discrete Fourier transform formula; Filtering the original frequency domain data using a Butterworth low-pass filter to obtain filtered frequency domain data; The filtered frequency domain data is converted into filtered time domain data using the improved inverse discrete Fourier transform formula; According to the difference between the original time domain data and the filtered time domain data, the monitoring data quality evaluation index is determined using the data quality evaluation formula; Determine the geological disaster monitoring data quality evaluation results based on the monitoring data quality evaluation index.
2. The method for evaluating the quality of geological disaster monitoring data according to claim 1, characterized in that: The improved discrete Fourier transform formula is: Among them, X(k) is the original frequency domain data, n is the time domain data sampling point, N is the total amount of monitoring data, x(n) is the original time domain data of geological disaster monitoring, i is the imaginary unit, k is the frequency domain data sampling point, and e is the natural base.
3. The method for evaluating the quality of geological disaster monitoring data according to claim 2, characterized in that: The improved inverse discrete Fourier transform formula is: Among them, x1(n) is the filtered time domain data of the improved inverse discrete Fourier transform, k c is the cutoff frequency of the Butterworth low-pass filter, and m is the order of the Butterworth low-pass filter.
4. The method for evaluating the quality of geological disaster monitoring data according to claim 3, characterized in that: The difference between the original time domain data and the filtered time domain data is: Among them, y(n) is the difference between the original time domain data and the filtered time domain data, is the filtered time domain data.
5. The method for evaluating the quality of geological disaster monitoring data according to claim 4, characterized in that: The data quality evaluation formula is: Among them, z is the monitoring data quality evaluation index, Q 0.75 [y(n)] is the corresponding value of the first 75% points within the difference value, Q 0.25 [y(n)] is the corresponding value of the first 25% points within the difference value, is the average value of the difference, Q 0.95 [y(n)] is the corresponding value of the first 95% points within the difference value, Q 0.05 [y(n)] is the corresponding value of the first 5% points within the difference value, std[y(n)] is the standard deviation of the difference value, and a is the standard error value of the monitoring data.
6. The method for evaluating the quality of geological disaster monitoring data according to claim 1, characterized in that: According to the monitoring data quality evaluation index, the geological disaster monitoring data quality evaluation results are determined, including: When the monitoring data quality evaluation index is greater than or equal to a first threshold value, and the monitoring data quality evaluation index is less than a second threshold value, it is determined that the geological disaster monitoring data quality evaluation result is low; the first threshold value is less than the second threshold value; When the monitoring data quality evaluation index is greater than or equal to the second threshold value, and the monitoring data quality evaluation index is less than the third threshold value, the geological disaster monitoring data quality evaluation result is determined to be medium; the second threshold value is less than the third threshold value; When the monitoring data quality evaluation index is greater than or equal to the third threshold and the monitoring data quality evaluation index is less than the fourth threshold, the geological disaster monitoring data quality evaluation result is determined to be high; the third threshold is less than the fourth threshold.
7. The method for evaluating the quality of geological disaster monitoring data according to claim 6, characterized in that: The first threshold is 0; the second threshold is 0.6; the third threshold is 0.9; and the fourth threshold is 1.
8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the geological disaster monitoring data quality evaluation method described in any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for evaluating the quality of geological disaster monitoring data described in any one of claims 1 to 7 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for evaluating the quality of geological disaster monitoring data described in any one of claims 1 to 7 is implemented.