A data quality detection method and system based on time series data

Abstract

The application relates to the technical field of data detection, and discloses a data quality detection method and system based on time sequence data. In a preset time period, data is acquired at fixed intervals, the time difference between adjacent data points is calculated, and the calculated time difference is compared with a set interval, so that it is judged whether the data is continuous and missing conditions are found in time, and the integrity and reliability of the overall acquisition process are ensured. Secondly, when the data is confirmed to be complete, the mean value of the data is calculated according to historical records, the absolute deviation of each data value from the mean value is detected, the data is divided into two categories of normal and abnormal according to a previously set deviation threshold, for the abnormal data, if the number of the abnormal data does not exceed an abnormal threshold, the mean value of the front and rear data is used for correction, the influence of the abnormal data on the overall data set is reduced, otherwise, the abnormality is considered to be serious, and data needs to be reacquired to ensure accuracy.

Description

Technical Field

[0001] This invention relates to the field of data inspection technology, specifically to a data quality inspection method and system based on time-series data. Background Technology

[0002] During the acquisition process, time series data may be affected by various factors such as equipment failure, sensor error, network transmission delay and external environmental interference, resulting in data loss, errors or abnormal fluctuations. Failure to detect and correct these problems in a timely manner may lead to deviations in data analysis results or even cause incorrect decisions.

[0003] In applications such as industrial monitoring, financial transactions, and weather forecasting, the real-time nature and accuracy of data are particularly critical. Any minor anomaly may lead to untimely system response or increased prediction errors, thereby affecting the stable operation of the entire process.

[0004] Time series data records the continuous state of a system or device over time. Its data quality directly affects the accuracy and reliability of subsequent analysis, prediction, and decision-making. Therefore, quality inspection of time series data is of great significance. Summary of the Invention

[0005] In view of the problems existing in the prior art, the purpose of the present invention is to provide a data quality detection method and system based on time series data, so as to detect the data quality of time series data.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a data quality detection method based on time-series data, comprising: Step 1, setting a time period and a time interval, acquiring data at fixed time intervals within the time period to obtain multiple time-series data, and acquiring the time difference between every two adjacent time-series data based on the multiple time-series data; Step 2, comparing the time difference with the time interval, if the time difference equals the time interval, it means there are no missing data between two adjacent time-series data; if the time difference is greater than the time interval, it means there are missing data between two adjacent time-series data; Step 3, counting the number of missing data between two adjacent time-series data, if the number of missing data is zero, it means the time-series data within the time period is complete and there are no omissions; if the number of missing data is greater than zero, it means the time-series data within the time period is incomplete and there are omissions; Step 4, under the premise that the time-series data is complete, acquiring the time-series data value, and acquiring the time-series data mean based on historical records, subtracting the time-series data mean from the time-series data value, and then processing it to obtain the time difference. Step 5: Set a threshold for time series data deviation. Compare the deviation of each time series data value with the threshold. If the deviation is less than or equal to the threshold, it means the deviation is within a reasonable range, and the data value is marked as normal. If the deviation is greater than the threshold, it means the deviation exceeds a reasonable range, and the data value is marked as abnormal. Step 6: Obtain the number of abnormal time series data values ​​and set an abnormality threshold. Compare the number of abnormal data values ​​with the threshold. If the number is less than or equal to the threshold, it means the abnormality is not severe within the time period, and the average of the two preceding and following data values ​​is used to replace each abnormal data value. If the number is greater than the threshold, it means the abnormality is severe within the time period, and the time series data is re-obtained.

[0007] In some implementations, the time interval between two adjacent abnormal time-series data values ​​is obtained and recorded as the abnormal time interval. When the number of abnormalities is less than or equal to the abnormal number threshold, the abnormal time interval is compared with the time interval, and different responses are derived based on the comparison results.

[0008] In some implementations, if the abnormal time interval is equal to the time interval, it means that the two adjacent abnormal time series data values ​​are continuous, and the abnormality is determined to be severe, and the time series data is reacquired; if the abnormal time interval is greater than the time interval, it means that the two adjacent abnormal time series data values ​​are not continuous, and the abnormality is maintained as not severe.

[0009] In some implementations, when the time series data within a time period is complete and the time series data values ​​are normal, all time series data values ​​are acquired again, and it is determined whether there are duplicate time series data values ​​among all the time series data values. Different responses are given based on the determination results.

[0010] In some implementations, if there are no duplicate time series data values, the time series data is kept intact and the time series data values ​​are normal; if there are duplicate time series data values, the number of duplicate time series data value groups is obtained, and at the same time, a threshold for the number of duplicate time series data value groups is set according to historical records. The number of duplicate time series data value groups is compared with the threshold for the number of duplicate time series data value groups, and different responses are given based on the comparison results.

[0011] In some implementations, if the number of repeated time series data sets is less than or equal to the threshold for the number of repeated time series data sets, it means that the number of repeated sets is within the normal range and does not affect the quality of the time series data; if the number of repeated time series data sets is greater than the threshold for the number of repeated time series data sets, it means that the number of repeated sets exceeds the normal range and affects the quality of the time series data, and then the repetition rate is further analyzed.

[0012] In some implementations, the specific process of further analyzing the repetition rate is to obtain the number of repetitive time-series data values, divide the number of repetitive time-series data values ​​by the total number of time-series data values ​​to obtain the repetition rate, set a repetition rate threshold, compare the repetition rate with the repetition rate threshold, and derive different responses based on the comparison results.

[0013] In some implementations, if the repetition rate is less than or equal to the repetition rate threshold, it means that the proportion of repetitive time series data values ​​is low, and the mean correction method is used to correct one of the repetitive time series data values; if the repetition rate is greater than the repetition rate threshold, it means that the proportion of repetitive time series data values ​​is high, and the time series data is reacquired.

[0014] This invention further provides a data quality detection system based on time-series data, which is used to execute the above-described method, comprising: a time difference acquisition module, which sets a time period and a time interval, acquires data at fixed time intervals within the time period to obtain multiple time-series data, and acquires the time difference between every two adjacent time-series data based on the multiple time-series data; an initial comparison module, which compares the time difference with the time interval, and if the time difference equals the time interval, it means that there are no missing data between two adjacent time-series data; if the time difference is greater than the time interval, it means that there are missing data between two adjacent time-series data; a statistical analysis module, which counts the number of missing data between two adjacent time-series data, and if the number of missing data is zero, it means that the time-series data within the time period is complete and there are no omissions; if the number of missing data is greater than zero, it means that the time-series data within the time period is incomplete and there are omissions; and a deviation acquisition module, which, under the premise that the time-series data is complete, acquires the time-series data values, and acquires the average value of the time-series data according to historical records, and after subtracting the average value from the time-series data values, passes the deviation acquisition module for positive values. The system first obtains the deviation of time-series data values. Then, a comparison module sets a threshold for this deviation and compares each deviation with that threshold. If the deviation is less than or equal to the threshold, it's within a reasonable range, and the data value is marked as normal. If the deviation exceeds the threshold, it's out of range, and the data value is marked as abnormal. Next, an anomaly analysis module obtains the number of abnormal data values ​​and sets an anomaly threshold. It compares the number of anomalies with this threshold. If the number is less than or equal to the threshold, the anomaly is not severe, and the average of the two preceding and following data values ​​is used to replace each abnormal value. If the number exceeds the threshold, the anomaly is severe, and the time-series data is re-obtained.

[0015] The present invention further provides a computer-readable storage medium storing a computer program, which is executed by a processor to implement the above-described data quality detection method based on time-series data.

[0016] The technical solution provided by this invention has the following advantages compared with the prior art:

[0017] This invention ensures that each stage of time-series data acquisition, correction, and repetition detection meets the expected quality standards through a multi-level detection process. First, data is acquired at fixed intervals within a preset time period. By calculating the time difference between adjacent data points and comparing it with the set interval, the continuity of the data is determined, and missing data is detected in a timely manner, ensuring the integrity and reliability of the overall acquisition process. Second, when the data is confirmed to be complete, the average data value is calculated based on historical records, and the absolute deviation of each data value from the average value is detected. Combined with a pre-set deviation threshold, the data is divided into normal and abnormal categories. For abnormal data, if the number does not exceed the abnormal threshold, the average of the preceding and following data is used for correction to reduce the impact of the abnormality on the overall data. Otherwise, it is considered a serious abnormality, and data needs to be re-acquired to ensure accuracy. Finally, after the overall data is deemed complete and the values ​​are normal (after necessary correction), the data is checked again for duplicate values. By counting the number of duplicate data groups and calculating the duplication rate, and comparing these indicators with predefined thresholds, it is determined whether the duplication phenomenon is within a reasonable range. If the number of duplicate groups or the duplication rate exceeds the normal threshold, a correction method is used for some duplicate data or the data is directly re-acquired, depending on the severity. Overall, this detection process not only helps to detect missing and abnormal data in the data collection process early and improve data quality, but also ensures that time-series data maintains high accuracy and reliability at critical moments by correcting duplicate data in a timely manner or re-collecting it. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the thought process of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It is understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.

[0021] The data quality detection method based on time-series data provided by this invention, such as... Figure 1 As shown, it includes:

[0022] The first step involves setting a predetermined time period and a data acquisition interval. Starting from the beginning of the time period, data is acquired at fixed intervals until the end of the time period, resulting in multiple time-series data points for that period. The time difference between each pair of adjacent time-series data points is then acquired and compared to the time interval. Different actions are taken based on the comparison results. If the time difference equals the time interval, there are no missing data points between adjacent time-series data points; if the time difference is greater than the time interval, there are missing data points between adjacent time-series data points. The number of missing data points between adjacent time-series data points is counted. If the number of missing data points is zero, the time-series data within the time period is complete and without omissions; if the number of missing data points is greater than zero, the time-series data within the time period is incomplete and contains omissions.

[0023] The second step involves obtaining the specific value of each time series data point once it's determined that the data is complete within a given time period. This value is recorded as the time series data value. The average value of these values ​​is then calculated based on historical records and recorded as the time series data mean. The deviation of each time series data point is obtained by subtracting the mean from the average value and performing absolute value processing. Furthermore, a time series data deviation threshold is set. The deviation of each time series data point is compared to this threshold, and different actions are taken based on the comparison results. If the deviation is less than or equal to the threshold, it indicates that the deviation is within a reasonable range, and the corresponding time series data point is marked as a normal time series data point. If the deviation exceeds the threshold, it indicates that the deviation exceeds a reasonable range, and the corresponding time series data point is marked as an abnormal time series data point.

[0024] The third step involves obtaining the number of abnormal time-series data values, denoted as the "abnormality count." Simultaneously, an abnormality count threshold is set based on the total number of all time-series data values ​​within a given time period. An abnormality count exceeding this threshold indicates a significant number of anomalies, rendering the time-series data within that period unusable. Under this condition, the abnormality count is compared to the abnormality count threshold, and different responses are determined based on the comparison results. If the abnormality count is less than or equal to the threshold, the abnormality of the time-series data within the time period is not severe. In this case, for each abnormal time-series data value, the average of its two preceding and following time-series data values ​​is used to replace it. If the abnormality count exceeds the threshold, the abnormality of the time-series data within the time period is severe. In this case, the time-series data is re-acquired.

[0025] For example, assuming a time period of 1 hour and a data acquisition interval of 10 minutes, the plan is to collect data at 00:00, 00:10, 00:20, 00:30, 00:40, 00:50, and 01:00 (7 times in total). In the first step, if the actual recorded times are 00:00, 00:10, 00:20, 00:40, 00:50, and 01:00, the time difference between adjacent records can be calculated. It is found that the interval between 00:20 and 00:40 is 20 minutes (greater than the expected 10 minutes), indicating a missing data point. The number of missing data points in this time period is 1. Moving to the second step, when judging the selected data records within this period (assuming only numerical detection is performed on the complete data period or missing parts are ignored), the specific value at each time point is obtained, for example, the recorded values ​​are 10, 12, and 11. The average of the data points 30, 13, and 12, calculated from historical data, is 16. The absolute deviation of each data point from 16 is then calculated, yielding deviation values ​​of 6, 4, 5, 14, 3, and 4. A time-series data deviation threshold of 5 is set. Deviations greater than 5 (i.e., 6 and 14) are marked as abnormal data, while deviations less than or equal to 5 are considered normal. The third step involves counting the number of abnormal data points. In this example, the number of abnormal data points is 2. Assuming the total number of data points in this period is 6 and the abnormal number threshold is set to 2, the number of abnormal data points equals the threshold, indicating the abnormality is not severe. At this point, each abnormal data point can be corrected using the average of the data points before and after it. For example, the abnormal data point represented by deviation 14 is 30, and the average of the two data points before and after it is (11+13) / 2=12. If the number of abnormal data points exceeds the threshold, it indicates a severe data abnormality, and the time-series data for this period needs to be reacquired.

[0026] When the number of anomalies is less than or equal to the anomaly threshold, the time interval between two adjacent abnormal time-series data values ​​is recorded as the abnormal time interval. This abnormal time interval is compared to the time interval, and different responses are determined based on the comparison results. If the abnormal time interval equals the time interval, it means the two adjacent abnormal time-series data values ​​are continuous. In this case, the anomaly is considered severe, and time-series data needs to be acquired again. If the abnormal time interval is greater than the time interval, it means the two adjacent abnormal time-series data values ​​are discontinuous. In this case, the anomaly is maintained at a less severe level. For example, if two adjacent abnormal data points appear at 00:20 and 00:30 respectively, the abnormal time interval between them is 10 minutes, equal to the expected time interval. This indicates that these two abnormal data points appear continuously, suggesting a relatively severe anomaly, and time-series data needs to be acquired again. However, if the abnormal data points appear at 00:20 and 00:40 respectively, with an interval of 20 minutes, exceeding the preset 10 minutes, it indicates that the abnormal data is discontinuous, and the anomaly is still not severe. The existing abnormal state is maintained, and this part of the data can be further corrected.

[0027] When the time-series data for a given time period is complete and the data values ​​are normal (non-serious abnormal data values ​​are considered normal after correction), all time-series data values ​​are retrieved again, and it is determined whether there are duplicate data values. If there are no duplicate data values, the time-series data remains complete and the values ​​are considered normal. If there are duplicate data values, the number of duplicate data value groups is determined. Simultaneously, a threshold for the number of duplicate data value groups is set based on historical records. The number of duplicate data value groups is compared with this threshold, and different actions are taken based on the comparison result. If the number of duplicate data value groups is less than or equal to the threshold, it means the number of duplicate groups is within the normal range and does not affect the quality of the time-series data. If the number of repeated time-series data sets exceeds the threshold, it indicates that the number of repeated sets exceeds the normal range, affecting the quality of the time-series data. In this case, the number of repeated time-series data sets is obtained, and the repetition rate is calculated by dividing the number of repeated time-series data sets by the total number of time-series data sets. A repetition rate threshold is set, and the repetition rate is compared with the threshold. Different actions are taken based on the comparison result. If the repetition rate is less than or equal to the threshold, it indicates that the proportion of repeated time-series data sets is low. In this case, the correction method described above is used to correct one of the repeated time-series data sets. If the repetition rate is greater than the threshold, it indicates that the proportion of repeated time-series data sets is high. In this case, the time-series data is reacquired. For example, if the threshold for the number of duplicate time-series data sets is set to 3, and the actual number of duplicate sets detected does not exceed 3, then these duplicates are considered to be within the normal range and will not significantly affect data quality. Conversely, if the number of duplicate sets exceeds 3, the severity of the duplicates needs to be further assessed. This involves counting all duplicate values ​​and dividing that number by the total number of time-series data sets in the entire period to obtain the duplicate percentage. Assuming a total of 100 data points, if 20 duplicate data points are detected, the duplicate percentage is 20%. Assuming the duplicate percentage threshold is set to 25%, a duplicate percentage of 20% is less than the threshold of 25%. The duplicate data is adjusted using the average correction method for data from previous and subsequent time points. If the duplicate percentage is 30%, exceeding 25%, then a high proportion of duplicate data is considered to have affected the accuracy and reliability of the time-series data. In this case, the data quality should be considered to have been significantly affected, and the time-series data needs to be acquired again.

[0028] In summary, the process begins by setting a fixed time period and data acquisition interval. Data is acquired at these intervals within the period, and the time difference between adjacent data points is calculated. If the time difference equals the set interval, the data is considered continuous; otherwise, it is considered missing. The number of missing data points is counted to determine data integrity. Assuming data integrity, the values ​​at each moment are checked. First, the overall mean is calculated based on historical data. Then, the absolute deviation of each data point from the mean is calculated, and a deviation threshold is set. Data with deviations within a reasonable range are marked as normal, while those exceeding the threshold are considered abnormal. Next, the number of abnormal data points is counted and compared with a preset abnormality threshold. If the number of abnormal data points does not exceed the threshold, the data is considered continuous. The average of the two points before and after the outlier is used for correction. At the same time, the time interval between adjacent outlier data is checked. If they occur consecutively, the outlier is serious and needs to be re-collected. Otherwise, the correction continues. When the data is considered complete and normal after the above processing, the full-cycle data is acquired again and the presence of duplicate values ​​is checked. If there are no duplicates, the status is maintained. If there are duplicates, the number of duplicate groups is counted and compared with the number of duplicate groups threshold. If the number of groups exceeds the threshold, the ratio of the number of duplicate data to the total number of data is calculated and compared with the set threshold. If the ratio is low, a correction method is used to adjust one of the data groups. Otherwise, if the ratio of duplicates is high, the data quality is deemed to be damaged and data needs to be re-collected.

[0029] The processes described above with reference to the flowcharts in the embodiments disclosed in this invention can be implemented as computer software programs. Embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having one or more wire segments, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless segments, wire segments, optical cables, RF, etc., or any suitable combination thereof.

[0030] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0031] Those skilled in the art should understand that the above description is only a specific embodiment of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application.

Claims

1. A data quality detection method based on time-series data, characterized in that, include: Step 1: Set the time period and time interval. Acquire data at fixed time intervals within the time period to obtain multiple time series data. Based on the multiple time series data, obtain the time difference between every two adjacent time series data. Step 2: Compare the time difference with the time interval. If the time difference equals the time interval, it means that there is no missing data between two adjacent time series data. If the time difference is greater than the time interval, it means that there is a missing data between two adjacent time series data. Step 3: Count the number of missing data between two adjacent time series data. If the number of missing data is zero, it means that the time series data within the time period is complete and there are no omissions. If the number of missing data is greater than zero, it means that the time series data within the time period is incomplete and there are omissions. Step 4: Provided that the time series data is complete, obtain the time series data values ​​and the average time series data value based on historical records. Subtract the average time series data value from the time series data value and then process it into a positive value to obtain the time series data value deviation. Step 5: Set a time series data deviation threshold. Compare the deviation of each time series data value with the time series data deviation threshold. If the deviation is less than or equal to the threshold, it means the deviation is within a reasonable range, and the time series data value it represents is marked as normal. If the deviation is greater than the threshold, it means the deviation exceeds a reasonable range, and the time series data value it represents is marked as abnormal. Step 6: Obtain the number of abnormal time series data values ​​and set an abnormal number threshold. Compare the abnormal number with the abnormal number threshold. If the abnormal number is less than or equal to the abnormal number threshold, it means that the abnormality of the time series data values ​​within the time period is not serious. Then, for each abnormal time series data value, use the average of the two time series data values ​​before and after it to replace it. If the number of anomalies exceeds the anomaly threshold, it indicates severe anomalies in the time series data within the time period, and the time series data is reacquired. The time interval between two adjacent anomaly time series data values ​​is recorded as the anomaly time interval. When the number of anomalies is less than or equal to the anomaly threshold, the anomaly time interval is compared with the time interval, and different actions are taken based on the comparison results. If the anomaly time interval equals the time interval, it indicates that the two adjacent anomaly time series data values ​​are continuous, and the anomaly is determined to be severe, and the time series data is reacquired. If the anomaly time interval is greater than the time interval, it indicates that the two adjacent anomaly time series data values ​​are not continuous, and the anomaly is maintained. If the situation is not serious, and the time series data within a time period is complete and the time series data values ​​are normal, then all time series data values ​​are retrieved again, and it is determined whether there are duplicate time series data values. Different actions are taken based on the determination result. If there are no duplicate time series data values, then the time series data is kept complete and the time series data values ​​are normal. If there are duplicate time series data values, then the number of duplicate time series data value groups is retrieved. At the same time, a threshold for the number of duplicate time series data value groups is set according to historical records. The number of duplicate time series data value groups is compared with the threshold for the number of duplicate time series data value groups, and different actions are taken based on the comparison result.

2. The data quality detection method based on time-series data according to claim 1, characterized in that, If the number of repeated time series data sets is less than or equal to the threshold for the number of repeated time series data sets, it means that the number of repeated sets is within the normal range and does not affect the quality of the time series data. If the number of repeated time series data sets is greater than the threshold for the number of repeated time series data sets, it means that the number of repeated sets exceeds the normal range and affects the quality of the time series data. In this case, further analysis of the repetition rate is needed.

3. The data quality detection method based on time-series data according to claim 2, characterized in that, Further analysis of the repetition rate involves obtaining the number of repetitive time-series data values, dividing the number of repetitive time-series data values ​​by the total number of time-series data values ​​to obtain the repetition rate, setting a repetition rate threshold, comparing the repetition rate with the repetition rate threshold, and determining different responses based on the comparison results.

4. The data quality detection method based on time-series data according to claim 3, characterized in that, If the repetition rate is less than or equal to the repetition rate threshold, it means that the proportion of repetitive time series data values ​​is low. In this case, the mean correction method is used to correct one of the repetitive time series data values. If the repetition rate is greater than the repetition rate threshold, it means that the proportion of repetitive time series data values ​​is high. In this case, the time series data is reacquired.

5. A data quality inspection system based on time-series data, used to perform the method described in any one of claims 1-4, characterized in that, include: The time difference acquisition module sets a time period and a time interval, acquires data at fixed time intervals within the time period, obtains multiple time series data, and acquires the time difference between every two adjacent time series data based on the multiple time series data. The initial comparison module compares the time difference with the time interval. If the time difference equals the time interval, it means there are no missing data between two adjacent time series data. If the time difference is greater than the time interval, it means there are missing data between two adjacent time series data. The statistical analysis module counts the number of missing data points between two adjacent time series data points. If the number of missing data points is zero, it means that the time series data within the time period is complete and there are no omissions. If the number of missing data points is greater than zero, it means that the time series data within the time period is incomplete and there are omissions. The deviation acquisition module acquires the time series data values ​​provided that the time series data is complete, and obtains the average value of the time series data based on historical records. The deviation of the time series data values ​​is obtained by subtracting the average value of the time series data from the time series data values ​​and then processing it as a positive value. The comparison module sets a threshold for time series data deviation. It compares the deviation of each time series data value with the threshold. If the deviation is less than or equal to the threshold, it means the deviation is within a reasonable range, and the time series data value it represents is marked as normal. If the deviation is greater than the threshold, it means the deviation exceeds a reasonable range, and the time series data value it represents is marked as abnormal. The anomaly analysis module acquires the number of abnormal time series data values ​​and sets an anomaly number threshold. It compares the number of anomalies with the anomaly number threshold. If the number of anomalies is less than or equal to the anomaly number threshold, it means that the time series data values ​​within the time period are not seriously abnormal. For each abnormal time series data value, the average of the two time series data values ​​before and after it is used to replace it. If the number of anomalies exceeds the anomaly threshold, it indicates that the time series data values ​​within the time period are severely abnormal, and the time series data should be reacquired.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a processor to implement a data quality detection method based on time-series data as described in any one of claims 1-4.

Images