Satellite telemetry data anomaly detection method based on DTW
Through the DTW-based satellite telemetry data abnormal detection method, the problem of failure to effectively detect and optimize satellite telemetry data in the prior art is solved, real-time data monitoring and abnormal detection are improved, and data accuracy and efficiency are improved.
Patent Information
- Application Number
- CN202411148244.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-21
AI Technical Summary
The prior art has failed to effectively detect, proofread and optimize the abnormality of satellite telemetry data, resulting in inaccurate data and low detection efficiency.
Adopt the abnormal detection method of satellite telemetry data based on DTW, including real-time acquisition, preprocessing, time series processing, DTW threshold setting, sliding detection, abnormal identification and post-processing, and optimize the DTW threshold and processing through simulation models. Strategy.
It improves the accuracy and efficiency of satellite telemetry data, realizes real-time monitoring of data, quality improvement, and abnormal detection sensitivity and coverage, and enhances the overall reliability and adaptability of data.
Smart Images

Figure CN119047176A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of remote sensing satellite data processing, and in particular to a satellite telemetry data anomaly detection method based on DTW. Background Art
[0002] Satellite telemetry data anomaly detection uses technical means to analyze satellite data, identify and process anomalies in the data, and ensure that the data is accurate and reliable. However, existing technologies often face problems such as failure to perform anomaly detection, lack of proofreading optimization and post-processing steps, resulting in inaccurate data, low detection efficiency, and inability to quickly process large amounts of data. Dynamic Time Warping (DTW) is an algorithm that measures the similarity of time series and is suitable for satellite telemetry data with complex spatiotemporal patterns. Using DTW for anomaly detection can improve detection accuracy, process complex patterns, meet real-time processing requirements, and enhance data reliability through subsequent proofreading and optimization. Therefore, the application of DTW in satellite telemetry data anomaly detection can overcome the shortcomings of existing technologies, improve the accuracy and efficiency of data processing, and provide guarantees for efficient use of data.
[0003] Chinese Patent Publication No.: CN104915568B discloses a satellite telemetry data anomaly detection method based on DTW, which relates to the field of satellite telemetry data anomaly detection. The satellite telemetry data is segmented with the amplitude angle mutation point as the mark, and then the mean DTW distance within each category in the time series containing the category label is calculated, and the time series x' to be detected for anomaly is obtained; the DTW distance between x' and the time series X containing the category label is calculated, and the minimum DTW distance dmin between x' and the quasi-class l' is determined, and whether x' is an abnormal sequence is determined based on the size relationship between dmin and. However, this scheme does not perform anomaly detection on satellite telemetry data, and does not perform proofreading, optimization and post-processing on satellite telemetry data anomaly detection, and cannot improve the efficiency of satellite telemetry data anomaly detection. Summary of the invention
[0004] To this end, the present invention provides a satellite telemetry data anomaly detection method based on DTW to overcome the problems in the prior art of inaccurate satellite telemetry data caused by failure to perform anomaly detection on satellite telemetry data, and failure to perform proofreading, optimization and post-processing on anomaly detection of satellite telemetry data, and low efficiency of anomaly detection of satellite telemetry data.
[0005] To achieve the above object, the present invention provides a satellite telemetry data anomaly detection method based on DTW, comprising:
[0006] Step S1, acquiring satellite telemetry data in real time to obtain real-time satellite telemetry data;
[0007] Step S2, preprocessing the real-time satellite telemetry data to obtain preprocessed satellite telemetry data;
[0008] Step S3, performing time series processing on the pre-processed satellite telemetry data to obtain time series satellite telemetry data;
[0009] Step S4, setting the DTW threshold according to normal data;
[0010] Step S5, inputting the historical data into the simulation model for abnormal analysis, and performing historical compensation on the normal data according to the abnormal analysis results of the historical data;
[0011] Step S6, inputting the real-time satellite telemetry data into the simulation model for abnormality analysis, and performing real-time compensation for normal data according to the abnormality analysis results of the real-time satellite telemetry data;
[0012] Step S7, optimizing the DTW threshold setting process according to the historical compensation times, the real-time compensation times and the normal data volume after compensation;
[0013] Step S8, performing sliding detection on the time series satellite telemetry data according to the DTW threshold;
[0014] Step S9, performing abnormal identification on the time series satellite telemetry data according to the sliding detection result to obtain abnormal satellite telemetry data;
[0015] Step S10, post-processing the abnormal satellite telemetry data according to the auxiliary identification data;
[0016] Step S11, identifying a key time period according to the number of abnormal times in the time period, and optimizing the time series processing process according to the key time period;
[0017] Step S12, performing compensation judgment on the key time period according to the decrease value of the number of abnormalities, and adjusting the optimization process of the time series processing process according to the compensation judgment result.
[0018] Furthermore, in step S2, when preprocessing the real-time satellite telemetry data, missing values and noise data in the real-time satellite telemetry data are processed to obtain cleaned data, and data standardization is performed on the cleaned data to obtain preprocessed satellite telemetry data.
[0019] Furthermore, in step S3, when performing time series processing on the preprocessed satellite telemetry data, the preprocessed satellite telemetry data is divided into time intervals according to preset unit time intervals, and the preprocessed satellite telemetry data after the time interval division are constructed into time series satellite telemetry data in chronological order.
[0020] Furthermore, in step S4, when setting the DTW threshold, the sum of the mean of normal data and a preset multiple of the standard deviation is calculated and used as the DTW threshold to be verified. The abnormal satellite telemetry data and the normal satellite telemetry data are cross-validated with the DTW threshold to be verified, and the cross-validation result is obtained. When the cross-validation result does not meet the verification condition, the preset multiple of the standard deviation is incrementally adjusted, and the DTW threshold to be verified is recalculated until the cross-validation result meets the verification condition and the DTW threshold to be verified is set as the DTW threshold.
[0021] Further, in the step S5, when performing historical compensation on normal data, historical data is acquired every other historical compensation cycle, and the historical data is input into the simulation model, normal data output by the simulation model is acquired, and the data is used as the abnormal analysis result of the historical data, and whenever the data volume of the abnormal analysis result of the historical data reaches a preset data volume, the normal data within the data volume of the abnormal analysis result of the historical data is compensated to the normal data of step S4, and the data volume of the abnormal analysis result of the historical data is cleared and recounted;
[0022] In step S6, when performing real-time compensation on normal data, real-time data is acquired every real-time compensation cycle, and the real-time data is input into the simulation model, normal data output by the simulation model is acquired, and used as the abnormal analysis result of the real-time data, and whenever the data volume of the abnormal analysis result of the real-time data reaches a preset data volume, the normal data within the data volume of the abnormal analysis result of the real-time data is compensated to the normal data of step S4, and the data volume of the abnormal analysis result of the real-time data is cleared and recounted.
[0023] Further, in the step S7, the DTW threshold optimization index mr is calculated according to the historical compensation times n1, the real-time compensation times n2, the DTW threshold iteration times n3 and the normal data volume nr after compensation, and mr=0.5×[(n1+n2+n3) / n0]+0.5×[nr / nr0] is set, n0 is a preset times parameter, nr0 is a preset data volume parameter, and the optimization index mr is compared with the preset optimization index mr0, and a judgment is made according to the optimization of the DTW threshold setting process, wherein:
[0024] When mr≤mr0, it is determined that the DTW threshold setting process is not optimized;
[0025] When mr>mr0, it is determined that the setting process of the DTW threshold is optimized, and the setting process of the DTW threshold is optimized until the DTW threshold to be verified of the last optimization iteration is set as the DTW threshold after reaching the preset number of optimization iterations;
[0026] In step S8, when performing sliding detection on the time series satellite telemetry data, DTW is used to calculate the DTW distance D between the time series satellite telemetry data in the sliding window and the reference time series satellite telemetry data, and the DTW distance D is output as the sliding detection result.
[0027] Furthermore, in step S9, when anomaly identification is performed on the time series satellite telemetry data, the DTW distance D is compared with the preset DTW distance D0, and anomaly identification is performed on the time series satellite telemetry data according to the comparison result, wherein:
[0028] When D≤D0, the time series satellite telemetry data at that moment is judged to be normal;
[0029] When D>D0, the time series satellite telemetry data at that moment is determined to be abnormal, and a collection of the time series satellite telemetry data at each abnormal moment in the time series satellite telemetry data is taken as abnormal satellite telemetry data.
[0030] Furthermore, in step S10, when the abnormal satellite telemetry data is post-processed, the auxiliary identification data at each moment is input into the post-processing analysis model, the post-processing analysis model outputs the post-processing analysis result, and the abnormal satellite telemetry data is post-processed according to the post-processing analysis result, wherein:
[0031] When the post-processing analysis result indicates that no post-processing is required, the abnormal satellite telemetry data at that moment will not be post-processed;
[0032] When the post-processing analysis result indicates that post-processing is required, the abnormal satellite telemetry data at that moment is post-processed, and the abnormal identification result at the corresponding moment in the abnormal satellite telemetry data at that moment is adjusted to normal time series satellite telemetry data at that moment.
[0033] Furthermore, in step S11, when optimizing the time series processing process, the number of anomalies in each preset identification time period is obtained, which is used as the number of anomalies in the time period p, the number of anomalies in the time period p is compared with the number of anomalies in the preset time period p0, and the key time period is identified according to the comparison result, wherein:
[0034] When p≤p0, determining that the preset identification time period is a non-critical time period;
[0035] When p>p0, the preset identification time period is determined to be a critical time period, and the preset unit time interval t of the critical time period is optimized. After optimization, the preset unit time interval is ts, and ts=t×μ is set, where μ is the time optimization coefficient and is set to 0.87<μ<1.
[0036] Further, in the step S12, after the preset identification time period is the key time period, the abnormal number decrease value L is compared with the preset abnormal number decrease value L0, and the compensation judgment situation of the key time period is judged according to the comparison result, wherein:
[0037] When L>L0, it is determined that no compensation judgment is performed on the critical time period;
[0038] When L≤L0, it is determined that a compensation judgment is performed on the critical time period, and the critical time period is adjusted to a non-critical time period.
[0039] Compared with the prior art, the beneficial effects of the present invention are that the method acquires satellite telemetry data in real time through step S1 to realize timely monitoring of the latest data, thereby improving the real-time and timeliness of the data; the method preprocesses the real-time satellite telemetry data through step S2 to realize data cleaning and standardization processing, thereby improving the quality and consistency of the data; the method performs time series processing on the preprocessed satellite telemetry data through step S3 to realize data serialization and structuring, thereby improving the accuracy and effectiveness of data analysis; the method sets the DTW threshold according to normal data through step S4 to realize abnormality detection The threshold is reasonably set to improve the sensitivity and accuracy of anomaly detection. The method inputs historical data into the simulation model for anomaly analysis through step S5, and performs historical compensation on normal data according to the anomaly analysis results of the historical data, so as to realize the correction of potential anomalies in the historical data, thereby improving the overall accuracy and reliability of the data. The method inputs real-time satellite telemetry data into the simulation model for anomaly analysis through step S6, and performs real-time compensation on normal data according to the anomaly analysis results of the real-time satellite telemetry data, so as to realize the timely correction of potential anomalies in the real-time data, thereby improving the accuracy of the real-time data. The method inputs historical data into the simulation model for anomaly analysis through step S7, and performs real-time compensation on normal data according to the anomaly analysis results of the real-time satellite telemetry data, so as to realize the timely correction of potential anomalies in the real-time data, thereby improving the accuracy of the real-time data. The number of compensations, the number of real-time compensations, and the amount of normal data after compensation optimize the setting process of the DTW threshold to achieve dynamic adjustment of the threshold setting, thereby improving the flexibility and adaptability of anomaly detection. The method performs sliding detection on the time series satellite telemetry data according to the DTW threshold in step S8 to achieve continuous monitoring of the data, thereby improving the coverage and continuity of anomaly detection. The method performs anomaly identification on the time series satellite telemetry data according to the sliding detection result in step S9 to obtain abnormal satellite telemetry data to achieve accurate identification of abnormal data, thereby improving the accuracy of anomaly detection. The method performs anomaly identification on the time series satellite telemetry data according to the auxiliary identification data in step S10. Abnormal satellite telemetry data is post-processed to achieve further evaluation and correction of the abnormal data, thereby improving the comprehensiveness and rationality of abnormal processing. The method identifies the key time period according to the number of abnormalities in the time period through step S11, and optimizes the time series processing process according to the key time period to achieve key monitoring of the key time period, thereby improving the abnormality detection capability of the important time period. The method performs compensation judgment on the key time period according to the decrease value of the number of abnormalities through step S12, and adjusts the optimization process of the time series processing process according to the compensation judgment result to achieve dynamic optimization of the detection and processing strategies, thereby improving the adaptability and effect of the overall method. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 Schematic diagram of the process of detecting satellite telemetry data anomaly based on DTW in this embodiment. DETAILED DESCRIPTION
[0041] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0042] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.
[0043] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.
[0044] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0045] See also Figure 1 As shown, it is a flow chart of the satellite telemetry data anomaly detection method based on DTW in this embodiment, and the method includes:
[0046] Step S1, acquiring satellite telemetry data in real time to obtain real-time satellite telemetry data;
[0047] Step S2, preprocessing the real-time satellite telemetry data to obtain preprocessed satellite telemetry data;
[0048] Step S3, performing time series processing on the pre-processed satellite telemetry data to obtain time series satellite telemetry data;
[0049] Step S4, setting the DTW threshold according to normal data;
[0050] Step S5, inputting the historical data into the simulation model for abnormal analysis, and performing historical compensation on the normal data according to the abnormal analysis results of the historical data;
[0051] Step S6, inputting the real-time satellite telemetry data into the simulation model for abnormality analysis, and performing real-time compensation for normal data according to the abnormality analysis results of the real-time satellite telemetry data;
[0052] Step S7, optimizing the DTW threshold setting process according to the historical compensation times, the real-time compensation times and the normal data volume after compensation;
[0053] Step S8, performing sliding detection on the time series satellite telemetry data according to the DTW threshold;
[0054] Step S9, performing abnormal identification on the time series satellite telemetry data according to the sliding detection result to obtain abnormal satellite telemetry data;
[0055] Step S10, post-processing the abnormal satellite telemetry data according to the auxiliary identification data;
[0056] Step S11, identifying a key time period according to the number of abnormal times in the time period, and optimizing the time series processing process according to the key time period;
[0057] Step S12, performing compensation judgment on the key time period according to the decrease value of the number of abnormalities, and adjusting the optimization process of the time series processing process according to the compensation judgment result.
[0058] Specifically, the method is set in a satellite telemetry data processing system, performs anomaly detection on satellite telemetry data through DTW, and performs proofreading, optimization and post-processing on the anomaly detection of satellite telemetry data, thereby improving the accuracy of satellite telemetry data and the efficiency of satellite telemetry data anomaly detection. The method acquires satellite telemetry data in real time through step S1 to realize timely monitoring of the latest data, thereby improving the real-time and timeliness of the data. The method pre-processes the real-time satellite telemetry data through step S2 to realize data cleaning and standardization processing, thereby improving the quality and consistency of the data. The method performs time series processing on the pre-processed satellite telemetry data through step S3 to realize data The method sets the DTW threshold according to the normal data in step S4 to achieve a reasonable setting of the anomaly detection threshold, thereby improving the sensitivity and accuracy of anomaly detection. The method inputs the historical data into the simulation model for anomaly analysis in step S5, and performs historical compensation on the normal data according to the anomaly analysis results of the historical data, so as to achieve correction of potential anomalies in the historical data, thereby improving the overall accuracy and reliability of the data. The method inputs the real-time satellite telemetry data into the simulation model for anomaly analysis in step S6, and performs real-time compensation on the normal data according to the anomaly analysis results of the real-time satellite telemetry data, so as to achieve correction of potential anomalies in the real-time satellite telemetry data. The method optimizes the setting process of the DTW threshold according to the number of historical compensations, the number of real-time compensations and the amount of normal data after compensation through step S7 to achieve dynamic adjustment of the threshold setting, thereby improving the flexibility and adaptability of anomaly detection. The method performs sliding detection on the time series satellite telemetry data according to the DTW threshold through step S8 to achieve continuous monitoring of the data, thereby improving the coverage and continuity of anomaly detection. The method performs anomaly identification on the time series satellite telemetry data according to the sliding detection result through step S9 to obtain abnormal satellite telemetry data, so as to achieve accurate identification of abnormal data, thereby improving the accuracy of anomaly detection. The method performs post-processing on the abnormal satellite telemetry data according to the auxiliary identification data through step S10 to achieve further evaluation and correction of the abnormal data, thereby improving the comprehensiveness and rationality of abnormal processing. The method identifies the key time period according to the number of abnormalities in the time period through step S11, and optimizes the time series processing process according to the key time period to achieve key monitoring of the key time period, thereby improving the abnormality detection capability of the important time period. The method performs compensation judgment on the key time period according to the decrease value of the number of abnormalities through step S12, and adjusts the optimization process of the time series processing process according to the compensation judgment result, so as to achieve dynamic optimization of the detection and processing strategies, thereby improving the adaptability and effect of the overall method.
[0059] Specifically, in step S1, the satellite telemetry data refers to information about various aspects of the earth's surface, atmosphere, ocean, and space environment, etc., which is remotely collected by sensors and instruments on artificial satellites. The embodiment does not limit the method for obtaining the satellite telemetry data. Those skilled in the art can freely set it according to actual conditions, and only need to meet the real-time acquisition requirements of satellite telemetry data. For example, it can be set that when the satellite telemetry data is acquired in real time, the satellite telemetry data with the latest timestamp in the satellite telemetry database is captured through the data acquisition channel, and it is acquired as real-time satellite telemetry data.
[0060] Specifically, in step S2, when preprocessing the real-time satellite telemetry data, missing values and noise data in the real-time satellite telemetry data are processed to obtain cleaned data, and data standardization is performed on the cleaned data to obtain preprocessed satellite telemetry data.
[0061] Specifically, the preprocessing includes processing missing values and noise data in real-time satellite telemetry data. In this embodiment, the missing values in the real-time satellite telemetry data are filled according to similar samples through the KNN algorithm. When filling, the distance between the real-time satellite telemetry data with missing values and the complete sample is calculated using the Euclidean distance, and the complete samples within the preset supplementary distance are selected as supplementary samples. The average value of the supplementary samples is calculated and used as the supplementary value of the real-time satellite telemetry data with missing values to supplement the missing values. In this embodiment, the noise data in the real-time satellite telemetry data is smoothed through Kalman filtering. The Kalman filtering refers to a recursive algorithm for estimating the state of a dynamic system from noisy observation data, and the cleaned data refers to the real-time satellite telemetry data after missing values and noise data.
[0062] Specifically, in step S2, the missing values and noise data in the real-time satellite telemetry data are processed to ensure the data quality of the real-time satellite telemetry data, and the cleaned data of different sensors are subjected to data standardization to facilitate subsequent data comparison.
[0063] Specifically, in step S3, when performing time series processing on the preprocessed satellite telemetry data, the preprocessed satellite telemetry data is divided into time intervals according to preset unit time intervals, and the preprocessed satellite telemetry data after the time interval division is constructed into time series satellite telemetry data in chronological order.
[0064] Specifically, the time interval division refers to dividing the pre-processed satellite telemetry data in time according to preset unit time intervals, dividing it once every preset unit time interval, and using the divided data set as the pre-processed satellite telemetry data after time interval division. This embodiment does not specifically limit the value of the preset unit time interval. Those skilled in the art can freely set it according to actual conditions, as long as it meets the requirements for time series processing of the pre-processed satellite telemetry data. For example, the preset unit time interval can be set to 1 minute.
[0065] Specifically, in step S3, by dividing the data into preset unit time intervals, the data can be made more structured and ordered, which is convenient for subsequent time series analysis and processing. At the same time, the time series satellite telemetry data is constructed in chronological order so that the data can reflect the changing trend in the time dimension, which helps to reveal the time patterns and laws hidden in the data. The processed data can better adapt to the time series analysis algorithm, thereby improving the accuracy of the analysis and the effectiveness of the prediction.
[0066] Specifically, in step S4, when setting the DTW threshold, the sum of the mean of normal data and the preset multiple of the standard deviation is calculated and used as the DTW threshold to be verified. The abnormal satellite telemetry data and the normal satellite telemetry data are cross-validated with the DTW threshold to be verified, and the cross-validation result is obtained. When the cross-validation result does not meet the verification condition, the preset multiple of the standard deviation is incrementally adjusted, and the DTW threshold to be verified is recalculated until the cross-validation result meets the verification condition and the DTW threshold to be verified is set as the DTW threshold.
[0067] Specifically, the normal data refers to satellite telemetry data in a normal state as a sample for setting the DTW threshold, the preset multiple refers to a multiple factor used to adjust the standard deviation, such as setting the preset multiple to 2, the abnormal satellite telemetry data refers to satellite telemetry time series data known to be collected under an abnormal or faulty state, which shows characteristics that deviate from the normal working state and is used to test whether the threshold can effectively detect the anomaly, the normal satellite telemetry data refers to satellite telemetry time series data known to be collected under a normal working state, and is used to ensure that the threshold does not produce false positives for normal data, the cross-validation refers to applying the DTW threshold to be verified to abnormal satellite telemetry data and normal satellite telemetry data, and evaluating its classification effect, the incremental adjustment refers to incremental adjustment of the preset multiple, such as 2 is increased to 3, and this embodiment does not specifically limit the type of the verification condition, such as it can be set to the minimum value of the normal data recognition rate and the abnormal data recognition rate, or the maximum value of the false alarm rate and the missed alarm rate.
[0068] Specifically, in step S4, by calculating the sum of the mean and standard deviation of the normal data, a reasonable DTW threshold can be preliminarily set, so that the threshold has statistical significance and reflects the overall characteristics of the normal data. The cross-validation process further ensures the accuracy and applicability of the threshold. By adjusting the preset multiples of the standard deviation, the DTW threshold can be dynamically optimized so that it can more effectively distinguish abnormal data from normal data, improve the accuracy of anomaly detection, and reduce the false alarm rate and missed alarm rate.
[0069] Specifically, in step S5, when performing historical compensation on normal data, historical data is acquired every historical compensation cycle, and the historical data is input into the simulation model, and normal data output by the simulation model is acquired and used as abnormal analysis results of the historical data. Whenever the data volume of the abnormal analysis results of the historical data reaches a preset data volume, the normal data within the data volume of the abnormal analysis results of the historical data is compensated to the normal data of step S4, and the data volume of the abnormal analysis results of the historical data is cleared and recounted.
[0070] Specifically, the historical compensation refers to a method of performing data compensation based on historical data, and the historical data refers to satellite telemetry data before the current time. The historical compensation period refers to a preset interval period for acquiring historical data, such as setting the historical compensation period to 7 days. The simulation model refers to a simulation model that identifies and outputs normal data after simulating historical data. This embodiment does not limit the construction method of the simulation model, and technicians in this field can freely set it according to actual conditions, as long as the recognition requirements for normal data are met. For example, the simulation model can be set to a deep learning model, and the historical data is used as the input value of the simulation model, and the recognition of normal data is used as the output value of the simulation model. The data volume of the abnormal analysis results of the historical data refers to the data volume of normal data output by the simulation model, and the preset data volume refers to the preset value for judging the data volume for compensation of normal data, such as the preset data volume can be set to 30M.
[0071] Specifically, in step S5, by periodically acquiring historical data and inputting it into the simulation model for analysis, anomalies and normal data in the historical data can be effectively identified and distinguished. The normal data provided by the simulation model is used to compensate for the current normal data set, thereby enriching the normal data sample and improving the robustness and accuracy of the model. The compensation mechanism ensures that the normal data set is continuously updated to reflect the latest system status and characteristics, which helps to more accurately set and adjust the DTW threshold. The process of clearing and recounting ensures the continuity of data processing and the real-time performance of the system, avoiding delays or computing pressure caused by data accumulation.
[0072] Specifically, in step S6, when performing real-time compensation on normal data, real-time data is acquired every real-time compensation cycle, and the real-time data is input into the simulation model, and normal data output by the simulation model is acquired and used as the abnormal analysis result of the real-time data. Whenever the data volume of the abnormal analysis result of the real-time data reaches a preset data volume, the normal data within the data volume of the abnormal analysis result of the real-time data is compensated to the normal data of step S4, and the data volume of the abnormal analysis result of the real-time data is cleared and recounted.
[0073] Specifically, the real-time compensation period refers to a preset interval period for acquiring real-time data. For example, the historical compensation period is set to 1 day, and the real-time data refers to satellite telemetry data acquired in real time.
[0074] Specifically, in step S6, by regularly acquiring real-time data and inputting it into the simulation model for analysis, the abnormal and normal data in the real-time data can be effectively distinguished. The normal data provided by the simulation model is used to compensate for the current normal data set, thereby enriching the normal data sample, improving the robustness and accuracy of the model, and ensuring the dynamic update of the normal data set.
[0075] Specifically, in step S7, the DTW threshold optimization index mr is calculated according to the historical compensation times n1, the real-time compensation times n2, the DTW threshold iteration times n3 and the normal data volume nr after compensation, and mr=0.5×[(n1+n2+n3) / n0]+0.5×[nr / nr0] is set, n0 is a preset times parameter, nr0 is a preset data volume parameter, and the optimization index mr is compared with the preset optimization index mr0, and a judgment is made according to the optimization of the DTW threshold setting process, wherein:
[0076] When mr≤mr0, it is determined that the DTW threshold setting process is not optimized;
[0077] When mr>mr0, it is determined that the setting process of the DTW threshold is optimized, and the setting process of the DTW threshold is optimized until the DTW threshold to be verified of the last optimization iteration is set as the DTW threshold after reaching a preset number of optimization iterations.
[0078] Specifically, the number of historical compensation times refers to the number of times historical data is obtained and compensated, the number of real-time compensation times refers to the number of times real-time data is obtained and compensated, the number of DTW threshold iterations refers to the number of iterations performed in the process of setting the DTW threshold, the amount of normal data after compensation refers to the total amount of normal data after compensation, the preset optimization index refers to a predetermined standard value used to determine whether the DTW threshold setting process needs to be optimized, such as setting the preset optimization index to 50, the preset optimization iteration number refers to the preset maximum number of iterations predetermined in the DTW threshold optimization process, such as setting the preset optimization iteration number to 10, the preset number parameter refers to the preset number parameter used to calculate the DTW threshold optimization index, such as setting it to 60, and the preset data amount parameter refers to the preset data amount parameter used to calculate the DTW threshold optimization index, such as setting it to 45.
[0079] Specifically, in step S7, by comprehensively considering the historical and real-time compensation times, the number of iterations, and the amount of normal data after compensation, the optimization requirements of the DTW threshold can be evaluated more comprehensively and accurately, and the optimization index mr can be set and compared, which helps to automatically determine whether the DTW threshold needs to be optimized and avoid human intervention, so as to improve the objectivity and efficiency of decision-making, make the threshold more in line with the actual data characteristics, and improve the accuracy and robustness of anomaly detection. At the same time, the preset number of optimization iterations is set to ensure that the optimization process is completed within a reasonable time and avoid the waste of computing resources caused by over-optimization.
[0080] Specifically, in step S8, when performing sliding detection on the time series satellite telemetry data, DTW is used to calculate the DTW distance D between the time series satellite telemetry data in the sliding window and the reference time series satellite telemetry data, and the DTW distance D is output as the sliding detection result.
[0081] Specifically, the sliding window refers to a window used to select data at a continuous time point in time series data processing. The sliding window is often used in time series analysis to divide long time series data into multiple short time periods, so that the window can be gradually slid to analyze each segment of data. For example, if there is a sliding window with a length of 7 days, then each sliding window will contain 7 consecutive days of data. As time goes on, the sliding window will gradually move forward. The reference time series satellite telemetry data refers to a standard data set for comparison with the current time series satellite telemetry data. The sliding detection refers to a method of performing segment-by-segment detection of time series data through a sliding window. During the sliding detection process, the data in the sliding window will be taken each time and compared with the reference time series data to detect and identify anomalies or changes in the data. The output result of the sliding detection is usually the detection result in each sliding window.
[0082] Specifically, in step S8, by applying the DTW algorithm, the nonlinear deformation and time offset problems in the time series data can be effectively handled, thereby improving the accuracy of detection. The DTW algorithm can flexibly match the changing trend in the time series, so that even in the presence of time delay or deformation, the similarity between time series can be accurately evaluated. By detecting the data in a sliding window manner, the changes in the time series data can be monitored in real time, and abnormal situations can be discovered in time. The DTW distance is output as a sliding detection result to provide a quantitative indicator to facilitate subsequent analysis and decision-making.
[0083] Specifically, in step S9, when anomaly identification is performed on the time series satellite telemetry data, the DTW distance D is compared with the preset DTW distance D0, and anomaly identification is performed on the time series satellite telemetry data according to the comparison result, wherein:
[0084] When D≤D0, the time series satellite telemetry data at that moment is judged to be normal;
[0085] When D>D0, the time series satellite telemetry data at that moment is determined to be abnormal, and a collection of the time series satellite telemetry data at each abnormal moment in the time series satellite telemetry data is taken as abnormal satellite telemetry data.
[0086] Specifically, the preset DTW distance refers to a preset value for anomaly identification of time series satellite telemetry data. For example, the preset DTW distance can be set to 10.
[0087] Specifically, in step S9, the preset threshold value D0 can be used to effectively distinguish normal and abnormal time series data, and the collection of time series satellite telemetry data at abnormal moments is managed uniformly as abnormal satellite telemetry data, which is helpful for subsequent data analysis and fault location.
[0088] Specifically, in step S10, when the abnormal satellite telemetry data is post-processed, the auxiliary identification data at each moment is input into the post-processing analysis model, the post-processing analysis model outputs the post-processing analysis result, and the abnormal satellite telemetry data is post-processed according to the post-processing analysis result, wherein:
[0089] When the post-processing analysis result indicates that no post-processing is required, the abnormal satellite telemetry data at that moment will not be post-processed;
[0090] When the post-processing analysis result indicates that post-processing is required, the abnormal satellite telemetry data at that moment is post-processed, and the abnormal identification result at the corresponding moment in the abnormal satellite telemetry data at that moment is adjusted to normal time series satellite telemetry data at that moment.
[0091] Specifically, the auxiliary identification data refers to information used as auxiliary input during anomaly detection and post-processing, such as other types of sensor data, environmental parameters, historical data, model prediction values, etc. The post-processing analysis model refers to a data analysis tool or algorithm used to further evaluate the abnormal satellite telemetry data that has been initially detected. This embodiment does not limit the type of the post-processing analysis model. Those skilled in the art can freely set it according to actual conditions. It only needs to meet the needs of further evaluating the abnormal satellite telemetry data. It can be set as a machine learning model, a statistical analysis method, a rule engine, or a model based on an expert system.
[0092] Specifically, in step S10, the auxiliary identification data at each moment is input into a post-processing analysis model, which outputs a post-processing analysis result, and the abnormal satellite telemetry data is post-processed according to the result to accurately distinguish which abnormal data requires further processing and which can be ignored. When the post-processing analysis result shows that no post-processing is required, unnecessary resource consumption and time waste are avoided, and the overall processing efficiency is improved. When the analysis result shows that post-processing is required, the abnormal data at that moment can be corrected in time, and the abnormal identification result can be adjusted to normal, thereby ensuring the accuracy and consistency of the data.
[0093] Specifically, in step S11, when optimizing the time series processing process, the number of anomalies in each preset identification time period is obtained, which is used as the number of anomalies in the time period p, and the number of anomalies in the time period p is compared with the number of anomalies in the preset time period p0, and the key time period is identified according to the comparison result, wherein:
[0094] When p≤p0, determining that the preset identification time period is a non-critical time period;
[0095] When p>p0, the preset identification time period is determined to be a critical time period, and the preset unit time interval t of the critical time period is optimized. After optimization, the preset unit time interval is ts, and ts=t×μ is set, where μ is the time optimization coefficient and is set to 0.87<μ<1.
[0096] Specifically, the preset identification time period refers to the preset value of the time interval for identifying the key time period, such as the preset identification time period can be set to 5 minutes. The number of abnormalities refers to the number of times that the time series satellite telemetry data at that moment is judged to be abnormal within the preset identification time period. The preset time period abnormality number refers to the preset value of the number of time period abnormalities for identifying the key time period, such as it can be set to 3 times.
[0097] Specifically, in step S11, when p≤p0, the preset identification time period is determined as a non-critical time period, thereby avoiding excessive processing of the non-critical time period and saving resources and time; and when p>p0, the preset identification time period is determined as a critical time period, and its preset unit time interval t is further optimized. The optimized time interval ts is adjusted by setting the μ value, thereby making the data collection and processing in the critical time period more refined and accurate.
[0098] Specifically, in step S12, after the preset identification time period is the key time period, the abnormal number decrease value L is compared with the preset abnormal number decrease value L0, and the compensation judgment of the key time period is judged according to the comparison result, wherein:
[0099] When L>L0, it is determined that no compensation judgment is performed on the critical time period;
[0100] When L≤L0, it is determined that a compensation judgment is performed on the critical time period, and the critical time period is adjusted to a non-critical time period.
[0101] Specifically, the abnormal number reduction value refers to the difference between the abnormal number of the current time period and the abnormal number of the time period before optimization, after the preset identification time period is regarded as the key time period and the preset unit time interval t of the key time period is optimized in step S11. The preset abnormal number reduction value refers to the preset value of the abnormal number reduction value for judging the compensation judgment of the key time period.
[0102] Specifically, in step S12, when the preset identification time period is determined to be a critical time period, the compensation judgment of the critical time period is further judged by comparing the abnormal number decrease value L with the preset abnormal number decrease value L0. When L>L0, the system determines that there is no need to perform compensation judgment on the critical time period, which can avoid unnecessary processing, save system resources and time, and ensure efficient operation of the system; and when L≤L0, the system determines that it is necessary to perform compensation judgment on the critical time period and adjust it to a non-critical time period. Such an adjustment mechanism can dynamically correct the identification of the time period to ensure the accuracy and flexibility of the identification. This method can effectively filter and correct misjudgments by judging the abnormal number decrease value, improve the accuracy of critical time period identification, and, through the dynamic adjustment of the critical time period, enable the system to respond to abnormal situations more flexibly and reduce the probability of false alarms and missed alarms. The introduction of the compensation judgment mechanism ensures that abnormal situations are discovered and handled in a timely manner during the critical time period, and can be quickly adjusted when the abnormalities are reduced, avoiding waste of system resources.
[0103] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.
Claims
1. A satellite telemetry data anomaly detection method based on DTW, characterized in that: include: Step S1, acquiring satellite telemetry data in real time to obtain real-time satellite telemetry data; Step S2, preprocessing the real-time satellite telemetry data to obtain preprocessed satellite telemetry data; Step S3, performing time series processing on the pre-processed satellite telemetry data to obtain time series satellite telemetry data; Step S4, setting the DTW threshold according to normal data; Step S5, inputting the historical data into the simulation model for abnormal analysis, and performing historical compensation on the normal data according to the abnormal analysis results of the historical data; Step S6, inputting the real-time satellite telemetry data into the simulation model for abnormality analysis, and performing real-time compensation for normal data according to the abnormality analysis results of the real-time satellite telemetry data; Step S7, optimizing the DTW threshold setting process according to the historical compensation times, the real-time compensation times and the normal data volume after compensation; Step S8, performing sliding detection on the time series satellite telemetry data according to the DTW threshold; Step S9, performing abnormal identification on the time series satellite telemetry data according to the sliding detection result to obtain abnormal satellite telemetry data; Step S10, post-processing the abnormal satellite telemetry data according to the auxiliary identification data; Step S11, identifying a key time period according to the number of abnormal times in the time period, and optimizing the time series processing process according to the key time period; Step S12, performing compensation judgment on the key time period according to the decrease value of the number of abnormalities, and adjusting the optimization process of the time series processing process according to the compensation judgment result.
2. The satellite telemetry data anomaly detection method based on DTW according to claim 1, characterized in that: In step S2, when preprocessing the real-time satellite telemetry data, missing values and noise data in the real-time satellite telemetry data are processed to obtain cleaned data, and data standardization is performed on the cleaned data to obtain preprocessed satellite telemetry data.
3. The satellite telemetry data anomaly detection method based on DTW according to claim 2 is characterized in that: In step S3, when performing time series processing on the preprocessed satellite telemetry data, the preprocessed satellite telemetry data is divided into time intervals according to preset unit time intervals, and the preprocessed satellite telemetry data after the time interval division is constructed into time series satellite telemetry data in chronological order.
4. The satellite telemetry data anomaly detection method based on DTW according to claim 1, characterized in that: In step S4, when setting the DTW threshold, the sum of the mean of normal data and the preset multiple of the standard deviation is calculated and used as the DTW threshold to be verified. The abnormal satellite telemetry data and the normal satellite telemetry data are cross-validated with the DTW threshold to be verified, and the cross-validation result is obtained. When the cross-validation result does not meet the verification condition, the preset multiple of the standard deviation is incrementally adjusted, and the DTW threshold to be verified is recalculated until the cross-validation result meets the verification condition and the DTW threshold to be verified is set as the DTW threshold.
5. The satellite telemetry data anomaly detection method based on DTW according to claim 1, characterized in that: In step S5, when performing historical compensation on normal data, historical data is acquired every other historical compensation cycle, and the historical data is input into the simulation model, normal data output by the simulation model is acquired, and the data is used as the abnormal analysis result of the historical data, and whenever the data volume of the abnormal analysis result of the historical data reaches a preset data volume, the normal data within the data volume of the abnormal analysis result of the historical data is compensated to the normal data of step S4, and the data volume of the abnormal analysis result of the historical data is cleared and recounted; In step S6, when performing real-time compensation on normal data, real-time data is acquired every real-time compensation cycle, and the real-time data is input into the simulation model, normal data output by the simulation model is acquired, and used as the abnormal analysis result of the real-time data, and whenever the data volume of the abnormal analysis result of the real-time data reaches a preset data volume, the normal data within the data volume of the abnormal analysis result of the real-time data is compensated to the normal data of step S4, and the data volume of the abnormal analysis result of the real-time data is cleared and recounted.
6. The satellite telemetry data anomaly detection method based on DTW according to claim 5, characterized in that: In the step S7, the DTW threshold optimization index mr is calculated according to the historical compensation times n1, the real-time compensation times n2, the DTW threshold iteration times n3 and the normal data volume after compensation nr, and mr=0.5×[(n1+n2+n3) / n0]+0.5×[nr / nr0] is set, n0 is a preset times parameter, nr0 is a preset data volume parameter, and the optimization index mr is compared with the preset optimization index mr0, and a judgment is made according to the optimization of the DTW threshold setting process, wherein: When mr≤mr0, it is determined that the DTW threshold setting process is not optimized; When mr>mr0, it is determined that the setting process of the DTW threshold is optimized, and the setting process of the DTW threshold is optimized until the DTW threshold to be verified of the last optimization iteration is set as the DTW threshold after reaching the preset number of optimization iterations; In step S8, when performing sliding detection on the time series satellite telemetry data, DTW is used to calculate the DTW distance D between the time series satellite telemetry data in the sliding window and the reference time series satellite telemetry data, and the DTW distance D is output as the sliding detection result.
7. The satellite telemetry data anomaly detection method based on DTW according to claim 6, characterized in that: In step S9, when anomaly identification is performed on the time series satellite telemetry data, the DTW distance D is compared with the preset DTW distance D0, and anomaly identification is performed on the time series satellite telemetry data according to the comparison result, wherein: When D≤D0, the time series satellite telemetry data at that moment is judged to be normal; When D>D0, the time series satellite telemetry data at that moment is determined to be abnormal, and a collection of the time series satellite telemetry data at each abnormal moment in the time series satellite telemetry data is taken as abnormal satellite telemetry data.
8. The satellite telemetry data anomaly detection method based on DTW according to claim 7, characterized in that: In step S10, when the abnormal satellite telemetry data is post-processed, the auxiliary identification data at each moment is input into the post-processing analysis model, the post-processing analysis model outputs the post-processing analysis result, and the abnormal satellite telemetry data is post-processed according to the post-processing analysis result, wherein: When the post-processing analysis result indicates that no post-processing is required, the abnormal satellite telemetry data at that moment will not be post-processed; When the post-processing analysis result indicates that post-processing is required, the abnormal satellite telemetry data at that moment is post-processed, and the abnormal identification result at the corresponding moment in the abnormal satellite telemetry data at that moment is adjusted to normal time series satellite telemetry data at that moment.
9. The satellite telemetry data anomaly detection method based on DTW according to claim 8, characterized in that: In step S11, when optimizing the time series processing process, the number of anomalies in each preset identification time period is obtained, which is used as the number of anomalies in the time period p, and the number of anomalies in the time period p is compared with the number of anomalies in the preset time period p0, and the key time period is identified according to the comparison result, wherein: When p≤p0, determining that the preset identification time period is a non-critical time period; When p>p0, the preset identification time period is determined to be a critical time period, and the preset unit time interval t of the critical time period is optimized. After optimization, the preset unit time interval is ts, and ts=t×μ is set, where μ is the time optimization coefficient and is set to 0.87<μ<1.
10. The satellite telemetry data anomaly detection method based on DTW according to claim 9, characterized in that: In the step S12, after the preset identification time period is the key time period, the abnormal number decrease value L is compared with the preset abnormal number decrease value L0, and the compensation judgment situation of the key time period is judged according to the comparison result, wherein: When L>L0, it is determined that no compensation judgment is performed on the critical time period; When L≤L0, it is determined that a compensation judgment is performed on the critical time period, and the critical time period is adjusted to a non-critical time period.
Citation Information
Patent Citations
Satellite abnormal condition detection system based on telemeasuring data
CN103646167A
Satellite telemetry data abnormity detection method based on DTW
CN104915568A
Power sale quantity prediction method and device based on leading analysis and factor compensation
CN107220764A
Gray release verification method, system, medium, device and program
CN114327561A
Environmental monitoring missing data compensation processing method
CN114547985A
Cited By
Power adapter anti-theft alarm system and method based on Internet of Things
CN119649574A
An anti-theft alarm system and method for a power adapter based on the Internet of Things
CN119649574B
Method for testing satellite test, operation and control data interface
CN120150794A
Current linkage early warning method used under coal mine underground multi-motor coupling effect
CN120652287A
Aircraft abnormal event detection method and system oriented to different voyage number scenes, and storage medium
CN121075183A
