Spacecraft telemetry data cleaning method and device

Abstract

The present disclosure provides a method and device for cleaning spacecraft telemetry data, which belongs to data processing technology. The method includes: acquiring telemetry data of a spacecraft; sorting the telemetry data in chronological order, and preprocessing the sorted telemetry data to obtain first intermediate data; based on the source information of the telemetry data and / or the key time information of the telemetry data, performing data cleaning on the first intermediate data to obtain cleaned data. The method provided by the present disclosure can comprehensively utilize strategies such as multi-station same-name data fusion, key time feature data retention, data drift threshold setting, and equal-interval sampling within the threshold to achieve effective cleaning of telemetry result data stored in the relational database of large spacecraft, while ensuring the integrity of data features and improving the efficiency of secondary analysis and utilization of historical data.

Description

Technical Field

[0001] The present disclosure belongs to the field of data processing technology, and specifically relates to a spacecraft telemetry data cleaning method and device, electronic equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the advancement of aerospace technology, the on-orbit operation time, TT&C communication coverage, information transmission volume, and transmission frequency of large spacecraft have all been greatly improved compared to previous generations. Consequently, the amount of TT&C data has also increased exponentially. This massive amount of TT&C data not only consumes significant storage resources, but also hinders rapid subsequent data analysis due to the large amount of duplicate data. Therefore, research on high-frequency TT&C data cleaning technology for large spacecraft is of great practical significance. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art and provide a spacecraft telemetry data cleaning method and device, a computer-readable storage medium and a computer program product that can maximize the release of storage resources and reduce storage costs.

[0004] To achieve the above objectives, in a first aspect, an embodiment of the present disclosure provides a method for cleaning spacecraft telemetry data, the method comprising:

[0005] Acquire telemetry data from spacecraft;

[0006] sorting the telemetry data in chronological order, and preprocessing the sorted telemetry data to obtain first intermediate data;

[0007] Based on the source information of the telemetry data and / or the key time information of the telemetry data, the first intermediate data is cleansed to obtain cleaned data.

[0008] In some embodiments, sorting the telemetry data in chronological order and preprocessing the sorted telemetry data to obtain first intermediate data includes:

[0009] Sorting the telemetry data in chronological order to obtain time series data;

[0010] Based on the preset data type flag field and data time period, the test data and temporary data in the time series data are deleted to obtain first intermediate data.

[0011] In some embodiments, the performing data cleaning on the first intermediate data based on the source information of the telemetry data and / or the time information of the telemetry data to obtain the cleaned data includes:

[0012] performing a first data cleansing on the first intermediate data based on the source information of the telemetry data to obtain first cleansed data;

[0013] The first cleaned data is cleaned for a second time according to the key time information of the telemetry data to obtain cleaned data.

[0014] In some embodiments, the source information of the telemetry data includes the on-device time of the transmission station corresponding to the telemetry data and the source packet identifier of the telemetry data, and performing a first data cleansing on the first intermediate data based on the source information of the telemetry data to obtain the first cleansed data includes:

[0015] determining whether there is data with the same name in the first intermediate data according to the on-device time and source packet identifier of the transmission station corresponding to each frame of data in the first intermediate data;

[0016] Deduplication processing is performed on data with the same name in the first intermediate data to obtain first cleaned data.

[0017] In some embodiments, performing deduplication processing on the data with the same name in the first intermediate data to obtain the first cleaned data includes:

[0018] For the data with the same name in the first intermediate data, only one copy of the data with the same name is retained, and all transmission station information corresponding to the data with the same name is saved to obtain first cleaned data.

[0019] In some embodiments, the key time information of the telemetry data includes a dispatch schedule and an important event schedule of the telemetry data, and performing a second data cleansing on the first cleaned data according to the key time information of the telemetry data to obtain cleaned data includes:

[0020] Obtain a dispatch schedule and a timeline of significant events for said telemetry data;

[0021] determining key frames in the first cleaned data based on a dispatch schedule and a key event schedule of the telemetry data;

[0022] Key frames in the first cleaned data are marked to obtain cleaned data.

[0023] In some embodiments, the method further comprises:

[0024] Based on the data drift threshold, the cleaned data is sampled to obtain the final cleaning result of the telemetry data.

[0025] In some embodiments, sampling the cleaned data based on the data drift threshold to obtain a final cleaning result of the telemetry data includes:

[0026] Determining a data drift threshold according to a change trend of the cleaned data;

[0027] Calculating the change value of any two adjacent frames of data in the cleaned data;

[0028] Comparing the change value of each two adjacent frames of data with the data drift threshold to obtain a first comparison result;

[0029] According to the first comparison result, a final cleaning result of the telemetry data is obtained.

[0030] In some embodiments, obtaining a final cleaning result of the telemetry data according to the first comparison result includes:

[0031] If the first comparison result is that the change value of the two adjacent frames of data exceeds the drift threshold, the two adjacent frames of data are used as the first retained data;

[0032] If the first comparison result is that the change value of the two adjacent frames of data is within the range of the drift threshold, the two adjacent frames of data are used as data to be retained;

[0033] Final cleaned data of the telemetry data is obtained according to the first retained data and the data to be retained.

[0034] In some embodiments, obtaining final cleaned data of the telemetry data based on the first retained data and the data to be retained includes:

[0035] Sampling the data to be retained based on a sampling period to obtain second retained data;

[0036] Final cleaned data of the telemetry data is obtained according to the first retained data and the second retained data.

[0037] In a second aspect, an embodiment of the present disclosure provides a spacecraft telemetry data cleaning device, the device comprising:

[0038] Acquisition module, used to obtain telemetry data of spacecraft;

[0039] a preprocessing module, configured to sort the telemetry data in chronological order and preprocess the sorted telemetry data to obtain first intermediate data;

[0040] The cleaning module is used to clean the first intermediate data based on the source information of the telemetry data and / or the time information of the telemetry data to obtain cleaned data.

[0041] In a third aspect, the present disclosure further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the computing method of the first aspect when executing the program.

[0042] In a fourth aspect, the present disclosure further provides a computer-readable storage medium storing a computer program for executing the computing method of the first aspect.

[0043] In a fifth aspect, the present disclosure further provides a computer program product, comprising a computer program / instruction, which implements the steps of the computing method of the first aspect when the computer program / instruction is executed by a processor.

[0044] The disclosed embodiment cleans the telemetry data according to the source information and key time information of the telemetry data, which can improve the efficiency of data cleaning and ensure that the key data corresponding to the key time will not be lost, thereby improving the utilization efficiency of the telemetry data for secondary analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 A flowchart of a spacecraft telemetry data cleaning method provided in an embodiment of the present disclosure;

[0047] Figure 2 A schematic diagram of a spacecraft telemetry data cleaning method provided by an embodiment of the present disclosure;

[0048] Figure 3 A structural block diagram of a spacecraft telemetry data cleaning device provided in an embodiment of the present disclosure;

[0049] Figure 4 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention is further described in detail and completely below with reference to the accompanying drawings and specific embodiments. It is apparent that the embodiments described are only a portion of the embodiments of the present disclosure, and not all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present disclosure without creative effort are within the scope of protection of the present disclosure.

[0051] In the field of space TT&C, traditional data storage methods parse raw TT&C data into structured data and use relational databases such as Oracle or MPP for real-time data storage. Due to the unique nature of large-scale space missions, all TT&C data must be stored completely during mission execution, with no room for data loss. Depending on the spacecraft design, various sensor data is periodically downloaded. Consequently, databases store multiple copies of data sent by multiple TT&C stations without performing data sampling and deduplication. After mission completion or at regular intervals, to free up storage resources, historical data is often compressed and migrated using database backup files. While this method of compressing historical data to free up storage resources to a certain extent, it also hinders data reuse. Querying and analyzing historical data often requires re-decompression and re-importing, significantly increasing workload complexity and reducing efficiency while also further tying up storage resources.

[0052] In order to improve the efficiency of secondary use of historical data, another technical solution for the long-term storage of aerospace measurement and control data is to export the data in segments and compress and dump it. During the dumping process, routine data cleaning such as data missing detection, outlier removal, and sampling deduplication is performed. When secondary use is required, small batches of data are decompressed and imported according to the user's query time period. This method provides a better solution for short-cycle data secondary analysis application scenarios, solving the problem of the inconvenience of reusing short-cycle data. However, conventional data cleaning cannot effectively perform substantial deduplication on aerospace measurement and control data. Simple sampling deduplication or smoothing processing will lead to the loss of data features at special moments such as remote control command execution and key state changes. It is not suitable for data usage scenarios unique to aerospace measurement and control. Moreover, for long-cycle data analysis scenarios, the problems faced by technical solution one still exist.

[0053] Based on the above-mentioned problems, in a first aspect, an embodiment of the present disclosure provides a method for cleaning spacecraft telemetry data.

[0054] Figure 1 This is a flow chart of a method for cleaning spacecraft telemetry data provided by an embodiment of the present disclosure. Figure 1 As shown, the data cleaning method specifically includes the following steps:

[0055] S11. Acquire telemetry data from the spacecraft. The telemetry data refers to data generated during the spacecraft's mission execution. The telemetry data includes at least telemetry parameter data, test data, and temporary data.

[0056] S12. Sort the telemetry data in chronological order, and pre-process the sorted telemetry data to obtain first intermediate data.

[0057] Specifically, sorting the acquired telemetry data in chronological order in advance can avoid multiple sorting of the telemetry data during post-processing, thereby improving the overall efficiency of data cleaning.

[0058] S13. Based on the source information of the telemetry data and / or the key time information of the telemetry data, the first intermediate data is cleaned to obtain cleaned data.

[0059] Specifically, spacecraft telemetry data is typically tracked synchronously using multiple transmission stations. The same data may be sent through different transmission stations, resulting in identical data. Furthermore, for spacecraft telemetry data, during mission execution, the resulting parameters remain largely unchanged, or undergo only minor changes, for the majority of the time. Only at critical moments do these parameters experience significant changes. Telemetry data that remains unchanged or undergoes only minor changes can be considered identical. However, the telemetry data corresponding to critical moments warrants special attention.

[0060] The present disclosure may perform data cleaning on the first intermediate data based on only the source information or key time information of the telemetry data, or may perform data cleaning based on both the source information or key time information of the telemetry data and the first intermediate data. The present disclosure does not limit this.

[0061] The disclosed embodiment cleans the telemetry data based on the source information and / or key time information of the telemetry data, which can improve the efficiency of data cleaning and ensure that the key data corresponding to the key time will not be lost, thereby improving the utilization efficiency of the telemetry data for secondary analysis.

[0062] In some embodiments, step S12 specifically includes the following steps:

[0063] S121. Sort the telemetry data in chronological order to obtain time series data.

[0064] S122. Based on a preset data type flag field and data time period, delete the test data and temporary data in the time series data to obtain first intermediate data.

[0065] Specifically, over 90% of a spacecraft's telemetry data consists of telemetry parameter results, while the remaining 10% consists of useless data such as test data and temporary data. Pre-set data type flags and data time periods can be pre-stored in a database table, allowing the entirety of useless telemetry data to be deleted by reading the database table. The data type flags can be used to identify test data. The corresponding time periods can be used to determine which data is temporary data generated during the spacecraft's mission.

[0066] The present disclosure can delete the test data and temporary data in the telemetry data as a whole according to the preset data type flag field and data time period, thereby reducing the difficulty of subsequent data cleaning.

[0067] In some embodiments, step S13, performing data cleaning on the first intermediate data based on the source information of the telemetry data and / or the time information of the telemetry data to obtain cleaned data, specifically includes the following steps:

[0068] S131. Perform a first data cleaning on the first intermediate data based on the source information of the telemetry data to obtain first cleaned data.

[0069] S132: Perform a second data cleaning on the first cleaned data according to the key time information of the telemetry data to obtain cleaned data.

[0070] Specifically, in the disclosed embodiment, multiple copies of data from the same source within the first intermediate data are first cleaned based on the source information of the telemetry data to produce first cleaned data. All data within this first cleaned data originate from different sources and represent distinct data sets. Furthermore, the data from different sources are cleaned a second time based on key time information, retaining only the key data. This setup not only effectively deduplicates the telemetry data but also ensures that key features within the data are not lost, thus ensuring the integrity of the data features.

[0071] In some embodiments, the source information of the telemetry data includes the on-board time of the transmission station corresponding to the telemetry data and the source packet identifier of the telemetry data.

[0072] Step S131: performing a first data cleansing on the first intermediate data based on the source information of the telemetry data to obtain first cleansed data, specifically comprising the following steps:

[0073] Step S311: Determine whether there is data with the same name in the first intermediate data according to the on-device time and source packet identifier of the transmission station corresponding to each frame of data in the first intermediate data.

[0074] Step S312: Deduplication is performed on the data with the same name in the first intermediate data to obtain first cleaned data.

[0075] Specifically, when multiple stations track telemetry data, data with the same name (i.e., identical data) may be sent through different stations. Due to the transmission latency constraints of the stations, the data arrives at different times for storage. Therefore, when stored, the data with the same name is stored as multiple entries in the database table. To address this situation, the disclosed embodiments identify the on-device time and source packet identifier of the transmitting station in the data record to determine whether the data meets the characteristics of data with the same name, and then perform deduplication on the data with the same name.

[0076] The on-board time of the transmission station refers to the time when the transmission station collects the telemetry data of the spacecraft, that is, the time when the spacecraft generates the telemetry data. Data with the same name have the same on-board time at the transmission station. The source packet identifier is used to identify the source of the telemetry data. Specifically, the same telemetry data will be copied into multiple copies and sent through different channels to prevent data loss when a transmission link device fails. Therefore, when the on-board time and source packet identifier corresponding to at least two pieces of telemetry data are the same, it can be determined that the at least two pieces of data are data with the same name.

[0077] In some embodiments, the process of determining whether there is data with the same name in step S311 specifically includes: performing a frame-by-frame comparison on any two adjacent frames of data in the first intermediate data, and if the device time and source packet identifier of the two adjacent frames of data in the first intermediate data are the same, then the two adjacent frames of data are determined to be data with the same name. In other words, the first intermediate data is compared frame by frame, and during the comparison process, null values ​​are skipped.

[0078] The disclosed embodiment determines whether two pieces of data are identical based on the device time and source packet identifier. This determination method is simple and the results are accurate. Of course, the disclosed embodiment can also determine whether two pieces of data are identical based on other methods. For example, by comparing the two pieces of data field by field, if each field of the two pieces of data is identical, the two pieces of data can be confirmed to be identical. The disclosed embodiment does not limit the method for determining whether two pieces of data are identical.

[0079] In some embodiments, step S312, deduplication processing is performed on the data with the same name in the first intermediate data to obtain first cleaned data, specifically including: for the data with the same name in the first intermediate data, only one copy of the data with the same name is retained, and all transmission station information corresponding to the data with the same name is saved to obtain the first cleaned data.

[0080] Among them, all the station information corresponding to the data with the same name refers to all the station identifiers of the data with the same name. When deduplicating the data with the same name, it is necessary to retain not only one copy of the data with the same name but also the station information of the data with the same name.

[0081] For example, the telemetry data includes 100 pieces of data, of which 20 pieces of data are from the same source and sent through different measurement stations. The first cleaned data obtained after the first cleaning only includes the other 80 pieces of data and one of the 20 pieces of data, as well as the 20 measurement station information corresponding to the one piece of data.

[0082] This setting can track the transmission status of all transmission stations, avoid timely processing when a transmission station fails, and thus ensure the integrity of telemetry data collection.

[0083] In some embodiments, retaining a copy of the data with the same name in step S312 can be specifically achieved by returning the index of any copy of the data with the same name in the first intermediate data, so that after all the data are cleaned, the cleaned data is obtained from the first intermediate data based on the returned index. This arrangement can avoid the introduction of duplicate data. For example, when comparing the first frame data and the second frame data, it is determined that the first frame data and the second frame data are data with the same name. At this time, the first frame data is retained (the second frame data can also be retained, and the first frame data is retained as an example here). The comparison continues. At this time, the third frame data is adjacent to the first frame data. The third frame data is compared with the retained first frame data. Similarly, the first frame data and the third frame data are data with the same name, and the first frame number is retained. At this point, the first frame data is retained repeatedly. In addition, in the embodiment of the present disclosure, the acquired telemetry data is cleaned multiple times, and duplicate data may also exist in the multiple data cleaning processes. Therefore, the present disclosure retains the index of the data, and after the data cleaning is completed, the cleaned data is uniformly obtained based on the index corresponding to the data.

[0084] In some embodiments, the key time information of the telemetry data includes an issuance schedule and an important event schedule of the telemetry data.

[0085] In step S132, performing a second data cleansing on the first cleaned data according to the key time information of the telemetry data to obtain cleaned data specifically includes the following steps:

[0086] Step S321: Obtain the dispatch schedule and important event schedule of the telemetry data.

[0087] Step S322: Determine key frames in the first cleaned data according to the command schedule and important event schedule of the telemetry data.

[0088] Step S323: Mark the key frames in the first cleaned data to obtain cleaned data.

[0089] Specifically, the telemetry data's command issuance timeline and key event timeline are mapped to the first cleaned data. Keyframe data corresponding to the command issuance time and key events in the first cleaned data are marked, thereby preserving the keyframe data and the multiple frames (preset values) before and after the keyframe data in the first cleaned data. The command issuance time is the time when the command is input into the spacecraft, and the spacecraft performs the corresponding task based on the received command. Key events refer to specific tasks performed by the spacecraft. The data corresponding to the command issuance time and key event times are feature point data in the telemetry data.

[0090] In some embodiments, after the telemetry data's command schedule and important event schedule are mapped to the first cleaned data, bfill and ffill can be used to perform T-frame padding on the command time, generating an index. The index of the frame data corresponding to the command time and important event time is then returned. After all cleaning steps are completed, the cleaned data can be obtained based on the returned index. This configuration can avoid the introduction of duplicate data in the cleaned data.

[0091] For example, among 100 pieces of data, the final returned results may be 1, 10, 11, 12, 12, 14, 15, and 50. In other words, according to the command schedule and the important event schedule, 10-15 frames of data are determined as feature point data.

[0092] In some data, the data cleaning method includes not only the above steps S11 to S13, but also step S14: sampling the cleaned data based on a data drift threshold to obtain the final cleaning result of the telemetry data.

[0093] Specifically, in the spacecraft field, if the difference between two frames of data is relatively small, they can be considered identical. For example, if a parameter value of a spacecraft changes from 5.1 to 5.2, it will not have any impact on the spacecraft's execution status. However, if a parameter value changes from 0.5 to 1.6, it may affect the flight status. Therefore, the disclosed embodiments clean at least some data using a data drift threshold, thereby improving the quality of spacecraft telemetry data and maximizing storage resources.

[0094] In some embodiments, step S14, sampling the cleaned data based on a data drift threshold to obtain a final cleansing result of the telemetry data, specifically includes the following steps:

[0095] S141 . Determine a data drift threshold according to a change trend of the cleaned data.

[0096] S142: Calculate the change value of any two adjacent frames of data in the cleaned data.

[0097] S143 : Compare the change value between every two adjacent frames of data with the data drift threshold to obtain a first comparison result.

[0098] S144. Obtain a final cleaning result of the telemetry data according to the first comparison result.

[0099] In some embodiments, step S144, obtaining a final cleaning result of the telemetry data based on the first comparison result, includes:

[0100] If the first comparison result shows that the change between the two adjacent frames of data exceeds the drift threshold, the two adjacent frames of data are used as first retained data. If the first comparison result shows that the change between the two adjacent frames of data is within the drift threshold, the two adjacent frames of data are used as data to be retained. The final cleaned data of the telemetry data is obtained based on the first retained data and the data to be retained.

[0101] Specifically, if the first comparison result shows that the change value between the two adjacent frames of data exceeds the drift threshold, it indicates that there is a significant difference between the two adjacent frames of data, and in this case, the two adjacent frames of data need to be retained. If the first comparison result shows that the change value between the two adjacent frames of data is within the drift threshold, it indicates that there is no significant difference between the two adjacent frames of data, and there is room for data cleaning.

[0102] In some embodiments, obtaining final cleaned data of the telemetry data based on the first retained data and the data to be retained includes:

[0103] Sampling the data to be retained based on a sampling period to obtain second retained data;

[0104] Final cleaned data of the telemetry data is obtained according to the first retained data and the second retained data.

[0105] Specifically, multiple frames of data whose change values ​​are within the range of the drift threshold cannot be completely cleaned up. The same data (data whose change values ​​are within the range of the drift threshold) needs to be sent all the time to facilitate subsequent research on the changing trend of the data, the time the parameter values ​​corresponding to the data are maintained, and so on.

[0106] In this embodiment of the present disclosure, the second retained data can be sampled at equal intervals based on a sampling period. Specifically, multiple frames of data with change values ​​within the drift threshold can be time-sequentially timestamped, and the timestamps can be divided by the sampling period to generate an index corresponding to the second retained data. It is understood that a corresponding index can also be generated for the first retained data. Finally, the final cleaned data can be obtained based on the indexes of the first and second retained data.

[0107] Figure 2 A schematic diagram of a spacecraft telemetry data cleaning method provided in an embodiment of the present disclosure.

[0108] like Figure 2 As shown in the figure, when cleaning telemetry data, the data is first preprocessed, removing useless data such as test data and temporary data. The preprocessed data is then compared row by row, skipping null values, and the data from multiple stations is fused. The fused result can also be used to generate an aggregate frame index. Secondly, feature point comparisons are performed on the preprocessed data using frame padding, and the T-frame indexes before and after the feature point are indexed. Finally, data in the preprocessed data with changes exceeding a drift threshold are retained, and the remaining data is periodically sampled to generate a sampling frame index. Finally, the aggregate frame index, the T-frame indexes before and after the feature point, and the sampling frame index are combined to generate the final cleaning result. The use of indexes for this combination is to avoid the introduction of duplicate data.

[0109] It can be understood that the above-mentioned aggregate frame index, T-frame index before and after the feature point, and sampling frame index can be cleaned separately according to the above steps, and finally the results of the three cleanings can be combined to obtain the final cleaned data. The three data cleaning methods can also be executed in a sequential and progressive manner. This disclosure does not impose any restrictions on this.

[0110] This paper uses a combination of strategies, such as multi-station homonymous data fusion, key time feature data retention, data drift threshold setting, and evenly spaced sampling within the threshold, to effectively clean telemetry data stored in relational databases of large spacecraft, while ensuring data feature integrity and improving the efficiency of secondary analysis of historical data. Specifically, this paper comprehensively analyzes the time points of remote control command transmission, key event judgment time points, and data jump time points (i.e., the time when the change between two adjacent frames of data exceeds the drift threshold). For these time points, a retention strategy is formulated based on actual application requirements to ensure that feature data characteristics are not lost.

[0111] Furthermore, the present disclosure can achieve compression of relational telemetry result data of large spacecraft through the comprehensive application of multiple cleaning strategies, release storage resources to the greatest extent, reduce storage costs, and improve the secondary utilization efficiency of space measurement and control relational data.

[0112] Furthermore, the present disclosure can improve the quality of aerospace tracking and control data and the efficiency of data query and analysis by effectively cleaning aerospace tracking and control data, laying a high-quality data foundation for long-term large-scale spacecraft tracking and control data analysis and mining, and carrying out machine learning.

[0113] Secondly, based on the same inventive concept, the disclosed embodiment also provides a spacecraft telemetry data cleaning device.

[0114] Figure 3 This is a structural block diagram of a spacecraft telemetry data cleaning device 300 provided in an embodiment of the present disclosure. Figure 3 As shown, the device 300 includes an acquisition module 301 , a pre-processing module 302 and a cleaning module 303 .

[0115] The acquisition module 301 is used to acquire telemetry data from a spacecraft. The preprocessing module 302 is used to sort the telemetry data in chronological order and preprocess the sorted telemetry data to obtain first intermediate data. The cleaning module 303 is used to clean the first intermediate data based on source information and / or key time information of the telemetry data to obtain cleaned data.

[0116] The apparatus 300 provided in the present disclosure can clean the first intermediate data based solely on the source information or key time information of the telemetry data, or can clean the first intermediate data based on both the source information and key time information of the telemetry data. The present disclosure is not limited in this regard. The embodiments of the present disclosure clean the telemetry data based on the source information and / or key time information of the telemetry data, thereby improving the efficiency of data cleaning and ensuring that key data corresponding to the key time is not lost, thereby improving the efficiency of secondary analysis of the telemetry data.

[0117] In some embodiments, the preprocessing module 302 includes a sorting unit and a deletion unit, wherein the sorting unit is used to sort the telemetry data in chronological order to obtain time series data; the deletion unit is used to delete the test data and temporary data in the time series data based on a preset data type flag field and data time period to obtain first intermediate data.

[0118] In some embodiments, the cleaning module 303 includes a first cleaning unit 313 and a second cleaning unit 323, wherein the first cleaning unit 313 is used to perform a first data cleaning on the first intermediate data based on the source information of the telemetry data to obtain first cleaned data; the second cleaning unit 323 is used to perform a second data cleaning on the first cleaned data based on the key time information of the telemetry data to obtain cleaned data.

[0119] In some embodiments, the source information of the telemetry data includes the on-device time of the transmission station corresponding to the telemetry data and the source packet identifier of the telemetry data. The first cleaning unit 313 includes a determination unit and a deduplication unit, wherein the determination unit is configured to determine whether there is data with the same name in the first intermediate data based on the on-device time and source packet identifier corresponding to each frame of data in the first intermediate data; and the deduplication unit is configured to dedupe data with the same name in the first intermediate data to obtain first cleaned data.

[0120] In some embodiments, the deduplication unit is specifically configured to retain only one copy of the data with the same name in the first intermediate data, and save all transmission station information corresponding to the data with the same name, to obtain first cleaned data.

[0121] In some embodiments, the key time information of the telemetry data includes the telemetry data's dispatch schedule and important event schedule, and the second cleaning unit 323 includes an acquisition unit, a key frame determination unit, and a second cleaning sub-unit. The acquisition unit is configured to acquire the telemetry data's dispatch schedule and important event schedule; the key frame determination unit is configured to determine key frames in the first cleaned data based on the telemetry data's dispatch schedule and important event schedule; and the second cleaning sub-unit is configured to mark the key frames in the first cleaned data to obtain cleaned data.

[0122] In some embodiments, the second cleaning unit 323 may also directly perform the cleaning steps for the first cleaned data as described above on the first intermediate data.

[0123] In some embodiments, the apparatus 300 further includes a sampling module 304 , which is configured to sample the cleaned data based on a data drift threshold to obtain a final cleaning result of the telemetry data.

[0124] In some embodiments, the sampling module 304 includes a trend determination unit, a calculation unit, a comparison unit, and a third cleaning unit. The trend determination unit is configured to determine a data drift threshold based on a change trend of the cleaned data; the calculation unit is configured to calculate a change value between any two adjacent frames of data in the cleaned data; the comparison unit is configured to compare the change value between any two adjacent frames of data with the data drift threshold to obtain a first comparison result; and the third cleaning unit is configured to obtain a final cleaning result of the telemetry data based on the first comparison result.

[0125] In some embodiments, the third cleaning unit is specifically used to use the two adjacent frames of data as the first retained data if the first comparison result is that the change value of the two adjacent frames of data exceeds the drift threshold; if the first comparison result is that the change value of the two adjacent frames of data is within the range of the drift threshold, then use the two adjacent frames of data as the data to be retained; and obtain the final cleaned data of the telemetry data based on the first retained data and the data to be retained.

[0126] In some embodiments, when the third cleaning unit executes the step of obtaining the final cleaned data of the telemetry data based on the first retained data and the data to be retained, it is specifically used to sample the data to be retained based on a sampling period to obtain second retained data; and obtain the final cleaned data of the telemetry data based on the first retained data and the second retained data.

[0127] In some embodiments, the first cleaning unit 313, the second cleaning unit 323 and the third cleaning unit mentioned in the device 500 all return the index of the corresponding cleaning result data in the first intermediate data during the data cleaning process, thereby avoiding duplicate data in the final cleaned data.

[0128] The apparatus 300 provided herein utilizes a first cleaning unit for fusing data with the same name from multiple stations, a second cleaning unit for retaining key time signature data, and a sampling module 304 for setting data drift thresholds and performing evenly spaced sampling within these thresholds. This effectively cleans telemetry data stored in a relational database of a large spacecraft, while ensuring data feature integrity and improving the efficiency of secondary analysis of historical data. The apparatus 300 comprehensively analyzes the time points at which remote control commands are sent, the time points at which key events are determined, and the time points at which data jumps (i.e., the time at which the change between two adjacent frames of data exceeds a drift threshold) and formulates a retention strategy for these data points based on actual application requirements to ensure that signature data features are not lost.

[0129] It can be understood that the specific details of the device provided in the embodiment of the present disclosure can refer to the specific details of the method embodiment of the first aspect mentioned above, and will not be repeated here.

[0130] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods described in the first aspect when executing the computer program.

[0131] Specifically, Figure 4 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present disclosure, such as Figure 4As shown, the electronic device 004 includes a processor 401 , a memory 402 and a bus 403 . The processor 401 and the memory 402 communicate with each other via the bus 403 .

[0132] The processor 401 is configured to call program instructions in the memory 402 to execute the methods provided by the above method embodiments.

[0133] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program of any one of the methods described in the first aspect.

[0134] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect above.

[0135] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0139] Specific embodiments are used in the present disclosure to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core ideas of the present disclosure. At the same time, for those skilled in the art, according to the ideas of the present disclosure, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present disclosure.

Claims

1. A spacecraft telemetry data cleaning method, characterized in that: The method comprises: Acquire telemetry data from spacecraft; Sorting the telemetry data in chronological order to obtain time series data; Based on a preset data type flag field and a data time period, test data and temporary data in the time series data are deleted to obtain first intermediate data, wherein the data type flag field identifies the test data, temporary data generated during the spacecraft mission execution is determined based on the data time period, and the test data and temporary data in the telemetry data are deleted as a whole based on the data type flag field and the data time period; performing data cleaning on the first intermediate data based on source information of the telemetry data and / or key time information of the telemetry data to obtain cleaned data, wherein the source information of the telemetry data includes an on-device time of a transmission station corresponding to the telemetry data, and a source packet identifier of the telemetry data; The step of performing data cleaning on the first intermediate data based on the source information of the telemetry data and / or the time information of the telemetry data to obtain cleaned data includes: performing a first data cleansing on the first intermediate data based on the source information of the telemetry data to obtain first cleansed data; performing a second data cleansing on the first cleaned data according to the key time information of the telemetry data to obtain cleaned data; The performing a first data cleaning on the first intermediate data based on the source information of the telemetry data to obtain first cleaned data includes: determining whether there is data with the same name in the first intermediate data according to the on-device time and source packet identifier of the transmission station corresponding to each frame of data in the first intermediate data; For data with the same name in the first intermediate data, only one copy of the data with the same name is retained, and all transmission station information corresponding to the data with the same name is saved to obtain first cleaned data, where all station information corresponding to the data with the same name refers to all station identifiers that collected the data with the same name; The method further comprises: Determining a data drift threshold according to a change trend of the cleaned data; Calculating the change value of any two adjacent frames of data in the cleaned data; Comparing the change value of each two adjacent frames of data with the data drift threshold to obtain a first comparison result; Obtaining a final cleaning result of the telemetry data according to the first comparison result; Obtaining a final cleaning result of the telemetry data according to the first comparison result includes: If the first comparison result is that the change value of the two adjacent frames of data exceeds the drift threshold, the two adjacent frames of data are used as the first retained data; If the first comparison result is that the change value of the two adjacent frames of data is within the range of the drift threshold, the two adjacent frames of data are used as data to be retained; Sampling the data to be retained based on a sampling period to obtain second retained data; Final cleaned data of the telemetry data is obtained according to the first retained data and the second retained data.

2. The method according to claim 1, characterized in that The key time information of the telemetry data includes a dispatch schedule and an important event schedule of the telemetry data. The second data cleaning is performed on the first cleaned data according to the key time information of the telemetry data to obtain cleaned data, including: Obtain a dispatch schedule and a timeline of significant events for said telemetry data; determining key frames in the first cleaned data based on a dispatch schedule and a key event schedule of the telemetry data; Key frames in the first cleaned data are marked to obtain cleaned data.

3. A spacecraft telemetry data cleaning device, characterized in that: The device comprises: Acquisition module, used to obtain telemetry data of spacecraft; A sorting unit, configured to sort the telemetry data in chronological order to obtain time series data; a deleting unit, configured to delete the test data and temporary data in the time series data based on a preset data type flag field and a data time period to obtain first intermediate data, wherein the data type flag field identifies the test data, the temporary data generated during the spacecraft mission execution is determined based on the data time period, and the test data and temporary data in the telemetry data are deleted as a whole based on the data type flag field and the data time period; a cleaning module, configured to clean the first intermediate data based on source information of the telemetry data and / or key time information of the telemetry data to obtain cleaned data, wherein the source information of the telemetry data includes an on-device time of a transmission station corresponding to the telemetry data and a source packet identifier of the telemetry data; The cleaning module includes a first cleaning unit and a second cleaning unit; The first cleaning unit is configured to perform a first data cleaning on the first intermediate data based on the source information of the telemetry data to obtain first cleaned data; The second cleaning unit is used to perform a second data cleaning on the first cleaning data according to the key time information of the telemetry data to obtain cleaned data; The first cleaning unit includes a determination unit and a deduplication unit, wherein the determination unit is used to determine whether there is data with the same name in the first intermediate data based on the on-device time and source packet identifier of the transmission station corresponding to each frame of data in the first intermediate data; the deduplication unit is used to retain only one copy of the data with the same name in the first intermediate data, and save all transmission station information corresponding to the data with the same name, to obtain first cleaned data, where all station information corresponding to the data with the same name refers to all station identifiers that collected the data with the same name; The device also includes: a sampling module; The sampling module includes a trend determination unit, a calculation unit, a comparison unit and a third cleaning unit; The trend determination unit is used to determine a data drift threshold according to a change trend of the cleaned data; The calculation unit is used to calculate the change value of any two adjacent frames of data in the cleaned data; The comparison unit is used to compare the change value of each two adjacent frames of data with the data drift threshold to obtain a first comparison result; The third cleaning unit is used to obtain a final cleaning result of the telemetry data according to the first comparison result; The third cleaning unit is specifically configured to, if the first comparison result shows that the change value of the two adjacent frames of data exceeds the drift threshold, use the two adjacent frames of data as first retained data; if the first comparison result shows that the change value of the two adjacent frames of data is within the range of the drift threshold, use the two adjacent frames of data as data to be retained; and obtain final cleaned data of the telemetry data based on the first retained data and the data to be retained; When the third cleaning unit executes the step of obtaining the final cleaned data of the telemetry data based on the first retained data and the data to be retained, it is specifically used to sample the data to be retained based on a sampling period to obtain second retained data; and obtain the final cleaned data of the telemetry data based on the first retained data and the second retained data.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to claim 1 or 2 is implemented.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program for executing the method according to claim 1 or 2.

6. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to claim 1 or 2 are implemented.

Images