Spacecraft telemetry data cleaning method and device

Abstract

The invention provides a spacecraft telemetry data cleaning method and device, and belongs to the data processing technology. The method comprises the following steps: acquiring telemetry data of a spacecraft; sorting the telemetry data according to a time sequence, and preprocessing the sorted telemetry data to obtain first intermediate data; and performing data cleaning on the first intermediate data based on information source information of the telemetry data and / or key time information of the telemetry data to obtain cleaned data. According to the method provided by the invention, strategies such as multi-station homonymy data fusion, key time feature data retention, data drift threshold setting and equal interval sampling within the threshold can be comprehensively utilized, effective cleaning of telemetering result data stored in a relational database of a large spacecraft can be realized, and meanwhile, data feature completeness is ensured; and the secondary analysis and utilization efficiency of the historical data is improved.

Description

Technical Field

[0001] The present disclosure belongs to the field of data processing technology, and specifically relates to a spacecraft telemetry data cleaning method and device, electronic equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of aerospace technology, the on-orbit operation time, measurement and control communication coverage, information transmission volume, and transmission frequency of large spacecraft have been greatly improved compared with the past. Correspondingly, the amount of measurement and control data has also shown an order of magnitude growth trend. The massive measurement and control data not only greatly occupies storage resources, but also the large amount of duplicate data is not convenient for subsequent rapid data analysis. Therefore, it is of great practical significance to study the high-frequency measurement and control data cleaning technology of large spacecraft. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art and to provide a spacecraft telemetry data cleaning method and device, a computer-readable storage medium and a computer program product that can release storage resources to the greatest extent and reduce storage costs.

[0004] In order to achieve the above-mentioned purpose, in a first aspect, an embodiment of the present disclosure provides a method for cleaning spacecraft telemetry data, the method comprising:

[0005] Acquire telemetry data from spacecraft;

[0006] Sorting the telemetry data in chronological order, and preprocessing the sorted telemetry data to obtain first intermediate data;

[0007] Based on the source information of the telemetry data and / or the key time information of the telemetry data, the first intermediate data is cleansed to obtain cleansed data.

[0008] In some embodiments, the sorting of the telemetry data in chronological order and preprocessing of the sorted telemetry data to obtain first intermediate data includes:

[0009] Sorting the telemetry data in chronological order to obtain time series data;

[0010] Based on the preset data type flag field and data time period, the test data and temporary data in the time series data are deleted to obtain the first intermediate data.

[0011] In some embodiments, the performing data cleaning on the first intermediate data based on the source information of the telemetry data and / or the time information of the telemetry data to obtain the cleaned data includes:

[0012] Performing a first data cleaning on the first intermediate data based on the source information of the telemetry data to obtain first cleaned data;

[0013] The first cleaned data is cleaned for a second time according to the key time information of the telemetry data to obtain cleaned data.

[0014] In some embodiments, the source information of the telemetry data includes the on-device time of the transmission station corresponding to the telemetry data and the source packet identifier of the telemetry data, and performing a first data cleaning on the first intermediate data based on the source information of the telemetry data to obtain the first cleaned data includes:

[0015] Determine whether there is data with the same name in the first intermediate data according to the on-device time and source packet identifier of the transmission station corresponding to each frame of data in the first intermediate data;

[0016] Deduplication processing is performed on data with the same name in the first intermediate data to obtain first cleaned data.

[0017] In some embodiments, performing deduplication processing on the data with the same name in the first intermediate data to obtain the first cleaned data includes:

[0018] For the data with the same name in the first intermediate data, only one copy of the data with the same name is retained, and all transmission station information corresponding to the data with the same name is saved to obtain first cleaned data.

[0019] In some embodiments, the key time information of the telemetry data includes a command schedule and an important event schedule of the telemetry data, and the second data cleaning of the first cleaned data according to the key time information of the telemetry data to obtain the cleaned data includes:

[0020] Obtaining a dispatch schedule and a timeline of significant events for said telemetry data;

[0021] Determining key frames in the first cleaning data according to the command schedule and the important event schedule of the telemetry data;

[0022] The key frames in the first cleaned data are marked to obtain cleaned data.

[0023] In some embodiments, the method further comprises:

[0024] Based on the data drift threshold, the cleaned data is sampled to obtain the final cleaning result of the telemetry data.

[0025] In some embodiments, sampling the cleaned data based on the data drift threshold to obtain the final cleaning result of the telemetry data includes:

[0026] Determining a data drift threshold according to a change trend of the cleaned data;

[0027] Calculate the change value of any two adjacent frames of data in the cleaned data;

[0028] Compare the change value of each two adjacent frames of data with the data drift threshold to obtain a first comparison result;

[0029] According to the first comparison result, a final cleaning result of the telemetry data is obtained.

[0030] In some embodiments, obtaining a final cleaning result of the telemetry data according to the first comparison result includes:

[0031] If the first comparison result is that the change value of the two adjacent frames of data exceeds the drift threshold, the two adjacent frames of data are used as the first reserved data;

[0032] If the first comparison result is that the change value of the two adjacent frames of data is within the range of the drift threshold, the two adjacent frames of data are used as data to be retained;

[0033] Final cleansing data of the telemetry data is obtained according to the first retained data and the data to be retained.

[0034] In some embodiments, obtaining final cleaned data of the telemetry data according to the first retained data and the data to be retained includes:

[0035] Sampling the data to be retained based on a sampling period to obtain second retained data;

[0036] Final cleansing data of the telemetry data is obtained according to the first retained data and the second retained data.

[0037] In a second aspect, an embodiment of the present disclosure provides a spacecraft telemetry data cleaning device, the device comprising:

[0038] An acquisition module, used to acquire telemetry data of the spacecraft;

[0039] A preprocessing module, used to sort the telemetry data in chronological order and preprocess the sorted telemetry data to obtain first intermediate data;

[0040] The cleaning module is used to clean the first intermediate data based on the source information of the telemetry data and / or the time information of the telemetry data to obtain cleaned data.

[0041] In a third aspect, the present disclosure further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the computing method of the first aspect described above is implemented.

[0042] In a fourth aspect, the present disclosure further provides a computer-readable storage medium storing a computer program for executing the computing method of the first aspect.

[0043] In a fifth aspect, the present disclosure further provides a computer program product, including a computer program / instruction, which implements the steps of the computing method of the first aspect when the computer program / instruction is executed by a processor.

[0044] The disclosed embodiment cleans the telemetry data according to the source information and key time information of the telemetry data, which can improve the efficiency of data cleaning and ensure that the key data corresponding to the key time will not be lost, thereby improving the utilization efficiency of the telemetry data for secondary analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 A flow chart of a method for cleaning spacecraft telemetry data provided by an embodiment of the present disclosure;

[0047] Figure 2 A schematic diagram of a spacecraft telemetry data cleaning method provided by an embodiment of the present disclosure;

[0048] Figure 3 A structural block diagram of a spacecraft telemetry data cleaning device provided in an embodiment of the present disclosure;

[0049] Figure 4 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail and completely in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0051] In the field of aerospace measurement and control, the traditional method of storing aerospace measurement and control data is to parse the original measurement and control data into structured data and use relational database products such as Oracle or MPP for real-time data storage. Due to the particularity of large-scale aerospace missions, all measurement and control data must be stored completely during the execution of the mission, and data loss is not allowed. According to the design of the spacecraft, various sensor data will be periodically and repeatedly transmitted. Therefore, multiple copies of data sent by multiple measurement and control stations will be stored in the database, and the data will not be sampled and deduplicated. After the mission is completed or after a fixed period, in order to release storage resources, the historical data is often compressed, stored and migrated as a whole using database backup files. The use of database backup files to compress and store historical data as a whole has achieved the effect of releasing storage resources to a certain extent, but it has caused difficulties in the reuse of data. When historical data needs to be queried and analyzed, it is often necessary to decompress and import the data again, which greatly increases the difficulty of work, reduces work efficiency, and occupies storage resources again.

[0052] In order to improve the efficiency of secondary use of historical data, another technical solution for the long-term storage of aerospace measurement and control data is to export the data in segments and compress and dump it. During the dumping process, routine data cleaning such as data missing detection, outlier removal, and sampling deduplication is performed. When secondary use is required, small batches of data are decompressed and imported according to the user's query time period. This method provides a better solution for short-cycle data secondary analysis application scenarios and solves the problem of the inconvenience of reusing short-cycle data. However, conventional data cleaning cannot effectively perform substantial deduplication on aerospace measurement and control data. Simple sampling deduplication or smoothing processing will result in the loss of data features at special moments such as remote control command execution and key state changes. It is not suitable for data usage scenarios unique to aerospace measurement and control. In addition, for long-cycle data analysis scenarios, the problems faced by technical solution one still exist.

[0053] Based on the above-mentioned problems, in a first aspect, an embodiment of the present disclosure provides a method for cleaning spacecraft telemetry data.

[0054] Figure 1 The following is a flow chart of a method for cleaning spacecraft telemetry data provided by an embodiment of the present disclosure. Figure 1 As shown, the data cleaning method specifically includes the following steps:

[0055] S11. Acquire telemetry data of the spacecraft. The telemetry data refers to data generated during the spacecraft's mission execution. The telemetry data at least includes telemetry parameter data, test data, and temporary data.

[0056] S12. Sort the telemetry data in chronological order, and preprocess the sorted telemetry data to obtain first intermediate data.

[0057] Specifically, sorting the acquired telemetry data in chronological order in advance can avoid multiple sorting of the telemetry data during post-processing, thereby improving the overall efficiency of data cleaning.

[0058] S13. Based on the source information of the telemetry data and / or the key time information of the telemetry data, the first intermediate data is cleansed to obtain cleansed data.

[0059] Specifically, the telemetry data of a spacecraft is usually tracked synchronously using multiple transmission stations. The same data may be sent through different transmission stations, and the resulting telemetry data contains the same data. In addition, for the telemetry data of a spacecraft, during the mission of the spacecraft, the telemetry parameter result data will basically not change for most of the time, or will only change slightly. Only at certain key time points will the telemetry parameter result data change significantly. Telemetry data that has not changed, or has only changed slightly, can be regarded as the same data. The telemetry data corresponding to the key time is the data that needs to be focused on.

[0060] The present disclosure may perform data cleaning on the first intermediate data based on only the source information or the key time information of the telemetry data, or may perform data cleaning on both the source information or the key time information of the telemetry data and the first intermediate data. The present disclosure does not limit this.

[0061] The disclosed embodiment cleans the telemetry data according to the source information and / or key time information of the telemetry data, which can improve the efficiency of data cleaning and ensure that the key data corresponding to the key time will not be lost, thereby improving the utilization efficiency of the telemetry data for secondary analysis.

[0062] In some embodiments, step S12 specifically includes the following steps:

[0063] S121. Sort the telemetry data in chronological order to obtain time series data.

[0064] S122. Based on a preset data type flag field and a data time period, delete the test data and temporary data in the time series data to obtain first intermediate data.

[0065] Specifically, more than 90% of the telemetry data of the spacecraft is telemetry parameter result data, and the remaining 10% of the data is useless data such as test data and temporary data. The preset data type flag field and data time period can be pre-stored in the database table, and the useless data in the telemetry data can be deleted as a whole by reading the database table. Among them, which data is test data can be identified according to the data type flag field. According to the time period corresponding to the data, it can be determined which data is temporary data generated during the spacecraft's mission.

[0066] The present disclosure can delete the test data and temporary data in the telemetry data as a whole according to the preset data type flag field and data time period, thereby reducing the difficulty of subsequent data cleaning.

[0067] In some embodiments, step S13, based on the source information of the telemetry data and / or the time information of the telemetry data, performing data cleaning on the first intermediate data to obtain cleaned data, specifically includes the following steps:

[0068] S131. Perform a first data cleaning on the first intermediate data based on the source information of the telemetry data to obtain first cleaned data.

[0069] S132. Perform a second data cleaning on the first cleaning data according to the key time information of the telemetry data to obtain cleaned data.

[0070] That is, in the disclosed embodiment, first, according to the source information of the telemetry data, multiple copies of data from the same source in the first intermediate data are cleaned to obtain the first cleaned data. All data in the first cleaned data come from different sources and belong to different data. Further, the data from different sources are cleaned for the second time according to the key time information, and only the key data in the data is retained. Such a setting can not only effectively deduplicate the telemetry data, but also ensure that the key features in the data are not lost, and ensure the integrity of the data features.

[0071] In some embodiments, the source information of the telemetry data includes the on-site time of the transmission station corresponding to the telemetry data and the source packet identifier of the telemetry data.

[0072] Step S131, performing a first data cleaning on the first intermediate data based on the source information of the telemetry data to obtain first cleaned data, specifically comprising the following steps:

[0073] Step S311: Determine whether there is data with the same name in the first intermediate data according to the on-device time and source packet identifier of the transmission station corresponding to each frame of data in the first intermediate data.

[0074] Step S312: deduplicate the data with the same name in the first intermediate data to obtain first cleaned data.

[0075] Specifically, when multiple stations track telemetry data, data with the same name (i.e., the same data) will be sent through different stations. Due to the constraints of the transmission delay of the station, the data arrives at different times for storage. Therefore, when storing, the data with the same name will be stored as multiple data in the database table. For such situations, the embodiment of the present disclosure determines whether the data meets the characteristics of the data with the same name by identifying the on-device time and source packet identifier of the transmitting station in the data record, and then deduplicates the data with the same name.

[0076] Among them, the on-board time of the transmission station refers to the time when the transmission station collects the telemetry data of the spacecraft, that is, the time when the spacecraft generates the telemetry data. The on-board time of the transmission station corresponding to the data with the same name is the same. The source packet identifier is used to identify the source of the telemetry data. Specifically, for the same telemetry data, it will be copied into multiple copies and sent through different channels to prevent data loss when a certain transmission link device fails. Therefore, when the on-board time and source packet identifier corresponding to at least two telemetry data are the same, it can be determined that the at least two data are data with the same name.

[0077] In some embodiments, the process of determining whether there is data with the same name in the first intermediate data in step S311 specifically includes: comparing any two adjacent frames of data in the first intermediate data frame by frame, and if there are two adjacent frames of data in the first intermediate data with the same device time and source packet identifier, then determining that the two adjacent frames of data are data with the same name. That is, comparing the first intermediate data frame by frame, and skipping null values ​​during the comparison process.

[0078] The embodiment of the present disclosure determines whether two pieces of data are data with the same name based on the time on the device and the source packet identifier. The determination method is simple and the result is accurate. Of course, the present disclosure can also determine whether two pieces of data are data with the same name based on other methods. For example, the two pieces of data are compared field by field. If each field of the two pieces of data is the same, it can be confirmed that the two pieces of data are data with the same name. The present disclosure does not limit the method for determining data with the same name.

[0079] In some embodiments, step S312, deduplication processing is performed on the data with the same name in the first intermediate data to obtain the first cleaned data, which specifically includes: for the data with the same name in the first intermediate data, only one copy of the data with the same name is retained, and all transmission station information corresponding to the data with the same name is saved to obtain the first cleaned data.

[0080] Among them, all the station information corresponding to the data with the same name refers to all the station identifiers of the data with the same name. When performing deduplication processing on the data with the same name, it is necessary to retain not only one copy of the data with the same name, but also the station information of the data with the same name.

[0081] For example, the telemetry data includes 100 pieces of data, of which 20 pieces of data are from the same source and sent through different measuring stations. Then the first cleaned data obtained after the first cleaning only includes the other 80 pieces of data and one of the 20 pieces of data, as well as the 20 measuring station information corresponding to the one piece of data.

[0082] This setting can track the transmission status of all transmission stations, avoid timely processing when a transmission station fails, and thus ensure the integrity of telemetry data collection.

[0083] In some embodiments, retaining one copy of the data with the same name in step S312 can be specifically achieved by returning the index of any copy of the data with the same name in the first intermediate data, so that after all the data are cleaned, the cleaned data is obtained from the first intermediate data according to the returned index. Such a setting can avoid the introduction of duplicate data. For example, when comparing the first frame data and the second frame data, it is determined that the first frame data and the second frame data are data with the same name. At this time, the first frame data is retained (the second frame data can also be retained, and the first frame data is retained as an example here), and the comparison is continued. At this time, the third frame data and the first frame data are adjacent data, and the third frame data is compared with the retained first frame data. Similarly, the first frame data and the third frame data are data with the same name, and the first frame number is retained. At this time, the first frame data is repeatedly retained. In addition, in the embodiment of the present disclosure, the acquired telemetry data is cleaned multiple times, wherein duplicate data may also exist in the multiple data cleaning processes. Therefore, the present disclosure retains the index of the data, and then obtains the cleaned data uniformly according to the index corresponding to the data after the data is cleaned.

[0084] In some embodiments, the critical time information of the telemetry data includes an issuance schedule and an important event schedule of the telemetry data.

[0085] In step S132, performing a second data cleaning on the first cleaned data according to the key time information of the telemetry data to obtain cleaned data specifically includes the following steps:

[0086] Step S321, obtaining the dispatching schedule and important event schedule of the telemetry data.

[0087] Step S322: Determine key frames in the first cleaned data according to the command schedule and important event schedule of the telemetry data.

[0088] Step S323: Mark the key frames in the first cleaned data to obtain cleaned data.

[0089] Specifically, the command timetable and important event timetable of the telemetry data are matched with the first cleaned data, and the key frame data corresponding to the command time and the important event in the first cleaned data are marked, so as to retain the key frame data in the first cleaned data and the multiple frames (preset value) before and after the key frame data. Among them, the command time is the time when the command is input into the spacecraft, and the spacecraft performs the corresponding task according to the received command. The important event refers to the specific task performed by the spacecraft. The data corresponding to the command time and the important event time are the feature point data in the telemetry data.

[0090] In some embodiments, after the command timetable and the important event timetable of the telemetry data are matched with the first cleaned data, T frame filling can be performed on the command time through bfill and ffill to generate an index, and the index of the frame data corresponding to the command time and the important event time is returned, so that after all the cleaning steps of the data are completed, the cleaned data can be obtained according to the returned index. This setting can avoid the introduction of duplicate data in the cleaned data.

[0091] For example, among 100 pieces of data, the final returned results may be 1, 10, 11, 12, 12, 14, 15, 50. That is to say, 10-15 frames of data are determined as feature point data according to the order schedule and the important event schedule.

[0092] In some data, the data cleaning method includes not only the above steps S11 to S13, but also step S14: based on the data drift threshold, sampling the cleaned data to obtain the final cleaning result of the telemetry data.

[0093] Specifically, in the field of spacecraft, if the difference between two frames of data is relatively small, the two frames of data can be considered to be the same data. For example, if a parameter value of a spacecraft changes from 5.1 to 5.2, it will not have any impact on the execution state of the spacecraft. If a parameter value changes from 0.5 to 1.6, it may affect the state of the aerospace. Therefore, the embodiment of the present disclosure cleans at least part of the data through the data drift threshold, which can improve the data quality of the telemetry data of the spacecraft and release storage resources to the greatest extent.

[0094] In some embodiments, step S14, sampling the cleaned data based on the data drift threshold to obtain the final cleaning result of the telemetry data, specifically includes the following steps:

[0095] S141. Determine a data drift threshold according to a change trend of the cleaned data.

[0096] S142, calculating the change value of any two adjacent frames of data in the cleaned data.

[0097] S143: Compare the change value between every two adjacent frames of data with the data drift threshold to obtain a first comparison result.

[0098] S144. Obtain a final cleaning result of the telemetry data according to the first comparison result.

[0099] In some embodiments, step S144, obtaining a final cleaning result of the telemetry data according to the first comparison result, includes:

[0100] If the first comparison result is that the change value of the two adjacent frames of data exceeds the drift threshold, the two adjacent frames of data are used as the first retained data. If the first comparison result is that the change value of the two adjacent frames of data is within the range of the drift threshold, the two adjacent frames of data are used as the data to be retained. According to the first retained data and the data to be retained, the final cleaned data of the telemetry data is obtained.

[0101] Specifically, if the first comparison result is that the change value of the two adjacent frames of data exceeds the drift threshold, it means that there is an obvious difference between the two adjacent frames of data, and the two adjacent frames of data need to be retained at the same time. If the first comparison result is that the change value of the two adjacent frames of data is within the range of the drift threshold, it means that there is no obvious difference between the two adjacent frames of data, and there is still room for data cleaning.

[0102] In some embodiments, obtaining final cleaned data of the telemetry data according to the first retained data and the data to be retained includes:

[0103] Sampling the data to be retained based on a sampling period to obtain second retained data;

[0104] Final cleansing data of the telemetry data is obtained according to the first retained data and the second retained data.

[0105] Specifically, multiple frames of data whose change values ​​are within the range of the drift threshold cannot be completely cleaned up, and the same data (data whose change values ​​are within the range of the drift threshold) needs to be sent all the time to facilitate subsequent research on the data's changing trend, the time the parameter value corresponding to the data is maintained, and so on.

[0106] Among them, the embodiment of the present disclosure can sample the second retained data at equal intervals based on the sampling period. Specifically, multiple frames of data whose change values ​​are within the range of the drift threshold can be generated with timestamps in chronological order, and the timestamps are divided by the sampling period to generate the index corresponding to the second retained data. It can be understood that the corresponding index can also be generated for the first retained data, and finally the final cleaned data is obtained based on the indexes of the first retained data and the second retained data.

[0107] Figure 2 A schematic diagram of a spacecraft telemetry data cleaning method provided in an embodiment of the present disclosure.

[0108] like Figure 2 As shown in the figure, when cleaning the telemetry data, the data is first preprocessed, that is, useless data such as test data and temporary data in the telemetry data is removed; then the preprocessed data is compared row by row, and the null values ​​are skipped, the multi-station data is fused, and the fused result can be used to generate an aggregate frame index; secondly, the preprocessed data can be compared with feature points through frame filling, and the T frames before and after the feature points are indexed; thirdly, the data in the preprocessed data that exceeds the drift threshold can be retained, and the other data can be sampled according to the period to generate a sampling frame index. Finally, the aggregate frame index, the T frame index before and after the feature point, and the sampling frame index are combined to generate the final cleaning result. The index is used for synthesis here to avoid introducing duplicate data.

[0109] It can be understood that the above-mentioned aggregate frame index, T-frame index before and after the feature point, and sampling frame index can be cleaned separately according to the above steps, and finally the three cleaning results are combined to obtain the final cleaned data. The three data cleaning methods can also be executed in a sequential and progressive manner, and the present disclosure does not impose any restrictions on this.

[0110] The disclosure can realize the effective cleaning of telemetry result data stored in the relational database of large spacecraft by comprehensively using strategies such as fusion of multi-station data with the same name, retention of key time feature data, setting of data drift threshold, and sampling at equal intervals within the threshold, while ensuring the integrity of data features and improving the efficiency of secondary analysis and utilization of historical data. Among them, the disclosure comprehensively analyzes the time point of sending remote control commands, the time point of judging key events, and the time point of data jump (that is, the data whose change value of two adjacent frames of data exceeds the drift threshold), and formulates a retention strategy for the above time point data according to actual application needs to ensure that the feature data features are not lost.

[0111] Furthermore, the present disclosure can realize the compression of relational telemetry result data of large spacecraft through the comprehensive application of multiple cleaning strategies, release storage resources to the greatest extent, reduce storage costs, and improve the secondary utilization efficiency of aerospace measurement and control relational data.

[0112] Furthermore, the present disclosure can improve the quality of aerospace tracking and control data and the efficiency of data query and analysis by effectively cleaning the data, thereby laying a high-quality data foundation for the analysis and mining of long-term large-scale spacecraft tracking and control data and conducting machine learning.

[0113] In a second aspect, based on the same inventive concept, the disclosed embodiment also provides a spacecraft telemetry data cleaning device.

[0114] Figure 3 FIG. 3 is a structural block diagram of a spacecraft telemetry data cleaning device 300 provided in an embodiment of the present disclosure. Figure 3 As shown, the device 300 includes an acquisition module 301 , a pre-processing module 302 and a cleaning module 303 .

[0115] The acquisition module 301 is used to acquire telemetry data of the spacecraft. The preprocessing module 302 is used to sort the telemetry data in chronological order and preprocess the sorted telemetry data to obtain first intermediate data. The cleaning module 303 is used to clean the first intermediate data based on the source information of the telemetry data and / or the key time information of the telemetry data to obtain cleaned data.

[0116] The device 300 provided in the present disclosure can perform data cleaning on the first intermediate data based on only the source information or the key time information of the telemetry data, or can perform data cleaning on the first intermediate data based on the source information or the key time information of the telemetry data at the same time. The present disclosure does not limit this. The embodiment of the present disclosure cleans the telemetry data based on the source information and / or the key time information of the telemetry data, which can improve the efficiency of data cleaning, and at the same time ensure that the key data corresponding to the key time will not be lost, thereby improving the utilization efficiency of the telemetry data for secondary analysis.

[0117] In some embodiments, the preprocessing module 302 includes a sorting unit and a deleting unit, wherein the sorting unit is used to sort the telemetry data in chronological order to obtain time series data; the deleting unit is used to delete the test data and temporary data in the time series data based on a preset data type flag field and data time period to obtain the first intermediate data.

[0118] In some embodiments, the cleaning module 303 includes a first cleaning unit 313 and a second cleaning unit 323, wherein the first cleaning unit 313 is used to perform a first data cleaning on the first intermediate data based on the source information of the telemetry data to obtain first cleaned data; the second cleaning unit 323 is used to perform a second data cleaning on the first cleaned data according to the key time information of the telemetry data to obtain cleaned data.

[0119] In some embodiments, the source information of the telemetry data includes the on-device time of the transmission station corresponding to the telemetry data and the source packet identifier of the telemetry data. The first cleaning unit 313 includes a determination unit and a deduplication unit, wherein the determination unit is used to determine whether there is data with the same name in the first intermediate data according to the on-device time and source packet identifier of the transmission station corresponding to each frame of data in the first intermediate data; the deduplication unit is used to perform deduplication processing on the data with the same name in the first intermediate data to obtain the first cleaned data.

[0120] In some embodiments, the deduplication unit is specifically used to retain only one copy of the data with the same name in the first intermediate data, and save all transmission station information corresponding to the data with the same name, so as to obtain the first cleaned data.

[0121] In some embodiments, the key time information of the telemetry data includes the dispatching schedule and the important event schedule of the telemetry data, and the second cleaning unit 323 includes an acquisition unit, a key frame determination unit, and a second cleaning subunit. The acquisition unit is used to acquire the dispatching schedule and the important event schedule of the telemetry data; the key frame determination unit is used to determine the key frames in the first cleaning data according to the dispatching schedule and the important event schedule of the telemetry data; and the second cleaning subunit is used to mark the key frames in the first cleaning data to obtain the cleaned data.

[0122] In some embodiments, the second cleaning unit 323 may also directly perform the cleaning steps for the first intermediate data as described above for the first cleaning data.

[0123] In some embodiments, the device 300 further includes a sampling module 304, and the sampling module 304 is used to sample the cleaned data based on a data drift threshold to obtain a final cleaning result of the telemetry data.

[0124] In some embodiments, the sampling module 304 includes a trend determination unit, a calculation unit, a comparison unit and a third cleaning unit. The trend determination unit is used to determine the data drift threshold according to the change trend of the cleaned data; the calculation unit is used to calculate the change value of any two adjacent frames of data in the cleaned data; the comparison unit is used to compare the change value of each two adjacent frames of data with the data drift threshold to obtain a first comparison result; the third cleaning unit is used to obtain the final cleaning result of the telemetry data according to the first comparison result.

[0125] In some embodiments, the third cleaning unit is specifically used to, if the first comparison result is that the change value of the two adjacent frames of data exceeds the drift threshold, then use the two adjacent frames of data as the first retained data; if the first comparison result is that the change value of the two adjacent frames of data is within the range of the drift threshold, then use the two adjacent frames of data as the data to be retained; based on the first retained data and the data to be retained, obtain the final cleaning data of the telemetry data.

[0126] In some embodiments, when the third cleaning unit executes the step of obtaining the final cleaned data of the telemetry data based on the first retained data and the data to be retained, it is specifically used to sample the data to be retained based on a sampling period to obtain second retained data; and obtain the final cleaned data of the telemetry data based on the first retained data and the second retained data.

[0127] In some embodiments, the first cleaning unit 313, the second cleaning unit 323 and the third cleaning unit mentioned in the device 500 all return the index of the corresponding cleaning result data in the first intermediate data during the data cleaning process, thereby avoiding duplicate data in the final cleaned data.

[0128] The device 300 provided by the present disclosure can realize the effective cleaning of the telemetry result data stored in the relational database of a large spacecraft by setting a first cleaning unit for merging data with the same name from multiple stations, a second cleaning unit for retaining key time feature data, and setting a sampling module 304 for setting data drift thresholds and sampling at equal intervals within the thresholds, while ensuring the integrity of data features and improving the efficiency of secondary analysis and utilization of historical data. Among them, the present disclosure comprehensively analyzes the time point of sending the remote control command, the time point of judging the key event, and the time point of data jump (that is, the data whose change value of two adjacent frames of data exceeds the drift threshold), and formulates a retention strategy for the above time point data according to actual application requirements to ensure that the feature data features are not lost.

[0129] It can be understood that the specific details of the device provided in the embodiment of the present disclosure can refer to the specific details of the method embodiment of the first aspect mentioned above, and will not be repeated here.

[0130] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in any one of the first aspects is implemented.

[0131] Specifically, Figure 4 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present disclosure, such as Figure 4As shown, the electronic device 004 includes a processor 401, a memory 402 and a bus 403. The processor 401 and the memory 402 communicate with each other via the bus 403.

[0132] The processor 401 is used to call the program instructions in the memory 402 to execute the methods provided by the above method embodiments.

[0133] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program of the method described in any one of the first aspects above.

[0134] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, comprising a computer program / instructions, which, when executed by a processor, implement the steps of any one of the methods described in the first aspect above.

[0135] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0139] The present disclosure uses specific embodiments to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method of the present disclosure and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present disclosure, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A spacecraft telemetry data cleaning method, characterized in that: The method comprises: Acquire telemetry data from spacecraft; Sorting the telemetry data in chronological order to obtain time series data; Based on a preset data type flag field and a data time period, the test data and temporary data in the time series data are deleted to obtain first intermediate data; Based on the source information of the telemetry data and / or the key time information of the telemetry data, the first intermediate data is cleansed to obtain cleansed data.

2. The method according to claim 1, characterized in that The step of performing data cleaning on the first intermediate data based on the source information of the telemetry data and / or the time information of the telemetry data to obtain cleaned data includes: Performing a first data cleaning on the first intermediate data based on the source information of the telemetry data to obtain first cleaned data; The first cleaned data is cleaned for a second time according to the key time information of the telemetry data to obtain cleaned data.

3. The method according to claim 2, characterized in that The information source information of the telemetry data includes the on-device time of the transmission station corresponding to the telemetry data and the source packet identifier of the telemetry data, and the first data cleaning is performed on the first intermediate data based on the information source information of the telemetry data to obtain the first cleaned data, including: Determine whether there is data with the same name in the first intermediate data according to the on-device time and source packet identifier of the transmission station corresponding to each frame of data in the first intermediate data; For the data with the same name in the first intermediate data, only one copy of the data with the same name is retained, and all transmission station information corresponding to the data with the same name is saved to obtain first cleaned data.

4. The method according to claim 2, characterized in that: The key time information of the telemetry data includes a command schedule and an important event schedule of the telemetry data, and the second data cleaning is performed on the first cleaned data according to the key time information of the telemetry data to obtain the cleaned data, including: Obtaining a dispatch schedule and a timeline of significant events for said telemetry data; Determining key frames in the first cleaning data according to the command schedule and the important event schedule of the telemetry data; The key frames in the first cleaned data are marked to obtain cleaned data.

5. The method according to claim 4, characterized in that The method further comprises: Determining a data drift threshold according to a change trend of the cleaned data; Calculate the change value of any two adjacent frames of data in the cleaned data; Compare the change value of each two adjacent frames of data with the data drift threshold to obtain a first comparison result; According to the first comparison result, a final cleaning result of the telemetry data is obtained.

6. The method according to claim 5, characterized in that Obtaining a final cleaning result of the telemetry data according to the first comparison result includes: If the first comparison result is that the change value of the two adjacent frames of data exceeds the drift threshold, the two adjacent frames of data are used as the first reserved data; If the first comparison result is that the change value of the two adjacent frames of data is within the range of the drift threshold, the two adjacent frames of data are used as data to be retained; Sampling the data to be retained based on a sampling period to obtain second retained data; Final cleansing data of the telemetry data is obtained according to the first retained data and the second retained data.

7. A spacecraft telemetry data cleaning device, characterized in that: The device comprises: An acquisition module, used to acquire telemetry data of the spacecraft; A sorting unit, used to sort the telemetry data in chronological order to obtain time series data; A deleting unit, configured to delete the test data and temporary data in the time series data based on a preset data type flag field and a data time period, to obtain first intermediate data; The cleaning module is used to clean the first intermediate data based on the source information of the telemetry data and / or the time information of the telemetry data to obtain cleaned data.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images