Satellite on-board data preprocessing system and method based on unsupervised learning
Patent Information
- Application Number
- CN202610494564.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-15
- Publication Date
- 2026-09-22
AI Technical Summary
[0038]相比传统人工方法，本方案有效实现了仅使用遥测帧固有特征，不依赖先验知识对遥测数据进行子帧长度识别与波道划分，以便后续进行工程值转化。相较于传统方法，本方法有效摆脱了对遥测帧XML配置文件的依赖，通过引入高度自动化的算法机制，在显著降低人工干预成本的同时，大幅提升了卫星遥测数据预处理的精度与效率。
Smart Images
Figure CN122802006A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of telemetry frame data processing technology. More specifically, it relates to a satellite on-board data preprocessing system and method based on unsupervised learning. Background Technology
[0002] Traditional telemetry frame data preprocessing methods, such as manually interpreting XML configuration files, are simple in principle and convenient to implement, making them suitable for offline satellite data analysis or low-speed, small-volume online data analysis. Therefore, they have long been the primary preprocessing technology used in my country's aerospace telemetry missions. However, with the rapid development of communication technology and the increasing demands for security, data transmission speeds have increased, making traditional manual data preprocessing methods insufficient to meet the parameter requirements of modern aerospace missions. Against this backdrop, intelligent data preprocessing methods have gradually become the mainstream research trend. However, existing intelligent telemetry data processing methods largely rely on prior knowledge, such as telemetry configuration files, making them unsuitable for processing semi-blind or fully blind telemetry data. Faced with these technical limitations, there is an urgent need to develop a new telemetry data preprocessing method that can achieve high precision and high automation without relying on prior knowledge, providing stronger technical support for subsequent data analysis. Summary of the Invention
[0003] To address the shortcomings of existing intelligent telemetry data processing methods that heavily rely on prior knowledge, this invention discloses a satellite on-board data preprocessing system based on unsupervised learning. This system preprocesses the on-board data before analysis and processing, including: a frame type division module, a subframe length identification module, a data channelization module, and a channel division module; wherein...
[0004] The frame type division module divides the satellite on-board data frame types based on the characteristics of telemetry frame data, and provides the identified telemetry frame data to the subframe length identification module and the data channelization module.
[0005] The subframe length identification module identifies and calculates the subframe length of the telemetry frame, and traverses the telemetry frame data output by the frame type division module. By identifying the inherent characteristics of the telemetry frame, it calculates and outputs the subframe length of the telemetry frame data.
[0006] The data channelization module receives the telemetry frame data transmitted by the frame type division module and the measurement frame data subframe length transmitted by the subframe length identification module, and performs channelization processing on the telemetry frame data according to the measurement frame data subframe length.
[0007] The channel partitioning module receives channelized telemetry frame data output by the data channelization module, uses an unsupervised learning algorithm to cluster the channelized telemetry frame data, and outputs the final processing result according to the channel clustering result and channel order.
[0008] Furthermore, the frame type division module is specifically used when the data transmission speed is within the range of 1MB / s, where the satellite on-board data includes telemetry frame data and ordinary frame data.
[0009] Furthermore, the telemetry frame structure is divided into horizontal row vector subframes and vertical column vector subframes. After channelization according to the correct subframe length, each column of data belongs to the same channel, and each channel always contains one or more columns of data.
[0010] Furthermore, the subframe length identification module uses a sliding window of length 1024 to traverse the telemetry frame data output by the frame type division module to record the byte index positions of all suspected alignment codes.
[0011] Furthermore, the subframe length identification module calculates the difference between adjacent indices and forms a set of differences. It then uses the calculated set of differences to solve for the index difference mode and obtains the subframe length based on the solved index difference mode.
[0012] Furthermore, the data channelization module is used to channelize the telemetry frame data provided by the frame type division module according to the subframe length of the measurement frame data provided by the subframe length identification module.
[0013] Furthermore, the channel partitioning module divides the channelized telemetry data output by the data channelization module into channels, extracts the statistical features, time series features, distribution and information theory features of the channelized telemetry data, uses the K-means unsupervised clustering algorithm to divide the channelized telemetry data into channels, and determines the optimal number of clusters.
[0014] The optimal number of clusters is determined using the following method:
[0015] For a given telemetry data column X={x1,x2,...,xn}, where each data point xᵢ is a d-dimensional statistical feature vector extracted from the byte position of binary telemetry data, n is the total number of data points, and d=22 is the feature dimension, the following steps are used to determine the optimal number of clusters;
[0016] S2A1: Setting the range of candidate cluster numbers; setting the search range for the number of clusters k [k_min, k_max], where k_min ≥ 2, k_max ≤ n. In this embodiment, k_min = 2, k_max = 20;
[0017] S2A2: Using k as the number of clusters, perform the K-means clustering algorithm on the feature dataset X to obtain the cluster partitions. and the corresponding cluster centers ;
[0018] S2A3, calculate the sum of squared intra-cluster errors WCSS(k) under this clustering partition:
[0019]
[0020] Where ||·|| represents Euclidean distance;
[0021] S2A4, calculate the average silhouette coefficient S(k) under this clustering partition, for each data point xᵢ∈C a (k), calculate:
[0022] Average intra-cluster distance:
[0023]
[0024] when When, define a(i) = 0,
[0025] Minimum average distance between clusters:
[0026]
[0027] The average profile coefficient is:
[0028]
[0029] S2A5, determining the optimal silhouette coefficient: Based on the S(k) sequence, determine the number of clusters that maximizes the silhouette coefficient.
[0030] .
[0031] Furthermore, the length of each telemetry full frame is fixed and unknown, while the number of subframes is not fixed and unknown.
[0032] This invention also provides a satellite on-board data preprocessing method based on unsupervised learning, comprising:
[0033] The raw satellite data collected is divided into frame types according to the inherent characteristics of telemetry frames to obtain telemetry frame data;
[0034] The length of a subframe in telemetry frame data is identified based on the mode feature of the index value.
[0035] The telemetry frame data is channelized according to the subframe length;
[0036] The K-means algorithm is used to divide the channelized telemetry data into channels and obtain a final result that approximates a normal frame structure.
[0037] Compared with the prior art, the present invention has the following advantages:
[0038] Compared to traditional manual methods, this approach effectively utilizes only the inherent features of telemetry frames, without relying on prior knowledge, to identify subframe lengths and divide channels in telemetry data for subsequent engineering value conversion. Compared to traditional methods, this approach effectively eliminates the dependence on telemetry frame XML configuration files. By introducing a highly automated algorithm mechanism, it significantly reduces the cost of manual intervention while greatly improving the accuracy and efficiency of satellite telemetry data preprocessing. Attached Figure Description
[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0040] Figure 1 This is a schematic diagram of a satellite on-board data preprocessing method based on unsupervised learning, provided in this application example.
[0041] Figure 2 This is a schematic diagram of the telemetry frame data structure provided in this application example.
[0042] Figure 3 This is a schematic diagram of frame type identification based on frame features provided in this application example.
[0043] Figure 4 This is a diagram showing the channelization result provided in this application example. Detailed Implementation
[0044] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0045] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0046] like Figure 1 As shown, this invention discloses a satellite on-board data preprocessing system based on unsupervised learning. Before processing and analyzing the satellite on-board data, the system preprocesses the on-board data, including: a frame type division module, a subframe length identification module, a data channelization module, and a channel division module; wherein,
[0047] The frame type division module divides the satellite on-board data frame types based on the characteristics of telemetry frame data, and provides the identified telemetry frame data to the subframe length identification module and the data channelization module.
[0048] The subframe length identification module identifies and calculates the subframe length of the telemetry frame, and traverses the telemetry frame data output by the frame type division module. By identifying the inherent characteristics of the telemetry frame, it calculates and outputs the subframe length of the telemetry frame data.
[0049] The data channelization module receives the telemetry frame data transmitted by the frame type division module and the measurement frame data subframe length transmitted by the subframe length identification module, and performs channelization processing on the telemetry frame data according to the measurement frame data subframe length.
[0050] The channel partitioning module receives channelized telemetry frame data output by the data channelization module, uses an unsupervised learning algorithm to cluster the channelized telemetry frame data, and outputs the final processing result according to the channel clustering result and channel order.
[0051] The frame type division module is specifically used when the data transmission speed is within the range of 1MB / s, where the satellite on-board data includes telemetry frame data and ordinary frame data.
[0052] The telemetry frame structure is divided into horizontal row vector subframes and vertical column vector subframes. After channelization according to the correct subframe length, each column of data belongs to the same channel, and each channel always contains one or more columns of data.
[0053] The subframe length identification module uses a sliding window of length 1024 to traverse the telemetry frame data output by the frame type classification module and record the byte index positions of all suspected alignment codes.
[0054] The subframe length identification module calculates the difference between adjacent indices and forms a set of differences. It then uses the calculated set of differences to solve for the index mode and obtains the subframe length based on the solved index mode.
[0055] The data channelization module is used to channelize the telemetry frame data provided by the frame type division module according to the subframe length of the measurement frame data provided by the subframe length identification module.
[0056] The channel partitioning module divides the channelized telemetry data output by the data channelization module into channels, and extracts the statistical features, time series features, distribution and information theory features of the channelized telemetry data. The K-means unsupervised clustering algorithm is then used to divide the channelized telemetry data into channels.
[0057] Each telemetry full frame contains subframes of fixed and unknown length, and the number of subframes is not fixed and unknown.
[0058] A satellite on-board data preprocessing method based on unsupervised learning divides the acquired raw satellite on-board data into frame types according to the inherent characteristics of telemetry frames to obtain telemetry frame data;
[0059] The length of a subframe in telemetry frame data is identified based on the mode feature of the index value.
[0060] The telemetry frame data is channelized according to the subframe length;
[0061] The K-means algorithm is used to divide the channelized telemetry data into channels and obtain a final result that approximates a normal frame structure.
[0062] Example:
[0063] A satellite on-board data preprocessing method based on unsupervised learning includes the following steps:
[0064] S1 extracts a 22-dimensional feature vector for each column of the channelized data matrix (i.e., each byte position to be analyzed). The features include mean, sample standard deviation, minimum value, maximum value, median, percentile, sample variance, proportion greater than the mean, number of unique values, information entropy, and value range distribution.
[0065] Specifically, for a telemetry frame column vector containing N sample values, with byte values ranging from 0 to 255, where:
[0066] The formula for calculating the mean is:
[0067]
[0068] The formula for calculating the sample standard deviation is:
[0069]
[0070] The formula for calculating sample variance is:
[0071]
[0072] The formula for calculating information entropy is:
[0073]
[0074] S2, Determining the number of clusters: The elbow rule is used to automatically determine the optimal number of clusters k, where:
[0075] Given a telemetry data sequence X = {x1, x2, ..., xn}, where each data point This is a d-dimensional statistical feature vector extracted from the byte positions of binary telemetry data, where n is the total number of data points (i.e., the number of byte positions analyzed), and d=22 is the feature dimension. The optimal number of clusters is determined through the following steps:
[0076] S2A1: Setting the range of candidate cluster numbers; setting the search range of the number of clusters k [k_min, k_max], where k_min≥2, k_max≤n. In this embodiment, k_min=2, k_max=20.
[0077] S2A2: Using k as the number of clusters, perform the K-means clustering algorithm on the feature dataset X to obtain the cluster partitions. and the corresponding cluster centers .
[0078] S2A3, calculate the sum of squared intra-cluster errors WCSS(k) under this clustering partition:
[0079]
[0080] Where ‖·‖ represents Euclidean distance.
[0081] S2A4, calculate the average silhouette coefficient S(k) under this cluster partition. For each data point ,calculate:
[0082] Average intra-cluster distance:
[0083]
[0084] when When, define .
[0085] Minimum average distance between clusters:
[0086]
[0087] The average profile coefficient is:
[0088]
[0089] S2A5, the optimal value of the silhouette coefficient is determined. Based on the S(k) sequence, the number of clusters that maximizes the silhouette coefficient is determined:
[0090]
[0091] S3, Final Clustering Execution. Using the determined optimal clustering number k, the final K-means clustering is performed on the telemetry data column X to obtain stable clustering results, which are used for subsequent protocol field structure analysis. Figure 4 The channelization result diagram provided for this application example.
[0092] Preferably, step S3 above includes the following sub-steps:
[0093] S3A1, Feature Standardization. Z-score standardization is performed on the feature matrix X:
[0094]
[0095] Where μj and σj are the mean and standard deviation of the j-th feature, respectively.
[0096] S3A2, K-means clustering execution. The K-means clustering algorithm is performed on the standardized feature matrix to minimize the objective function:
[0097]
[0098] Preferably, the specific execution process includes the following sub-steps:
[0099] Initialization: Randomly select k data points from the standardized feature matrix as initial cluster centers, where k is the optimal number of clusters determined by step S2.
[0100] Iterative optimization: Repeat sub-steps (a) and (b) until the cluster centers no longer change or the preset maximum number of iterations is reached.
[0101] Sub-step (a), the allocation step: For each data point xi in the feature matrix, calculate its Euclidean distance to all cluster centers μj(t), and assign it to the nearest cluster Cj(t). Preferably, the allocation rule is as follows:
[0102]
[0103] Sub-step (b), update step: Based on the current cluster partitioning, recalculate the centroid (mean) of each cluster as the new cluster center:
[0104]
[0105] S3A3, Clustering Result Validation. Calculate the silhouette coefficient S(k) of the final clustering result to validate the clustering quality. When S(k) > 0.5, the clustering result is considered to have good intra-cluster compactness and inter-cluster segregation.
[0106] S3A4, Channel splicing. According to the final cluster division {C1,C2,…,Ck} obtained from S3A2, each cluster Cj, j∈1~k corresponds to a physical channel, and the data points contained therein correspond to specific columns (byte positions) in the original channelized telemetry data matrix.
[0107] In step S3A4, the original data columns corresponding to all data points belonging to the same cluster Cj, j∈1~k, are extracted and reorganized in column index order from the two-dimensional data matrix output by the data channelization module. Specifically, the columns within each cluster are arranged according to their original order in the data matrix (or sorted according to their statistical characteristics such as the median), thereby mapping the abstract categories obtained from clustering to physical channel sequences with clear engineering significance. Finally, k channel data blocks are obtained, each channel containing continuous data columns with similar statistical characteristics, completing the conversion from unlabeled binary data to structured channel data, providing direct input for subsequent engineering value interpretation and analysis.
[0108] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0109] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0110] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.
[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0112] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A satellite on-board data preprocessing system based on unsupervised learning, characterized in that, Before processing and analyzing the satellite's on-board data, the system performs preprocessing on the data, including: a frame type division module, a subframe length identification module, a data channelization module, and a channel division module; among which, The frame type division module divides the satellite on-board data frame types based on the characteristics of telemetry frame data, and provides the identified telemetry frame data to the subframe length identification module and the data channelization module. The subframe length identification module identifies and calculates the subframe length of the telemetry frame, and traverses the telemetry frame data output by the frame type division module. By identifying the inherent characteristics of the telemetry frame, it calculates and outputs the subframe length of the telemetry frame data. The data channelization module receives the telemetry frame data transmitted by the frame type division module and the measurement frame data subframe length transmitted by the subframe length identification module, and performs channelization processing on the telemetry frame data according to the measurement frame data subframe length. The channel partitioning module receives channelized telemetry frame data output by the data channelization module, uses an unsupervised learning algorithm to cluster the channelized telemetry frame data, and outputs the final processing result according to the channel clustering result and channel order.
2. The satellite on-board data preprocessing system based on unsupervised learning according to claim 1, characterized in that, The frame type division module is specifically used when the data transmission speed is within the range of 1MB / s, where the satellite on-board data includes telemetry frame data and ordinary frame data.
3. The satellite on-board data preprocessing system based on unsupervised learning according to claim 1, characterized in that, The telemetry frame structure is divided into horizontal row vector subframes and vertical column vector subframes. After channelization according to the correct subframe length, each column of data belongs to the same channel, and each channel always contains one or more columns of data.
4. The satellite on-board data preprocessing method based on unsupervised learning according to claim 1, characterized in that, The subframe length identification module uses a sliding window of length 1024 to traverse the telemetry frame data output by the frame type classification module and record the byte index positions of all suspected alignment codes.
5. The satellite on-board data preprocessing method based on unsupervised learning according to claim 4, characterized in that, The subframe length identification module calculates the difference between adjacent indices and forms a set of differences. It then uses the calculated set of differences to solve for the index mode and obtains the subframe length based on the solved index mode.
6. The satellite on-board data preprocessing method based on unsupervised learning according to claim 1, characterized in that, The data channelization module is used to channelize the telemetry frame data provided by the frame type division module according to the subframe length of the measurement frame data provided by the subframe length identification module.
7. The satellite on-board data preprocessing method based on unsupervised learning according to claim 1, characterized in that, The channel partitioning module divides the channelized telemetry data output by the data channelization module into channels, extracts the statistical features, time series features, distribution and information theory features of the channelized telemetry data, uses the K-means unsupervised clustering algorithm to divide the channelized telemetry data into channels, and determines the optimal number of clusters.
8. The satellite on-board data preprocessing method based on unsupervised learning according to claim 7, characterized in that, The optimal number of clusters is determined using the following method: For a given telemetry data column X={x1,x2,...,xn}, where each data point xᵢ is a d-dimensional statistical feature vector extracted from the byte position of binary telemetry data, n is the total number of data points, and d=22 is the feature dimension, the following steps are used to determine the optimal number of clusters; S2A1: Setting the range of candidate cluster numbers; setting the search range for the number of clusters k [k_min, k_max], where k_min ≥ 2, k_max ≤ n. In this embodiment, k_min = 2, k_max = 20; S2A2: Using k as the number of clusters, perform the K-means clustering algorithm on the feature dataset X to obtain the cluster partitions. and the corresponding cluster centers ; S2A3, calculate the sum of squared intra-cluster errors WCSS(k) under this clustering partition: ， Where ||·|| represents Euclidean distance; S2A4, calculate the average silhouette coefficient S(k) under this clustering partition, for each data point xᵢ∈C a (k), calculate: Average intra-cluster distance: ， When |C a When (k)|=1, define a(i)=0. Minimum average distance between clusters: ， The average profile coefficient is: ， S2A5, determining the optimal silhouette coefficient: Based on the S(k) sequence, determine the number of clusters that maximizes the silhouette coefficient. 。 9. The satellite on-board data preprocessing method based on unsupervised learning according to claim 1, characterized in that, Each telemetry full frame contains subframes of fixed and unknown length, and the number of subframes is not fixed and unknown.
10. A satellite on-board data preprocessing method based on unsupervised learning, characterized in that... include: The raw satellite data collected is divided into frame types according to the inherent characteristics of telemetry frames to obtain telemetry frame data; The length of a subframe in telemetry frame data is identified based on the mode feature of the index value. The telemetry frame data is channelized according to the subframe length; The K-means algorithm is used to divide the channelized telemetry data into channels and obtain a final result that approximates a normal frame structure.