A satellite signal group event mining method based on FLCSS and k-medoids

Abstract

The present invention relates to a satellite signal group event mining method based on FLCSS and k-medoids, including: extracting time series data corresponding to the timestamps of various events occurring in each signal, performing similarity calculation on the time series data of different signals, and obtaining a similarity matrix; performing k-medoids clustering on the similarity matrix to obtain a number of cluster clusters corresponding to the division of a number of signal group events. The present invention uses dynamic programming to calculate the FLCSS algorithm from the overall perspective of the sequence, while making the common subsequence found as long as possible and also making the distance between each pair of common points as small as possible, avoiding the unreasonable situation that each pair of common points may be at the boundary of the distance threshold caused by the original algorithm; further optimizing the results of k-medoids clustering, making the maximum use of the original data, filling in the missed clusters, and removing samples that do not belong to any cluster.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a satellite signal group event mining method based on FLCSS and k-medoids. Background Art

[0002] In recent years, with the rapid development of aerospace technology, the number of satellites launched has increased rapidly, and the amount of detected signal data has also increased. On the other hand, the rapid development of the computer industry and the improvement of software and hardware environment have provided a good environment for the implementation of complex algorithms. Therefore, using computers to automatically mine valuable information from large-scale satellite signal data is the general trend of development. The detection of similar behavior events in groups is one of the most valuable applications.

[0003] The existing methods for detecting similar behaviors in groups mainly extract features, define similarity measurement formulas between samples, and then use clustering methods. However, for time series signal data, the similarity is difficult to measure due to the different sequence spans and number of sample points. Among the existing methods, greedy algorithms are difficult to consider the sequence as a whole and are prone to miss better solutions. RNN and its variant neural network algorithms lack stability and have poor interpretability. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a satellite signal group event mining method based on FLCSS and k-medoids to solve the shortcomings of the prior art.

[0005] The purpose of the present invention is achieved through the following technical solutions: a satellite signal group event mining method based on FLCSS and k-medoids, which specifically includes the following contents:

[0006] For each signal, the time series data corresponding to the timestamps of various events are extracted, the similarity of the time series data of different signals is calculated, and clustering is performed based on this similarity, and finally several clusters are obtained, corresponding to the division of several signal group events.

[0007] Furthermore, the satellite signal group event mining method comprises the following steps:

[0008] A. Query the IDs of all signals in a specified time window and their corresponding event types and event occurrence timestamps, and classify the event types and event occurrence timestamps by signal ID. Let the number of queried signal IDs be N. Classify the data corresponding to each signal ID by event type, and sort them in ascending order by event timestamp.

[0009] B. Taking the "out-of-link" event type as an example, at this time, each signal has its "out-of-link" event timestamp sequence data. The similarity between all signals is calculated using this time series data to obtain an N×N "out-of-link" similarity matrix.

[0010] C. For other event types, similarity matrices are calculated in sequence according to step B. Each similarity matrix is ​​multiplied by the weight of the corresponding event type to obtain a new matrix. Then, the new matrices are added and normalized to obtain the final N×N similarity matrix.

[0011] D. Set the minimum number of centroids to 2 and the maximum number of centroids to max_k. The first centroid is randomly selected from the sample, and the other centroids are selected in sequence using the kmeans++ idea, that is, the sample with the smallest maximum similarity to the existing centroid is selected as the new centroid. Set limit1 as a threshold. If the similarity between the selected centroid and any existing centroid is greater than limit1, no new centroid will be added. At this time, the maximum number of centroids max_k is updated to the current number of existing centroids.

[0012] E. Traverse from the minimum number of centroids 2 to the maximum number of centroids max_k, and generate cn groups of centroids for each centroid. The generation method is: the first group is generated by kmeans++, and the remaining cn-1 groups are generated randomly. A total of (max_k-2+1)×cn groups of centroids are generated.

[0013] F. Use the similarity matrix to perform k-medoids clustering on each group of centroids to obtain (max_k-2+1)×cn groups of results, and use the silhouette score to select the best result.

[0014] G. Use threshold limit2 to further optimize the best result of step F to determine the final centroids and corresponding cluster samples.

[0015] Furthermore, the similarity is calculated using time series data; the calculation method is: using an improved LCSS algorithm to calculate the longest common subsequence of two time series, and using the similarity evaluation formula to calculate the similarity based on the subsequence length and the distance difference within the common point pair.

[0016] Furthermore, the formula of the improved LCSS algorithm (ie, FLCSS) is as follows:

[0017]

[0018] f(x)=0.5+(1-x / r)*0.5

[0019] in:

[0020] A and B are the time series data of signal A and signal B respectively, A={a1,a2,…,at}, B={b1,b2,…,b i};

[0021] dist(a t ,b i ) is a t , b i The absolute value of the difference between the corresponding timestamps;

[0022] f is the function of dist, defined as shown in the formula f(x);

[0023] r is the distance threshold parameter in the FLCSS algorithm.

[0024] Furthermore, the similarity evaluation formula is as follows:

[0025]

[0026] in

[0027] FLCSS(A,B) is the longest common subsequence length of two signals;

[0028] len A ,len B are the sequence lengths of signal A and signal B respectively;

[0029] sum(d) is the sum of the timestamp differences of each pair of corresponding points in the public sequence.

[0030] Furthermore, the threshold value limit2 is used for further optimization; the optimization method is:

[0031] A. Eliminate samples from each cluster whose similarity with the corresponding centroid is less than limit2. If there is only one sample left after the cluster is eliminated, the similarity between this sample and the centroids of other clusters will be queried in turn. If there is a similarity greater than limit2, this sample will be included in the cluster. If there are multiple similarities greater than limit2, it will be included in the cluster with the largest similarity; if the similarities are all less than or equal to limit2, this sample will be eliminated and the cluster will be deleted at the same time.

[0032] B. Each remaining cluster reselects its own centroid according to the similarity matrix, that is, finds a sample in the cluster with the highest sum of similarities with other samples in the cluster as the centroid, obtains a new centroid set, and reselects samples in each cluster for all data based on the new centroid set, finds the centroid c with the highest similarity among the centroids with a similarity greater than limit2 for each sample x, and assigns this sample x to the cluster corresponding to the centroid c. Samples with similarities with all centroids less than or equal to limit2 are assigned to the elimination set T.

[0033] C. Determine a centroid st from the set T according to the similarity matrix, that is, the sum of the similarities between st and other samples in T is the largest, find samples in T with a similarity greater than the threshold limit2 and cluster them into one category, and the remaining samples are classified into the T' set.

[0034] D. If the number of samples in T' is 0 or st cannot find any samples with a similarity greater than limit2, then end; otherwise, set set T' to T and repeat step C.

[0035] Preferably, the threshold value limit1=0.6, the number of centroid groups cn=12, and the maximum number of centroids max_k=min(25, N / 2).

[0036] Preferably, the parameter r=160 (seconds).

[0037] Preferably, the threshold value limit2=0.35.

[0038] The present invention has the following advantages:

[0039] 1. The present invention improves the LCSS algorithm and uses dynamic programming to perform calculations from the perspective of the entire sequence. While making the common subsequence found as long as possible, it also makes the distance between each pair of common points as small as possible, thus avoiding the unreasonable situation in which each pair of common points is at the boundary of the distance threshold due to the original algorithm's pursuit of the longest subsequence.

[0040] 2. Based on the results of the improved LCSS algorithm, a similarity calculation formula is defined to measure the similarity from two perspectives: the proportion of common points and the degree of similarity within the common points. The formula ensures that its value is between 0 and 1, where 1 means that the two series completely overlap. This provides a reasonable and interpretable new method for measuring the similarity of two time series.

[0041] 3. By further optimizing the results of k-medoids clustering, each cluster of the clustering results can be made more compact and the centroid position more reasonable. At the same time, the original data can be used to the maximum extent, the clusters that may be missed can be supplemented, and the samples that do not belong to any cluster can be eliminated. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the process of the present invention;

[0043] Figure 2 Schematic diagram of the FLCSS algorithm of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application provided below in conjunction with the drawings is not intended to limit the scope of protection of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application. The present invention is further described below in conjunction with the drawings.

[0045] The present invention specifically relates to a satellite signal group event mining method based on FLCSS and k-medoids: extracting time series data corresponding to timestamps of various events occurring in each signal, performing similarity calculation on time series data of different signals, performing clustering based on the similarity, and finally obtaining a plurality of clusters corresponding to the division of a plurality of signal group events.

[0046] like Figure 1 As shown, specifically including the following:

[0047] A. Preprocess the original data and use the improved LCSS algorithm to calculate the similarity matrix. The formula of the improved LCSS algorithm (i.e., FLCSS) is as follows:

[0048]

[0049] f(x)=0.5+(1-x / r)*0.5

[0050] in:

[0051] A and B are the time series data of signal A and signal B respectively, A={a1,a2,…,a t}, B={b1,b2,…,b i};

[0052] dist(a t ,b i ) is a t , b i The absolute value of the difference between the corresponding timestamps;

[0053] f is the function of dist, defined as shown in the formula f(x);

[0054] r is the threshold parameter of the FLCSS algorithm.

[0055] The improvement of this formula is that f(dist(a t ,b i )) replaces 1 in the original formula, and f(dist(a t ,b i )) is the distance between common points. When the distance is 0, its value is 1, otherwise it is less than 1, see the definition of f(x). In this way, the formula can take into account both the length of the common sequence and the distance between common points, so that while searching for the longest possible common sequence matching method, it can also try to ensure that the distance between common points is not too far.

[0056] The FLCSS algorithm can calculate the length and content information of the common subsequence. Compared with the original LCSS algorithm, FLCSS can better balance the maximization of the common sequence length and the minimization of the corresponding sample distance by introducing the f function.

[0057] Furthermore, preprocessing the original data specifically includes:

[0058] Query the IDs of all signals in a specified time window and their corresponding event types and event occurrence timestamps, and classify the event types and event occurrence timestamps by signal ID. Let the number of queried signal IDs be N. Classify the data corresponding to each signal ID by event type, and sort them in ascending order by event timestamp.

[0059] B. Calculate the similarity matrix based on the FLCSS result. The similarity matrix calculation formula is as follows:

[0060]

[0061] in:

[0062] FLCSS(A,B) is the longest common subsequence length of two signals, such as Figure 2 As shown;

[0063] len A ,len B are the sequence lengths of signal A and signal B respectively;

[0064] sum(d) is the sum of the timestamp differences of each pair of corresponding points in the public sequence; Figure 2 shown.

[0065] By directly using the s(A,B) formula, since FLCSS(A,B) is the length of the subsequence, that is, it is a number, just substitute this number into s(A,B). In addition, r has been defined before and can be any number, depending on the actual situation; if the meaning of sum(d) is difficult to understand in textual description, please refer to Figure 2 , Figure 2The d in the equation is the distance between a pair of common points. There are 6 pairs of common points in total. The sum of their corresponding d is sum(d).

[0066] This formula calculates the similarity between the time series of the same event type of two signals. The similarity matrix (N×N) is formed by calculating the similarity between all signals. This matrix is ​​a symmetric matrix with a diagonal of 1. Taking the "out of connection" event type as an example, the calculation results are as follows:

[0067]

[0068] Different event types obtain their own matrices in the same way, and then all matrices are summed according to the weight of each event type to obtain the final similarity matrix, as shown in the following example:

[0069] S=0.6*S 出联 +0.3*S 消失 +0.1*S 规格变化

[0070] C. Use the similarity matrix S to cluster multiple groups of initial centroids with different centroid positions and numbers, and select the best clustering result based on the silhouette score.

[0071] D. Use threshold limit2 for further optimization, set limit2 = 0.35. Eliminate samples in each cluster whose similarity with the corresponding centroid is less than limit2. If there is only one sample left after the cluster is eliminated, query the similarity between this sample and the centroids of other clusters in turn. If there is a similarity greater than limit2, then this sample is included in the cluster. If there are multiple similarities greater than limit2, then it is included in the cluster with the largest similarity; if the similarities are all less than or equal to limit2, then this sample is eliminated and the cluster is deleted at the same time.

[0072] E. Each remaining cluster reselects its own centroid according to the similarity matrix, that is, finds a sample in the cluster with the highest sum of similarities with other samples in the cluster as the centroid, obtains a new centroid set, and reselects samples in each cluster for all data based on the new centroid set, finds the centroid c with the highest similarity among the centroids with similarity greater than limit2 for each sample x, and assigns this sample x to the cluster corresponding to the centroid c. Samples with similarity less than or equal to limit2 with all centroids are assigned to the elimination set T.

[0073] F. Determine a centroid st from the set T according to the similarity matrix, that is, the sum of the similarities between st and other samples in T is the largest, find samples in T with a similarity greater than the threshold limit2 and cluster them into one category, and the remaining samples are classified into the T' set.

[0074] G. If the number of samples in T' is 0 or st cannot find any samples with a similarity greater than limit2, then end; otherwise, set set T' to T and repeat step F.

[0075] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A satellite signal group event mining method based on FLCSS and k-medoids, characterized by: The mining method comprises: S1. Extract the time series data of the timestamps corresponding to the occurrence of various events of each signal, and calculate the similarity of the time series data of different signals through the FLCSS algorithm to obtain a similarity matrix; S2, performing k-medoids clustering on the similarity matrix to obtain a number of clusters corresponding to the division of a number of signal group events; The time series data of the timestamps corresponding to the occurrence of various events of each signal are extracted, and the similarity of the time series data of different signals is calculated by the FLCSS algorithm to obtain a similarity matrix including: S11, querying the IDs of all signals in a set time window and their corresponding event types and event occurrence timestamps, and classifying and sorting the event types and event occurrence timestamps according to the signal IDs, and setting the number of queried signal IDs to N; S12, calculating the similarity between two signals according to the time series data of a certain event type, and obtaining an N×N similarity matrix of this event type; S13, repeat step S12 to obtain an N×N similarity matrix of all event types, multiply each similarity matrix by the weight of the corresponding event type to obtain a new matrix, and add and normalize each new matrix to obtain an N×N final similarity matrix; The method for calculating similarity based on time series data includes: calculating the longest common subsequence of two time series according to the FLCSS algorithm, and calculating the similarity using a similarity evaluation formula based on the length of the subsequence and the difference in distance within the common point pair; The FLCSS algorithm formula includes: f(x)=0.5+(1-x / r)*0.5 Where A and B are the time series data of signal A and signal B respectively, A={a1,a2,…,a t }, B={b1,b2,…,b i }; dist(a t ,b i ) is a t , b i The absolute value of the difference between the corresponding timestamps; f is the function of dist, defined as shown in the f(x) formula; r is the distance threshold parameter in the original LCSS algorithm.

2. The satellite signal group event mining method based on FLCSS and k-medoids according to claim 1, characterized in that: The k-medoids clustering is performed on the similarity matrix to obtain a number of clusters, and the divisions corresponding to the number of signal group events include: S21, set the minimum number of centroids to 2, the maximum number of centroids to max_k, and use the sample with the smallest maximum similarity to the existing centroids as the selection principle for the new centroid. If the similarity between the selected centroid and any existing centroid is greater than the threshold limit1, no new centroid will be added, and the maximum number of centroids max_k will be updated to the current number of existing centroids; S22, traverse from the minimum number of centroids 2 to the maximum number of centroids max_k, each centroid number is generated according to the kmeans++ algorithm according to the first group, and the remaining cn-1 groups are randomly generated to generate cn groups of centroids, and a total of (max_k-2+1)×cn groups of centroids are generated; S23, use the similarity matrix to perform k-medoids clustering on each group of centroids, obtain (max_k-2+1)×cn groups of results, and select the best result by the silhouette coefficient; S24. The best result is optimized by threshold limit2 to improve the cohesion within the cluster and fill in the clusters that may be missed, and finally determine each centroid and the corresponding cluster sample.

3. The satellite signal group event mining method based on FLCSS and k-medoids according to claim 2, characterized in that: The optimization of the best result by threshold limit2 includes: A1. Eliminate samples from each cluster whose similarity with the corresponding centroid is less than the threshold limiti2. If there is only one sample left in the cluster after the sample is eliminated, query the similarity of this sample with the centroids of other clusters in turn. If there is a similarity greater than the threshold limiti2, then this sample is included in the cluster. If there are multiple similarities greater than the threshold limit2, then it is included in the cluster with the largest similarity. If there are similarities less than or equal to the threshold limit2, then this sample is eliminated and the cluster is deleted at the same time. A2. Each remaining cluster reselects its own centroid according to the similarity matrix, that is, finds a sample in the cluster with the highest sum of similarities with other samples in the cluster as the centroid, obtains a new centroid set, and reselects samples in each cluster for all data based on the new centroid set. For each sample x, find the centroid c with the highest similarity among the centroids with a similarity greater than limit2, and assign this sample x to the cluster corresponding to the centroid c. Samples with similarities with all centroids less than or equal to limit2 are assigned to the elimination set T. A3. Determine a centroid st from the set T according to the similarity matrix, that is, the sum of the similarities between st and other samples in T is the largest, find samples in T with a similarity greater than the threshold limit2 and cluster them into one category, and the remaining samples are classified into the set T'; A4. If the number of samples in T' is 0 or st cannot find any samples with a similarity greater than limit2, then end; otherwise, set set T' to T and repeat step A3.

4. The satellite signal group event mining method based on FLCSS and k-medoids according to claim 1, characterized in that: The similarity evaluation formula includes: Where FLCSS(A,B) is the longest common subsequence length of the two signals; len A ,len B are the sequence lengths of signal A and signal B respectively; sum(d) is the sum of the timestamp differences of each pair of corresponding points in the public sequence.

5. The satellite signal group event mining method based on FLCSS and k-medoids according to claim 2, characterized in that: The threshold value limit1=0.6, the threshold value limit2=0.35, the number of centroid groups cn=12, and the maximum number of centroids max_k=min(25, N / 2).

Images