Deep clustering method for selectively enhancing training data
By introducing a selective data enhancement method in deep embedding clustering, using the probability distribution function and the discrete exponential data selectively amplify favorable sample features, the problem of low clustering efficiency of high-dimensional data is solved, and the clustering effect is improved and Reduced computational complexity.
Patent Information
- Application Number
- CN202411022621.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-26
AI Technical Summary
In high-dimensional and large-scale datasets, the accuracy and efficiency of existing clustering algorithms have decreased due to the curse of dimensions, and it is difficult to locate key samples from unlabeled data and perform selective data augmentation, resulting in high computational complexity and affect clustering Effect.
A deep clustering method for selective enhancement of training data is proposed. The selective data augmentation SeDA method is introduced into the deep embedding clustering DEC. It uses two probability distribution functions PDF and data discrete index DDI related to Sigmoid to selectively amplify The sample features that are conducive to clustering reduce computational complexity.
The clustering effect is significantly improved, the computational complexity is reduced, and the effectiveness and efficiency of the algorithm are verified in UCR benchmarking and practical applications, especially in time series classification and energy consumption datasets, which outperform traditional methods.
Smart Images

Figure CN119025940A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data enhancement, and in particular to a deep clustering method for selectively enhancing training data. Background Art
[0002] Clustering is an unsupervised learning method used to mine relevant information between data samples. Clustering techniques include hierarchical methods, partition-based methods, density-based methods, grid-based methods, model-based methods, and graph-based clustering. Clustering algorithms usually require iterative calculation of similarity, density, or specific statistical distribution measures in the data space to determine the cluster centers. These algorithms perform well on medium-sized and low-dimensional data. However, when analyzing high-dimensional and large-scale data sets with a large number of data samples, the accuracy and efficiency of the above algorithms will drop sharply due to the curse of dimensionality.
[0003] The clustering performance of DEC can be greatly improved by combining DA, but since DEC-DA enhances all data samples without selection, its high computational complexity limits its application in real-world problems. If it is possible to identify key samples that only affect the clustering results from the original data and appropriately amplify the distribution characteristics of these key samples, the clustering effect can be significantly improved while reducing the computational complexity. Unfortunately, it is currently very difficult to locate key samples from unlabeled data. How to reasonably select samples for data enhancement and better present the data distribution characteristics remains a challenge.
[0004] Therefore, it is necessary to provide a deep clustering method with selective enhancement of training data to solve the above problems. Summary of the invention
[0005] The purpose of this invention is to provide a deep clustering method for selective enhancement of training data. Two typical sample distribution patterns are defined. The distribution pattern is determined by extracting the distribution characteristics of the samples. According to the determined pattern, samples are probabilistically selected and enhanced under a probability distribution function. The effectiveness and efficiency of the proposed algorithm are verified through competitive performance in UCR benchmark tests and practical applications.
[0006] To achieve the above object, the present invention provides a deep clustering method for selectively enhancing training data. In the deep embedded clustering DEC, a selective data enhancement SeDA method for clustering time series is embedded to form DEC-SeDA. In DEC-SeDA, two probability distribution functions PDF related to Sigmoid are proposed to perform selective data enhancement, and a data dispersion index DDI is proposed to reflect the data distribution characteristics.
[0007] The specific steps include:
[0008] S1: In deep embedding clustering DEC, feature selection and feature extraction are performed. Feature selection amplifies the distribution characteristics of clustering, and feature extraction is achieved through autoencoders;
[0009] S2: In the DEC-SeDA framework, time series scaling and selective data augmentation are performed. The time series scaling is performed in equal proportions in each dimension, and samples are selected for data augmentation to selectively amplify features that are more conducive to clustering.
[0010] Preferably, in step S1, feature extraction in DEC specifically includes the following steps:
[0011] S11: The original data X is projected into the latent space Z to deal with the curse of dimensionality;
[0012] S12: A deep autoencoder AE is initialized and the parameters are optimized by iteratively comparing the auxiliary target distribution and minimizing the divergence KLD;
[0013] S13: Soft assignments are calculated between embedded samples and cluster centers Depth mapping function f θ is updated to optimize cluster centers;
[0014] In step S13, based on the auxiliary target distribution, initializing the cluster center includes the following steps:
[0015] S131: Data is passed through the initialized autoencoder AE to obtain embedded data samples;
[0016] S132: Perform analysis in feature space Z using standard k-means clustering to obtain k initial cluster centers Reduce the high dimensionality of the original data to a low dimensionality that is conducive to clustering;
[0017] The soft distribution is the Student's t distribution, described as:
[0018]
[0019] In the formula, the embedding point z i =f θ (x i )∈Z represents x i ∈X is projected into a low-dimensional space, α and q ij is the degrees of freedom of the Student’s t distribution, q ij represents the probability of assigning sample i to cluster j, μ j represents the cluster center;
[0020] S133: Measure the embedding point Z i and cluster center μ jThe similarity between them is used to construct the optimization target of KL clustering loss;
[0021] S134: Soft allocation q i , clustering assignment s and auxiliary distribution p i , the difference between the student distribution t is expressed as:
[0022]
[0023] In the formula,
[0024] p ij First, q i The quadratic power is calculated and then normalized by the frequency of each cluster:
[0025]
[0026] Cluster center μ j The parameters θ of the autoencoder AE are optimized by stochastic gradient descent SGD with momentum, and the embedding point z i The feature space embedding and cluster center μ j The gradient of the loss L is calculated as follows:
[0027]
[0028] In the formula, To find the sign of partial derivative;
[0029] Preferably, in step S1, in DEC-DA, feature selection specifically includes the following steps:
[0030] Given a set of data samples Where D and n represent the dimension and number of data samples respectively, and each data sample is Converted to an enhanced version, where T represents the transformation and the loss function of data enhancement DA is expressed as:
[0031]
[0032] Where fW and gU represent the encoder and decoder respectively; the encoder and decoder are fully connected networks or convolutional networks;
[0033] After connecting data enhancement DA and deep embedding clustering DEC, the overall loss function of DEC-DA is:
[0034] L=αL r +βL c ,
[0035] Where L c= dist(y, t) represents the distance between the output y and the student distribution t, where y and t represent the output and target of the feature learning model, α is the degree of freedom of the student t distribution, and β is the weight;
[0036] The target t is calculated using the original sample X, and the output y is calculated using the enhanced sample calculate.
[0037] Preferably, in step S2, SeDA automatically selects the enhanced data by comparing the value of the data dispersion index DDI with χ, wherein in the scaling of the time series, the target t is calculated by using the clean sample X, and the output y is calculated by the clean sample X and the enhanced sample The union of is calculated, Effectively enhance the time series x i Transformed by the following scaling operations:
[0038]
[0039] in is the ratio of increase or decrease of all elements. The increase or decrease of dimensions is carried out simultaneously. The above formula can be rewritten as:
[0040]
[0041] in Specifically norm(1,σ), which generates normally distributed nonzero numbers around 1.
[0042] Preferably, in step S2, the selective data enhancement Selective DA divides the sample into internal and external parts by measuring the distance between the sample and the mean center of the clean data. Given an effective enhancement sequence x i and mean center μ,x i The distance between and μ is denoted as λ i , sort all values λ and put λ i The ascending ranking is recorded as rank i ; x i Located in the inner part, x i Located in the outer part, where n represents the number of data samples;
[0043] The probability density function PDF of the shifted Sigmoid function is used for selective data enhancement. When the samples in the internal part are enhanced, the PDF is: When the samples in the outer part are enhanced,
[0044] If the DDI is lower than the threshold χ, the external samples are enhanced, and if the DDI is higher than the threshold χ, the internal samples are enhanced. The threshold χ is 0.15; the PDF is selected as follows:
[0045]
[0046] Under the control of PDF, SeDA is executed as follows:
[0047]
[0048] Among them, RF(0,1) represents a random floating point number between 0 and 1, and NULL means no data enhancement.
[0049] Therefore, the present invention adopts the above-mentioned deep clustering method with selective enhancement of training data, which has the following beneficial effects:
[0050] (1) The present invention proposes a data dispersion index (DDI) that reflects the characteristics of data distribution. It is insensitive to the offset caused by sampling errors and shows potential in selecting samples that are beneficial to data enhancement.
[0051] (2) This paper proposes a selective data augmentation (SeDA) method for clustering time series. In DEC-SeDA, two probability distribution functions (PDFs) related to Sigmoid are proposed to perform selective data augmentation. Compared with the current method of enhancing all original data, SeDA only enhances samples that are valuable and conducive to clustering.
[0052] (3) The proposed SeDA is effective on time series classification (TSC). DEC-SeDA shows effectiveness on 128 UCR datasets and is significantly more efficient than DEC-DA considering selected samples on five energy consumption-based datasets.
[0053] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is a flow chart of a deep clustering method for selectively enhancing training data of the present invention;
[0055] Figure 2 It is the probability enhancement density map of the inner part and the probability enhancement density map of the outer part in the present invention;
[0056] Figure 3 is the NMI distribution graph of 30 runs of DEC-SeDA on 50 benchmarks in Example 1 of the present invention;
[0057] Figure 4is a graph showing different clustering performances of DEC-SeDA in two evaluation indicators in Example 2 of the present invention;
[0058] Figure 5 It is a comparison chart of the time consumption of DEC, DEC-SeDA and DEC-DA on the energy consumption data set in the second embodiment of the present invention. DETAILED DESCRIPTION
[0059] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.
[0060] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.
[0061] The words "include" or "comprises" and the like used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms "inside", "outside", "upper", "lower", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly specified and limited, the terms "attachment" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral body; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0062] like Figure 1 As shown, the present invention provides a deep clustering method for selectively enhancing training data. In the deep embedded clustering DEC, a selective data enhancement SeDA method for clustering time series is embedded and coupled into DEC-SeDA. In DEC-SeDA, two probability distribution functions PDF related to Sigmoid are proposed to perform selective data enhancement, and a data dispersion index DDI is proposed to reflect the data distribution characteristics.
[0063] The specific steps include:
[0064] S1: In the deep embedding clustering DEC, feature selection and feature extraction are performed, feature selection amplifies the distribution characteristics of the clustering, and feature extraction is realized through the autoencoder; in step S1, feature extraction in DEC specifically includes the following steps:
[0065] S11: The original data X is projected into the latent space Z to deal with the curse of dimensionality;
[0066] S12: A deep autoencoder AE is initialized and the parameters are optimized by iteratively comparing the auxiliary target distribution and minimizing the divergence KLD;
[0067] S13: Soft assignments are calculated between embedded samples and cluster centers Depth mapping function f θ is updated to optimize cluster centers;
[0068] In step S13, based on the auxiliary target distribution, initializing the cluster center includes the following steps:
[0069] S131: Data is passed through the initialized autoencoder AE to obtain embedded data samples;
[0070] S132: Perform analysis in feature space Z using standard k-means clustering to obtain k initial cluster centers Reduce the high dimensionality of the original data to a low dimensionality that is conducive to clustering;
[0071] The soft distribution is the Student's t distribution, described as:
[0072]
[0073] In the formula, the embedding point z i =f θ (x i )∈Z represents x i ∈X is projected into a low-dimensional space, α and q ij is the degrees of freedom of the Student’s t distribution, q ij represents the probability of assigning sample i to cluster j, μ j represents the cluster center;
[0074] S133: Measure the embedding point Z i and cluster center μ j The similarity between them is used to construct the optimization target of KL clustering loss;
[0075] S134: Soft allocation q i , clustering assignment s and auxiliary distribution p i , the difference between the student distribution t is expressed as:
[0076]
[0077] In the formula,
[0078] p ij First, q iThe quadratic power is calculated and then normalized by the frequency of each cluster:
[0079]
[0080] Cluster center μ j The parameters θ of the autoencoder AE are optimized by stochastic gradient descent SGD with momentum, and the embedding point z i The feature space embedding and cluster center μ j The gradient of the loss L is calculated as follows:
[0081]
[0082] In the formula, To find the sign of partial derivative;
[0083] In step S1, in DEC-DA, feature selection specifically includes the following steps:
[0084] Given a set of data samples Where D and n represent the dimension and number of data samples respectively, and each data sample is Converted to an enhanced version, where T represents the transformation and the loss function of data enhancement DA is expressed as:
[0085]
[0086] Where fW and gU represent the encoder and decoder respectively; the encoder and decoder are fully connected networks or convolutional networks;
[0087] After connecting data enhancement DA and deep embedding clustering DEC, the overall loss function of DEC-DA is:
[0088] L=αL r +βL c ,
[0089] Where L c = dist(y, t) represents the distance between the output y and the student distribution t, where y and t represent the output and target of the feature learning model, α is the degree of freedom of the student t distribution, and β is the weight;
[0090] The target t is calculated using the original sample X, and the output y is calculated using the enhanced sample calculate.
[0091] S2: In the DEC-SeDA framework, time series scaling and selective data augmentation are performed. The time series scaling is performed in equal proportions in each dimension, and samples are selected for data augmentation to selectively amplify features that are more conducive to clustering.
[0092] In step S2, SeDA automatically selects augmented data by comparing the value of the data dispersion index DDI with χ, where in the scaling of the time series, the target t is calculated by taking the clean sample X, and the output y is calculated by taking the clean sample X and the augmented sample The union of is calculated, Effectively enhance the time series x i Transformed by the following scaling operations:
[0093]
[0094] in is the ratio of increase or decrease of all elements. The increase or decrease of dimensions is carried out simultaneously. The above formula can be rewritten as:
[0095]
[0096] in Specifically norm(1,σ), which generates normally distributed nonzero numbers around 1.
[0097] In step S2, Selective Data Augmentation (DA) divides the sample into internal and external parts by measuring the distance from the sample to the mean center of the clean data. Given an effective enhanced sequence x i and mean center μ,x i The distance between and μ is denoted as λ i , sort all values λ and put λ i The ascending ranking is recorded as rank i ; x i Located in the inner part, x i Located in the outer part, where n represents the number of data samples;
[0098] The probability density function PDF of the shifted Sigmoid function is used for selective data enhancement. When the samples in the internal part are enhanced, the PDF is: When the samples in the outer part are enhanced, the PDF is These two probability density functions are Figure 2 The simulation was performed in .
[0099] If the DDI is lower than the threshold χ, the external samples are enhanced because when When the sample is distributed in a thin spherical layer, the effect of enhancing the inner and outer samples will be similar; when When it is low, the outer samples should be enhanced as described above. If the DDI is higher than the threshold χ, the inner samples are enhanced, and the threshold χ is 0.15; the PDF is selected as follows:
[0100]
[0101] Under the control of PDF, SeDA is executed as follows:
[0102]
[0103] Among them, RF(0,1) represents a random floating point number between 0 and 1, and NULL means no data enhancement.
[0104] The distance between the mean and median centers is used to measure the distribution regularity of the data, which helps to initially understand the degree of sample deviation relative to the mean center. First, The metric value of , where ρ is the mean center of the data sample, max(λ i ) is all λ i Divide by max(λ i ), dist(μ,ρ) can be normalized.
[0105] If the sample is close to the mean center, dist(μ,ρ) is relatively low, and max(λ i ) is relatively high, in this case, you need to add a Mutually exclusive additional indices, which are combined with Multiplying them can significantly reduce its sensitivity, and the value range of this new indicator is limited to (0,1).
[0106] To achieve the above purpose, refer to The value of reflects the distribution strength of the sample. Assuming that the space surrounded by the outermost sample is approximately a sphere, the distribution space of the entire data set can be approximated as a spherical layer, and It can be used to estimate the thickness of the spherical layer. When the sample distribution is sparse, is larger; on the contrary, when the samples are densely distributed, Smaller. The indicator is expressed as It is called the Data Dispersion Index (DDI), which has the following characteristics:
[0107] In extreme cases, if the sample distribution is too dense, then is close to zero, if the mean is very close to the median, then In both cases, the value of DDI is close to zero.
[0108] The values of DDI are located in a smaller range than (0, 1). Specifically, the distribution of DDI values is similar to an approximate quadratic function, where the maximum value is located in the middle and the minimum value is located on both sides of the independent variable.
[0109] Experimental setup
[0110] Six groups of experiments are conducted to evaluate the performance of the proposed method. The first group of experiments verifies the effectiveness of DEC-SeDA and ten new variants of deep clustering algorithms. The second group of experiments demonstrates the effectiveness of SeDA compared with DEC. Further extended experiments are given in Section 4. The third group of experiments illustrates the efficiency of SeDA. The fourth group of experiments verifies the effectiveness of the parameter χ. The fifth group of experiments studies the sensitivity of the parameter σ, and the sixth group studies the time consumption of the algorithm. The process is as follows:
[0111] Dataset Resources: To illustrate the effectiveness of DEC-SeDA, the largest repository of time series datasets, namely the UCR / UEA archive, was used, which contains 128 univariate datasets and hourly load curves simulated by specific regions. All 128 univariate datasets in the UCR / UEA archive were used. Among all the energy consumption datasets, six energy consumption types were selected, such as the power consumption of fans and indoor lights. Each dataset contains buildings of the same energy consumption type, such as supermarkets, schools, hospitals, apartments, offices, restaurants, warehouses, etc. For the experiments, 5000 samples were randomly selected from all the original data.
[0112] Performance indicators: Since the UCR dataset has labels and the energy consumption dataset is not labeled, the external evaluation indicator normalized mutual information (NMI) and two internal evaluation indicators, including the silhouette coefficient (SC) and the Davidson-Bondage index (DBI), are used to evaluate the clustering performance of these two types of datasets. NMI is used to measure the similarity between two clustering results, and can determine the accuracy of community division compared with the standard division. Its value range is 0 to 1, and the higher the value, the more accurate the clustering. SC is used to describe the similarity between the sample and other clusters. Its value range is [-1, +1], and the higher the value, the more accurate the clustering. DBI calculates the intra-cluster distance ratio of all clusters, and a lower DBI indicates better clustering performance.
[0113] In order to thoroughly compare the performance of all clustering algorithms in case study 1, the NMI of all algorithms on the UCR benchmark is compared using Friedman test to obtain their overall ranking.
[0114] Comparison of algorithms and parameter settings: To comprehensively demonstrate the performance of DEC-SeDA in time series clustering, some state-of-the-art clustering algorithms are compared. Considering that deep clustering algorithms include network architecture, pre-loss, and clustering loss, these state-of-the-art clustering algorithms are generated by combining components of recently proposed network architecture, pre-loss, and clustering loss. For the network architecture, a fully connected neural network layer is adopted. For the pre-loss, triplet loss and variational autoencoder (VAE) are adopted respectively. For the clustering loss, deep embedding clustering (DEC), deep embedding regularized clustering (DEPICT), deep temporal clustering (DTCR)
[55] , and IDEC
[56] are used respectively. Among them, triplet loss has the advantage of only requiring an architecture containing the encoder. Removing the decoder can save computational resources because its parameters do not need to be trained, but this also eliminates the problem of decoder architecture design.
[0115] The parameters are set as follows: The structure of the encoder is a three-layer artificial neural network, which includes and Three encoders. The maximum number of iterations to optimize the encoder weights is 1000, and p is calculated. ij and q ij The update interval is 200; the tolerance threshold for stopping training is 0.001; SGD is used to pre-train AE with a learning rate of 0.1 and a momentum of 0.9; the number of epochs and batch size of pre-training are 100 and 256, respectively.
[0116] Embodiment 1
[0117] DEC-SeDA was independently run 30 times on the UCR benchmark. According to the Friedman test, the average NMI index ranking of all algorithms is shown in Table I. From the Friedman test results given in Table I, it can be concluded that:
[0118] The proposed DEC-SeDA ranks the highest among all algorithms, while the original VAE-DEC ranks the lowest. Although the triples are used to assist in training the encoder specialized for time series, the clustering performance can only be slightly improved. This result proves that the proposed SeDA effectively improves the performance of the DEC algorithm.
[0119] Triplet does not improve the clustering performance of all algorithms. When working with DEPICT and DEC, triple helps improve the clustering performance, while for other algorithms, triple is useless.
[0120] In terms of clustering loss, DTCR and DEC achieve the best and worst performance respectively, which means that K-means based clustering loss is better than KL based clustering loss. In addition, IDEC has a slight advantage over DEPICT.
[0121] The NMI distribution of DEC-SeDA after 30 runs on 50 randomly selected benchmarks is shown in Figure 3 As shown in Figure 1, the overall distribution and dispersion of the data, such as the minimum and maximum values, each quantile, the median, and outliers, are included. It can be observed that the spread of NMI is small and there are almost no outliers in the experimental results. In addition, results with NMI greater than 0.5 were obtained on about half of the benchmarks. Therefore, DEC-SeDA performs reliably in terms of clustering stability and effectiveness.
[0122] Table I Average ranking of algorithms
[0123]
[0124] Embodiment 2
[0125] DEC-SeDA and DEC were run independently 30 times on the energy consumption data, and the experimental results are shown in Table II. The neural network architectures in both DEC and DEC-SeDA use AE instead of VAE. DEC-SeDA is more effective than DEC on all problems. In terms of SC, DEC-SeDA has the most significant improvements on D5 and D4, which are 182% and 16%, respectively. The possible reasons for this performance difference are as follows: D4 is a dataset where samples are distributed in clusters, and it is relatively easy to search for the initial cluster centers in the search space. Usually the optimization algorithm can directly find the optimum, which makes it difficult to further improve the clustering performance.
[0126] Table I Comparison of performance indicators between IDEC-SeDA and DEC
[0127]
[0128] Efficiency of SeDA: To demonstrate the efficiency of SeDA, the sample-specific improvement (MSI) metrics of DEC-DA and DEC-SeDA over DEC are given as follows. The MSI metric is calculated by the ratio of the improved performance metric to the number of samples participating in data augmentation. Obviously, the sample-specific SC and DBI improvements of DEC-SeDA are significantly better than those of DEC-DA. On D3, D4, and D5, DEC-SeDA even doubles the sample-specific improvements SC and DBI of DEC-DA, which means that DEC-SeDA can achieve the same effect as DEC-DA even if only half of the samples are augmented. This result proves the necessity of data augmentation on selected samples. The sample-specific improvement performance metrics are shown in Table III;
[0129] Parameter Validity: To verify the validity of the parameters, DEC-SeDA(inside) and DEC-SeDA(outside) with data samples enhanced separately are compared on the energy consumption dataset as shown in Table IV. In addition, to verify the rationality of dividing the dataset into internal and external parts, DEC-SeDA(Rand) with selective DA using randomly selected samples is also tested. The data samples are evenly divided into internal and external parts, i.e., each part contains half of all samples, and DEC-SeDA(Rand) enhances the random half of the samples. These DEC-SeDA variants are independently run 30 times, and the average results of SC and DBI are given. The DDI value of each dataset is also provided. By setting mutually exclusive factors, the value of DDI is stabilized in the range of 0.08 to 0.19. For datasets with χ<0.15, including D1, D2, D3, and D5, DEC-SeDA(outside) outperforms DEC-SeDA(inside); while for D4 dataset with χ>0.15, DEC-SeDA(inside) exceeds DEC-SeDA(outside). This result is consistent with the setting of χ in this paper, indicating that the use of selective data samples for augmentation is necessary and beneficial.
[0130] Sensitivity analysis: In addition to the parameter χ, the parameter σ also affects the clustering performance. In order to test the sensitivity of the parameter, the parameter σ is set from 0 to 0.06 with an interval of 0.05, and the corresponding performance of DEC-SeDA is evaluated. The experimental results are shown in Figure 4 Obviously, DEC-SeDA is insensitive to the changes in parameters, which means that the proposed SeDA is easy to operate. This is consistent with the previous theoretical assumption that if each dimension of the data sample is scaled in the same ratio, the augmented data will help to quickly optimize the weights of the autoencoder.
[0131] Computational load of SeDA: The time consumption of DEC, DEC-DA and DEC-SeDA is as follows: Figure 5 As shown in Figure 3. On the tested dataset, the time consumption of DEC-DA is almost twice that of DEC. As the number of original and enhanced samples increases, the time consumption caused by DA will be higher. Compared with DA, SeDA significantly reduces the time consumption. Since half of the data samples are still enhanced, the time consumption can be further reduced if fewer samples are selected.
[0132] Table III Sample-specific improved performance metrics
[0133]
[0134] Table IV: Comparison of performance metrics when executing SEDA using internal and external data samples
[0135]
[0136] Experimental results show that SeDA in this paper is effective on time series classification (TSC). DEC-SeDA shows effectiveness on 128 UCR datasets and is significantly more efficient than DEC-DA considering selected samples on five energy consumption-based datasets.
[0137] Therefore, the present invention adopts the above-mentioned deep clustering method with selective enhancement of training data, defines two typical sample distribution patterns, determines the distribution pattern by extracting the distribution characteristics of the samples, and probabilistically selects and enhances samples under a probability distribution function according to the determined pattern. The effectiveness and efficiency of the proposed algorithm are verified through competitive performance in UCR benchmark tests and practical applications.
[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.
Claims
1. A deep clustering method with selective enhancement of training data, characterized by: In deep embedding clustering DEC, the selective data enhancement SeDA method for clustering time series is embedded to form DEC-SeDA. In DEC-SeDA, two probability distribution functions PDF related to Sigmoid are proposed to perform selective data enhancement, and the data dispersion index DDI is proposed to reflect the data distribution characteristics; The specific steps include: S1: In deep embedding clustering DEC, feature selection and feature extraction are performed. Feature selection amplifies the distribution characteristics of clustering, and feature extraction is achieved through autoencoders; S2: In the DEC-SeDA framework, time series scaling and selective data augmentation are performed. The time series scaling is performed in equal proportions in each dimension, and samples are selected for data augmentation to selectively amplify features that are more conducive to clustering.
2. The deep clustering method for selectively enhancing training data according to claim 1, characterized in that: In step S1, feature extraction in DEC specifically includes the following steps: S11: The original data X is projected into the latent space Z to deal with the curse of dimensionality. S12: A deep autoencoder AE is initialized and the parameters are optimized by iteratively comparing the auxiliary target distribution and minimizing the divergence KLD; S13: Soft assignments are calculated between embedded samples and cluster centers Depth mapping function f θ is updated to optimize cluster centers; In step S13, based on the auxiliary target distribution, initializing the cluster center includes the following steps: S131: Data is passed through the initialized autoencoder AE to obtain embedded data samples; S132: Perform analysis in feature space Z using standard k-means clustering to obtain k initial cluster centers Reduce the high dimensionality of the original data to a low dimensionality that is conducive to clustering; The soft distribution is the Student's t distribution, described as: In the formula, the embedding point z i =f θ (x i )∈Z represents x i ∈X is projected into a low-dimensional space, α and q ij is the degrees of freedom of the Student’s t distribution, q ij represents the probability of assigning sample i to cluster j, μ j represents the cluster center; S133: Measure the embedding point Z i and cluster center μ j The similarity between them is used to construct the optimization target of KL clustering loss; S134: Soft allocation q i , clustering assignment s and auxiliary distribution p i , the difference between the student distribution t is expressed as: In the formula, p ij First, q i The quadratic power is calculated and then normalized by the frequency of each cluster: Cluster center μ j The parameters θ of the autoencoder AE are optimized by stochastic gradient descent SGD with momentum, and the embedding point z i The feature space embedding and cluster center μ j The gradient of the loss L is calculated as follows: In the formula, To find the sign of the partial derivative.
3. The deep clustering method for selectively enhancing training data according to claim 1, characterized in that: In step S1, in DEC-DA, feature selection specifically includes the following steps: Given a set of data samples Where D and n represent the dimension and number of data samples respectively, and each data sample is Converted to an enhanced version, where T represents the transformation and the loss function of data enhancement DA is expressed as: Where fW and gU represent the encoder and decoder respectively; the encoder and decoder are fully connected networks or convolutional networks; After connecting data enhancement DA and deep embedding clustering DEC, the overall loss function of DEC-DA is: L=αL r +βL c , Where L c = dist(y,t) represents the distance between the output y and the student distribution t, y represents the output of the feature learning model, α is the degree of freedom of the student t distribution, and β is the weight; The target t is calculated using the original sample X, and the output y is calculated using the enhanced sample calculate.
4. The deep clustering method for selectively enhancing training data according to claim 3, characterized in that: In step S2, SeDA automatically selects augmented data by comparing the value of the data dispersion index DDI with χ, where in the scaling of the time series, the target t is calculated by taking the clean sample X, and the output y is calculated by taking the clean sample X and the augmented sample The union of is calculated, Effectively enhance the time series x i Transformed by the following scaling operations: in is the ratio of increase or decrease of all elements. The increase or decrease of dimensions is carried out simultaneously. The above formula can be rewritten as: in Specifically norm(1,σ), which generates normally distributed nonzero numbers around 1.
5. The deep clustering method for selectively enhancing training data according to claim 4, characterized in that: In step S2, Selective Data Augmentation (DA) divides the sample into internal and external parts by measuring the distance from the sample to the mean center of the clean data. Given an effective enhanced sequence x i and mean center μ,x i The distance between and μ is denoted as λ i , sort all values λ and put λ i The ascending ranking is recorded as rank i ; x i Located in the inner part, x i Located in the outer part, where n represents the number of data samples; The probability density function PDF of the shifted Sigmoid function is used for selective data enhancement. When the samples in the internal part are enhanced, the PDF is: When the samples in the outer part are enhanced, the PDF is If the DDI is lower than the threshold χ, the external samples are enhanced, and if the DDI is higher than the threshold χ, the internal samples are enhanced. The threshold χ is 0.15; the PDF is selected as follows: Under the control of PDF, SeDA is executed as follows: Among them, RF(0,1) represents a random floating point number between 0 and 1, and NULL means no data enhancement.
