A spectral clustering algorithm combined with a self-encoder, a device, an apparatus and a storage medium

Abstract

The present application relates to the technical field of unsupervised learning of computer technology, and discloses a spectral clustering algorithm combined with a self-encoder, a device, equipment and a storage medium.The algorithm comprises the following steps: calculating a Laplacian matrix corresponding to target data; taking a preset cluster number, the target data and the Laplacian matrix as inputs, and learning a feature corresponding to the target data through a self-encoder; a total loss function used by the self-encoder during learning is obtained according to a loss function of spectral clustering, a sparsity constraint and a loss function of the self-encoder itself; and the feature is clustered based on a preset clustering algorithm to obtain a clustering result.The present application uses a self-encoder to replace a feature decomposition step in an existing spectral clustering algorithm, and improves a loss function, so that the self-encoder can take into account target optimization of the original spectral clustering algorithm, and solves the technical problem of high operation time complexity of feature decomposition of the Laplacian matrix in the existing spectral clustering algorithm.

Description

TECHNICAL FIELD

[0001] The present application relates to the unsupervised learning technical field of computer technology, and particularly relates to a spectral clustering algorithm combined with a self-encoder, a device, equipment and a storage medium. BACKGROUND

[0002] Spectral clustering is a common clustering algorithm based on spectral graph theory. The process of the spectral clustering algorithm is to first obtain a similarity matrix and a Laplacian matrix by calculating the similarity between data, then perform eigenvalue decomposition on the Laplacian matrix to obtain the features corresponding to the original data, and finally perform clustering on the obtained features to obtain the final clustering result. The spectral clustering algorithm has the advantages of low sensitivity to sample shape, convergence to global optimal solution and good support for high-dimensional data compared with the traditional K-means algorithm. However, since the eigenvalue decomposition of the Laplacian matrix is required, the time complexity of this operation is O(n3), and when facing massive data, the time cost of this operation will greatly increase.

[0003] The self-encoder is a neural network used in unsupervised learning, which takes the input data as a label to learn the representation of the input data. The self-encoder is often applied to dimensionality reduction and anomaly detection of data. SUMMARY

[0004] The present application provides a spectral clustering algorithm combined with a self-encoder, a device, equipment and a storage medium, which improves the existing spectral clustering algorithm by combining the self-encoder, and solves the technical problem of high time complexity of the operation of eigenvalue decomposition of the Laplacian matrix in the existing spectral clustering algorithm.

[0005] The present application provides a spectral clustering algorithm combined with a self-encoder, a device, equipment and a storage medium, which improves the existing spectral clustering algorithm by combining the self-encoder, and solves the technical problem of high time complexity of the operation of eigenvalue decomposition of the Laplacian matrix in the existing spectral clustering algorithm.

[0006] Calculate the Laplacian matrix corresponding to the target data;

[0007] Take the preset number of clusters, the target data and the Laplacian matrix as inputs, and learn the features corresponding to the target data through the self-encoder; the total loss function used by the self-encoder during learning is obtained according to the loss function of spectral clustering, the sparsity constraint and the loss function of the self-encoder itself;

[0008] Cluster the features based on the preset clustering algorithm to obtain a clustering result.

[0009] According to an implementable manner of the first aspect of the present application, the total loss function is:

[0010] Loss=Lr+αLs+βLe

[0011] In the formula, Loss represents a total loss function used when the autoencoder is learned, Lr represents a spectral clustering loss, L represents a sparsity constraint, and a and β are preset tuning parameters, and Ls represents an autoencoder loss.

[0012] According to an implementable manner of the first aspect of the present application, the loss function of the spectral clustering is:

[0013] Ls = Tr(F T LF)

[0014] In the formula, Tr represents, F represents features corresponding to the target data, L represents the Laplacian matrix, F T is the transpose of F, and Tr(F T LF) represents the trace of the matrix F T LF.

[0015] According to an implementable manner of the first aspect of the present application, the sparsity constraint is:

[0016]

[0017] In the formula, n is the number of the target data, k is the preset number of clusters, δ(·) is a softmax function, and f ij is the similarity between the i th data point in the target data and the j th cluster.

[0018] According to an implementable manner of the first aspect of the present application, the loss function of the autoencoder itself is:

[0019]

[0020] In the formula, n is the number of the target data, g ij represents the output of the j th layer of the encoder when the i th data point is input, x ij represents the output of the j th layer of the decoder when the i th data point is input, and N is the number of layers of the encoder and the decoder.

[0021] According to an implementable manner of the first aspect of the present application, the features are clustered based on a preset clustering algorithm to obtain a clustering result, which includes:

[0022] The features are clustered based on a K-means clustering algorithm to obtain a clustering result.

[0023] According to an implementable manner of the first aspect of the present application, the algorithm further includes:

[0024] Before calculating the Laplacian matrix corresponding to the target data, the input raw data is preprocessed to obtain the target data; the preprocessing includes outlier cleaning and data normalization processing.

[0025] The second aspect of the application provides a spectral clustering device combined with a self-encoder, comprising:

[0026] A calculation module is configured to calculate a Laplacian matrix corresponding to target data.

[0027] A feature learning module is configured to take a preset cluster number, the target data and the Laplacian matrix as inputs, and learn the features corresponding to the target data through a self-encoder; a total loss function used by the self-encoder during learning is obtained according to a spectral clustering loss function, a sparsity constraint and a loss function of the self-encoder itself.

[0028] A clustering module is configured to cluster the features based on a preset clustering algorithm to obtain a clustering result.

[0029] Loss=Lr+αLs+βLe

[0030] In the formula, Loss represents the total loss function used by the self-encoder during learning, Lr represents the spectral clustering loss, L represents the sparsity constraint, and α and β are preset tuning parameters; Ls represents the self-encoder loss.

[0031] According to an implementable manner of the second aspect of the application, the spectral clustering loss function is:

[0032] Ls=Tr(F T LF)

[0033] In the formula, Tr represents, F represents the features corresponding to the target data, L represents the Laplacian matrix, F T is the transpose of F, and Tr(F T LF) represents the trace of the matrix F T LF.

[0034] According to an implementable manner of the second aspect of the application, the sparsity constraint is:

[0035]

[0036] In the formula, n is the number of the target data, k is the preset cluster number, δ(·) is a softmax function, and f ij is the similarity between the i th data point in the target data and the j th cluster.

[0037] According to an implementable manner of the second aspect of the present application, the loss function of the autoencoder itself is:

[0038]

[0039] where n is the number of the target data, g ij represents the output of the jth layer of the encoder when the ith data point is taken as the input, x ij represents the output of the jth layer of the decoder when the ith data point is taken as the input, and N is the number of layers of the encoder and the decoder.

[0040] According to an implementable manner of the second aspect of the present application, the clustering module comprises:

[0041] a clustering unit configured to cluster the features based on a K-means clustering device to obtain a clustering result.

[0042] According to an implementable manner of the second aspect of the present application, the device further comprises:

[0043] a preprocessing module configured to preprocess the input raw data to obtain the target data before calculating the Laplacian matrix corresponding to the target data, wherein the preprocessing comprises outlier cleaning and data normalization processing.

[0044] The third aspect of the present application provides a spectral clustering device combined with an autoencoder, comprising:

[0045] a memory configured to store instructions, wherein the instructions are used to implement the spectral clustering algorithm combined with the autoencoder according to any one of the implementable manners described above;

[0046] a processor configured to execute the instructions in the memory.

[0047] The fourth aspect of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the spectral clustering algorithm combined with the autoencoder according to any one of the implementable manners described above.

[0048] As can be seen from the above technical solutions, the present application has the following advantages:

[0049] The algorithm of the application comprises: calculating a Laplacian matrix corresponding to target data; taking a preset cluster number, the target data and the Laplacian matrix as inputs, and learning a feature corresponding to the target data through a self-encoder; a total loss function used by the self-encoder during learning is obtained according to a loss function of spectral clustering, a sparsity constraint and a loss function of the self-encoder itself; clustering the feature based on a preset clustering algorithm to obtain a clustering result; the application uses the self-encoder to replace a feature decomposition step in an existing spectral clustering algorithm, and additionally adds the loss function of spectral clustering and the sparsity constraint to the loss function of the self-encoder itself, so that the self-encoder can learn the feature while taking into account the target optimization of the original spectral clustering algorithm, can reduce the time cost of the traditional spectral clustering algorithm when facing massive data, and solves the technical problem of high operation time complexity of the feature decomposition of the Laplacian matrix in the existing spectral clustering algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0051] Figure 1 A flowchart of a spectral clustering algorithm combined with a self-encoder is provided for an optional embodiment of the present application.

[0052] Figure 2 A network structure diagram of a self-encoder is provided for an optional embodiment of the present application.

[0053] Figure 3 A structure connection block diagram of a spectral clustering device combined with a self-encoder is provided for an optional embodiment of the present application.

[0054] Reference signs:

[0055] 1-computing module; 2-feature learning module; 3-clustering module. DETAILED DESCRIPTION

[0056] The embodiments of the present application provide a spectral clustering algorithm, device, equipment and storage medium combined with a self-encoder, which are used to solve the technical problem of high operation time complexity of feature decomposition of a Laplacian matrix in an existing spectral clustering algorithm.

[0057] In order to make the application purposes, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the embodiments described below are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0058] The present application provides a spectral clustering algorithm combined with a self-encoder.

[0059] Please refer to Figure 1 , Figure 1 The flow chart of the spectral clustering algorithm combined with the self-encoder provided by the embodiments of the present application is shown.

[0060] The spectral clustering algorithm combined with the self-encoder provided by the embodiments of the present application comprises:

[0061] Step S1, calculating a Laplacian matrix corresponding to the target data.

[0062] In an implementable manner, the Laplacian matrix corresponding to the target data is calculated, and the calculation is specifically performed as follows:

[0063] The distance between each two of all the data is calculated and a distance matrix is generated;

[0064] A weight matrix is calculated according to the distance matrix;

[0065] A degree matrix is calculated according to the weight matrix;

[0066] Based on the obtained degree matrix, a Laplacian matrix is obtained through matrix multiplication.

[0067] As an implementation, when the distance between each two of all the data is calculated, the distance between each two of all the data is the Euclidean distance.

[0068] As an implementation, when the weight matrix is calculated according to the distance matrix, the calculation is performed according to the following formula:

[0069]

[0070] In the formula, w ij is the similarity between data i and data j, d(x i , x j ) is the distance between data i and data j, and σ 2 is the feature dimension of the data.

[0071] As an implementation, when the degree matrix is calculated according to the weight matrix, the calculation of the degree of each data is performed according to the following formula:

[0072]

[0073] In the formula, d is the degree of data i, and n is the number of target data. ii

[0074] As an implementation, when the Laplacian matrix is obtained through matrix multiplication, the calculation is specifically performed according to the following formula:

[0075]

[0076] In the formula, D represents a degree matrix, and W represents a weight matrix.

[0077] In step S2, the preset number of clusters, the target data and the Laplacian matrix are taken as inputs, and the feature corresponding to the target data is obtained through self-encoder learning; the total loss function used by the self-encoder during learning is obtained according to a loss function of spectral clustering, a sparsity constraint and a loss function of the self-encoder itself.

[0078] In an implementable manner, the total loss function is as follows:

[0079] Loss = Lr + aLs + bLe

[0080] In the formula, Loss represents the total loss function used by the self-encoder during learning, Lr represents a spectral clustering loss, L represents a sparsity constraint, a and b are both preset tuning parameters, and Ls represents a self-encoder loss.

[0081] The network structure diagram of the self-encoder is as shown in Figure 2 Win, Wf, Wout and Wg are all network parameters of the self-encoder. The learning process of the self-encoder can be considered to have two parts: an encoding process F = f(X) and a decoding process G = G(f(X)) for generating reconstruction. In the encoding process, there is an encoder hidden layer Hin in the auto-encoder, which can generate an encoding to represent the input X. In the decoding process, there is a decoder hidden layer Hout in the auto-encoder, which can decode the result obtained by encoding to obtain the decoder result G.

[0082] In an implementable manner, the loss function of spectral clustering is as follows:

[0083] Ls = Tr(F T LF)

[0084] In the formula, Tr represents, F represents the feature corresponding to the target data, L represents the Laplacian matrix, F T is the transpose of F, and Tr(F T LF) represents the trace of the matrix F T LF.​

[0085] In an implementable manner, the sparsity constraint is:

[0086]

[0087] wherein n is the number of the target data, k is the preset number of clusters, δ(·) is a softmax function, f ij is the similarity of the i th data point in the target data and the j th cluster.

[0088] When a vector only contains one non-zero item, the entropy of the vector is equal to zero. In this case, the corresponding data point belongs to the cluster with the same data point as the non-zero item. In this embodiment, the feature F with better clustering effect is obtained by adding the sparsity constraint.

[0089] In an implementable manner, the loss function of the autoencoder itself is:

[0090]

[0091] wherein n is the number of the target data, g ij represents the output of the j th layer of the encoder when the i th data point is input, x ij represents the output of the j th layer of the decoder when the i th data point is input, and N is the number of layers of the encoder and the decoder.

[0092] In the above embodiments of the present application, the feature decomposition step in the traditional spectral clustering algorithm is replaced by using the autoencoder, and two loss functions consistent with the optimization objective of spectral clustering are additionally added to the loss function of the autoencoder itself: the loss function of spectral clustering and the sparsity constraint, so that the autoencoder can perform feature learning while considering the objective optimization of the spectral clustering algorithm.

[0093] Step S3: clustering the features based on a preset clustering algorithm to obtain a clustering result.

[0094] In an implementable manner, the clustering of the features based on the preset clustering algorithm to obtain a clustering result comprises:

[0095] clustering the features based on a K-means clustering algorithm to obtain a clustering result.

[0096] In other implementable manners, the preset clustering algorithm adopts a fuzzy clustering algorithm.

[0097] In an implementable manner, the algorithm further comprises:

[0098] Before calculating the Laplacian matrix corresponding to the target data, the original input data is preprocessed to obtain the target data; the preprocessing includes outlier cleaning and data normalization processing.

[0099] The above embodiments of the present application can reduce the time cost of the traditional spectral clustering algorithm when facing massive data, and solve the technical problem of high operation time complexity of the existing spectral clustering algorithm for feature decomposition of the Laplacian matrix.

[0100] The present application also provides a spectral clustering device combined with a self-encoder, which can be used to execute the spectral clustering algorithm combined with a self-encoder according to any one of the above embodiments of the present application.

[0101] Please refer to Figure 3 , Figure 3 The structure connection block diagram of the spectral clustering device combined with a self-encoder according to an embodiment of the present application is shown.

[0102] The spectral clustering device combined with a self-encoder according to an embodiment of the present application comprises:

[0103] The computing module 1 is configured to calculate the Laplacian matrix corresponding to the target data.

[0104] The feature learning module 2 is configured to take the preset cluster number, the target data and the Laplacian matrix as inputs, and learn the features corresponding to the target data through a self-encoder; the total loss function used by the self-encoder during learning is obtained according to the loss function of spectral clustering, the sparsity constraint and the loss function of the self-encoder itself.

[0105] The clustering module 3 is configured to cluster the features based on a preset clustering algorithm to obtain a clustering result. In an implementable manner, the total loss function is:

[0106] Loss=Lr+αLs+βLe

[0107] In the formula, Loss represents the total loss function used by the self-encoder during learning, Lr represents the spectral clustering loss, L represents the sparsity constraint, and α and β are preset tuning parameters. Ls represents the self-encoder loss.

[0108] In an implementable manner, the loss function of spectral clustering is:

[0109] Ls=Tr(F T LF)

[0110] In the formula, Tr represents, F represents the features corresponding to the target data, L represents the Laplacian matrix, F T is the transpose of F, and Tr(F T LF) represents the matrix FT LF's trace.

[0111] In an implementable manner, the sparsity constraint is:

[0112]

[0113] where n is the number of target data, k is the preset number of clusters, δ(·) is a softmax function, f ij is the similarity between the i th data point in the target data and the j th cluster.

[0114] In an implementable manner, the loss function of the autoencoder itself is:

[0115]

[0116] where n is the number of target data, g ij represents the output of the j th layer of the encoder when the i th data point is input, x ij represents the output of the j th layer of the decoder when the i th data point is input, and N is the number of layers of the encoder and the decoder.

[0117] In an implementable manner, the clustering module 3 comprises:

[0118] A clustering unit configured to cluster the features based on a K-means clustering device to obtain a clustering result.

[0119] In an implementable manner, the device further comprises:

[0120] A preprocessing module configured to preprocess the input raw data to obtain the target data before calculating the Laplacian matrix corresponding to the target data; the preprocessing comprises outlier cleaning and data normalization processing.

[0121] The present application also provides a spectral clustering device combined with an autoencoder, comprising:

[0122] A memory configured to store instructions; wherein the instructions are configured to implement the spectral clustering algorithm combined with the autoencoder according to any one of the above embodiments;

[0123] A processor configured to execute the instructions in the memory.

[0124] The present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the spectral clustering algorithm combined with the autoencoder according to any one of the above embodiments.

[0125] Those skilled in the art can clearly understand the specific working process of the apparatus, device, module and unit described above for the convenience and brevity of description, can refer to the corresponding process in the foregoing method embodiments, and the specific benefits of the apparatus, device, module and unit described above can refer to the corresponding benefits in the foregoing method embodiments, which will not be described here.

[0126] In several embodiments provided in the present application, it should be understood that the disclosed apparatus, device and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. Taking the division of the modules as an example, the division can be a logical function division, and there can be another division manner in actual implementation. For example, a plurality of modules or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, apparatuses or modules, and can be electrical, mechanical or in other forms.

[0127] The modules described as separated components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, i.e., can be located in one place, or can be distributed on a plurality of network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0128] In addition, each functional module in the various embodiments of the present application can be integrated in one processing module, or each module can exist physically independently, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0129] When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that makes a contribution to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0130] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A spectral clustering algorithm combined with a self-encoder, characterized in that, The method comprises the following steps: calculating a Laplacian matrix corresponding to target data; learning features corresponding to the target data by using a self-encoder with the preset number of clusters, the target data and the Laplacian matrix as inputs; a total loss function used by the self-encoder during learning is obtained according to a loss function of spectral clustering, a sparsity constraint and a loss function of the self-encoder itself; performing clustering on the features based on a preset clustering algorithm to obtain a clustering result; the total loss function is: ； wherein, denotes the total loss function employed by the autoencoder during learning, denotes the spectral clustering loss, denotes the sparsity constraint, and are preset tuning parameters, denotes the autoencoder loss; the loss function of spectral clustering is: ； wherein, denotes a feature corresponding to the target data, denotes the Laplacian matrix, is the transpose of denotes the trace of the matrix . the sparsity constraint is: ； In the formula, The quantity of the target data. The preset number of clusters, For the softmax function, For the target data, the first The data point and the first The similarity of each cluster. 2.The spectral clustering algorithm with self-encoder of claim 1, wherein, the loss function of the self-encoder itself is: ； wherein is the number of the target data, denotes the output of the encoder's layer when the th data point is used as input, denotes the output of the decoder's layer when the th data point is used as input, is the number of layers of the encoder and the decoder. 3.The spectral clustering algorithm with self-encoder of claim 1, wherein, the method further comprises the following steps: performing clustering on the features based on a K-means clustering algorithm to obtain a clustering result.

4. The spectral clustering algorithm with self-encoder according to claim 1, characterized in that, The method further comprises the following steps: performing preprocessing on input original data to obtain the target data before calculating the Laplacian matrix corresponding to the target data; the preprocessing comprises outlier cleaning and data normalization processing.

5. A spectral clustering apparatus incorporating a self-encoder, characterized by, The method comprises the following steps: a calculating module, configured to calculate a Laplacian matrix corresponding to target data; a feature learning module, configured to learn features corresponding to the target data by using a self-encoder with the preset number of clusters, the target data and the Laplacian matrix as inputs; a total loss function used by the self-encoder during learning is obtained according to a loss function of spectral clustering, a sparsity constraint and a loss function of the self-encoder itself; a clustering module, configured to perform clustering on the features based on a preset clustering algorithm to obtain a clustering result; the total loss function is: ； wherein, denotes the total loss function employed by the autoencoder during learning, denotes the spectral clustering loss, denotes the sparsity constraint, and are preset tuning parameters, denotes the autoencoder loss; the loss function of spectral clustering is: ； wherein, denotes a feature corresponding to the target data, denotes the Laplacian matrix, is the transpose of denotes the trace of the matrix ​​ the sparsity constraint is: ； In the formula, The quantity of the target data. The preset number of clusters, For the softmax function, For the target data, the first The data point and the first The similarity of each cluster.

6. A spectral clustering device incorporating a self-encoder, characterized by, The method comprises the following steps: a memory, configured to store instructions; wherein the instructions are used to implement the spectral clustering algorithm combined with a self-encoder according to any one of claims 1-4; a processor, configured to execute the instructions in the memory.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the spectral clustering algorithm combined with a self-encoder according to any one of claims 1-4.

Images