Electroencephalogram signal clustering method based on Mama architecture and comparative learning

Abstract

The invention discloses an electroencephalogram signal clustering method based on a Mama framework and comparative learning, and belongs to the crossing field of engineering application and information science, and the method comprises the following steps: collecting and preprocessing label-free electroencephalogram data; the method comprises the following steps of: constructing a Mama-based feature extractor, dividing an input electroencephalogram signal into a plurality of slices by adopting an electroencephalogram signal slice embedding strategy so as to obtain fine-grained local information, and modeling a potential context relationship by utilizing a slice perception scanning mechanism; the method comprises the following steps: constructing an original data view and an enhanced data view, and respectively inputting samples of the two data views into two Mamba feature extractors with shared network parameters for feature embedding; processing the feature pairs of the two views using an instance projector, constructing weighted instance level contrast learning wherein the distance in the original data space provides weight information; processing feature pairs of the two views by using a clustering projector, and constructing a clustering distribution discrimination contrast learning branch and a semantic perception contrast learning branch; and finally, performing joint training by using the comparative learning branches, and outputting a high-quality clustering result by ensuring feature, clustering distribution and semantic consistency.

Description

Technical Field

[0001] This invention relates to a Mamba-based EEG signal clustering method, which is a clustering method for EEG signals and belongs to the interdisciplinary field of engineering applications and information science. Background Technology

[0002] Electroencephalogram (EEG) signals are complex, non-stationary, low signal-to-noise ratio (SNR), and non-invasive physiological signals used to record the brain's electrical activity. EEG can provide valuable insights into a variety of neurophysiological processes, such as cognition, emotion, sleep, and neurological disorders. Furthermore, EEG signals are widely used in brain-computer interfaces, such as motor imagery detection, robotic arm control, and virtual reality. Despite significant progress in EEG research, most existing studies focus on supervised tasks requiring extensive labeling expertise. However, labeling EEG signals is resource-intensive and time-consuming in real-world scenarios, posing a major challenge to EEG analysis. Therefore, developing advanced clustering algorithms has become a research focus, as they do not require human supervision or labeling. Moreover, clustering-based techniques can facilitate the identification of meaningful patterns and potential representations in EEG.

[0003] Mamba is an emerging state-space model that demonstrates significant performance in modeling sequential data and capturing long-term dependencies through state transitions, while improving operational efficiency through the use of convolutional computations. Recently, visual Mamba has achieved a good balance between performance and efficiency across various tasks, highlighting its enormous potential in computer vision applications. Furthermore, Mamba has begun exploratory trials in time series analysis, showing significant potential to outperform Transformer backbone networks.

[0004] Contrastive learning plays a crucial role in clustering tasks. By constructing positive and negative sample pairs, it forces the model to bring similar samples closer together and push dissimilar samples further apart, thereby learning more discriminative feature representations. In clustering, these discriminative features help the model better identify the inherent structure and distribution patterns of the data, resulting in more accurate and meaningful clustering results. For example, through contrastive learning, the model can learn which features are important factors in distinguishing different categories and which features are noise or redundant information. This allows clustering algorithms to effectively discover potential patterns and clusters in data even without supervised information. Furthermore, contrastive learning can enhance the model's robustness to data, enabling it to maintain good clustering performance even when faced with complex and varied data distributions. In summary, contrastive learning provides a powerful learning mechanism for unsupervised clustering tasks, significantly improving the accuracy and reliability of clustering. Summary of the Invention

[0005] The purpose of this invention is to provide a clustering method for EEG signals that combines the Mamba architecture and contrastive learning, in order to solve the problems of scarce EEG signal labels, time-consuming and labor-intensive manual labeling, and insufficient accuracy of existing clustering algorithms.

[0006] A clustering method for EEG signals based on the Mamba architecture and contrastive learning, the technical solution of which includes:

[0007] S1 collects and preprocesses unlabeled EEG data. S2 constructs a powerful and general-purpose Mamba-based EEG feature extractor. This feature extractor employs an EEG slice embedding strategy, dividing the input EEG signal into multiple slices to obtain fine-grained local information and utilizing a slice-aware scanning mechanism to model potential contextual relationships. S3 constructs two data views using the original EEG samples and their corresponding augmented samples. Samples from both data views are input into two Mamba feature extractors with shared network parameters for feature embedding. S4 processes the feature pairs from the two views using an instance projector, constructing a weighted instance-level contrastive learning model, where distance in the original data space provides weight information. S5 processes the feature pairs from the two views using a clustering projector, constructing a clustering assignment discriminative contrastive learning branch and a semantic-aware contrastive learning branch. S6 trains using S4 and S5, and outputs the clustering results through the clustering projector in S5.

[0008] S1 includes artifact removal and normalization of EEG data to avoid the effects of noisy data and data scale differences.

[0009] S2 includes building a Mamba-based feature extractor. Mamba is a type of state-space model, and state-space models are mathematical models used to describe and analyze the behavior of continuous systems. State-space models use a hidden state... Map a one-dimensional sequence or function x(t)∈R to a response y(t)∈R.

[0010]

[0011] in, These are evolutionary parameters. and These are projection parameters.

[0012] Mamba is a discrete version of the continuous state-space model described above. It introduces a time-scale parameter Δ, transforming A and B into discrete parameters A and B. This zero-order-preserving transformation can be expressed as:

[0013]

[0014] Then, the parameters can be discretized using the step size Δ, as shown below:

[0015]

[0016] Finally, the output of Mamba can be viewed as the result of a convolution operation.

[0017]

[0018] in, Let L represent the convolution kernel, and L be the length of the sequence x.

[0019] It is worth noting that Mamba implements a selective scanning mechanism. Specifically, the input sequence... (Where B is the batch size, L is the sequence length, and D is the number of channels) is converted into parameters. and Its inherent contextual information retrieval capability is enhanced by introducing a length dimension L.

[0020] Based on the Mamba architecture, this invention proposes a Mamba feature extraction framework for EEG signal feature learning. Instead of directly using EEG signal recordings as input, this framework employs a simple mechanism called EEG slice embedding to segment the EEG signal into equal-length slices. Furthermore, this feature extraction framework also utilizes a slice-aware scanning mechanism to arrange the input sequence of EEG slices.

[0021] The slice-aware scanning mechanism divides the raw EEG signal into equal-length slices and uses a linear feature extractor and positional encoding to obtain embedded features.

[0022] Given an input EEG signal It contains D channels and T time steps. EEG slice embedding uses d one-dimensional convolutions (kernel size k, stride k) to segment the EEG signal into L slices, where L = ceil(T / k), and ceil(·) represents the floor function. Subsequently, to preserve the temporal relationships in the original EEG signal, a positional encoding P is introduced. The entire process can be represented as follows:

[0023]

[0024] in, Ψ represents the input sequence, where P corresponds to the embedding position. conv Ψ represents a one-dimensional convolution operation. reshape Transform the sequence size from d×L to L×d.

[0025] While Mamba-based models perform well on time-series tasks, they do not consider contextual slice dependencies, which is crucial for deploying Mamba in EEG signal clustering. To address this, this invention utilizes a slice-aware scanning mechanism in Mamba-based feature extraction to model local contextual information between EEG slices. Specifically, three copies of the input EEG slice are made to establish three different slice scan sequences: forward slice scan, reverse slice scan, and shuffled slice scan. Subsequently, Mamba blocks with shared parameters are used to model the dependencies between these three slice scan sequences. Finally, the sequence embedding features are restored to the correct order (forward slice scan) and merged into the output. The slice-aware scanning mechanism can be represented as follows:

[0026]

[0027] in, This represents the output of the i-th (i∈1,2,...,N) Mamba layer or the input of the (i-1)-th Mamba block. yes Sequence embedding features. These represent three slice sequences derived from forward slice scan, reverse slice scan, and shuffled slice scan, respectively. Ψ inv and Ψ shu They are and The reverse operation and the shuffling operation. It is the i-th standard Mamba layer, with shared parameters θ. i For ease of description, the backbone network consisting of N Mamba layers will be referred to simply as...

[0028] S3 involves using data augmentation methods to construct EEG sample pairs to obtain both the original and augmented perspectives. Specifically, given an EEG signal x... i Its augmented sample can be defined as x′ i This results in EEG sample pairs containing the original signal and its enhanced version. Deeper features are extracted from these EEG sample pairs using N Mamba layers with a shared parameter Θ. This process can be represented as:

[0029]

[0030] Among them, v i and v′ i Through the Mamba backbone network with parameter Θ From x i and x′ i Deep feature pairs extracted from it. Attached Figure Description

[0031] Figure 1 This is a technical flowchart of the present invention.

[0032] Figure 2 This is the network architecture diagram of the present invention.

[0033] Figure 3 This is a feature extraction architecture diagram based on Mamba. Detailed Implementation

[0034] The invention will be further described below with reference to the accompanying drawings. This invention proposes a method for clustering electroencephalogram (EEG) signals based on the Mamba architecture and contrastive learning, comprising the following steps:

[0035] S1 collects and preprocesses unlabeled EEG data. S2 constructs a powerful and general-purpose EEG feature extractor based on Mamba. This feature extractor employs an EEG slice embedding strategy, dividing the input EEG signal into multiple slices to obtain fine-grained local information and utilizing a slice-aware scanning mechanism to model potential contextual relationships. S3 constructs two data views using the original EEG samples and their corresponding augmented samples. Samples from both data views are then input into two Mamba feature extractors with shared network parameters for feature embedding. S4 uses an instance projector to process the feature pairs from the two views, constructing a weighted instance-level contrastive learning model, where distance in the original data space provides weight information. S5 uses a clustering projector to process the feature pairs from the two views, constructing a clustering assignment discriminative contrastive learning branch and a semantic-aware contrastive learning branch. S6 trains using S4 and S5, and outputs the clustering results through the clustering projector in S5.

[0036] S1 includes artifact removal and normalization of EEG data to avoid the effects of noisy data and data scale differences.

[0037] S2 includes building a Mamba-based feature extractor. Mamba is a type of state-space model, and state-space models are mathematical models used to describe and analyze the behavior of continuous systems. State-space models use a hidden state... Map a one-dimensional sequence or function x(t)∈R to a response y(t)∈R.

[0038]

[0039] in, These are evolutionary parameters. and These are projection parameters.

[0040] Mamba is a discrete version of the continuous state-space model described above. It introduces a time-scale parameter Δ, transforming A and B into discrete parameters A and B. This zero-order-preserving transformation can be expressed as:

[0041]

[0042] The parameters can be discretized using the step size Δ, as shown below:

[0043]

[0044] The output of Mamba can be considered as the result of a convolution operation:

[0045]

[0046] in, Let L represent the convolution kernel, and L be the length of the sequence x.

[0047] It is worth noting that Mamba implements a selective scanning mechanism. Specifically, the input sequence... (Where B is the batch size, L is the sequence length, and D is the number of channels) is converted into parameters. and Its inherent contextual information retrieval capability is enhanced by introducing a length dimension L.

[0048] A Mamba feature extraction framework for learning EEG signal features is proposed. Instead of directly using EEG signal recordings as input, this framework employs a simple mechanism called EEG slice embedding to segment the EEG signal into equal-length slices. Furthermore, the feature extraction framework utilizes a slice-aware scanning mechanism to arrange the input sequence of EEG slices.

[0049] The slice-aware scanning mechanism divides the raw EEG signal into equal-length slices and uses a linear feature extractor and positional encoding to obtain embedded features.

[0050] Given an input EEG signal It contains D channels and T time steps. EEG slice embedding uses d one-dimensional convolutions (kernel size k, stride k) to segment the EEG signal into L slices, where L = ceil(T / k), and ceil(·) represents the floor function. Subsequently, to preserve the temporal relationships in the original EEG signal, a positional encoding P is introduced. The entire process can be represented as follows:

[0051]

[0052] in, Ψ represents the input sequence, where P corresponds to the embedding position. conv Ψ represents a one-dimensional convolution operation.reshape Transform the sequence size from d×L to L×d.

[0053] While Mamba-based models perform well on time-series tasks, they do not consider contextual slice dependencies, which is crucial for deploying Mamba in EEG signal clustering. To address this, this invention utilizes a slice-aware scanning mechanism in a Mamba-based feature extractor to model local contextual information between EEG slices. Specifically, three copies of the input EEG slice are made to establish three different slice scan sequences: forward slice scan, reverse slice scan, and shuffled slice scan. Subsequently, Mamba blocks with shared parameters are used to model the dependencies between these three slice scan sequences. Finally, the sequence embedding features are restored to the correct order (forward slice scan) and merged into the output. The slice-aware scanning mechanism can be represented as follows:

[0054]

[0055] in, This represents the output of the i-th (i∈1,2,...,N) Mamba layer or the input of the (i-1)-th Mamba block. yes Sequence embedding features. These represent three slice sequences derived from forward slice scan, reverse slice scan, and shuffled slice scan, respectively. Ψ inv and Ψ shu They are and The reverse operation and the shuffling operation. It is the i-th standard Mamba layer, with shared parameters θ. i For ease of description, the backbone network consisting of N Mamba layers will be referred to simply as...

[0056] S3 involves using data augmentation methods to construct EEG sample pairs to obtain both the original and augmented perspectives. Specifically, given an EEG signal x... i Its augmented sample can be defined as x′ i This results in EEG sample pairs containing the original signal and its enhanced version. Deeper features are extracted from these EEG sample pairs using N Mamba layers with a shared parameter Θ. This process can be represented as:

[0057]

[0058] Among them, v i and v′ i Through the Mamba backbone network with parameter Θ From xi and x′ i Deep feature pairs extracted from it.

[0059] S4 involves using an instance-level contrastive learning strategy to maximize the consistency of positive sample pairs in the embedding space and reduce the similarity of negative sample pairs to learn feature representations. However, in mini-batch training scenarios, similar EEG signals may be assigned as negative sample pairs, leading to the separation of their learned features in the embedding space. To address this, this invention proposes a weighted instance-level contrastive learning strategy that assigns weights to the contrastive loss by calculating the distance between EEG samples in the data space.

[0060] Given a minibatch of size B, perform data augmentation on each EEG sample to obtain a set {x1, ..., x2} containing 2B samples. B ,x′1,...,x B′}, where x′ i x represents i The augmented samples, i∈{1,2,...,B}. Specifically, for x... i It forms 2B-1 sample pairs with other samples, where only (x) i , x′ i ) are positive sample pairs, and the remaining 2B-2 are negative sample pairs. Then, for the negative sample pairs (x) a x b The instance contrast loss defines a distance-based weight w(·, ·):

[0061]

[0062] Where a, b∈[1,2,...,B], d w (·,·) is used to measure the dynamic time-warped distance between EEG sample pairs, τ w This represents the temperature coefficient. It's important to note that only the distance between EEG sample pairs in the raw (unenhanced) data view is calculated. For w(x′) a x b ), w(x a , x′ b ) and w(x′ a , x′ b ), set its value to be the same as w(x) a x b )same.

[0063] A two-layer nonlinear fully connected neural network g is used. I (·) as an instance-level projector to extract feature embeddings z i =g I (v iThis invention uses cosine similarity to measure the distance between feature embeddings and calculates the weighted instance-level contrastive loss as follows:

[0064]

[0065] Where d(·,·) is used to calculate the cosine distance, while τ1 controls the temperature coefficient of weighted instance-level contrastive learning.

[0066] S5 constructs a cluster-level contrastive learning model that combines an assignment-discriminative contrastive branch and a semantic-aware contrastive branch. Specifically, a clustering projector maps input samples to these two branches. The assignment-discriminative branch maintains the consistency of the cluster assignment distribution, while the semantic-aware branch uses k-means to calculate cluster semantic information, thereby forming a group-instance distinction. The goal is for these two branches to synergistically enhance the accuracy and robustness of cluster assignment.

[0067] In the assignment and discrimination branch, the learned features are mapped to C-dimensional probability vectors, where C equals the number of clusters. Specifically, for x... i Through a two-layer nonlinear fully connected layer g a (·) Output the softmax assignment probability p i =g a (v i ),in Similarly, p′ i =g a (v′ i ) represents x′ i The probability of allocation.

[0068] Given and Let {x1, x2, ..., x} represent respectively. B} and {x1, x2, ..., x} B The probability distribution matrix of}, q=[q1…q C ] B×C and q′=[q′1…q′ C ] B×C It is a restatement of p and p′. Then, q... i and q′ i The sample can be represented as {x1, x2, ..., x...} B} and {x′1, x′2, ..., x′ B} represents the probability of being assigned to cluster i. For a good clustering model, it is expected that positive cluster pairs will be assigned similarly, while the assignments of different cluster pairs will show differences. Therefore, q i The allocation level loss can be expressed as:

[0069]

[0070] Where τ2 represents the temperature coefficient of the assignment-discrimination-comparison learning.

[0071] To avoid getting trapped in local optima, assignment-level regularization is introduced:

[0072]

[0073] Where, q ji and q′ ji They are q i and q′ i The j-th element. and The total loss of the assigned discriminant branch can be expressed as:

[0074]

[0075] In the semantic-aware contrastive branch, the instance features and cluster centers of the clustering projector are used to perform cluster group-instance contrastive analysis on the original view and the enhanced view. Specifically, a two-layer nonlinear fully connected neural network g is employed. s (·), in cluster-level contrastive learning, we obtain instance feature representations s = {s1, ..., s2}. B} and s′={s′1,...,s′ B}, where s i =g s (v i ) and s′ i =g s (v′ i Furthermore, k-means calculates K cluster centers for the original view and the enhanced view, respectively, denoted as {μ1, ..., μ...}. K} and {μ′1,...,μ′ K Next, semantic pseudo-labels are assigned to samples of both views based on the nearest cluster centers. Finally, the semantic-aware contrastive loss can be expressed as:

[0076]

[0077] Where τ3 is the temperature coefficient of semantic perception contrast loss, and u(i) and u′(i) represent s i and s′ i The corresponding cluster center index.

[0078] S6 includes the training objective of the clustering method in this invention. Instance-level contrastive and cluster-level contrastive are considered: on the one hand, similarity weights from the data space are introduced in instance-level contrastive learning; on the other hand, assignment-discrimination loss and semantic-aware contrastive loss are considered in cluster-level contrastive learning. The objective function can be expressed as:

[0079]

Claims

1. A method for electroencephalogram (EEG) signal clustering based on Mamba architecture and contrastive learning, the main features comprising the following steps: S1: Collecting and preprocessing unlabeled EEG data; S2 constructing a powerful and general Mamba-based EEG feature extractor; The feature extractor adopts an EEG slice embedding strategy to divide the input EEG signal into multiple slices to obtain fine-grained local information, and uses a slice-aware scanning mechanism to model the potential contextual relationship; S3: Constructing two data views using the original EEG samples and their corresponding augmented samples, and inputting the samples of the two data views into two Mamba networks with shared network parameters for feature embedding; S4: Using an instance projector to process the feature pairs of the two views to construct a weighted instance-level contrastive learning, wherein the distance in the original data space provides weight information; S5: Using a clustering projector to process the feature pairs of the two views to construct a clustering assignment discriminative contrastive learning branch and a semantic-aware contrastive learning branch; S6: Training using S4 and S5, and outputting the clustering result through the clustering projector in S5.

2. The electroencephalogram signal clustering method based on Mamba architecture and contrastive learning according to claim 1, characterized in that, Step S2: Constructing a powerful and general EEG feature extractor based on Mamba: S2 comprises constructing a Mamba-based feature extractor; Mamba belongs to a kind of state space model, and state space model is used to describe and analyze the mathematical model of continuous system behavior;State space model maps a hidden state mapping a one-dimensional sequence or function x(t) e R to a response y(t) e R; wherein is an evolution parameter, and is a projection parameter; Mamba is a discrete version of the above continuous state space model, which introduces a time scale parameter Δ to convert A and B into discrete parameters A and B; this conversion based on zero-order hold can be represented as: Where the parameter can be discretized by the step Δ as follows: The output of Mamba can be regarded as the result of convolution operation: wherein, denotes a convolution kernel, L is the length of the sequence x; Notably, Mamba implements a selective scanning mechanism; specifically, the input sequence is converted to parameters and by introducing the length dimension L to enhance its inherent contextual information retrieval capability; Based on the Mamba architecture, the present application proposes a Mamba feature extraction framework for EEG signal feature learning; instead of directly taking EEG signal records as input, the framework adopts a simple mechanism called EEG slice embedding to divide the EEG signal into equal-length slices; In addition, the feature extraction framework also adopts a slice-aware scanning mechanism to arrange the input sequence of EEG slices; The slice-aware scanning mechanism divides the original EEG signal into equal-length slices and uses a linear feature extractor and position encoding to obtain embedded features; Given an input electroencephalogram signal where D is the number of channels and T is the number of time steps, the EEG slice embedding uses d one-dimensional convolutions (kernel size k, stride k) to split the electroencephalogram signal into L slices, where L = ceil(T / k), ceil(·) denotes the ceiling function; subsequently, to preserve the temporal relationship in the original electroencephalogram signal, a position encoding P is introduced; the entire process can be represented as follows: wherein, denotes an input sequence, P corresponds to a positional embedding; Ψ conv denotes a one-dimensional convolution operation, Ψ reshape transforms the sequence size from d x L to L x d; Although Mamba-based models perform well in time series tasks, they do not consider contextual slice correlation, which is crucial for deploying Mamba in EEG signal clustering; To this end, the present application uses a slice-aware scanning mechanism in the Mamba-based feature extractor to model the contextual local information between EEG slices; Specifically, three copies of the input EEG slice are duplicated to establish three different slice scanning sequences, namely forward slice scanning, reverse slice scanning and shuffled slice scanning; Then, a Mamba block with shared parameters is used to model the dependency of the three slice scanning sequences; Finally, the sequence embedding features are restored to the correct order (forward slice scanning) and combined into the output; The slice-aware scanning mechanism can be represented as follows: wherein, denotes the output of the i-th (i ∈ 1, 2,..., N) Mamba layer or the input of the i-1-th Mamba block; is a sequence embedding feature of denote three kinds of slice sequences from forward slice scanning, reverse slice scanning and shuffle slice scanning, respectively; inv and shu are and inverse operation and shuffle operation of is the i-th standard Mamba layer, sharing parameters θ i ; for the convenience of description, the backbone network containing N Mamba layers is simply referred to as 3. The electroencephalogram signal clustering method based on Mamba architecture and contrastive learning according to claim 1, characterized in that, Step S4: Using an instance projector to process the feature pairs of the two views to construct a weighted instance-level contrastive learning, wherein the distance in the original data space provides weight information: S4 includes learning feature representation by maximizing consistency of positive sample pairs in embedding space and reducing similarity of negative sample pairs using instance-level contrastive learning strategy; However, in the small batch training scene, similar EEG signals may be assigned as negative sample pairs, so that the features learned by them in the embedding space are separated from each other; In this regard, the present application proposes a weighted instance-level contrastive learning strategy, which assigns weights to the contrastive loss by calculating the distance of EEG samples in the data space; Given a mini-batch of size B, data augmentation is performed on each EEG sample to obtain a set of 2B samples {x1,..., x B , x B′} where x i denotes the augmented sample of x i , i ∈ {1, 2,..., B}; in particular, for x i , it forms 2B - 1 sample pairs with other samples, where only (x i , x i ) is a positive sample pair, and the rest of 2B - 2 are negative sample pairs; then, for the instance of negative sample pair (x a , x b ), a distance-based weight w(·, ·) is defined for the contrastive loss: where a, b ∈ [1, 2,..., B], d w (·, ·) for measuring dynamic time warping distance between pairs of EEG samples, τ w denotes the temperature coefficient; note that only distances between pairs of EEG samples under the raw data view (non-augmented) are computed; for w(x a , x b ), w(x a , x b ) and w(x a , x b ), their values are set to be the same as w(x a , x b ); We use a two-layer nonlinear fully connected neural network gI(·) as the instance-level projector to extract the feature embedding z i = g I (v i ); we use cosine similarity to measure the distance between the feature embeddings and compute the weighted instance-level contrastive loss as follows: Where d(·,·) is used to calculate the cosine distance, and τ1 controls the temperature coefficient of weighted instance-level contrastive learning.

4. The electroencephalogram signal clustering method based on Mamba architecture and contrastive learning according to claim 1, characterized in that, Step S5 uses an instance projector to process the feature pairs of the two views to construct weighted instance-level contrastive learning, wherein the distance in the original data space provides weight information: S5 constructs cluster-level contrastive learning, which combines the assignment discriminative contrast branch and the semantic-aware contrast branch; Specifically, the input samples are mapped into the two branches using a cluster projector, wherein the assignment discriminative branch maintains the consistency of the cluster assignment distribution, and the semantic-aware branch calculates the cluster semantic information using k-means, thereby forming group-instance differentiation; It is expected that the two branches can synergistically enhance the accuracy and robustness of the cluster assignment; In the assignment discriminative branch, the features learned by the feature extractor are mapped to a C-dimensional probability vector, where C equals the number of clusters; specifically, for x i The softmax assignment probabilities p i = g a (v i ) are output by a two-layer nonlinear fully connected layer g a (·) where Similarly, p′ i = g a (v′ i ) represents the assignment probabilities for x′ i ; Given and denote the assignment probability matrices of {x1, x2,..., x B} and {x1, x2,..., x B}, respectively, q = [q1...g C ] B×C and q' = [q'1...q'g C ] B×C are re-expressions of p and p'; then, q i and q' i can represent the assignment probabilities of samples {x1, x2,..., x B} and {x'1, x'2,..., x'g B} belonging to cluster i; for an excellent clustering model, the assignments of positive cluster pairs are expected to be similar, while the assignments of different cluster pairs exhibit differences; therefore, the assignment level loss of q i can be expressed as: Where τ2 represents the temperature coefficient of the assignment discriminative contrastive learning; In order to avoid falling into local optimum, assignment-level regularization is introduced: where q ji and q′ ji are the jth elements of q i and q′ i respectively; and The total loss assigned to the discrimination branch can be expressed as: In the semantic-aware contrast branch, the instance features and their cluster centers by the cluster projector are utilized to perform cluster-group instance contrast between the original and augmented views; specifically, a two-layer nonlinear fully connected neural network g s (·) is adopted to obtain the instance feature representations s = {s1,..., s B} and s′ = {s′1,..., s′ B} in the cluster-level contrast learning, where s i = g s (v i ) and s′ i = g s (v′ i ); in addition, k-means is used to compute K cluster centers for the original and augmented views, denoted as {μ1,..., μ K} and {μ′1,..., μ′ K}, respectively; then, the semantic pseudo labels are assigned to the samples of the two views according to the nearest cluster centers; finally, the semantic-aware contrast loss can be represented as: where τ3 is the temperature coefficient of semantic-aware contrastive loss, u(i) and u'(i) represent s i and s' i corresponding cluster center index.

Images