An industrial fault detection method based on inhibiting contrast learning dimension collapse

Abstract

This invention belongs to the field of industrial safety technology and discloses an industrial fault detection method based on suppressing dimensional collapse in contrastive learning. It restores the geometric structure of the feature space through a local-global collaborative optimization mechanism and introduces a dynamic temperature parameter adjustment strategy based on content-style decoupling to balance the model's attention to features of different dimensions, avoiding excessive suppression of the style dimension and effectively improving the accuracy of fault detection. This invention effectively alleviates the dimensional collapse problem and improves feature quality. Through local and global collaborative optimization, the high-dimensional distribution of the feature space is guaranteed from a geometrical perspective, preventing fault features from degenerating into low-dimensional subspaces and significantly improving the expressive power and richness of features. This invention has excellent generalization ability, is not specifically dependent on the physical modes of the input data, and can be widely applied to fault detection tasks of various complex industrial multivariate time-series data, including bearing vibration signals and chemical process parameters.

Description

Technical Field

[0001] This invention relates to the field of industrial safety technology, and in particular to an industrial fault detection method based on suppressing dimensional collapse of contrastive learning. Background Technology

[0002] With the deepening of digital industrial transformation, modern industrial equipment is equipped with massive amounts of sensors, generating large-scale multivariate time series data. To reduce reliance on manual labeling, self-supervised contrastive learning is widely used for feature extraction from industrial data. However, contrastive learning methods are prone to dimensional collapse in the feature space during the optimization of deep neural networks. This means that the extracted feature embedding vectors often degenerate and collapse into a low-dimensional subspace. This phenomenon prevents the feature distribution from fully utilizing the capacity of the high-dimensional geometric space, causing different types of fault features to overlap in the low-dimensional space, thus severely reducing the model's ability to distinguish minor faults. This collapse problem is even more severe in industrial scenarios. The variable operating conditions of industrial equipment (such as speed and load fluctuations) manifest as rich style dimensions (i.e., dimensions that are more prone to change). Existing loss functions, such as the classic InfoNCE loss, suffer from inductive bias, tending to over-suppress these non-content-dimensional style information. This suppression exacerbates the collapse phenomenon, limiting the robustness and accuracy of the model under complex operating conditions. The formula is expressed as: To mitigate the dimensionality collapse problem, existing technologies primarily employ two independent perspectives: Firstly, from a global perspective, existing methods focus on optimizing the overall distribution of the feature space, thereby achieving global distribution optimization. These methods attempt to force features to exhibit an isotropic distribution in the global space through global feature decorrelation or regularization, thus eliminating redundant correlations between dimensions. Secondly, from a local perspective, existing methods mainly focus on optimizing the geometric relationships between samples at a local scale. These methods constrain the distribution structure of samples within their local neighborhoods by introducing geometric indices such as Local Intrinsic Dimension (LID), preventing the degradation of the local feature space. However, existing technologies typically separate local and global optimization, failing to theoretically elucidate and utilize the synergistic effect between the two. Furthermore, regarding the issue of excessive suppression of style dimensions in the aforementioned industrial scenarios, existing methods lack targeted dynamic adjustment mechanisms, failing to effectively retain information on operational changes that have potential value for fault diagnosis. This limitation makes it difficult for existing methods to fundamentally solve the dimensionality collapse problem when facing complex and ever-changing industrial data. Summary of the Invention

[0003] This invention addresses the issues of dimensionality collapse and excessive feature suppression in contrastive learning for industrial fault detection, proposing an industrial fault detection method, LG-CoST, to suppress dimensionality collapse. This invention restores the geometric structure of the feature space through a local-global collaborative optimization mechanism and introduces a dynamic temperature parameter adjustment strategy based on content-style decoupling. This balances the model's focus on features of different dimensions, avoiding excessive suppression of the style dimension and effectively improving the accuracy of fault detection.

[0004] The technical solution of this invention is as follows: An industrial fault detection method based on suppressing dimensional collapse of contrastive learning, the specific steps of which are as follows: Step 1: Data preprocessing; The collected multivariate time series data of industrial equipment is preprocessed. Based on the characteristics of industrial data, time-domain data augmentation operations are performed to inject Gaussian noise, perform random amplitude scaling, and replace time segments to simulate sensor bias and operating condition fluctuations and obtain a time-domain augmented view. Fourier transform is performed on the original multivariate time series data through frequency-domain augmentation operations. The spectral components between different samples are swapped in the frequency domain and then inversely transformed to restore the data, generating a hybrid view with periodic regularity. Finally, subsequences are extracted proportionally from dense and sparse regions, and a time interval masking strategy is used to eliminate the influence of sampling non-uniformity on training. The final output consists of two sets of augmented views with complementary features. Step 2: Construct a temporal convolutional encoder as a feature extraction network; the temporal convolutional encoder integrates an input projection layer, a temporal masking module, and a core convolutional architecture; the two enhanced views obtained in Step 1 are mapped to a high-dimensional space through the input projection layer, and dynamic masking is performed on specific time periods through the aforementioned temporal masking module; multi-scale temporal correlations are captured through the core convolutional architecture inside the temporal convolutional encoder to generate feature representations with translation invariance; the obtained feature representations are used for local view optimization and global view optimization, respectively; Step 3: Local viewpoint optimization; Receive the feature representation output from step 2, and introduce local dimension regularization; optimize the local geometry of the feature space by calculating and increasing the local intrinsic dimension of the feature representation in the feature space; add the increased local intrinsic dimension of the samples as an explicit regularization term to the total loss; Step 4: Global perspective optimization; Step 4.1: Decorrelate global features; The feature representation output from step 2 is received as the global embedding; a global feature decorrelation mechanism is introduced, which constrains the covariance matrix of the global embedding to converge to the identity matrix, eliminates redundant correlations between dimensions, and forces the dimensions to be independent. Step 4.2: Feature decoupling based on variance shift; To distinguish between the content dimension, which represents the essential characteristics of faults, and the style dimension, which represents the characteristics of changes in operating conditions, decoupling is achieved through a metric based on variance offset. The preprocessed original industrial time-series data and the enhanced view generated in step 1 are acquired and fed into the temporal convolutional network encoder described in step 2, thereby outputting the original feature representation and the enhanced feature representation respectively. Subsequently, the variance change of each feature dimension before and after data augmentation is calculated. If the variance change of a certain dimension after data augmentation is greater than its original variance, then that feature dimension is determined to be a style dimension; otherwise, it is considered a content dimension. Step 4.3: Dynamic temperature adjustment based on characteristic amplitude; The design incorporates dynamic temperature parameters, calculates the L2 norm of each dimension in the output feature representation of step 2 as a feature amplitude index, and assigns attention scores based on these indexes, thereby mapping sequence-level global attention to the feature dimension level. The temperature parameters in the contrastive loss function are then dynamically adjusted based on the attention scores. : Assign lower temperature parameters to the suppressed style dimension, while maintaining higher temperature parameters for the content dimension; Step 5: Calculate total loss and train the model; By combining local dimensionality regularization and global contrastive loss, a total loss function is constructed; a temporal convolutional encoder is trained based on the total loss function; after training, the parameters of the temporal convolutional encoder are frozen, and a fault classifier is trained using different labeled data according to different scenario requirements to achieve accurate detection of different faults.

[0005] The beneficial effects of this invention are: This invention effectively alleviates the dimensionality collapse problem and improves feature quality. Through coordinated optimization of local and global features, it ensures a high-dimensional distribution of the feature space from a geometrical perspective, preventing faulty features from degenerating into low-dimensional subspaces and significantly improving the expressive power and richness of features.

[0006] To address the issue of dimensional collapse caused by excessive suppression of data changes in existing technologies, this invention employs a dynamic temperature adjustment mechanism to avoid excessive compression of the style dimension (representing distribution changes under different operating conditions). This allows the model to extract essential fault features while retaining necessary information on operating condition changes, thereby maintaining the high-rank property of the feature space and significantly improving the model's robustness in complex and variable industrial environments.

[0007] This invention has excellent generalization ability, does not have a specific dependence on the physical mode of the input data, and does not require the design of manual features for specific signal forms. Therefore, it can be widely applied to fault detection tasks of various complex industrial multivariate time series data, including bearing vibration signals and chemical process parameters. Attached Figure Description

[0008] Figure 1 This is the overall flowchart of the LG-CoST invention.

[0009] Figure 2 This is a global view optimization structure diagram of the LG-CoST of the present invention.

[0010] Figure 3 This is a diagram showing the local viewpoint optimization structure of the LG-CoST of the present invention.

[0011] Figure 4 This is a flowchart for using pre-trained code to perform a fault detection task. Detailed Implementation

[0012] An industrial fault detection method based on suppressing dimensional collapse of contrastive learning, comprising the following steps: Step 1: Data preprocessing; The collected multivariate time series data of industrial equipment is preprocessed. Based on the characteristics of industrial data, Gaussian noise injection, random amplitude scaling, and time segment permutation are performed using time-domain data augmentation operations to simulate sensor bias and operating condition fluctuations and obtain a time-domain augmented view. Fourier transform is performed on the original multivariate time series data through frequency-domain augmentation operations, and the spectral components between different samples are exchanged in the frequency domain and inversely transformed to restore the data, generating a hybrid view with periodic regularity. Finally, subsequences are extracted proportionally from dense and sparse regions, and a time interval masking strategy is used to eliminate the influence of sampling non-uniformity on training, ultimately outputting two sets of augmented views with complementary features. The time-domain transformation unit is generated using methods such as time-domain jittering, scaling, or masking.

[0013] Step 2: Construct a temporal convolutional encoder as a feature extraction network; The temporal convolutional encoder integrates an input projection layer, a temporal masking module, and a core convolutional architecture; the resulting feature representations are used for local viewpoint optimization and global viewpoint optimization, respectively.

[0014] Step 3: Local viewpoint optimization; At the local perspective optimization level, local dimension regularization is introduced, aiming to optimize the local geometry of the feature space by increasing the local intrinsic dimension (LID) of the samples. The LID enhancement of the samples is added as an explicit regularization term to the total loss.

[0015] Step 4: Global perspective optimization; Step 4.1: Decorrelate global features; At the global perspective optimization level, a global feature decorrelation mechanism is first introduced. By constraining the global embedding covariance matrix to converge to the identity matrix, redundant correlations between dimensions are eliminated, and the independence between dimensions is forced to avoid the dimension collapse problem.

[0016] Step 4.2: Feature decoupling based on variance shift; To distinguish between the essential characteristics of faults (content dimension) and the characteristics of changes in operating conditions (style dimension), the impact of data augmentation on feature dimensions is decoupled. The change in variance of a feature dimension before and after data augmentation is calculated. If the variance of a feature dimension shifts significantly after data augmentation (i.e., the change brought about by data augmentation is greater than the original variance), then the feature dimension is determined to be a style dimension that is significantly affected by data augmentation; otherwise, it is considered a robust content dimension.

[0017] Step 4.3: Dynamic temperature adjustment based on characteristic amplitude; To address the issue of dimensional collapse caused by excessive suppression of the style dimension during model optimization, a dynamic temperature parameter is designed, utilizing the feature dimension... Norm-based attention scores are assigned, mapping global attention to the feature dimension level. The temperature parameter in the contrastive loss function is then dynamically adjusted based on these attention scores. The suppressed style dimension is assigned a lower temperature parameter, while the content dimension is maintained at a higher temperature parameter. The lower temperature increases the difficulty for the model to distinguish negative samples, thus generating a larger gradient. This mechanism forces the model to refocus on these style dimensions during the optimization process, preventing them from being over-compressed.

[0018] Step 5: Calculate total loss and train the model; A total loss function is constructed by combining local dimensionality regularization and global contrastive loss. The encoder is then trained using this total loss function. After training, the encoder parameters are frozen, and fault classifiers are trained using different labeled data according to different scenario requirements to achieve accurate detection of various faults.

[0019] The specific embodiments of the present invention are further described below.

[0020] Step 1: Data processing; For each input sample Two enhanced views are generated by employing temporal jitter and frequency domain mixing as enhancement methods. and .

[0021] Step 2: Construct a temporal convolutional encoder as a feature extraction network to optimize the feature extraction network through local-global collaborative optimization; The feature extraction network is used to extract high-dimensional feature representations from the augmented view. The specific network structure is as follows: 1. Input Projection Layer: A fully connected layer is set up to map the input augmented view to hidden layer feature vectors (typically 64 dimensions), which serve as the input to the subsequent backbone network. The final backbone network output dimension is set to 320.

[0022] 2. Temporal Masking Module: During the training phase, a random binary mask is directly applied to the original input sequence. The mask positions follow a Bernoulli distribution (default masking rate 50%), and the masked time steps are set to zero before input projection. This masked sequence is then passed through a temporal convolutional network encoder, which is required to reconstruct these masked original features using contextual information.

[0023] 3. Convolutional Architecture: Composed of 11 stacked residual blocks (including 10 hidden layers and 1 output layer). Each residual block contains two 1D dilated convolutional layers. The expansion rate of the layer residual block is set to This exponential growth rate allows the network to possess a receptive field covering the entire time series while maintaining a controllable number of parameters, thus effectively capturing long-range temporal dependencies. The network employs a pre-activation structure, where the GELU activation function is placed before each convolutional layer.

[0024] 4. Feature Output: The last convolutional block of the TCN encoder directly maps the hidden layer features to the target dimension. This output is directly used as the final feature representation. Used for subsequent local and global optimization and collaborative optimization.

[0025] Step 3: Local viewpoint optimization; The feature representations output from step 2 are received, and the local intrinsic dimension of the samples is increased as an explicit regularization term added to the total loss, aiming to enrich the local geometry of the feature manifold. Specifically, the method of moments is used to estimate each feature representation (i.e., the sample embedding vector) output from step 2. The LID value. For a batch of size N, calculate... Find the distances to its k nearest neighbors and estimate its local intrinsic dimension. To prevent local representations from collapsing into a low-dimensional manifold, the goal is to maximize the geometric mean of these local dimensions. Therefore, the local regularization loss is defined as: Step 4: Global perspective optimization; Step 4.1: Decorrelate global features; The feature representation output from step 2 is received as a global embedding. First, the received... The dimensional feature vector is divided into 3D feature vectors of size 1. The input batch is divided into several groups, and ZCA whitening is applied independently within each group. For the decentralized input batch... The output calculation formula for ZCA whitening is: in, It is the covariance matrix The eigenvalue diagonal matrix, These are the corresponding orthogonal eigenvector matrices, thus ensuring that the output covariance matrix is ​​the identity matrix. To further eliminate the limitations of fixed grouping and enhance the global decorrelation effect, a random permutation mechanism is introduced. That is, before executing DBN, the order of feature dimensions is shuffled by random permutation, and restored by inverse permutation after whitening, thereby achieving more thorough feature decorrelation.

[0026] Step 4.2: Feature decoupling and dynamic temperature adjustment; 1. Calculate variance shift: Statistically calculate the variance of features within a batch before and after data augmentation. For the _____, ... 3D features, calculate the variance of the original features and enhanced feature variance Calculate the offset: 2. Dimensional Partitioning: A relative threshold strategy is used for dimensional partitioning: if the variance drift caused by data augmentation on a certain feature dimension exceeds its inherent variance, then the following condition is met: If the feature dimension is positively affected, it is determined to be a style dimension that is significantly affected by the enhancement; otherwise, it is considered a robust content dimension.

[0027] 3. Dynamic Temperature Adjustment: After decoupling and globally decorrelating the content and style dimensions, features are input into the Transformer module to obtain attention scores for different dimensions. Feature calculation... Norm and normalization yield attention score Calculate the temperature parameter using the following formula: in, These are the preset upper and lower temperature limits. The temperature range for the style dimension is set to the lower range (e.g., ...). The temperature range for the content dimension is in the higher range (e.g.) This mechanism generates lower temperature parameters for the suppressed style dimension, thereby forcing the model to focus on the style dimension and avoiding excessive suppression of the style dimension.

[0028] Step 5: Loss Calculation; The total loss of the LG-CoST framework consists of the global contrastive loss and the local dimension regularization term: in, It is a hyperparameter that balances the two types of loss (default 0.01). Using a decoupled InfoNCE approach, dynamic temperature parameters are applied to both the content and style dimensions: Step 6: Fault detection task; The encoder network is trained using the aforementioned total loss function until convergence. After training, the encoder parameters are fixed, and a linear classifier is trained. In actual fault detection tasks, data from different faulty devices are input into the network, features are extracted, and the classifier outputs fault diagnosis results (e.g., normal, rotor imbalance, bearing inner ring wear, bearing outer ring crack), thereby guiding on-site personnel in handling and maintenance.

[0029] The model in this invention is implemented using the PyTorch framework, and the objective function consists of a global contrastive loss and a LID regularization term. During training, to balance feature consistency and diversity, the weight coefficient of the contrastive loss is set to 0.05, and the coefficient of the LID regularization term is set to 0.01. For the optimizer, the Adam optimizer is used, with a learning rate of 0.001 and a weight decay rate of 0.0005. The default training cycle is 30 epochs, and the batch size is set to 32 to ensure that the model can fully learn the features in the data.

[0030] The device system on which the method of this invention is running is Ubuntu 16.04, the CPU model is Intel Xeon CPU E5-2650v4 @ 2.20GHz, the GPU is NVIDIA GeForce 4090, and the video memory is 24G.

Claims

1. An industrial fault detection method based on suppressing dimensional collapse of contrastive learning, characterized in that, The specific steps are as follows: Step 1: Data preprocessing; The collected multivariate time series data of industrial equipment is preprocessed; based on the characteristics of industrial data, time-domain data augmentation operations are performed to inject Gaussian noise, perform random amplitude scaling and time segment permutation to simulate sensor bias and operating condition fluctuations and obtain a time-domain augmented view. The original multivariate time series data is subjected to Fourier transform through frequency domain enhancement operations. The spectral components between different samples are swapped in the frequency domain and then inversely transformed to restore the data, generating a hybrid view with periodic regularity. Finally, subsequences are extracted proportionally from dense and sparse regions, and a time interval masking strategy is used to eliminate the impact of sampling non-uniformity on training, ultimately outputting two sets of enhanced views with complementary features. Step 2: Construct a temporal convolutional encoder as a feature extraction network; the temporal convolutional encoder integrates an input projection layer, a temporal masking module, and a core convolutional architecture; the two enhanced views obtained in Step 1 are mapped to a high-dimensional space through the input projection layer, and dynamic masking is performed on specific time periods through the aforementioned temporal masking module; multi-scale temporal correlations are captured through the core convolutional architecture inside the temporal convolutional encoder to generate feature representations with translation invariance; the obtained feature representations are used for local view optimization and global view optimization, respectively; Step 3: Local viewpoint optimization; Receive the feature representation output from step 2, and introduce local dimension regularization; optimize the local geometry of the feature space by calculating and increasing the local intrinsic dimension of the feature representation in the feature space; add the increased local intrinsic dimension of the samples as an explicit regularization term to the total loss; Step 4: Global perspective optimization; Step 4.1: Decorrelate global features; The feature representation output from step 2 is used as a global embedding. Introduction The global feature decorrelation mechanism eliminates redundant correlations between dimensions and forces dimensions to be independent by constraining the covariance matrix of the global embedding to converge to the identity matrix. Step 4.2: Feature decoupling based on variance shift; To distinguish between the content dimension representing the essential characteristics of faults and the style dimension representing the characteristics of changes in operating conditions, a metric based on variance offset is used for decoupling. The preprocessed original industrial time-series data and the enhanced view generated in step 1 are obtained and input into the temporal convolutional network encoder described in step 2, thereby outputting the original feature representation and the enhanced feature representation respectively. Subsequently, the variance change of each feature dimension before and after data augmentation is calculated; if the variance change of a certain dimension after data augmentation is greater than its original variance, then the feature dimension is determined to be a style dimension, otherwise it is considered a content dimension. Step 4.3: Dynamic temperature adjustment based on characteristic amplitude; The design incorporates dynamic temperature parameters, calculates the L2 norm of each dimension in the output feature representation of step 2 as a feature amplitude index, and assigns attention scores based on these indexes, thereby mapping sequence-level global attention to the feature dimension level. The temperature parameters in the contrastive loss function are then dynamically adjusted based on the attention scores. : Assign lower temperature parameters to the suppressed style dimension, while maintaining higher temperature parameters for the content dimension; Step 5: Calculate total loss and train the model; By combining local dimensionality regularization and global contrastive loss, a total loss function is constructed; a temporal convolutional encoder is trained based on the total loss function; after training, the parameters of the temporal convolutional encoder are frozen, and a fault classifier is trained using different labeled data according to different scenario requirements to achieve accurate detection of different faults.

Images