Deep sea manned submersible unsupervised anomaly detection method and system based on multi-modal feature fusion

Abstract

The invention relates to a deep sea manned submersible unsupervised anomaly detection method and system based on multi-modal feature fusion, and the method comprises the steps: carrying out the initial feature extraction of preprocessed multi-modal signal data, and obtaining a local time sequence feature; performing multi-scale frequency domain enhancement on the local time sequence characteristics through a frequency domain enhancement module to obtain frequency domain enhancement characteristics; performing long-term dependency relationship modeling on the frequency domain enhancement features to obtain long-distance dependency features; performing context enhancement on the frequency domain enhancement feature by using a fusion type attention coding module to obtain a coding feature; performing multi-modal feature fusion on the frequency domain enhancement feature, the long-distance dependence feature and the coding feature to obtain a fusion feature; performing state modeling on the fusion features to obtain global hidden state features, and performing reconstruction restoration on the global hidden state features to obtain reconstruction output; and carrying out parameter optimization by adopting a joint loss function, and carrying out continuous iterative training to obtain a trained anomaly detection model.

Description

TECHNICAL FIELD

[0001] The present application relates to a deep-sea manned submersible unsupervised anomaly detection method and system based on multi-modal feature fusion, belonging to the field of artificial intelligence. BACKGROUND

[0002] In high-risk and strong task demand tasks such as scientific experiments, submarine geological structure research, deep-sea shipwreck archaeology and submarine equipment maintenance, as the complexity of deep-sea exploration tasks continues to increase, the submersible system is becoming increasingly complex in structure, control, power, sensing, communication and other aspects, and the requirements for system safety, reliability and intelligence level are also increasing. In order to ensure the stable operation of the submersible in the extreme environment of deep-sea high pressure, strong corrosion and limited communication, modern deep-sea manned submersibles are usually equipped with a large number of sensor modules to monitor the equipment state, environmental parameters, operation behavior and other key data in real time. These sensors include mechanical structure stress sensors, water-tight cabin pressure sensors, propulsion system current and voltage monitors, inertial measurement units (IMU), depth sensors, water temperature and salinity sensors, underwater acoustic communication state monitors and other types. In a long time of operation, these sensors will generate a large amount of high-dimensional, time-series multi-modal data, providing a wealth of information sources for state assessment, fault diagnosis and safety warning. However, due to the non-repeatability of the deep-sea environment and the irreversibility of the task, once the submersible fails in the task, it may cause expensive economic losses or even endanger the safety of personnel. Therefore, the fault detection and health management (FDIHM, Fault Detection, Isolation and Health Management) technology has become an important part of the design of the submersible.

[0003] Currently, traditional submersible fault detection mainly relies on rule-based or expert system methods, combining preset thresholds, rule libraries, or supervised learning models based on limited historical samples for state judgment. These methods perform well in some typical fault patterns, but have the following outstanding problems: Lack of deep-sea actual fault samples: Due to the high cost and high risk of deep-sea operations, actual fault samples are extremely scarce, making it difficult to collect labeled data covering multiple abnormal patterns, which limits the application of traditional supervised machine learning algorithms in fault identification; Strong dependence on expert knowledge and experience rules: The traditional model construction process often needs to rely heavily on experienced engineers to manually set fault rules or thresholds, which has strong subjectivity and limitations, and is not easy to generalize or adapt to changes in task scenarios; Data modalities are complex and susceptible to interference: Deep-sea sensor data often has high dimensionality, multi-modality, high noise, and non-stationary distribution characteristics, making it difficult for traditional feature extraction methods to extract features with stable discrimination ability, affecting detection accuracy; Real-time and deployment problems: During task execution, the anomaly detection system needs to have strong real-time response capability and edge deployment capability to facilitate independent operation under the limited computing resource conditions of the submersible, and traditional methods often fail to meet these requirements.

[0004] In recent years, with the development of deep learning and artificial intelligence technology, unsupervised anomaly detection methods have attracted widespread attention due to their ability to adapt to complex data structures without the need for large amounts of labeled data. In particular, in safety-critical fields such as industrial Internet of Things, aerospace, and medical diagnosis, unsupervised methods have gradually shown superior performance in small sample, weak label, or zero label environments. In deep-sea manned submersible applications, unsupervised anomaly detection methods have particular application potential; the basic idea is to learn the distribution pattern of sensor data of the submersible system in the normal operating state, establish a "baseline model" of normal behavior, and identify abnormal data points that deviate from the normal distribution during operation to achieve early warning and fault identification. Typical unsupervised anomaly detection techniques include but are not limited to: Autoencoder: by learning the low-dimensional embedding and reconstruction error of data, samples with poor reconstruction ability are considered potential anomalies; Variational Autoencoder (VAE) and Generative Adversarial Network (GAN): use probabilistic generative models to fit the normal data distribution and identify low-probability samples as anomalies; Distance or density-based methods: such as LOF (Local Outlier Factor), KNN, Isolation Forest, etc.; Time series modeling methods: such as LSTM and Transformer-based prediction models for anomaly identification based on prediction error of time series; Graph Neural Network (GNN) methods: used to model the topology and collaborative relationship between multiple sensors; Multimodal fusion methods: by fusing different types of sensor data, improve the generalization and robustness of the model.

[0005] Although the above unsupervised detection methods have achieved good results in some industrial data sets, there are still several challenges in deep-sea manned submersible applications. Multimodal data fusion difficulty: deep-sea submersible sensors are of various types, and data is heterogeneous and has time delay differences, how to construct an effective multimodal feature fusion mechanism is one of the key problems; Complex normal operating state distribution: submersible tasks are diverse and are greatly affected by the marine environment, resulting in significant non-stationarity in the normal data distribution, which leads to insufficient generalization ability of the baseline model; Deployment feasibility problem: limited by the computing resources, bandwidth, and energy consumption of the submersible, traditional large models cannot be directly deployed in the actual system, and lightweight, deployable unsupervised detection algorithms need to be designed;

[0006] In summary, deep-sea manned submersible unsupervised anomaly detection still faces multiple challenges such as data scarcity, complex tasks, and limited computing resources, and there is an urgent need for an unsupervised anomaly detection technology solution that combines multimodal adaptability, high robustness, and deployment feasibility to achieve intelligent monitoring, early warning, and task safety assurance of the deep-sea manned submersible operating state. SUMMARY

[0007] To overcome the deficiencies of the prior art, the present application provides a deep-sea manned submersible unsupervised anomaly detection method and system based on multi-modal feature fusion, aiming to solve the technical problems existing in the prior art, such as: lack of abnormal samples, complex data modalities, difficulty in extracting deep temporal patterns, and difficulty in deploying models. The method is based on a deep neural network to build a fusion multi-module anomaly detection model, which comprehensively utilizes reconstruction error, attention mechanism, modal redundancy and semantic consistency for joint modeling, effectively improving the perception and discrimination ability of potential abnormal states.

[0008] The technical solutions of the present application are as follows: The first aspect of the present application provides a deep-sea manned submersible unsupervised anomaly detection method based on multi-modal feature fusion, comprising: Collecting multi-modal signal data of the deep-sea submersible and preprocessing; constructing an anomaly detection model, inputting the preprocessed multi-modal signal data into the anomaly detection model for training, obtaining a trained anomaly detection model, and inputting the multi-modal signal data of the deep-sea submersible into the trained anomaly detection model for anomaly detection of the deep-sea submersible; Constructing an anomaly detection model and inputting preprocessed multi-modal signal data into the anomaly detection model for training, comprising: Step 1: Perform initial feature extraction on the preprocessed multi-modal signal data to obtain local temporal features; Step 2: Perform multi-scale frequency domain enhancement on the local temporal features through a frequency domain enhancement module to obtain frequency domain enhanced features; The frequency domain enhancement module (FEM) includes multi-branch convolution operation and fusion operation, which extracts implicit frequency domain disturbance features through multi-branch convolution operation, and fuses the frequency domain disturbance features through fusion operation; Step 3: Model the long-term dependency relationship of the frequency domain enhanced features to obtain long-distance dependency features; Step 4: Use a fusion attention encoding module to enhance the context of the frequency domain enhanced features to obtain encoded features; The fusion attention encoding module models the correlation between features through a series of attention and prior attention, enhancing the ability to capture local dependencies; Step 5: Perform multi-modal feature fusion on the frequency domain enhanced features, long-distance dependency features and encoded features to obtain fusion features; Step 6: Model the state of the fusion features to obtain global hidden state features, and reconstruct the global hidden state features to obtain reconstruction output; Step 7: Use a joint loss function to optimize the parameters, and through continuous iterative training, obtain a trained anomaly detection model.

[0009] According to the application, the multimodal signal data of the deep-sea submersible is collected and preprocessed, including: The multimodal sensor signal data of the deep-sea submersible in multiple working environments is collected, including time domain signals and frequency domain signals, the time domain signals including depth, temperature, attitude signals, acceleration, angular velocity, and the frequency domain signals including acoustic positioning signals, pressure pulsation signals, noise signals, and angular vibration signals. The multimodal sensor signal data is subjected to data cleaning, including missing value processing and outlier processing, to generate input signals Wherein B is the batch size, T is the time window length, and C represents the number of channels. Further preferably, the preprocessed multimodal signal data is subjected to initial feature extraction to obtain local time sequence features, including: The input signals are inputted for initial feature extraction to obtain local time sequence features, as follows: ; Wherein, The local time sequence features are represented by ReLU, the activation function, and Conv1D represents the one-dimensional convolution layer operation, which is defined as: ; Wherein, is the convolution kernel parameter, K is the convolution kernel size, C represents the number of channels, is the input signal, b represents the batch index, t represents the current time step index, i.e., the tth time position in the output, represents the index of the input channel, k represents the index of the convolution kernel in the time dimension, i.e., the kth point in the time window, is the bias, and the output The convolution layer captures the local dependence features of adjacent time points, and improves the parameter efficiency through weight sharing.

[0010] According to the application, the local time sequence characteristics are subjected to multi-scale frequency domain enhancement through a frequency domain enhancement module to obtain frequency domain enhanced features, including: The frequency domain enhancement module aims to mine the frequency domain disturbance features and multi-scale periodic structure information hidden in the original sensor signals, solve the problem of insufficient sensitivity of traditional time sequence modeling to frequency dependence, and improve the recognition ability of the abnormal detection system to hidden periodic anomalies; the module can capture frequency-related abnormal patterns that are difficult to model by traditional time sequence methods, adaptively capture frequency domain abnormal features by learning convolution kernel weights, and improve detection sensitivity; The multi-branch convolution operation includes two branches, the obtained local time sequence features are inputted into the two branches respectively, convolution, normalization and activation function processing are sequentially performed, and frequency domain disturbance features are obtained, as follows: ; ; wherein, and denote two frequency domain disturbance features obtained, denote local timing features, i.e. , B is batch size, C is channel number, T is time series length, denote batch normalization, denote one-dimensional convolution with a convolution kernel K, denote a dilated convolution, K is a convolution kernel size (K=3), d is a convolution kernel sampling interval (d=2); The two frequency domain disturbance features obtained are fused using a fusion operation to obtain a frequency domain enhanced feature: ; ; wherein, denote a frequency domain enhanced feature, denote a spliced feature, denote a splicing operation, denote one-dimensional convolution with a convolution kernel K1, denote a convolution kernel size (K1=1), denote an output channel number.

[0011] According to the present application, the frequency domain enhanced feature is preferably modeled for long-term dependency to obtain a long-distance dependency feature; comprising: A Mamba module is introduced to strengthen the sequence internal feature and model the long-term dependency; Mamba is a new structure with higher sequence modeling efficiency than Transformer, which can effectively make up for the problem of performance decline of traditional Transformer in modeling long sequence; the MAMBA module can effectively fuse the timing correlation and cross-modal semantic dependency between different modalities, and enhance the robustness and generalization ability of the model; The Mamba module includes a splicing fusion operation, a convolution operation, a state space operation and a dynamic gating operation; The frequency domain enhanced feature is divided into M feature sequences, and a splicing fusion operation is performed, as follows: ; wherein, denote a spliced feature, denote a feedforward network of the Mth feature sequence, denote the Mth feature sequence, M denotes the number of feature sequences of the frequency domain enhanced feature; Calculate the mapping input: ; wherein, represents the mapping input, represents the weight matrix, represents the dimension of the feature vector at each time step, ⊙ represents element multiplication, is the Sigmoid activation function, is the activation function, and represents the bias vector; Use convolution operation to map the input Further extract local features and perform nonlinear mapping through the anomaly-aware GELU activation mechanism: = ; At the same time, an abnormal enhancement activation mechanism is adopted for the mapping input as follows: = ; in, represents the mapping feature, Represents enhanced features, Conv represents convolution operation, represents the activation function; Use state-space operations (optionally SSM) to process long sequences of mapped features: ; in, represents the hidden state at time t, represents the hidden state at time t-1, Represents the mapping feature at time t, A, It uses trainable parameters and sliding window convolution optimization to accelerate calculations and support long sequence modeling. Extract high-response channels through dynamic gating operations and obtain long-range dependency features: ; in, represents the activation function, represents the weight vector, represents the bias vector, represents the hidden state sequence, Indicates long-distance dependency features.

[0012] Preferably, according to the present invention, a fusion attention encoding module is used to perform context enhancement on the frequency domain enhancement feature to obtain the encoding feature; comprising: The fusion attention encoding module includes encoding operations and anomaly attention operations; Anomalous attention manipulations include serial attention and prior attention; A fused attention encoding module is used to contextually enhance long-distance dependency features. The fused attention encoding module (Anomaly Transformer Encoder) adopts an improved Transformer structure with an embedded AnomalyAttention mechanism. It models the correlation between normal and potentially abnormal behaviors based on local attention in a time window. The output of the fused attention encoding module includes encoding results, attention series information, prior attention distribution and variance information, providing a basis for subsequent anomaly judgment. The frequency domain enhancement feature is encoded as follows: ; in, represents the input encoding features, Represent word embeddings; Calculate the query vector, value vector, and key vector of the encoding result: , ; in, represents the query vector, represents the key vector, V represents the value vector, 、 、 represents the weight matrix; Compute the series attention features: ; in, represents the serial attention feature, represents the dimension of the key vector G; Calculate the prior attention features: ; in, represents the prior attention matrix, i.e., the prior attention feature, is the variance, i and j represent the positions of two time steps respectively. The prior matrix introduces a priori knowledge based on position relationship to enhance the model's ability to capture local dependencies; The calculated prior attention matrix and serial attention features Fusion generates the final attention weight matrix , as shown below: ; The encoding result is calculated as follows: ; in, is the coding feature.

[0013] According to the application, the frequency domain enhanced features, long distance dependent features and coding features are fused by multi-modal feature fusion to obtain fused features, which include: The frequency domain enhanced features, long distance dependent features and coding features are fused by a multi-modal feature fusion module (WeightedFeatureFusion), and the fusion weight parameters are obtained by training; compared with the traditional simple splicing method, the module can realize effective fusion of more consistent semantics and more complementary features, which helps to improve the stability and robustness of the model. The frequency domain enhanced features are set as , the long distance dependent features are set as , and the coding features are set as ; The global weight of each feature is calculated as ; wherein, represents the global weight of the i-th feature; t represents the time step; represents the i-th feature in the three types of features, and T represents the length of the time window; The linear mapping weight is calculated as ; wherein, represents the linear mapping result of the i-th feature, represents the weight matrix, represents the bias term; The normalization processing is performed as ; wherein, represents the normalized weight of the i-th feature, represents the linear mapping result of the j-th feature; Finally, the fused features are obtained as ; wherein, represents the fused features.

[0014] According to the application, the fused features are state modeled to obtain global hidden state features, and the global hidden state features are reconstructed to obtain reconstruction output, which includes: The fused features are feature extracted as ; wherein, represents the hidden state of the current time step t, represents the hidden state of the time step t−1, represents a bidirectional GRU network (Gated Recurrent Unit) operation; compressing the time dimension and obtaining a final state representation to obtain a global hidden state feature: ; wherein, represents a global hidden state feature, is a hidden layer dimension, represents a hidden state at the last time step; ; reconstructing and restoring the global hidden state feature, and the reconstruction and restoration calculation process is: ; wherein, represents a reconstruction output, represents a decoding operation; calculating an error between the input signal X and the reconstruction output, as follows: ; wherein, represents a reconstruction error, represents an input signal, represents a reconstruction output, B, T, and C are batch size, time window length, and channel number, respectively, are the current batch size, the current time window length, and the current channel number, respectively; If there is a significant deviation between the original input, i.e., the input signal, and the reconstruction output, i.e., the reconstruction error is greater than a set threshold (ε), , the input signal is regarded as an abnormal signal, and the output detection result is abnormal, otherwise it is a normal signal.

[0015] Further preferably, a joint loss function is used for parameter optimization, and after continuous iterative training, a trained anomaly detection model is obtained; including: constructing a reconstruction loss : ; wherein, represents an input signal, represents a reconstruction output; constructing a normalized prior distribution : ; wherein, is a normalized prior distribution, represents the total length of the sequence, represents an original prior distribution value at time step v, i.e., an initial probability distribution, represents an original prior distribution value at time step u; Constructing series consistency loss (series loss): ; in, represents the series consistency loss, is the normalized prior distribution, represents the Kullback-Leibler divergence, represents the model prediction distribution at time step u (the model's probabilistic prediction of the sequence features at time step u based on the current input signal and previously learned knowledge); Constructing prior stability loss: ; in, represents the prior stability loss, Express Smoothing and contrast enhancement, represents the original prior distribution value at time step u; Define two relative objective loss functions, including minimization objective and maximization objective: the minimization objective (used to update the generation network) is: ; The maximization objective (for stable distribution training) is: ; in, To adjust the parameters; During training, by minimizing the target loss and maximize the target loss Optimize and continuously iterate the training to obtain a trained anomaly detection model.

[0016] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the processor implements the steps of an unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion.

[0017] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of an unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion.

[0018] A second aspect of the present invention provides an unsupervised anomaly detection system for deep-sea manned submersibles based on multimodal feature fusion, comprising: The model construction module is configured to: collect multi-modal signal data of the deep-sea submersible and pre-process the multi-modal signal data; construct an anomaly detection model, input the pre-processed multi-modal signal data into the anomaly detection model for training, obtain the trained anomaly detection model, and input the multi-modal signal data of the deep-sea submersible into the trained anomaly detection model for anomaly detection of the deep-sea submersible; The initial feature extraction module is configured to: perform initial feature extraction on the pre-processed multi-modal signal data to obtain local time sequence features; The frequency domain enhancement module is configured to perform multi-scale frequency domain enhancement on the local time sequence features through the frequency domain enhancement module to obtain frequency domain enhanced features; The frequency domain enhancement module includes a multi-branch convolution operation and a fusion operation, the multi-branch convolution operation extracts implicit frequency domain disturbance features, and the fusion operation fuses the frequency domain disturbance features; The long-term dependency modeling module is configured to: model long-term dependency relationships of the frequency domain enhanced features to obtain long-distance dependency features; The context enhancement module is configured to: use a fusion attention encoding module to perform context enhancement on the frequency domain enhanced features to obtain encoded features; The fusion attention encoding module models the relevance between features through a series of attentions and prior attentions to enhance the ability to capture local dependencies; The feature fusion module is configured to: perform multi-modal feature fusion on the frequency domain enhanced features, the long-distance dependency features and the encoded features to obtain fused features; The reconstruction module is configured to: model the state of the fused features to obtain global hidden state features, and reconstruct and restore the global hidden state features to obtain reconstruction output; The optimization module is configured to: perform parameter optimization using a joint loss function, and obtain the trained anomaly detection model through continuous iterative training.

[0019] The beneficial effects of the present application are: 1. Multi-modal structure fusion enhances detection robustness: The present application fuses various feature extraction mechanisms in time domain, frequency domain, structural domain and attention domain, so that the model has stronger stability and adaptability when facing high noise and multi-source heterogeneous data in the marine environment.

[0020] 2. Introducing Mamba module structure optimizes long sequence modeling capability: Compared with the traditional Transformer, Mamba has better sequence modeling capability and computing efficiency, and is suitable for processing long-term historical data analysis in submersible tasks.

[0021] 3. The reconstruction mechanism realizes zero-sample anomaly detection: by constructing a reconstruction model of the normal state, it does not rely on any labeled abnormal samples, overcomes the problem of scarcity of abnormal samples in deep sea tasks, and realizes unsupervised anomaly detection.

[0022] 4. Feature fusion learnable weight mechanism: the designed weighted fusion module can adaptively adjust the importance of different source features, avoiding the subjective bias caused by manual setting of fusion strategy.

[0023] 5. End-to-end modeling and deployment friendliness: the overall model supports end-to-end training and inference, and can be trimmed into a lightweight version for deployment on edge computing platforms, meeting the real-time and resource-constrained deployment requirements of deep sea tasks.

[0024] 6. Strong interpretability, easy to trace anomalies: the attention map and prior distribution information output by the encoder can be used to analyze the key areas that the model focuses on, assisting engineers in anomaly tracing and analysis. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 The flowchart of the deep sea manned submersible unsupervised anomaly detection method based on multi-modal feature fusion of the present application; Figure 2 The data processing flowchart based on projection and selective SSM of the present application; Figure 3 The data processing flowchart based on attention mechanism of the present application. DETAILED DESCRIPTION

[0026] The present application will be further described below by way of examples and in conjunction with the accompanying drawings, but is not limited thereto.

[0027] Example 1 The deep sea manned submersible unsupervised anomaly detection method based on multi-modal feature fusion, as shown in Figure 1 , comprises: Collecting multi-modal signal data of the deep sea submersible and performing preprocessing; constructing an anomaly detection model, inputting the preprocessed multi-modal signal data into the anomaly detection model for training, obtaining a trained anomaly detection model, and inputting the multi-modal signal data of the deep sea submersible into the trained anomaly detection model for anomaly detection of the deep sea submersible; Constructing an anomaly detection model and inputting preprocessed multi-modal signal data into the anomaly detection model for training comprises: Step 1: performing initial feature extraction on the preprocessed multi-modal signal data to obtain local time series features; Step 2: performing multi-scale frequency domain enhancement on the local time series features through a frequency domain enhancement module to obtain frequency domain enhanced features; The frequency domain enhancement module (FEM) includes a multi-branch convolution operation and a fusion operation, the multi-branch convolution operation extracts implicit frequency domain disturbance features, and the fusion operation fuses the frequency domain disturbance features; Step 3: Long-term dependency modeling is performed on the frequency domain enhancement features to obtain long-distance dependency features; Step 4: A fusion attention encoding module is used to perform context enhancement on the frequency domain enhancement features to obtain encoding features; The fusion attention encoding module models the correlation between features through a series of attentions and prior attentions to enhance the ability to capture local dependencies; Step 5: The frequency domain enhancement features, long-distance dependency features, and encoding features are fused to obtain fusion features; Step 6: State modeling is performed on the fusion features to obtain global hidden state features, and the global hidden state features are reconstructed to obtain reconstruction output; Step 7: A joint loss function is used for parameter optimization, and after continuous iterative training, a trained anomaly detection model is obtained.

[0028] Embodiment 2 The multi-modal feature fusion-based deep-sea manned submersible unsupervised anomaly detection method according to Embodiment 1, the difference lies in: Collecting multi-modal signal data of the deep-sea submersible and preprocessing; including: Collecting multi-modal sensor signal data of the deep-sea submersible in multiple working environments, including time domain signals and frequency domain signals, the time domain signals including depth, temperature, attitude signals, acceleration, angular velocity, the frequency domain signals including acoustic positioning signals, pressure pulsation signals, noise signals, angular vibration signals; Data cleaning of the multi-modal sensor signal data, including missing value processing and outlier processing, to generate input signals Wherein B is the batch size, T is the time window length, and C represents the number of channels; Performing initial feature extraction on the preprocessed multi-modal signal data to obtain local time series features; including: Inputting the input signal into the initial feature extraction to obtain the local time series features, as follows: ; Wherein, The local time series features, ReLU represents an activation function, Conv1D represents a one-dimensional convolution layer operation, and is defined as: ; Wherein, is a convolution kernel parameter, K is a convolution kernel size, and C represents the number of channels, For the input signal, b represents the batch index, t represents the current time step index, that is, the t-th time position in the output, represents the index of the input channel, k represents the index of the convolution kernel in the time dimension, that is, the k-th point in the time window, is a bias, and the output The convolutional layer captures the local dependent features of adjacent time points, and improves the parameter efficiency through weight sharing.

[0029] The local time sequence characteristics are enhanced in the frequency domain by a frequency domain enhancement module to obtain frequency domain enhancement features; including: The frequency domain enhancement module aims to mine the frequency domain disturbance features and multi-scale periodic structure information hidden in the original sensor signal, solve the problem of insufficient sensitivity of traditional time sequence modeling to frequency dependence, and improve the recognition ability of the anomaly detection system to hidden periodic anomalies; The module can capture frequency-related abnormal patterns that are difficult to model by traditional time sequence methods, adaptively capture frequency domain anomaly features by learning convolution kernel weights, and improve detection sensitivity; The multi-branch convolution operation includes two branches, and the obtained local time sequence features are input into the two branches respectively, and are sequentially processed by convolution, normalization and activation function to obtain frequency domain disturbance features, as follows: ; ; Among them, and represent the two obtained frequency domain disturbance features, represents the local time sequence feature, that is, , B is the batch size, C is the number of channels, and T is the length of the time sequence, represents batch normalization, represents one-dimensional convolution with a convolution kernel K, represents a dilated convolution, K is the size of the convolution kernel (K=3), and d is the sampling interval of the convolution kernel (d=2); The two obtained frequency domain disturbance features are fused using a fusion operation to obtain frequency domain enhancement features: ; ; Among them, represents the frequency domain enhancement feature, represents the spliced feature, represents the splicing operation, represents one-dimensional convolution with a convolution kernel K1, represents the size of the convolution kernel (K1=1), represents the number of output channels.

[0030] Model the long-term dependency of frequency domain enhancement features to obtain long-distance dependency features; Figure 2 Shown, including: The Mamba module is introduced to strengthen the internal features of the sequence and model long-term dependencies. As an emerging structure with higher sequence modeling efficiency than the Transformer, Mamba can effectively compensate for the problem of the traditional Transformer's reduced performance in modeling long sequences. The MAMBA module can effectively integrate the temporal correlation between different modalities and cross-modal semantic dependencies, enhancing the robustness and generalization ability of the model. The Mamba module includes splicing and fusion operations, convolution operations, state space operations, and dynamic gating operations; The frequency domain enhancement features are divided into M feature sequences and concatenated and fused as follows: ; in, Represents the characteristics of splicing fusion, represents the feedforward network of the Mth feature sequence, represents the Mth feature sequence, and M represents the number of feature sequences of frequency domain enhancement features; Compute the mapping input: ; in, represents the mapping input, represents the weight matrix, represents the dimension of the feature vector at each time step, ⊙ represents element multiplication, is the Sigmoid activation function, is the activation function, and represents the bias vector; Use convolution operation to map the input Further extract local features and perform nonlinear mapping through the anomaly-aware GELU activation mechanism: = ; At the same time, an abnormal enhancement activation mechanism is adopted for the mapping input as follows: = ; in, represents the mapping feature, Represents enhanced features, Conv represents convolution operation, represents the activation function; Use state-space operations (i.e., selective SSM) to process long sequences of mapped features: ; wherein, denotes the hidden state at time t, denotes the hidden state at time t-1, denotes the mapping feature at time t, A, is a trainable parameter, a sliding window convolution optimization is used to accelerate the calculation, and long sequence modeling is supported; Long distance dependency features are obtained by extracting high response channels through a dynamic gating operation: ; wherein, denotes an activation function, denotes a weight vector, denotes a bias vector, denotes a hidden state sequence, denotes a long distance dependency feature.

[0031] The frequency domain enhanced feature is context enhanced using a fusion attention encoding module to obtain an encoded feature; as shown in Figure 3 , including: The fusion attention encoding module includes an encoding operation and an anomaly attention operation (Anomaly Attention); The anomaly attention operation includes a series of attentions and prior attentions; The long distance dependency feature is context enhanced using a fusion attention encoding module, and the fusion attention encoding module (Anomaly Transformer Encoder) adopts an improved Transformer structure, which is embedded with an Anomaly Attention mechanism. The local attention based on the time window models the relevance between normal behavior and potential abnormal behavior; the fusion attention encoding module outputs include an encoding result, attention series information, prior attention distribution and variance information, which provide a basis for subsequent anomaly judgment; The frequency domain enhanced feature is encoded as follows: ; wherein, denotes an input encoded feature, denotes a word embedding; The query vector, value vector and key vector of the encoding result are calculated: , ; wherein, denotes a query vector, denotes a key vector, and V denotes a value vector, , , denotes a weight matrix; calculating series attention features: ; wherein, denotes series attention features, denotes the dimension of the key vector G; calculating prior attention features: ; wherein, denotes a prior attention matrix, i.e., prior attention features, is a variance, i and j represent the positions of two time steps, respectively, the prior matrix introduces a priori knowledge based on the position relationship, to enhance the ability of the model to capture local dependencies; fuses the calculated prior attention matrix with the series attention features to generate the final attention weight matrix , as follows: ; calculating the encoding result, as follows: ; wherein, is an encoding feature.

[0032] fusing the frequency domain enhanced features, the long distance dependency features and the encoding features to obtain the fused features; including: using a multi-modal feature fusion module (WeightedFeatureFusion) to weight fuse the frequency domain enhanced features, the long distance dependency features and the encoding features, and obtaining the fused weight parameters through training; compared with the traditional simple splicing method, this module can realize effective fusion of more consistent semantics and more complementary features, which helps to improve the stability and robustness of the model; setting the frequency domain enhanced features as , the long distance dependency features as , and the encoding features as ; calculating the global weight for each feature: ; wherein, denotes the global weight of the i-th feature; t denotes the time step; denotes the i-th feature in the three types of features, and T is the length of the time window; calculating the linear mapping weight: ; wherein, denotes a linear mapping result of the i-th type of feature, denotes a weight matrix, denotes a bias term; normalization processing is performed: ; wherein, denotes a normalized weight of the i-th type of feature, denotes a linear mapping result of the j-th type of feature; a fusion feature is finally obtained: ; wherein, denotes the fusion feature.

[0033] state modeling is performed on the fusion feature to obtain a global hidden state feature, and the global hidden state feature is reconstructed to obtain a reconstruction output; including: feature extraction is performed on the fusion feature, as follows: ; wherein, denotes a hidden state of a current time step t, denotes a hidden state of a time step t−1, denotes a bidirectional GRU (gated recurrent unit) operation; a time dimension is compressed to obtain a final state representation, and a global hidden state feature is obtained: ; wherein, denotes the global hidden state feature, is a hidden layer dimension, denotes a hidden state of a last time step ; the global hidden state feature is reconstructed, and a reconstruction calculation process is as follows: ; wherein, denotes a reconstruction output, denotes a decoding operation; an error between an input signal X and the reconstruction output is calculated, as follows: ; wherein, denotes a reconstruction error, denotes an input signal, denotes a reconstruction output, B, T, and C are batch size, time window length, and channel number, respectively, are a current batch size, a current time window length, and a current channel number, respectively; If there is a significant deviation between the original input, that is, the input signal, and the reconstructed output, that is, the reconstruction error is greater than the set threshold ( ), the input signal is considered an abnormal signal and the output detection result is abnormal, otherwise it is a normal signal.

[0034] The joint loss function is used for parameter optimization. After continuous iterative training, a trained anomaly detection model is obtained; including: Constructing reconstruction loss : ; in, represents the input signal, represents the reconstruction output; Constructing a normalized prior distribution : ; in, is the normalized prior distribution, Indicates the total length of the sequence, Represents the original prior distribution value at time step v, that is, the initial probability distribution, represents the original prior distribution value at time step u; Constructing series consistency loss (series loss): ; in, represents the series consistency loss, is the normalized prior distribution, represents the Kullback-Leibler divergence, represents the model prediction distribution at time step u (the model's probabilistic prediction of the sequence features at time step u based on the current input signal and previously learned knowledge); Constructing prior stability loss: ; in, represents the prior stability loss, Express Smoothing and contrast enhancement, represents the original prior distribution value at time step u; Define two relative objective loss functions, including minimization objective and maximization objective: the minimization objective (used to update the generation network) is: ; The maximization objective (for stable distribution training) is: ; wherein, is an adjustment parameter; During training, optimization is performed by minimizing the target loss and maximizing the target loss After continuous iterative training, a trained anomaly detection model is obtained.

[0035] Mamba module: a new (LLM) architecture that integrates structured state models to manage long data sequences. Combining the best features of RNNs, LSTMs, and continuous-time models, it can effectively model long-term dependencies.

[0036] Bidirectional GRU network: a recurrent neural network that combines gated recurrent units (GRUs) with a bidirectional structure. It captures past and future context information at each time step by using forward and backward GRU layers, controls information flow using update and reset gates, effectively models long-term dependencies, and is suitable for tasks that require context understanding such as speech recognition and text tagging.

[0037] The effectiveness of the present application is verified on the SWAT, OWN-1, and OWN-2 datasets. The SWAT is a public dataset that records various operations and events that occur in industrial control systems at water treatment plants. These records can be sensor readings, control instructions, device state changes, etc. OWN-1 and OWN-2 are different levels of collected data. The performance of the anomaly detection method in the present application is evaluated using Accuracy, Precision, Recall, and F-score as evaluation indicators, as shown in Table 1:

[0038] Table 1: Anomaly detection performance of different datasets

[0039] Among them, the method in the present application achieves high accuracy, precision, and F-score in the OWN-1 dataset, which are 0.9926, 0.9711, and 0.9854, respectively; and high Recall in the OWN-1 and OWN-2 datasets, which is 1.0000. The experimental results demonstrate the effectiveness and robustness of the method in the present application, which has good self-adaptability and detection ability for unknown anomalies.

[0040] Embodiment 3 A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the unsupervised anomaly detection method based on multi-modal feature fusion of the deep-sea manned submersible in embodiment 1 or 2 when executing the computer program.

[0041] Embodiment 4 A computer-readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the steps of the deep-sea manned submersible unsupervised anomaly detection method based on multi-modal feature fusion of embodiment 1 or 2.

[0042] Embodiment 5 The deep-sea manned submersible unsupervised anomaly detection system based on multi-modal feature fusion comprises: The model construction module is configured to collect multi-modal signal data of the deep-sea submersible and pre-process the multi-modal signal data; construct an anomaly detection model, input the pre-processed multi-modal signal data into the anomaly detection model for training, obtain the trained anomaly detection model, and input the multi-modal signal data of the deep-sea submersible into the trained anomaly detection model for anomaly detection of the deep-sea submersible; The initial feature extraction module is configured to perform initial feature extraction on the pre-processed multi-modal signal data to obtain local time sequence features; The frequency domain enhancement module is configured to perform multi-scale frequency domain enhancement on the local time sequence features through the frequency domain enhancement module to obtain frequency domain enhancement features; The frequency domain enhancement module includes a multi-branch convolution operation and a fusion operation, which extracts implicit frequency domain disturbance features through the multi-branch convolution operation and fuses the frequency domain disturbance features through the fusion operation; The long-term dependency modeling module is configured to model the long-term dependency of the frequency domain enhancement features to obtain long-distance dependency features; The context enhancement module is configured to use a fusion attention encoding module to perform context enhancement on the frequency domain enhancement features to obtain encoded features; The fusion attention encoding module models the relevance between features through a series of attentions and prior attentions to enhance the ability to capture local dependencies; The feature fusion module is configured to perform multi-modal feature fusion on the frequency domain enhancement features, the long-distance dependency features, and the encoded features to obtain fusion features; The reconstruction module is configured to model the state of the fusion features to obtain global hidden state features, and reconstruct and restore the global hidden state features to obtain reconstruction output; The optimization module is configured to perform parameter optimization using a joint loss function, and obtain the trained anomaly detection model through continuous iterative training.

Claims

1. An unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion, characterized by: include: Collect multimodal signal data from deep-sea submersibles and perform preprocessing; An anomaly detection model is constructed, and the preprocessed multimodal signal data is input into the anomaly detection model for training to obtain a trained anomaly detection model. The multimodal signal data of the deep-sea submersible is input into the trained anomaly detection model to perform anomaly detection on the deep-sea submersible; Build an anomaly detection model and input the preprocessed multimodal signal data into the anomaly detection model for training, including: Step 1: Perform initial feature extraction on the preprocessed multimodal signal data to obtain local time series features; Step 2: Use the frequency domain enhancement module to perform multi-scale frequency domain enhancement on the local time series characteristics to obtain frequency domain enhanced features; The frequency domain enhancement module (FEM) includes multi-branch convolution operations and fusion operations. The multi-branch convolution operation extracts the implicit frequency domain perturbation features, and the fusion operation fuses the frequency domain perturbation features. Step 3: Model the long-term dependency of the frequency domain enhanced features to obtain long-distance dependency features; Step 4: Use the fusion attention encoding module to perform context enhancement on the frequency domain enhancement features to obtain the encoding features; The fused attention encoding module models the correlation between features through serial attention and prior attention, enhancing the ability to capture local dependencies; Step 5: Perform multimodal feature fusion on the frequency domain enhancement features, long-distance dependency features and coding features to obtain fused features; Step 6: Perform state modeling on the fusion features to obtain the global hidden state features, and reconstruct and restore the global hidden state features to obtain the reconstructed output; Step 7: Use the joint loss function to optimize parameters. After continuous iterative training, a trained anomaly detection model is obtained.

2. The unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion according to claim 1 is characterized in that: Collect multimodal signal data from deep-sea submersibles and perform preprocessing, including: Collect multimodal sensor signal data from deep-sea submersibles in multiple operating environments, including time domain signals and frequency domain signals. Time domain signals include depth, temperature, attitude signals, acceleration, and angular velocity. Frequency domain signals include acoustic positioning signals, pressure pulsation signals, noise signals, and angular vibration signals. Perform data cleaning on multimodal sensor signal data, including missing value processing and outlier processing, to generate input signals , where B is the batch size, T is the time window length, and C represents the number of channels; Further preferably, the pre-processed multimodal signal data is subjected to initial feature extraction to obtain local time series features; including: The input signal is input for initial feature extraction to obtain local time series features, as shown below: ； in, Represents local temporal features, ReLU represents the activation function, and Conv1D represents the one-dimensional convolutional layer operation.

3. The unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion according to claim 2 is characterized in that: The frequency domain enhancement module performs multi-scale frequency domain enhancement on the local time series characteristics to obtain frequency domain enhancement features; including: The multi-branch convolution operation consists of two branches. The local time series features are input into the two branches respectively, and convolution, normalization and activation function processing are performed in sequence to obtain the frequency domain perturbation features, as shown below: ； ； in, and Represents the two obtained frequency domain perturbation features, Represents the local temporal characteristics, i.e. , B is the batch size, C is the number of channels, T is the time series length, represents batch normalization, represents one-dimensional convolution, represents the dilated convolution, K is the convolution kernel size, and d is the convolution kernel sampling interval; The two obtained frequency domain perturbation features are fused using the fusion operation to obtain the frequency domain enhancement feature: ； ； in, represents the frequency domain enhancement feature, Represents the splicing feature, Represents a splicing operation, represents one-dimensional convolution, represents the convolution kernel size, Indicates the number of output channels.

4. The unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion according to claim 3 is characterized in that: Model the long-term dependency of frequency domain enhancement features to obtain long-distance dependency features; including: The Mamba module is introduced to strengthen the internal features of the sequence and model long-term dependencies; The Mamba module includes splicing and fusion operations, convolution operations, state space operations, and dynamic gating operations; The frequency domain enhancement features are divided into M feature sequences and concatenated and fused as follows: ； in, Represents the characteristics of splicing fusion, represents the feedforward network of the Mth feature sequence, represents the Mth feature sequence, and M represents the number of feature sequences of frequency domain enhancement features; Compute the mapping input: ； in, represents the mapping input, represents the weight matrix, represents the dimension of the feature vector at each time step, ⊙ represents element multiplication, is the Sigmoid activation function, is the activation function, and represents the bias vector; Use convolution operation to map the input Further extract local features and perform nonlinear mapping through abnormal enhancement activation mechanism: = ； At the same time, an abnormal enhancement activation mechanism is adopted for the mapping input as follows: = ； in, represents the mapping feature, Represents enhanced features, Conv represents convolution operation, represents the activation function; Use state-space operations to process long sequences of mapped features: ； in, represents the hidden state at time t, represents the hidden state at time t-1, Represents the mapping feature at time t, A, is a trainable parameter; Extract high-response channels through dynamic gating operations and obtain long-range dependency features: ； in, represents the activation function, represents the weight vector, represents the bias vector, represents the hidden state sequence, Indicates long-distance dependency features.

5. The unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion according to claim 4 is characterized in that: The fusion attention encoding module is used to perform context enhancement on the frequency domain enhancement features to obtain the encoding features; including: The fused attention encoding module includes encoding operations and abnormal attention operations; Anomalous attention manipulations include serial attention and prior attention; The frequency domain enhancement feature is encoded as follows: ； in, represents the input encoding features, Represent word embeddings; Calculate the query vector, value vector, and key vector of the encoding result: ， ； in, represents the query vector, represents the key vector, V represents the value vector, 、 、 represents the weight matrix; Compute the series attention features: ； in, represents the serial attention feature, represents the dimension of the key vector G; Calculate the prior attention features: ； in, represents the prior attention matrix, i.e., the prior attention feature, is the variance, i and j represent the positions of two time steps respectively; The calculated prior attention matrix and serial attention features Fusion generates the final attention weight matrix , as shown below: ； The encoding result is calculated as follows: ； in, is the coding feature.

6. The unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion according to claim 5 is characterized in that: The frequency domain enhancement features, long-distance dependency features and coding features are fused into multimodal features to obtain fused features; including: Use the multimodal feature fusion module to perform weighted fusion of frequency domain enhancement features, long-distance dependency features, and coding features, and obtain the fusion weight parameters through training; Set the frequency domain enhancement feature to , the long-distance dependency feature is set to , the encoding feature is set to ; Calculate the global weight for each feature: ； in, represents the global weight of the i-th feature; t represents the time step; represents the i-th feature among the three types of features, and T is the length of the time window; Calculate the linear mapping weights: ； in, Represents the linear mapping result of the i-th category feature, represents the weight matrix, represents the bias term; Perform normalization: ； in, represents the normalized weight of the i-th category feature, Represents the linear mapping result of the j-th category feature; Finally, the fusion features are obtained: ； in, Indicates fusion features.

7. The unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion according to claim 6, characterized in that: Perform state modeling on the fusion features to obtain the global hidden state features, and reconstruct and restore the global hidden state features to obtain the reconstructed output; include: Feature extraction is performed on the fused features as follows: ； in, represents the hidden state at the current time step t, represents the hidden state at time step t−1, Represents a bidirectional GRU network; Compress the time dimension and obtain the final state representation to obtain the global hidden state features: ； in, represents the global hidden state feature, is the hidden layer dimension, Indicates the last time step The hidden state of The global hidden state features are reconstructed and restored. The reconstruction and restoration calculation process is: ； in, represents the reconstruction output, Indicates a decoding operation; Calculate the error between the input signal X and the reconstructed output as follows: ； in, represents the reconstruction error, represents the input signal, Represents the reconstruction output, B, T, and C are the batch size, time window length, and number of channels, respectively. are the current batch size, the current time window length, and the current number of channels respectively; If there is a significant deviation between the original input, i.e., the input signal, and the reconstructed output, i.e., the reconstruction error is greater than the set threshold, then the input signal is considered an abnormal signal and the output detection result is abnormal; otherwise, it is a normal signal. Further preferably, a joint loss function is used for parameter optimization, and a trained anomaly detection model is obtained through continuous iterative training; including: Constructing reconstruction loss : ； in, represents the input signal, represents the reconstruction output; Constructing a normalized prior distribution : ； in, is the normalized prior distribution, Indicates the total length of the sequence, Represents the original prior distribution value at time step v, that is, the initial probability distribution, represents the original prior distribution value at time step u; Constructing the series consistency loss: ； in, represents the series consistency loss, is the normalized prior distribution, represents the Kullback-Leibler divergence, represents the model prediction distribution at time step u; Construct a priori stability loss: ； in, represents the prior stability loss, Express Smoothing and contrast enhancement, represents the original prior distribution value at time step u; Define the loss function of two relative objectives, including minimization objective and maximization objective: the minimization objective is: ； The maximization objective is: ； in, To adjust the parameters; During training, by minimizing the target loss and maximize the target loss Optimize and continuously iterate the training to obtain a trained anomaly detection model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion as described in any one of claims 1-7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the unsupervised anomaly detection method for deep-sea manned submersibles based on multimodal feature fusion as described in any one of claims 1 to 7 are implemented.

10. Unsupervised anomaly detection system for deep-sea manned submersibles based on multimodal feature fusion, characterized by: include: The model building module is configured to: collect multimodal signal data of the deep-sea submersible and perform preprocessing; An anomaly detection model is constructed, and the preprocessed multimodal signal data is input into the anomaly detection model for training to obtain a trained anomaly detection model. The multimodal signal data of the deep-sea submersible is input into the trained anomaly detection model to perform anomaly detection on the deep-sea submersible; The initial feature extraction module is configured to: perform initial feature extraction on the preprocessed multimodal signal data to obtain local time series features; A frequency domain enhancement module is configured to perform multi-scale frequency domain enhancement on the local time series characteristics through the frequency domain enhancement module to obtain frequency domain enhanced features; The frequency domain enhancement module includes multi-branch convolution operations and fusion operations. The multi-branch convolution operation is used to extract implicit frequency domain perturbation features, and the fusion operation is used to fuse the frequency domain perturbation features. The long-term dependency modeling module is configured to: perform long-term dependency modeling on the frequency domain enhanced features to obtain long-distance dependency features; The context enhancement module is configured to: use the fusion attention encoding module to perform context enhancement on the frequency domain enhancement features to obtain encoding features; The fused attention encoding module models the correlation between features through serial attention and prior attention, enhancing the ability to capture local dependencies; The feature fusion module is configured to: perform multimodal feature fusion on the frequency domain enhancement feature, the long-distance dependency feature and the coding feature to obtain a fused feature; The reconstruction module is configured to: perform state modeling on the fusion features, obtain the global hidden state features, and reconstruct and restore the global hidden state features to obtain the reconstructed output; The optimization module is configured to use a joint loss function to optimize parameters and obtain a trained anomaly detection model through continuous iterative training.

Images