Multi-scale dual-path attention mechanism schizophrenia diagnosis method based on comparative learning

Abstract

The invention provides a schizophrenia diagnosis method of a multi-scale double-path attention mechanism based on comparative learning. The model adopts an end-to-end deep learning architecture, firstly performs systematic data preprocessing on an original EEG signal, then adopts a double-view data enhancement strategy, then extracts spatio-temporal features through a convolution feature extraction block, adopts sliding windows for parallel processing, independently applies MSDPA to each window, and then inputs MSDPA output features into a time sequence convolution network module. And finally, integrating the multi-window features through an average fusion strategy, inputting the multi-window features into a classifier and a projector at the same time, and outputting an HC / SZ binary classification diagnosis result by adopting joint loss optimization, thereby providing technical support for early discovery and precise treatment of schizophrenia.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of auxiliary diagnosis of mental illness, and a schizophrenia diagnosis method based on a multi-scale dual-path attention mechanism of contrast learning. BACKGROUND

[0002] Schizophrenia (SZ) hinders brain development and seriously damages thought, emotional expression, and perception of reality. Most studies show that brain structure and function abnormalities have a great impact, but the cause of SZ is still unclear. EEG-SZ automatic diagnosis using artificial intelligence (AI) methods includes traditional machine learning (ML) and deep learning (DL), which includes preprocessing, feature extraction and selection, and feature classification, of which feature extraction is the most important part. In traditional machine learning, the features extracted from EEG signals are mainly divided into: 1. time domain, 2. frequency domain, 3. time-frequency, and 4. nonlinearity.

[0003] Application No. CN202410855167.X discloses a method and system for constructing a schizophrenia recognition model based on EEG fuzzy entropy and a continuous convolutional neural network, and a computer storage medium. The method process is as follows: S1. Preprocess the EEG brain wave signal to obtain the processed EEG signal. S2. Extract different rhythm wave bands from each processed EEG signal to obtain the fuzzy entropy corresponding to each rhythm wave band. S3. Construct a three-dimensional feature matrix containing spatial information of the EEG signal. S4. The neural network performs convolution and uses different attention mechanisms to output different recognition results. S5. Different voting methods are used to combine different recognition results to obtain the final recognition result output by the model.

[0004] The application number CN202510182565.4 discloses a youth schizophrenia deep learning recognition system and method based on signal optimization preprocessing. The method process is as follows: S1. Extract data from multiple public schizophrenia electroencephalogram data sets to obtain a data set. S2. Data preprocessing: select F7, F3, F4, F8, T3, C3, Cz, C4, T4, T5, P3, Pz, P4, T6, O1, and O2 electroencephalogram lead channels, and standardize the data of each subject, and uniformly intercept 1 minute of resting state electroencephalogram signals; then resample these signals to 128Hz to ensure efficient processing and transmission of data, and remove low and high frequency noise through a 0.5-30Hz bandpass filter to retain the most clinically significant signal components; finally, convert the electroencephalogram signals of each channel of each subject into frequency domain information to generate a frequency spectrum, i.e. an FFT spectrum. S3. A deep learning model based on convolutional neural network (CNN model, VGG-16 model and ChronoNet model) is used to classify and identify the electroencephalogram signals. S4. The trained deep learning model is used to identify and classify youth schizophrenia patients.

[0005] Disadvantages: Electroencephalogram (EEG) carries information about brain network connectivity and can be used to detect brain abnormalities that suggest schizophrenia. A key deficiency of existing research is the lack of a robust mechanism that can simultaneously capture local and long-range EEG features, which are crucial for distinguishing schizophrenia-related abnormalities.

[0006] Existing methods lack robustness design for individual differences, and robust models are crucial for learning generalizable patterns. Schizophrenia patients have significant individual differences, and single classification loss training is difficult to learn feature representation with good generalization ability. There is a lack of effective contrast learning mechanism and data enhancement strategy.

[0007] Existing attention mechanisms mainly focus on single-scale features and cannot effectively capture abnormal patterns of different time scales in EEG signals of schizophrenia patients. SUMMARY

[0008] To overcome the shortcomings of the prior art in weak cross-subject generalization ability, single-scale feature capture of attention mechanism, and lack of long-range temporal dependence modeling, the present application proposes a schizophrenia diagnosis method based on a multi-scale dual-path attention mechanism of contrast learning (MSDPA-ContrastNet).

[0009] A schizophrenia diagnosis method based on a multi-scale dual-path attention mechanism of contrast learning, comprising: Step S1: data preprocessing; The specific process is as follows: the original continuous EEG signal is segmented without overlap, with a single frame duration of 25 seconds, to obtain multiple frames of EEG signal for each subject. The dimension of each frame signal is "time sample number × channel number", i.e., 6250×19. The z-score and L2 method are used to normalize each EEG signal frame.

[0010] Step S2: Establish a comparative learning framework. The specific process is as follows: Dual-view data augmentation: Enhancement of the standardized EEG signal X norm Gaussian noise injection, time translation, and amplitude scaling are performed sequentially to obtain a moderately enhanced viewpoint X. aug1 Based on the operation of enhancing the viewpoint to a moderate intensity, an additional frequency domain masking operation is added to generate the viewpoint X with a stronger intensity enhancement. aug2 See formulas (1) and (2): (1) (2) in 1 represents the Gaussian noise injected into the normalized signal, δ1 represents the time shift, and α1 represents the amplitude scaling factor. 2 indicates higher injected noise intensity, δ2 indicates a larger time shift amplitude, α2 indicates a wider amplitude adjustment range, and FreqMask indicates a frequency domain mask function.

[0011] This invention proposes a diagnostic method for schizophrenia based on a multi-scale dual-path attention mechanism (MSDPA-ContrastNet) using contrastive learning. The model employs an end-to-end deep learning architecture. First, it systematically preprocesses the raw EEG signals. Then, it uses a dual-view data augmentation strategy. Next, it extracts spatiotemporal features through convolutional feature extraction blocks, employing parallel processing with sliding windows. Each window independently applies the multi-scale dual-path attention mechanism (MSDPA). The MSDPA output features are then input into a temporal convolutional network (TCN) module. Finally, an averaging fusion strategy integrates the multi-window features, simultaneously inputting them into a classifier and projector. Joint loss optimization is used to output a binary HC / SZ diagnostic result, providing technical support for the early detection and precise treatment of schizophrenia.

[0012] This invention evaluates the performance of the proposed model using a dataset from the Institute of Psychiatry and Neurology in Warsaw, Poland. Experimental results show that the highest classification accuracy reaches 99.5% in the classification task between schizophrenic patients (SZ) and healthy controls (HC). This indicates that the model can accurately diagnose schizophrenia and provide strong support for early intervention and treatment. Attached Figure Description

[0013] Figure 1 This forms the overall framework of the schizophrenia diagnostic method of the present invention.

[0014] Figure 2 This is a structural diagram of the multi-scale dual-path attention mechanism of the present invention. Detailed Implementation

[0015] like Figure 1 As shown, a diagnostic method for schizophrenia based on a multi-scale dual-path attention mechanism using contrastive learning includes: Step S1: Data Preprocessing Dataset: Data from the Institute of Psychiatry and Neurology in Warsaw, Poland (Olejarczyk and Jernajczyk, 2017). First, the raw continuous EEG signals were segmented without overlap, with each frame lasting 25 seconds (calculated at a sampling rate of 250 Hz, each frame contains 250 Hz × 25 s = 6250 time samples). This resulted in multi-frame EEG signals for each subject, with each frame having a dimension of "time sample number × channel number", i.e., 6250 × 19. Next, z-scores and the L2 method were used to normalize each EEG signal frame.

[0016] Step S2: Comparative Learning Framework Dual-view data augmentation: Enhancement of the standardized EEG signal X norm Gaussian noise injection, time translation, and amplitude scaling are performed sequentially to obtain a moderately enhanced viewpoint X. aug1 Based on the operation of enhancing the viewpoint to a moderate intensity, an additional frequency domain masking operation is added to generate the viewpoint X with a stronger intensity enhancement. aug2 See formulas (1) and (2): (1) (2) in 1 represents the Gaussian noise injected into the normalized signal, δ1 represents the time shift, and α1 represents the amplitude scaling factor. 2 indicates higher injected noise intensity, δ2 indicates a larger time shift amplitude, α2 indicates a wider amplitude adjustment range, and FreqMask indicates a frequency domain mask function.

[0017] Projector: The blended feature F after passing through SW (sliding window) fused By combining fully connected layers, batch normalization, and activation functions, low-dimensional intermediate features are mapped, as shown in equations (3) and (4). (3) (4) In formula (3), Dense 128 (F fused This indicates a fully connected layer with an output dimension of 128 (Dense). 128 ) to fuse high-dimensional features F fused Compressed to 128 dimensions; BatchNorm(·) is a batch normalization operation used to standardize the distribution of the output features of the fully connected layer; ReLU(·) is an activation function that introduces nonlinear transformation to enhance the model's ability to express features, ultimately obtaining the intermediate feature h1. In formula (4), Dense 128 (h1) indicates that h1 is subjected to a quadratic linear transformation through a second fully connected layer with an output dimension of 128, further optimizing the feature distribution and obtaining unnormalized low-dimensional features z. raw .

[0018] To eliminate the influence of feature magnitude on subsequent similarity calculations, z raw L2 normalization is performed to obtain the final low-dimensional embedding feature z, and the mathematical expression is shown in formula (5): (5) Contrast loss: For each sample X in the batch i The two corresponding enhanced perspectives are X. (i) aug1 and X (i) aug2 After processing by the encoder, the feature representation z is obtained. i and z i+N Where N is the batch size. A positive sample pair is defined as (z... i , z i+N ) and (z i+N , z i This refers to different enhancement perspectives of the same sample. Based on the L2 normalized feature z, the cosine similarity between features is calculated as shown in formula (6): (6) Since z has been L2 normalized, s ij = z i T z j The temperature parameter τ is applied for scaling, as shown in formula (7): (7) For each positive sample pair (z) i , z i+N ), calculate its "relative similarity loss" among all samples in the batch, as shown in formula (8): (8) To maintain symmetry, while calculating (z) i+N , z i The loss is shown in formula (9): (9) The total batch contrast loss L is obtained by averaging the losses of all positive sample pairs within the batch. contrast As shown in formula (10): (10) Joint Loss Optimization: Classification Loss L clf Using cross-entropy, the total loss L total The weighted sum of the classification loss and the contrastive loss is shown in Equation (11): (11) Where λ clf =0.9, λ contrast =0.1.

[0019] Encoder: The enhanced EEG signal is subjected to preliminary feature extraction through convolution as shown in formula (12): (12) ConvBlock(·) includes temporal convolution and spatially separable convolution, used to extract basic spatiotemporal features. Temporal convolution extracts frequency band features along the time dimension through a one-dimensional convolution kernel, while spatially separable convolution extracts spatial patterns through translead spatial convolution.

[0020] The convolutional features are divided into multiple time windows using a sliding window: F w = {F w_1 F w_2 ,...,F w_n}, where F w_i = Window(F conv Let i) represent the feature of the i-th window, and n be the number of windows. The sliding window module divides a continuous time series into multiple overlapping time segments, and each window is processed independently, which is convenient for capturing feature patterns at different time scales.

[0021] Multi-Scale Dual-Path Attention (MSDPA) module: Network structure as follows Figure 2 As shown. First, the input feature sequence is reshaped into a spatial format, and then a surrogate token A is generated through a convolution operation. tokens The proxy token, as a compact representation of global information, can effectively reduce computational complexity and capture long-range dependencies. A QKV transformation is performed on the two paths, as shown in equations (13), (14), and (15). The self-attention path captures local feature relationships, while the proxy attention path captures global feature relationships.

[0022] (13) (14) (15) Among them, W Q W K W V These are the learnable weight matrices for the query, key, and value in the self-attention path, used to weight the input features F. w_i Perform a linear transformation; Q T K T V T These are the transformation results of the query, key, and value in the self-attention path, respectively; W Q ′、W K ′、W V '' represents the learnable weight matrix for query, key, and value in the agent attention path, used to adjust the agent token A. tokens Perform a linear transformation; Q A K A V A These are the transformation results of the query, key, and value in the agent attention path, respectively.

[0023] The multi-scale dual-path attention mechanism module performs multi-scale Top-K selection using four ratios (50%, 67%, 75%, and 80%) to capture feature patterns at different time scales. These four ratios are 50%, 67%, 75%, and 80%, respectively; smaller ratios are used to capture fine-grained local features, while larger ratios are used to capture coarse-grained global patterns. For each ratio k... ratios [r], where r is the scale index, r∈{1,2,3,4}, corresponding to the four proportions of 50%, 67%, 75%, and 80% respectively; calculate the Top-K mask and apply the Softmax function to obtain the attention weights, as shown in (16), (17), and (18).

[0024] (16) (17) (18) In formula (16), Q T K T These are the transformation results of the query and key in the self-attention path, respectively, Q. A This represents the transformed result of the query in the proxy attention path. τ is a learnable temperature parameter used to adjust the sharpness of the attention distribution. S TS is the attention score of the self-attention path, reflecting the relationship between input tokens. A This is the proxy attention score for the proxy attention path, which measures the strength of the association between the proxy token and the input token. This calculation method avoids scaling to the square root of the dimension as in traditional Transformers, and instead adaptively adjusts the attention strength by learning parameters.

[0025] In formula (17), N is the sequence length, k ratios [r] represents the four percentages: 50%, 67%, 75%, and 80%. This indicates rounding down, k = N · k ratios [r] Let A be the number of Top-K values ​​at the r-th scale, and TopKMask represent the Top-K masking function, retaining the first k maximum values ​​and setting the rest to negative infinity. Softmax transforms the attention scores into a probability distribution. T (r) Represents the attention weight of the self-attention path at the r-th scale; in formula (18), A A (r) This represents the attention weight of the agent's attention path at the r-th scale.

[0026] The outputs of the self-attention path and the proxy attention path at different scales are processed through learnable weights w. r Weighted fusion is performed to aggregate multi-scale information, as shown in Equation (19); then, it is processed through another set of learnable weights w. a and w b The aggregated output H of the self-attention path is fused T fused The aggregated output H of the agent attention path A fused H attention As shown in formula (20); In order to enhance the feature representation capability of the model and promote gradient flow, it is fused with a residual path processed by depthwise Conv2D to obtain the final output, as shown in formulas (21) and (22).

[0027] (19) (20) (twenty one) (twenty two) Formula (21) represents the input feature F w_iAfter reshaping, depthwise separable convolution is performed to obtain the result H of the residual path after depthwise separable convolution processing. conv Formula (22) represents the fusion of self-attention path and surrogate attention path H. attention and H conv The summation results are then fully connected to prevent overfitting, yielding the final output A of the multi-scale dual-path attention mechanism module. output .

[0028] Next, the Temporal Convolutional Network (TCN) module is used. The TCN module captures temporal dependencies at different time scales through dilated convolutions. The design of increasing dilation rate allows the model to learn temporal patterns gradually from local to global. After N repetitions of MSDPA and TCN processing, all window features are fused, as shown in formulas (23) and (24): (twenty three) (twenty four) Where Conv1Ddil=1, Conv1Ddil=2, and Conv1Ddil=4 represent one-dimensional convolution operations with different dilation rates of 1, 2, and 4, respectively, used to capture temporal dependencies at different time scales; ELU represents the exponential linear unit activation function, used to introduce nonlinearity and enhance the expressive power of the model; through the above operations, the output feature T after processing by the temporal convolutional network module through a single window can be obtained. output (i) After all windows are processed by a temporal convolutional network, the output features T of all windows are... output (i) By performing average fusion, a globally unified feature representation F is obtained. fused .

[0029] This invention utilizes a dataset from the Warsaw Institute of Psychiatry and Neurology in Poland, comprising 14 patients with SZ and 14 healthy controls. The GPU used is an NVIDIA RTX 2080 Ti, and the deep learning framework employed is the Keras library. During model training and testing, participants are arranged in ascending order of their participant IDs, and a 5-fold cross-validation method is used. During training, it is ensured that participants in the training and test sets do not overlap, and that participants in the test set do not overlap in any of the five training iterations. The total number of training epochs is set to 50, with a batch size of 128. The initial learning rate is set to 0.0001, and the learning rate decays by 3% every 10 epochs. The loss function uses joint loss (cross-entropy loss with a weight of 0.9 + contrastive loss with a weight of 0.1), the optimizer is Adam, and dropout is set to 0.3. On the test set, the confusion matrix, accuracy, recall, precision, F1-score, AUC, and Cohen's Kappa coefficient are calculated to evaluate model performance.

Claims

1. A diagnostic method for schizophrenia based on a multi-scale dual-path attention mechanism using contrastive learning, characterized in that... include: Step S1: Data preprocessing; Step S2: Establish a comparative learning framework.

2. The method for diagnosing schizophrenia according to claim 1, characterized in that... The data preprocessing in step S1 above is as follows: the original continuous EEG signal is segmented without overlap, with a single frame duration of 25 seconds, to obtain multiple frames of EEG signal for each subject. The dimension of each frame signal is "time sample number × channel number", i.e., 6250 × 19. The z-score and L2 method are used to normalize each EEG signal frame.

3. The method for diagnosing schizophrenia according to claim 2, characterized in that... The specific process of establishing the contrastive learning framework in step S2 above is as follows: Dual-view data augmentation: Enhancement of the standardized EEG signal X norm Gaussian noise injection, time translation, and amplitude scaling are performed sequentially to obtain a moderately enhanced viewpoint X. aug1 Based on the operation of enhancing the viewpoint to a moderate intensity, an additional frequency domain masking operation is added to generate the viewpoint X with a stronger intensity enhancement. aug2 See formulas (1) and (2): (1) (2) in 1 represents the Gaussian noise injected into the normalized signal, δ1 represents the time shift, and α1 represents the amplitude scaling factor. 2 indicates higher injected noise intensity, δ2 indicates a larger time shift amplitude, α2 indicates a wider amplitude adjustment range, and FreqMask indicates a frequency domain mask function.

4. The method for diagnosing schizophrenia according to claim 3, characterized in that... Establish a projector: merging the features F after passing through the sliding window. fused By combining fully connected layers, batch normalization, and activation functions, low-dimensional intermediate features are mapped, as shown in formulas (3) and (4): (3) (4) In formula (3), Dense 128 (F fused () indicates a Dense fully connected layer with an output dimension of 128. 128 High-dimensional fusion feature F fused Compressed to 128 dimensions; BatchNorm(·) is a batch normalization operation used to standardize the distribution of the output features of the fully connected layer; ReLU(·) is an activation function that introduces nonlinear transformation to enhance the model's ability to express features, and finally obtains the intermediate feature h1; In formula (4), Dense 128 (h1) indicates that h1 is subjected to a quadratic linear transformation through a second fully connected layer with an output dimension of 128, further optimizing the feature distribution and obtaining unnormalized low-dimensional features z. raw ; (5) To eliminate the influence of feature magnitude on subsequent similarity calculations, z raw L2 normalization is performed to obtain the final low-dimensional embedding feature z, and the mathematical expression is shown in formula (5).

5. The method for diagnosing schizophrenia according to claim 4, characterized in that... Establish contrastive loss: for each sample X in the batch i The two corresponding enhanced perspectives are X. (i) aug1 and X (i) aug2 After processing by the encoder, the feature representation z is obtained. i and z i+N Where N is the batch size; a positive sample pair is defined as (z i , z i+N ) and (z i+N , z i ), that is, different enhancement perspectives of the same sample; based on the L2 normalized feature z, the cosine similarity between features is calculated as shown in formula (6): (6) Since z has been L2 normalized, s ij = z i T z j The scaling is performed using the temperature parameter τ, as shown in formula (7): (7) For each positive sample pair (z) i , z i+N ), calculate its "relative similarity loss" among all samples in the batch, as shown in formula (8): (8) To maintain symmetry, while calculating (z) i+N , z i The loss is shown in formula (9): (9) The total batch contrast loss L is obtained by averaging the losses of all positive sample pairs within the batch. contrast As shown in formula (10): (10) Joint Loss Optimization: Classification Loss L clf Using cross-entropy, the total loss L total The weighted sum of the classification loss and the contrastive loss is shown in Equation (11): (11) among them clf =0.9,λ contrast =0.1。 6. The method for diagnosing schizophrenia according to claim 5, characterized in that... Encoder establishment: Preliminary feature extraction is performed on the enhanced EEG signal through convolution, as shown in formula (12): (12) ConvBlock(·) includes temporal convolution and spatial depth separable convolution, which are used to extract basic spatiotemporal features. Temporal convolution extracts frequency band features along the time dimension through a one-dimensional convolution kernel, while spatial depth separable convolution extracts spatial patterns through cross-lead spatial convolution.

7. The method for diagnosing schizophrenia according to claim 6, characterized in that... The convolutional features are divided into multiple time windows using a sliding window: F w = {F w_1 F w_2 ,...,F w_n }, where F w_i = Window(F conv , i) represents the feature of the i-th window, and n is the number of windows; the sliding window module divides the continuous time series into multiple overlapping time segments, and each window is processed independently, which is convenient for capturing feature patterns at different time scales.

8. The method for diagnosing schizophrenia according to claim 7, characterized in that... A multi-scale dual-path attention mechanism module is established: first, the input feature sequence is reshaped into a spatial format, and then a surrogate token A is generated through convolution operations. tokens The proxy token, as a compact representation of global information, can effectively reduce computational complexity and capture long-range dependencies. The QKV transformation is performed on the two paths, as shown in formulas (13)(14)(15). The self-attention path captures local feature relationships, while the proxy attention path captures global feature relationships. (13) (14) (15) Among them, W Q W K W V These are the learnable weight matrices for the query, key, and value in the self-attention path, used to weight the input features F. w_i Perform a linear transformation; Q T K T V T These are the transformation results of the query, key, and value in the self-attention path, respectively; W Q ′、W K ′、W V '' represents the learnable weight matrix for query, key, and value in the agent attention path, used to adjust the agent token A. tokens Perform a linear transformation; Q A K A V A These are the transformation results of the query, key, and value in the agent attention path, respectively. The multi-scale dual-path attention mechanism module performs multi-scale Top-K selection using four ratios (50%, 67%, 75%, and 80%) to capture feature patterns at different time scales. These four ratios are 50%, 67%, 75%, and 80%, respectively; smaller ratios are used to capture fine-grained local features, while larger ratios are used to capture coarse-grained global patterns. For each ratio k... ratios [r], where r is the scale index, r∈{1,2,3,4}, corresponding to the four proportions of 50%, 67%, 75%, and 80% respectively; calculate the Top-K mask and apply the Softmax function to obtain the attention weights, as shown in (16), (17), and (18); (16) (17) (18) In formula (16), Q T K T These are the transformation results of the query and key in the self-attention path, respectively, Q. A This represents the transformed result of the query in the proxy attention path. τ is a learnable temperature parameter used to adjust the sharpness of the attention distribution. S T S is the attention score of the self-attention path, reflecting the relationship between input tokens. A It is the proxy attention score of the proxy attention path, which measures the strength of the association between the proxy token and the input token; In formula (17), N is the sequence length, k ratios [r] represents the four percentages: 50%, 67%, 75%, and 80%. This indicates rounding down, k = N · k ratios [r] Let A be the number of Top-K values ​​at the r-th scale, and TopKMask represent the Top-K masking function, retaining the first k maximum values ​​and setting the rest to negative infinity. Softmax transforms the attention scores into a probability distribution. T (r) Represents the attention weight of the self-attention path at the r-th scale; in formula (18), A A (r) This represents the attention weights of the agent's attention path at the r-th scale; The outputs of the self-attention path and the proxy attention path at different scales are processed through learnable weights w. r Weighted fusion is performed to aggregate multi-scale information. The calculation formula is shown in (19), and the aggregated output H of the self-attention path is obtained. T fused The aggregated output H of the agent attention path A fused ; (19) Then, through another set of learnable weights w a and w b The aggregated output H of the self-attention path is fused T fused The aggregated output H of the agent attention path A fused H attention The calculation is shown in formula (20); (20) To enhance the model’s feature representation capabilities and promote gradient flow, it is fused with a residual path processed by depthwise separable convolution to obtain the final output, as shown in formulas (21) and (22). (21) (22) Wherein, formula (21) represents the input feature F w_i After reshaping, depthwise separable convolution is performed to obtain the result H of the residual path after depthwise separable convolution processing. conv Formula (22) represents the fusion of self-attention path and surrogate attention path H. attention and H conv The summation results are then fully connected to prevent overfitting, yielding the final output A of the multi-scale dual-path attention mechanism module. output . Next, the model enters the temporal convolutional network module. This module captures temporal dependencies at different time scales through dilated convolutions. The design of increasing dilation rate enables the model to learn temporal patterns gradually from local to global. After N repetitions of the multi-scale dual-path attention mechanism module and the temporal convolutional network module, all window features are fused, as shown in formulas (23) and (24). (23) (24) Where Conv1Ddil=1, Conv1Ddil=2, and Conv1Ddil=4 represent one-dimensional convolution operations with different dilation rates of 1, 2, and 4, respectively, used to capture temporal dependencies at different time scales; ELU represents the exponential linear unit activation function, used to introduce nonlinearity and enhance the expressive power of the model; through the above operations, the output feature T after processing by the temporal convolutional network module through a single window can be obtained. output (i) After all windows are processed by a temporal convolutional network, the output features T of all windows are... output (i) By performing average fusion, a globally unified feature representation F is obtained. fused .

Images