A model architecture of time-frequency joint analysis and multi-scale feature enhancement learning

Abstract

The application belongs to the technical field of time series data analysis, and relates to a model architecture of time-frequency joint analysis and multi-scale feature enhancement learning, which comprises an encoder, a feature learning module and a classification head; the encoder converts input original time series data into first feature representation through value embedding, position embedding and time feature embedding, and takes the first feature representation as input of the feature learning module; the feature learning module combines fast Fourier transform and continuous wavelet transform, adjusts decomposition levels of time-frequency analysis through an adaptive aggregation algorithm, comprehensively captures data features, and outputs second feature representation after processing by the feature learning module; and the classification head maps the input second feature representation to a target category space through an activation function, a Dropout layer and a full connection layer, so as to realize a classification task; the application controls parameter quantity and calculation quantity within a certain range, and is suitable for real-time medical monitoring and low-resource devices.

Description

Technical Field

[0001] This invention belongs to the field of medical time series data analysis technology, specifically involving a model architecture for time-frequency joint analysis and multi-scale feature enhancement learning. Background Technology

[0002] Medical time-series data often exhibit complex non-stationarity and multi-dimensional characteristics, posing significant challenges to traditional time-series analysis methods. In recent years, time-frequency domain deep learning methods have been proposed for the automated analysis of various physiological signals. Researchers employ various feature extraction techniques to transform biomedical signals in the time domain into joint time-frequency domain signals, aiming to achieve better task performance. Fourier transform and wavelet transform are among the commonly used methods. For example, using FFT to extract frequency features from electrocardiogram (ECG) signals has achieved good results in the abnormal classification of cardiac patients; similarly, the statistical characteristics of wavelet coefficients have been used to characterize the time-frequency distribution of electroencephalogram (EEG) signals, enabling classification for different tasks. However, FFT has limitations when processing non-stationary signals, easily losing time-domain information and struggling to capture transient features. Although CWT has good localization capabilities in time-frequency analysis, its limitation in frequency domain resolution may lead to poor global feature extraction.

[0003] Medical time-series data often exhibits complex patterns that are both short-term and long-term recurring. Therefore, comprehensively considering these patterns is crucial for accurate classification. The main modeling challenge lies in effectively capturing the complex temporal patterns of each variable (including short-term local patterns and long-term global patterns) and the complex interdependencies between variables. Two-dimensional convolutional neural networks (CNNs) have performed well in tasks such as signal and image feature extraction and disease detection. With the improvement of computing power and the advancement of neural network architecture, recent research has gradually focused on methods such as CNNs, RNNs, and Transformers. However, these methods each have their limitations. For example, due to the locality of convolution kernels, CNNs struggle to fully simulate long-term dependencies in time series, especially when processing long sequences, where local convolution operations may "isolate" information, thus limiting their performance. Although RNN-type methods have achieved significant results, they often encounter the vanishing and exploding gradient problems, struggle to effectively capture long-term dependencies, and their sequential computation paradigm also affects their performance. Transformer-based models have been introduced to better capture long-term dependencies in time series, but their high computational complexity and memory requirements make them difficult to apply to long-sequence modeling. Furthermore, their inherently superior attention mechanism is difficult to directly identify reliable dependencies from scattered time points. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a model architecture for time-frequency joint analysis and multi-scale feature enhancement learning.

[0005] To achieve the objectives of this invention, the following technical solutions are adopted.

[0006] A model architecture for joint time-frequency analysis and multi-scale feature enhancement learning is disclosed. The architecture includes an encoder, a feature learning module, and a classification head. The encoder transforms the input raw time-series data into a first feature representation through value embedding, position embedding, and temporal feature embedding. After processing by the feature learning module, a second feature representation is output. The classification head maps the second feature representation to the target class space through an activation function, a Dropout layer, and a fully connected layer to achieve the classification task.

[0007] The feature learning module includes a Fast Fourier Transform (FFT) module, a Continuous Wavelet Transform (CWT) module, and two multi-scale feature enhancement learning modules. The FFT module performs frequency domain analysis on the first feature representation to capture global and periodic features, and then passes it to one multi-scale feature enhancement learning module. The CWT module performs time domain analysis on the first feature representation to capture local changes and abnormal fluctuations, and then passes it to another multi-scale feature enhancement learning module. Each multi-scale feature enhancement learning module includes a feature attention module, a MSDC module, and an LCWD module. The feature attention module adaptively adjusts the weight of each input feature, enabling the model to automatically focus on high-weight features. The MSDC module uses multidimensional differential convolution to extract local information from high-weight features and capture abnormal signals. The LCWD module uses a strip convolution kernel to capture long-range dependencies from high-weight features to express lesion region features. The global and local features are fused by multiplying the lesion region features and the abnormal signal to form a second feature representation. The multidimensional differential convolution captures boundary features in the signal by calculating the local gradient of the signal.

[0008] As a preferred embodiment of the present invention, the first feature represents essential features, temporal order, and temporal dependency.

[0009] As a preferred embodiment of the present invention, residual connections are introduced into the multi-scale feature enhancement learning module to improve the expressive power of features.

[0010] As a preferred embodiment of the present invention, the output result of the multi-scale feature enhancement learning module is:

[0011] .

[0012] As a preferred embodiment of the present invention, the specific processing procedure of the Fast Fourier Transform (FFT) module is as follows:

[0013] Assuming the input signal Where T is the time step of the signal, and C is the number of channels, i.e., the number of time series features; the frequency domain signal is obtained by performing FFT on the signal. This refers to the frequency domain representation; the calculation formula is as follows:

[0014] ;

[0015] in, It includes frequency components in complex form; the top K principal frequency components are extracted from the FFT result. and the corresponding amplitude The calculation formula is as follows:

[0016] ;

[0017] Frequency and period are reciprocals of each other, that is... Period length From frequency components The calculated extracted frequency components and period length When combined into a two-dimensional matrix, it will eventually form a shape of , The matrix; its representation is as follows:

[0018] ;

[0019] Output matrix have and Two dimensions, representing period length and frequency respectively; then the frequency is normalized:

[0020] ;

[0021] in, The normalized frequency weights.

[0022] As a preferred embodiment of the present invention, the specific processing procedure of the continuous wavelet transform module (CWT) is as follows:

[0023] CWT uses wavelet functions Extracting feature representations at different scales For input signals The formula for CWT is:

[0024] ;

[0025] in, This represents the feature representation at the m-th scale; the corresponding frequency information is extracted using CWT. And normalize the frequency:

[0026] ;

[0027] in, The normalized frequency weights.

[0028] As a preferred embodiment of the present invention, the specific processing procedure of the feature attention module FA is as follows:

[0029] Assume that the feature representations after FFT and CWT are both First, after passing through the feature attention module, the calculation is as follows:

[0030] ;

[0031] Where GAP and GMP represent global average pooling and global max pooling, respectively, two 1×1 convolutions are then performed to learn the importance of each feature. The dimensionality is first reduced and then increased, and the calculation is as follows:

[0032] ;

[0033] ReLU and Sigmoid are the activation functions. Finally, the results of global average pooling and max pooling are fused and weighted to adjust the input features, calculated as follows:

[0034] ;

[0035] in, This indicates point-by-point multiplication. This is the output after feature weighting.

[0036] As a preferred embodiment of the present invention, the specific processing procedure of the MSDC module is as follows:

[0037] Entering the MSDC module, the MSDC module captures local changes and performs the following calculations:

[0038] ;

[0039] in, Represents a 1×1 convolution. This represents the combined operation of BatchNorm and SiLU activation functions, i.e. , It's a multidimensional difference convolution. Output result. The system is divided into three parts based on the channel, each requiring different operations. The division formula is as follows:

[0040] ;

[0041] in Represents the number of channels, 0.4 As reserved The result of the convolution is 0.3. Assigned to 5×5 central difference convolution 0.3 Assigned to 7×7 central difference convolution The calculation formulas for each part are as follows:

[0042]

[0043] The three features are concatenated, and then... Further fusion and dimensionality reduction are performed, and the calculation is as follows:

[0044] .

[0045] As a preferred embodiment of the present invention, the specific processing procedure of the LCWD module is as follows:

[0046] Upon entering the LCWD module, the LCWD module captures long-range dependencies and calculates them as follows:

[0047] ;

[0048] in: and It is a strip-shaped central difference convolution used to capture long-range dependencies.

[0049] As a preferred embodiment of the present invention, the expression representing the second feature is:

[0050] ;

[0051] in, This indicates point-by-point multiplication, achieving a weighted mechanism.

[0052] Beneficial effects: 1. By combining Fast Fourier Transform (FFT) and Continuous Wavelet Transform (CWT), the decomposition level of time-frequency analysis is adjusted through an adaptive aggregation algorithm to comprehensively capture data features;

[0053] 2. A multi-scale feature enhancement learning module is designed, comprising feature attention, MSDC, and LCWD modules. The feature attention module adaptively adjusts feature weights to focus on key features; the MSDC module utilizes multi-scale differential convolution to extract local information from multiple directions and scales, keenly capturing weak anomalous signals; the LCWD module uses long strip convolution kernels to capture long-range dependencies; and through a weighted mechanism, weighted fusion of global and local features is achieved, effectively improving the model's classification performance.

[0054] 3. By employing multidimensional differential convolution, the local and boundary features of the signal are captured with keen sensitivity, improving the feature representation of the lesion area, enhancing the disease localization and diagnostic capabilities, and improving the overall performance of the model;

[0055] 4. The overall model design keeps the number of parameters and computation within a certain range, making it suitable for real-time medical monitoring and low-resource equipment. Attached Figure Description

[0056] Figure 1 This is a model architecture diagram of the present invention. Detailed Implementation

[0057] The present invention will be further described in conjunction with the embodiments and accompanying drawings.

[0058] In medical time-series data analysis, effectively integrating time-domain and frequency-domain information, dynamically focusing on key information, efficiently learning multi-scale features, and controlling computational complexity within a certain range are pressing issues. Therefore, this invention proposes a model architecture for joint time-frequency analysis and multi-scale feature enhancement learning in a classification task scenario for medical time-series data. The model architecture diagram is shown below. Figure 1 As shown, the architecture mainly consists of an encoder, an FWTBlock feature learning module, and a classification head. The encoder transforms the original time-series data into low-dimensional, high-information feature representations through value embedding, position embedding, and temporal feature embedding. These representations preserve the essential features, temporal order, and temporal dependencies of the data, providing rich information input for subsequent model learning. After processing by the FWTBlock feature learning module, the classification head maps the feature representations to the target class space through activation functions, Dropout layers, and fully connected layers, thus completing the classification task.

[0059] The proposed time-frequency joint analysis method is supported by the FWTBlock module. The FWTBlock module, as a feature learning module, performs FFT and CWT on the input signal after it has been processed by the encoder. FFT is used for frequency domain analysis to capture global and periodic features. First, the frequency components and their amplitudes of K major periods are extracted. Then, these frequency components are transformed into two-dimensional matrices according to the period length and passed as input to the multi-scale feature enhancement learning module. The processed K outputs are adaptively aggregated with their corresponding frequency amplitudes, with higher amplitude frequency components receiving higher weights, thus highlighting important periods. CWT is used for time domain analysis to capture local variations and anomalous fluctuations. First, M feature representations and their corresponding frequency information of the input signal at different scales are extracted using wavelet functions, and the frequency weights for each scale are calculated. The M feature representations are then passed to the multi-scale feature enhancement learning module for enhancement. The output results are adaptively aggregated according to the frequency weights, fusing feature information from different frequencies. Finally, the aggregated results of FFT and CWT are used as input to the classification head. To avoid losing key information during the weighting process and to accelerate model convergence, residual connections are introduced.

[0060] The multi-scale feature enhancement learning module of this invention accepts feature representations processed by FFT and CWT, respectively. This module consists of a feature attention module, an MSDC module, and an LCWD module. The feature representations first pass through the feature attention module, which adaptively adjusts the weights of each feature, enabling the model to automatically focus on features more important for the classification task. Specifically, it uses global average pooling and max pooling to obtain global information for each feature, then learns the importance of each feature through two 1×1 convolutions, and outputs weight values ​​ranging from 0 to 1 using the Sigmoid activation function. These weight values ​​are multiplied by the original input features for weighted adjustment. The output processed by the feature attention module then flows through the MSDC and LCWD modules. The MSDC module captures local variations in the data. Specifically, the input signal first undergoes 1×1 convolution for channel fusion and dimension mapping, followed by BatchNorm normalization and SiLU activation, then multidimensional difference convolution. The data is then proportionally divided into channels, with 5×5 and 7×7 central difference convolutions performed. The convolution results are concatenated proportionally with the original data, and finally, a 1×1 convolution is used for channel fusion and dimensionality reduction output. The LCWD module focuses on long-range dependencies of the signal. Specifically, it first performs channel fusion and dimension mapping using 1×1 convolution, then central difference convolutions using 1×11 and 11×1 kernels, followed by a 1×1 convolution. A Sigmoid activation function is used to obtain the weights of features under global dependencies. The output of the LCWD module is multiplied by the output of the MSDC module. Through a weighting mechanism, each feature dimension output by the MSDC module is influenced by global information, introducing residual connections to further enhance the expressive power of the features.

[0061] I. Time-Frequency Joint Analysis

[0062] 1. FFT:

[0063] FFT is used to extract frequency domain features of a signal, especially to capture periodic variations. The specific application steps are as follows:

[0064] Assuming the input signal Where T is the time step of the signal, and C is the number of channels, i.e., the number of time series features. The frequency domain signal is obtained by performing an FFT on the signal. This refers to the frequency domain representation. The calculation formula is as follows:

[0065] ;

[0066] in, It includes frequency components in complex form. The top K principal frequency components are extracted from the FFT result. and the corresponding amplitude The calculation formula is as follows:

[0067] ;

[0068] Frequency and period are reciprocals of each other, that is... Period length From frequency components The calculated extracted frequency components and period length When combined into a two-dimensional matrix, it will eventually form a shape of , The matrix is ​​represented as follows:

[0069] ;

[0070] Thus, the output matrix have and Two dimensions, representing period length and frequency, respectively. The frequency is then normalized:

[0071] ;

[0072] in, The normalized frequency weights.

[0073] 2. CWT:

[0074] CWT is used for time-domain analysis to capture local variations and anomalous fluctuations in signals. CWT utilizes wavelet functions. Extracting feature representations at different scales For the input signal The formula for CWT is:

[0075] ;

[0076] in, This represents the feature representation at the m-th scale. The corresponding frequency information is extracted using CWT. And normalize the frequency:

[0077] ;

[0078] in, The normalized frequency weights.

[0079] The features obtained after FFT processing are obtained through the above steps. Features after CWT processing These will be passed to the multi-scale feature enhancement learning module respectively, and outputs will be obtained respectively. and The fusion is performed through weighted aggregation, and residual connections are introduced. The final result is expressed as follows:

[0080] ;

[0081] II. Multi-scale Feature Reinforcement Learning

[0082] 1. Feature Attention Module:

[0083] Assume that the feature representations after FFT and CWT are both First, after passing through the feature attention module, the calculation is as follows:

[0084] ;

[0085] Where GAP and GMP represent global average pooling and global max pooling, respectively, two 1×1 convolutions are then performed to learn the importance of each feature. The dimensionality is first reduced and then increased, and the calculation is as follows:

[0086] ;

[0087] ReLU and Sigmoid are the activation functions. Finally, the results of global average pooling and max pooling are fused and weighted to adjust the input features, calculated as follows:

[0088] ;

[0089] in, This indicates point-by-point multiplication. This is the output after feature weighting.

[0090] Multidimensional difference convolution:

[0091] Unlike traditional convolution, this invention innovatively employs multidimensional differential convolution, specifically tailored to the characteristics of medical time-series data. Medical time-series data is a sequence of signals that varies over time, typically containing weak and transient anomalous signals. These local variations are often key clues for disease diagnosis. Differential convolution, by calculating the local gradient of the signal, more sensitively captures boundary features within the signal, making the feature representation of lesion areas clearer. This significantly improves disease localization and diagnostic capabilities, further enhancing the overall performance of the model.

[0092] The multidimensional differential convolutions (DEConv) in the MSDC module of the model include central differential convolution (CDConv), angular differential convolution (ADConv), horizontal differential convolution (VDConv), vertical differential convolution (HDConv), and ordinary convolution (VConv). Differential convolutions enhance the gradient-level information of the data by modifying the weights at different positions of the convolution kernel, while ordinary convolutions supplement the intensity-level information of the data. To reduce the number of parameters and inference time, the results of the five parallel convolutions are combined into a single standard convolution representation through convolution additivity. The specific description is as follows:

[0093] Let the input features be The five types of convolution are calculated as follows:

[0094] ;

[0095] in, and This represents the weights and biases corresponding to each type of convolution. This is a convolution operation. A linear combination of the five convolution results is:

[0096] ;

[0097] By merging the weight matrices and leveraging the additivity of convolutions, it is simplified to a single standard convolution:

[0098]

[0099] After simplification, The calculation still retains the effect of multidimensional difference convolution, but significantly reduces the number of parameters and computational cost.

[0100] 2. MSDC Module:

[0101] Upon entering MSDC, the MSDC module captures local changes and performs the following calculations:

[0102] ;

[0103] in, Represents a 1×1 convolution. This represents the combined operation of BatchNorm and SiLU activation functions, i.e. , It's a multidimensional difference convolution. Output result. The system is divided into three parts based on the channel, each requiring different operations. The division formula is as follows:

[0104] ;

[0105] in Represents the number of channels, 0.4 As reserved The result of the convolution is 0.3. Assigned to 5×5 central difference convolution 0.3 Assigned to 7×7 central difference convolution The calculation formulas for each part are as follows:

[0106] ;

[0107] The three features are concatenated, and then... Further fusion and dimensionality reduction are performed, and the calculation is as follows:

[0108] ;

[0109] 3. LCWD module:

[0110] The LCWD module captures long-range dependencies and calculates them as follows:

[0111] ;

[0112] in and It is a strip-shaped central difference convolution used to capture long-range dependencies.

[0113] Ultimately, the output of the LCWD module will be multiplied by the output of the MSDC module. That is, the output of the LCWD module will be multiplied by the base product. Provides global information for the output of the MSDC module. Point-by-point weighting is performed to combine global and local features. The specific calculation is as follows:

[0114] ;

[0115] in, This indicates point-by-point multiplication, achieving a weighted mechanism.

[0116] As a verification example of the present invention:

[0117] 1. Dataset Construction

[0118] Electrical impedance tomography (EIT) is a novel, non-invasive, radiation-free, and regional continuous lung ventilation monitoring technique that can be performed in real time at the bedside. With informed consent, the EIT dataset used in this invention was obtained from the Second Affiliated Hospital of Soochow University. The data was unlabeled, and we categorized it into five classes based on the GOLD standard. The specific dataset details are shown in Table 1.

[0119] Table 1. Dataset and Number of Categories

[0120]

[0121] 2. Evaluation Indicators

[0122] When evaluating the experimental performance of this invention, common metrics for classification tasks were selected: accuracy, macro-precision, macro-recall, macro-F1 score, macro-specificity, macro-AUC, and geometric mean (G-mean).

[0123] (1) Accuracy:

[0124] Accuracy measures the proportion of samples that correctly predict the outcome out of the total sample. The formula for calculating accuracy is as follows:

[0125] ;

[0126] Wherein, TP (True Positives) represents the number of samples predicted as positive and actually being positive. TN (True Negatives) represents the number of samples predicted as negative and actually being negative. FP (False Positives) represents the number of samples predicted as positive and actually being negative. FN (False Negatives) represents the number of samples predicted as negative and actually being positive.

[0127] (2) Macro-Precision:

[0128] Macro-average precision is a simple average of the precision for each class. Precision refers to the proportion of samples predicted as positive that are actually positive. The formula for calculating macro-average precision is as follows:

[0129] ;

[0130] in, Indicates the first The number of true positives in a class. Indicates the first The number of false positives in a class. This indicates the total number of categories.

[0131] (3) Macro-Recall:

[0132] Macro-average recall is a simple average of the recall rates for each class. Recall refers to the proportion of samples that were actually positive but were predicted to be positive. The formula for macro-average recall is as follows:

[0133] ;

[0134] in, Indicates the first The number of true positives in a class. Indicates the first The number of false negatives for a class. This indicates the total number of categories.

[0135] (4) Macro-average F1 score:

[0136] The macro-average F1 score is a simple average of the F1 scores for each class. The F1 score is the harmonic mean of precision and recall, representing the model's ability to balance precision and recall. The formula for calculating the macro-average F1 score is as follows:

[0137] ;

[0138] in, Indicates the first The accuracy of the class. Indicates the first Recall rate of the class. This indicates the total number of categories.

[0139] (5) Macro-specificity:

[0140] Macro-average specificity is a simple average of the specificity for each class. Specificity refers to the proportion of samples that are predicted to be negative out of all samples that are actually negative. The formula for calculating macro-average specificity is as follows:

[0141] ;

[0142] in, Indicates the first The number of true negatives of a class. Indicates the first The number of false positives in a class. This indicates the total number of categories.

[0143] (6) Macro-AUC:

[0144] AUC (Area Under the Curve) refers to the area under the ROC curve, representing the model's ability to distinguish between different classes. A higher AUC value indicates better model performance. The AUC ranges from 0 to 1, where 1 represents a perfect model and 0.5 indicates the model has no discriminatory power. Macro-AUC is obtained by simply averaging the AUC values ​​for each class. The formula for calculating macro-AUC is as follows:

[0145] ;

[0146] in, It is the first The AUC value of the class. This indicates the total number of categories.

[0147] (7) Geometric Mean (G-Mean):

[0148] The geometric mean (G-mean) is used to measure model performance, especially when dealing with class imbalance problems, as it reflects the balance performance of each class. The calculation formula is as follows:

[0149] ;

[0150] in, It is the first Class sensitivity (or recall). It is a very small constant (usually 10 ... ), used to prevent zero values ​​from occurring when taking the logarithm.

[0151] 3. Experimental Environment

[0152] The experimental environment is a fundamental condition for conducting experiments. The details of this experimental environment are as follows:

[0153] Table 2 Experimental Environment

[0154]

[0155] 4. Parameter Settings

[0156] Table 3 Experimental Settings

[0157]

[0158] 5. Experimental Results

[0159] As described in the preceding sections, this experiment uses the private dataset EIT. Table 4 shows that our model demonstrates superiority over several state-of-the-art methods, including Pyraformer (ICLR'22), TimesNet (ICLR'23), Crossformer (ICLR'23), Reformer (ICLR'20), Transformer (NeurIPS'17), ITransformer (ICLR'24), Informer (AAAI'21), LightTS (arXiv'22), MICN (ICLR'23), and DLinear (AAAI'23). These methods primarily include those based on CNNs and Transformers.

[0160] Furthermore, we compared the number of parameters and computational cost of our model, as shown in Table 5. Although our model does not have an advantage in the number of parameters compared to other models, we successfully kept the computational cost at a low level and demonstrated superior classification performance, showcasing a high balance between efficiency and performance. This improvement is of great significance in practical applications.

[0161] Table 4 Comparison of Seven Evaluation Indicators

[0162]

[0163] Table 5 Comparison of Model Parameter Quantity and Computational Cost

[0164]

[0165] 6. Other materials

[0166] Algorithm Complexity Analysis: Computational complexity is an important factor to consider when evaluating the performance of this invention. This method effectively controls computational complexity through reasonable algorithm design while integrating time-frequency joint analysis and multi-scale feature enhancement learning methods.

[0167] For the FFT part, its computational complexity mainly depends on the length of the input signal. Assuming the input signal length is N, the time complexity of FFT is typically O(NlogN). In this invention, by extracting the first K main frequency components, the amount of data processed subsequently is reduced to some extent, thereby alleviating the computational burden. CWT has relatively high computational complexity because it requires analysis of the signal at different scales. However, this invention reduces unnecessary computational overhead without losing key information by performing normalization and adaptive aggregation on the extracted features. In the multi-scale feature enhancement learning module, the feature attention module mainly involves global average pooling, max pooling, and convolution operations. The computational complexity of these operations is relatively low, mainly depending on the sequence length and feature dimension. Although the multidimensional differential convolution in the MSDC module uses various types of convolution operations, it merges the results of five parallel convolutions into a single standard convolution representation through convolution additivity, greatly reducing the number of parameters and computational complexity. Its computational complexity has certain advantages compared to traditional convolution, enabling fast and effective extraction of local features when processing medical time-series data. In the LCWD module, a long strip convolutional kernel is used to capture long-range dependencies. By reasonably setting the kernel size and stride, sufficient long-range information can be obtained while avoiding excessive computation. Its computational complexity is related to the size of the input features and the parameters of the convolutional kernel, and is generally within an acceptable range.

[0168] In summary, through a series of algorithm optimizations and module designs, this invention effectively controls the algorithm complexity while ensuring high classification performance, making it a promising candidate for application in real-time medical monitoring and low-resource devices.

[0169] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A model architecture of time-frequency joint analysis and multi-scale feature enhancement learning, characterized in that: The model architecture comprises an encoder, a feature learning module and a classification head, the encoder converts input raw time series data into a first feature representation through value embedding, position embedding and time feature embedding, outputs a second feature representation after processing by the feature learning module, and the classification head maps the second feature representation to a target category space through an activation function, a Dropout layer and a fully connected layer to realize a classification task; wherein: The feature learning module comprises a fast Fourier transform module, a continuous wavelet transform module and two multi-scale feature enhancement learning modules, the fast Fourier transform module performs frequency domain analysis on the first feature representation, captures global and periodic features, and passes them to a multi-scale feature enhancement learning module; the continuous wavelet transform module performs time domain analysis on the first feature representation, captures local changes and abnormal fluctuation features, and passes them to another multi-scale feature enhancement learning module; each multi-scale feature enhancement learning module comprises a feature attention module, an MSDC module and an LCWD module, the feature attention module adjusts the weight of each input feature adaptively, so that the model can automatically focus on high-weight features; the MSDC module extracts local information from high-weight features using multi-dimensional difference convolution to capture abnormal signals; the LCWD module uses a long strip convolution kernel to capture long-range dependencies from high-weight features to express lesion region features; the global and local features are weighted and fused by point multiplication of the lesion region features and the abnormal signals to form the second feature representation; wherein: the multi-dimensional difference convolution captures boundary features in the signal by calculating the local gradient of the signal.

2. The model architecture of claim 1, wherein: The first feature representation comprises essential features, time order and time dependence.

3. The model architecture of claim 2, wherein: Residual connections are introduced in the multi-scale feature enhancement learning module to improve the expression ability of features.

4. The model architecture of claim 3, wherein: The specific processing process of the fast Fourier transform module FFT is as follows: Assume the input signal where T is the time step of the signal, and C is the number of channels, i.e., the number of time series features. performing an FFT on the signal to obtain a frequency domain signal , i.e. a frequency domain representation; The calculation formula is as follows: ； wherein, contains the frequency components of the form of the retest; the first K main frequency components are extracted from the FFT result and the corresponding amplitude ; the calculation formula is as follows: ； The frequency and the period are reciprocal, i.e. The length of the period is calculated from the frequency component The extracted frequency component and the length of the period are combined into a two-dimensional matrix, which ultimately forms a matrix of the shape , The representation is as follows: ； Output matrix Having And Two dimensions, representing period length and frequency, respectively; and normalizing the frequency: ； wherein, is the normalized frequency weight.

5. The model architecture of claim 4, wherein: The specific processing process of the continuous wavelet transform module CWT is as follows: CWT by wavelet functions extracting feature representations at different scales ; for an input signal The formula for the CWT is: ； wherein, denotes the feature representation at the mth scale; through CWT, the corresponding frequency information is extracted and the frequency is normalized: ； wherein, is the normalized frequency weight.

6. The model architecture of claim 5, wherein: The output result of the multi-scale feature enhancement learning module is as follows: 。 7. The model architecture of claim 6, wherein: The specific processing process of the feature attention module FA is as follows: Let the feature representation after FFT and CWT processing be First, pass through the feature attention module, and calculate as follows: ； Wherein GAP and GMP represent global average pooling and global maximum pooling respectively, followed by two 1x1 convolutions to learn the importance of each feature, and then dimension reduction and dimension increase, the calculation is as follows: ； Wherein, ReLU and Sigmoid are activation functions, and the results of global average pooling and maximum pooling are finally fused to weight the input features, the calculation is as follows: ； wherein, denotes point-wise multiplication, is the output after feature weighting.

8. The model architecture of claim 7, wherein: The specific processing process of the MSDC module is as follows: Into the MSDC module, the MSDC module captures local changes, calculates as follows: ； where, represents 1 x 1 convolution, represents the combined operation of BatchNorm and SiLU activation function, i.e. , is a multi-dimensional differential convolution, and the output result is divided into three parts by channel, and different operations are performed respectively, and the division formula is as follows: ； where represents the number of channels, 0.4 as reserved as a result of convolution, 0.3 assigned to 5x5 center difference convolution , 0.3 assigned to 7x7 center difference convolution , the formula for each part is as follows: ； The three parts of features are spliced, and the spliced features are calculated by Further fusion dimension reduction is calculated as follows: 。 9. The model architecture of claim 8, wherein: The specific processing process of the LCWD module is as follows: Into the LCWD module, the LCWD module captures long-range dependencies, computes as follows: ； wherein: and are centred-difference convolutions in strips to capture long-range dependencies.

10. The model architecture of claim 9, wherein: The expression of the second feature representation is as follows: ； wherein, represents point-wise multiplication, implementing the weighting mechanism.

Images