A medical time series classification method based on time-aware multiple-instance learning

Abstract

The application discloses a medical time sequence classification method based on time-aware multi-instance learning, and belongs to the technical field of signal processing. The method comprises the following steps: acquiring a medical time sequence collected by a wearable sensor; and inputting the medical time sequence into a medical time sequence classification model, wherein the medical time sequence classification model comprises a time-aware module, a multi-granularity block embedding module, a multi-granularity attention module and a classification output unit. The method is based on time sequence input, sequentially processed through the time-aware module, the multi-granularity block embedding module and the multi-granularity attention module, and finally outputs a classification label result, so that automatic evaluation of Parkinson's disease is realized.

Description

Technical Field

[0001] This invention relates to the field of signal processing technology, and specifically to a medical time series classification method based on time-aware multi-instance learning. Background Technology

[0002] Parkinson's disease is a common neurodegenerative disorder, and its clinical manifestations typically include motor and non-motor symptoms such as tremor, bradykinesia, rigidity, postural and gait abnormalities, and sleep disturbances. Because Parkinson's disease is a long-term progressive disease, early identification, continuous monitoring, and assessment of disease severity are crucial for assisting clinical diagnosis, intervention decisions, and disease management.

[0003] Traditional Parkinson's disease assessment primarily relies on clinical scales, physician observation, and periodic outpatient examinations. These methods suffer from high subjectivity, low monitoring frequency, and difficulty in reflecting changes in movement during daily life. In recent years, with the development of wearable sensors, collecting multi-channel medical time-series data such as triaxial acceleration and triaxial angular velocity using wrist-worn devices has become an important technological direction for auxiliary assessment of Parkinson's disease. This type of data can continuously record changes in movement during daily activities such as walking and drinking, providing an objective data basis for assessing the severity of Parkinson's disease.

[0004] In the field of medical time series analysis, Transformer-based models have been widely applied to tasks such as time series forecasting, classification, and anomaly detection. Existing models typically divide the time series into different block representations and utilize attention mechanisms to learn temporal dependencies. For example, existing techniques such as Autoformer (Wu H, Xu J, Wang J, et al. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting[J]. Advances in neural information processing systems, 2021, 34: 22419-22430.) are mainly aimed at long-term time series forecasting tasks, modeling temporal changes through decomposition structures or autocorrelation mechanisms. Existing technology 2: iTransformer (Liu Y, Hu T, Zhang H, et al. itransformer: Inverted transformers are effective for time series forecasting[C] / / International conference on learning representations. 2024,2024: 11116-11140.) treats the complete time series of a single channel as a block representation to highlight the relationship between channel dimensions. Existing technology 3: PatchTST (Nie Y, Nguyen NH, Sinthong P, et al. A time series is worth 64 words: Long-term forecasting with transformers[J]. arXiv preprintarXiv:2211.14730, 2022.) divides a single-channel time series into several time segments and performs attention modeling based on the segments.

[0005] However, the aforementioned existing technologies still have the following shortcomings when applied to wearable medical time series classification, especially in assessing the severity of Parkinson's disease: First, most existing methods are designed with time series prediction as their main goal. When directly applied to medical time series classification, they are difficult to fully adapt to the characteristics of sparse, localized, and significantly individual-differentiated symptom signals in disease assessment scenarios.

[0006] Second, some methods use individual cross-channel time points as block representations. Although they can preserve instantaneous channel information, they easily ignore coarse-grained time structures over longer time ranges, making it difficult to effectively express the periodic changes in daily actions such as walking and drinking.

[0007] Third, some methods use the complete sequence of a single channel as a block representation. While this is beneficial for modeling the overall relationship between channels, it can easily obscure fine-grained temporal dynamics, leading to insufficient expression of local symptom evidence.

[0008] Fourth, some block-based time series models only perform block division and modeling within a single channel, failing to integrate information from multiple sensor channels simultaneously in each block. Therefore, they are insufficient in modeling the coordinated changes between multiple medical time series such as acceleration and angular velocity.

[0009] Fifth, existing Transformer-type time series models typically employ a single-granularity, block-based representation, making it difficult to simultaneously characterize symptom features at different time granularities. Since Parkinson's disease-related motor manifestations may be distributed across different time scales and frequency bands, using only a single granularity can easily lead to the loss of symptom-specific information.

[0010] Sixth, evidence of clinically relevant symptoms is often sparse and localized, and the most discriminative time segments usually lack fine annotation. Existing fully supervised classification methods typically classify the entire time series as a whole, making it difficult to effectively aggregate symptom evidence from key local segments, and also difficult to model the correlation between different symptom evidence segments.

[0011] Therefore, existing technologies still lack an effective method for time series classification in wearable medicine that can simultaneously achieve periodic feature modeling, multi-granularity time feature expression, cross-channel dependency modeling, and relevant multi-instance aggregation of local symptom evidence to improve the accuracy and robustness of Parkinson's disease severity assessment. Summary of the Invention

[0012] The purpose of this invention is to overcome the shortcomings of the prior art and provide a medical time series classification method based on time-aware multi-instance learning for the classification of medical time series data collected by wearable sensors, which is particularly suitable for application scenarios such as Parkinson's disease severity assessment, Parkinson's disease assisted screening, and home continuous monitoring.

[0013] To achieve the above objectives, the technical solution of the present invention is as follows: A medical time series classification method based on time-aware multi-instance learning, the method comprising: Acquire medical time series data collected by wearable sensors; The medical time series is input into a medical time series classification model, which includes a time-aware module, a multi-granularity block embedding module, a multi-granularity attention module, and a classification output unit. The time-aware module generates an enhanced time-series representation based on the input medical time series; The multi-granularity block embedding module divides and embeds the time series enhanced representation across channels according to multiple time granularities to obtain block embedding representations and corresponding routing tags at multiple granularities. The multi-granularity attention module obtains the fused medical time series representation based on the block embedding representation and the corresponding routing tags; The classification output unit outputs classification labels based on the fused medical time series representation.

[0014] Optionally, the time-aware module generates an enhanced time-series representation based on the input medical time series, including: The time-aware module extends the one-dimensional time changes in the medical time series into a multi-period two-dimensional representation, and performs two-dimensional feature extraction, one-dimensional reconstruction and adaptive aggregation on the multi-period two-dimensional representation to generate an enhanced time series representation.

[0015] Optionally, the process further includes: before inputting the medical time series data into the medical time series classification model. The medical time series is then filtered, normalized, segmented, and padded.

[0016] Optionally, the time-aware module is configured with M layers, and the value of M is set according to the actual task; each layer of the time-aware module takes the time series enhancement representation output by the previous layer as input and outputs a new time series enhancement representation, which is then used as the input of the next layer.

[0017] Optionally, the step of dividing and embedding the time series augmentation representation across channels according to multiple time granularities to obtain block-embedded representations at multiple granularities includes: Set the block length set as ;in, Indicates the number of particles. Indicates the first The block length corresponding to each granularity; for each granularity The time series augmentation representation is padded along the time axis to increase the time length. able to be Divisible; dividing the time series into Non-overlapping windows: Each window contains all of them simultaneously. Data from each channel; Indicates the number of channels; For the At each granularity level, each cross-channel block is flattened into a vector and converted into a block embedding representation of uniform dimension through a linear mapping. Position and granularity identifiers are then added to the block embedding representation to obtain the _th_ granularity. The insertion location identifier and the block embedding representation after the granularity identifier are as follows:

[0018] in, Indicates location identification information; Indicates the first Granularity identification information for each granularity; Indicates the first The number of blocks at each granularity, where D represents the dimension of the embedded token in each block.

[0019] Optionally, a routing tag can be constructed for each granularity:

[0020] in, Indicates the first Each granularity corresponds to a routing tag. Used to provide granularity for routing tags The number of blocks below, Provide granular identity information to make different route tags distinguishable; Get the set corresponding to the block length of and .

[0021] The optional multi-granularity attention module includes intra-granularity attention and inter-granularity attention; The intra-granularity attention is used to model the relationships between blocks within the same granularity at the same time. The intergranularity attention is used to model the relationship between different time granularities to obtain a fused medical time series representation.

[0022] Optional intra-granularity attention is used to model the relationships between blocks within the same time granularity, including: For granularity That is, block length , granularity The block embedding representation after adding position identifiers and granularity identifiers With the corresponding route tag To splice together, form ; in, This represents the combined sequence after chunk embedding and routing tagging; [] indicates that the concatenated result is treated as a whole. This indicates concatenation along the dimension of token quantity; right Perform granular intragranular attention computation while updating the block embedding representation and route tags.

[0023] Optionally, the inter-granularity attention is used to model the relationship between different time granularities, including: After completing the intra-granularity attention calculation, the route tag corresponding to each granularity already contains the global information for that granularity; collect the route tags corresponding to all granularities as follows:

[0024] Where U represents the routing matrix formed by concatenating routing tags of all granularities; This represents the routing tag for the first granularity; This represents the routing tag at the nth granularity. Each routing tag is made to pay attention to all routing tags, and inter-granular attention calculation is performed to obtain the fused medical time series representation.

[0025] Optionally, the classification output unit includes a fully connected layer and a normalized classification function, used to output the probability values ​​of each category, and output the category with the highest probability value as the classification label.

[0026] Compared with existing medical time series classification methods, this invention achieves better classification accuracy, generalization ability, and symptom feature expression ability in the medical time series classification task of Parkinson's disease through the collaborative design of a time-aware module, a multi-granularity block embedding module, and a multi-granularity attention module; its beneficial effects are mainly reflected in the following aspects.

[0027] 1. Capable of explicitly modeling periodic features in medical time series. Parkinson's disease patients often exhibit obvious periodic movement patterns in daily activities such as walking and drinking. However, traditional one-dimensional time series modeling methods mainly focus on changes between adjacent time points, making it difficult to explicitly express intra-period and inter-period changes.

[0028] This invention employs a time-aware module to extract dominant periods from time series data and transforms one-dimensional time series into a multi-period two-dimensional representation. This allows the model to simultaneously learn local variations within periods and alignment variations between periods. This design more fully expresses periodic cues in Parkinson's disease-related motor abnormalities, thereby improving classification performance.

[0029] The ablation experiment results show that, on the PADS dataset corresponding to drinking activity data from consumer-grade wrist-worn smartwatches, in a binary classification task between Parkinson's disease subjects and healthy subjects, using only the basic backbone network and using a basic Transformer encoder classification network without the time-aware module, multi-granularity block embedding module, and multi-granularity attention module as a control model, the accuracy, macro-mean precision, macro-mean recall, and macro-mean F1 score were 69.27%, 54.68%, 54.18%, and 53.42%, respectively. After adding the time-aware module, the four indicators improved to 79.15%, 70.92%, 65.83%, and 66.90%, with the macro-mean F1 score improving by 13.48%. This indicates that the time-aware module has a significant enhancing effect on the periodic characteristics in the medical time series of Parkinson's disease.

[0030] 2. Capable of capturing symptom-related features across multiple time scales. Parkinson's disease-related motor abnormalities can be distributed across different time scales, such as short-duration tremor, moderate-duration bradykinesia, and long-duration rhythmic abnormalities. Existing single-granularity segmentation methods are insufficient to simultaneously cover multiple time scales, easily leading to the loss of symptom-specific information.

[0031] This invention utilizes a multi-granularity block embedding module to segment and embed time-series augmented representations at multiple granularities, enabling the model to extract features across different time window lengths. Segmentation at different granularities is equivalent to observing medical time series from multiple time scales, helping to reveal symptom-related clues across different time bands.

[0032] Experimental results show that on the PADS dataset, after adding a multi-granularity block embedding module on top of the time-aware module, the macro-average recall increased from 65.83% to 67.81%, and the macro-average F1 score increased from 66.90% to 68.68%. This indicates that the multi-granularity block embedding module can enhance the model's sensitivity to evidence related to Parkinson's disease.

[0033] 3. Capable of modeling intra-granular and inter-granular relationships. Existing methods typically model time series segments at a single granularity, making it difficult to effectively represent the relationships between different time granularities. For Parkinson's disease assessment, symptom presentations at different time scales are not isolated but may be interconnected.

[0034] This invention employs a multi-granularity attention module, setting up intra-granularity attention and inter-granularity attention respectively. Intra-granularity attention is used to learn the relationships between blocks at the same temporal granularity, while inter-granularity attention is used to learn the interaction relationships between representations at different temporal granularities. Therefore, the model can not only capture local dynamics within a single scale but also model cross-scale symptom relationships.

[0035] Experimental results show that, on the PADS dataset, the macro-average precision increased from 70.07% to 72.23% after adding a multi-granularity attention module, and the macro-average F1 score reached 69.37%. This indicates that modeling intra-granular and inter-granular relationships helps reduce misclassifications caused by pseudo-local fluctuations and improves the reliability of classification results.

[0036] 4. Applicable to different types of wrist-worn wearable sensors This invention can directly process medical time-series data, such as triaxial acceleration signals and triaxial angular velocity signals, collected by wrist-worn wearable sensors, without relying on expensive medical imaging equipment or complex clinical testing devices. Therefore, this invention is suitable for application in consumer-grade wrist-worn smartwatches, research-grade high-precision wrist-worn sensors, mobile terminals, or remote health monitoring platforms.

[0037] In experimental verification, this invention uses two types of wrist-worn wearable sensor data for testing: one type is data collected by a consumer-grade wrist-worn smartwatch, corresponding to the PADS dataset (see reference 1: Varghese J, Brenner A, Fujarski M, et al. Machine Learning in the Parkinson's disease smartwatch (PADS) dataset[J]. npj Parkinson's Disease, 2024, 10(1): 9.). The other type is data collected by a research-grade high-precision wrist-worn sensor, corresponding to the Shimmer dataset (see reference 2: Wang X, Peng X, Xu Z, et al. PDWearML: Leveraging Daily Activities for Fast Parkinson's Disease Severity Assessment with Wearable Machine Learning[J]. IEEE Transactions on Biomedical Engineering, 2025.). Both contain six channels for triaxial acceleration and triaxial angular velocity, but differ in sampling rate, sensor accuracy, and time series length. Experimental results show that the present invention can achieve good classification performance under different equipment types and different sampling conditions, demonstrating strong equipment adaptability and generalization ability.

[0038] In the PADS water-drinking activity binary classification task between Parkinson's disease subjects and healthy subjects, the accuracy, macro-average precision, macro-average recall, and macro-average F1 score of this invention reached 79.74%, 72.23%, 68.40%, and 69.37%, respectively. Compared with the best values ​​among 10 strong baseline methods such as Autoformer, PatchTST, and iTransformer, the accuracy was improved by 6.79%, and the macro-average F1 score was improved by 5.68%.

[0039] In the Shimmer water-drinking activity's binary classification task between Parkinson's disease subjects and healthy subjects, the accuracy, macro-average precision, macro-average recall, and macro-average F1 score of this invention reached 96.83%, 97.20%, 95.16%, and 95.94%, respectively. Compared with the best values ​​among 10 best baseline methods such as Autoformer, PatchTST, and iTransformer, the accuracy is improved by 16.44%, and the macro-average F1 score is improved by 21.98%.

[0040] The above results show that the present invention is applicable not only to medical time series collected by consumer-grade wrist-worn smartwatches, but also to medical time series collected by research-grade high-precision wrist-worn sensors, demonstrating good cross-device adaptability and practical application value.

[0041] 5. Capable of adapting to medical time-series scenarios where symptom evidence is sparse and localized. Evidence of Parkinson's disease-related symptoms is not typically evenly distributed across the entire time series, but rather tends to concentrate in a few key time segments. Traditional holistic classification methods are easily influenced by numerous non-critical segments, making it difficult to highlight truly significant local evidence.

[0042] This invention employs a multi-instance learning modeling approach. First, in the time-aware module, the input medical time series undergoes multi-period two-dimensional expansion, two-dimensional feature extraction, one-dimensional reconstruction, and adaptive aggregation to generate an enhanced time series representation. Then, in the multi-granularity block embedding module, the enhanced time series representation is treated as a package and divided into cross-channel time blocks according to multiple time granularities, with each cross-channel time block serving as an in-package instance. Further, in the multi-granularity attention module, the relationship between in-package instances at the same time granularity is modeled through intra-granularity attention, and instance-level symptom evidence at that granularity is aggregated through routing labels. Finally, inter-granularity attention fuses routing labels from different time granularities to obtain a package-level medical time series representation, which is then used for classification output.

[0043] 6. It can be put into practical application based on low-cost wrist-worn devices. This invention enables Parkinson's disease-related classification based on medical time-series data collected by a wrist-worn sensor, without relying on large medical equipment or complex clinical testing devices. Especially in the consumer-grade wrist-worn smartwatch scenario, this invention can utilize triaxial acceleration and triaxial angular velocity data collected by low-cost devices for classification analysis, making it suitable for scenarios such as Parkinson's disease assisted screening, home monitoring, disease follow-up, and rehabilitation assessment.

[0044] Furthermore, this invention can also be extended to data acquired by research-grade high-precision wrist-worn sensors for clinical research, model validation, and severity assessment. Therefore, this invention combines low-cost application value with high-precision research value.

[0045] In summary, the present invention has at least the following beneficial effects: First, the ability to express periodic features in medical time series is enhanced through the time-aware module; Second, the multi-granularity block embedding module improves the model's ability to capture symptom features at different time scales; Third, joint modeling of intra-granularity and inter-granularity relationships was achieved through a multi-granularity attention module; Fourth, the multi-instance learning approach improves the model's ability to aggregate sparse and localized symptom evidence; Fifth, this invention achieves classification results superior to existing baseline methods on both consumer-grade smartwatch data and research-grade high-precision wrist-worn sensor data, demonstrating better accuracy, generalization ability, and cross-device adaptability. Sixth, this invention enables medical time series classification based on a low-cost wrist-worn sensor, and is applicable to practical application scenarios such as Parkinson's disease screening, home monitoring, assisted assessment, and long-term follow-up. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the overall process of the medical time series classification method based on time-aware multi-instance learning proposed in this application. Figure 2 Diagram of the time-aware Transformer model architecture; Figure 3 This is a schematic diagram illustrating the working principle of the time-sensing module. Figure 4 This is a schematic diagram illustrating the working principle of the multi-granularity block embedding module; Figure 5 This is a schematic diagram illustrating the working principle of the multi-granularity attention module. Detailed Implementation

[0047] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] See Figure 1 As shown, the medical time series classification method based on time-aware multi-instance learning provided in this application mainly includes the following steps: Acquire medical time series data collected by wearable sensors.

[0049] The medical time series is input into a medical time series classification model, which includes a time-aware module, a multi-granularity block embedding module, a multi-granularity attention module, and a classification output unit; for example... Figure 2 As shown, the time-aware Transformer model framework consists of a time-aware module, a multi-granularity block embedding module, and an opinion multi-granularity attention module.

[0050] The time-aware module generates an enhanced time-series representation based on the input medical time series. Thus, addressing the difficulty of existing medical time-series models in explicitly representing periodic motion changes, this application establishes a time-aware module that extracts the dominant period from the time series, converting the one-dimensional medical time series into a multi-period two-dimensional representation, and further generating an enhanced time-series representation. This enables the model to simultaneously express intra-period changes and inter-period changes, enhancing its ability to model Parkinson's disease-related movement patterns in periodic daily activities such as walking and drinking.

[0051] The multi-granularity block embedding module divides and embeds the time series augmented representation across channels according to multiple time granularities, obtaining block embedding representations and corresponding routing tags at multiple granularities. Thus, addressing the problem that existing methods often employ single-granularity block representation, making it difficult to characterize symptom features at different time scales, this application sets up a multi-granularity block embedding module to divide and embed the time series augmented representation at multiple granularities. Through block representations at different time granularities, this application can capture symptom-related features at multiple time scales, thereby improving the ability to express motion anomaly patterns at different frequency bands and durations. Simultaneously, addressing the problem that existing block methods are insufficient in modeling the dependencies between multi-channel sensor data, this application adopts a cross-channel block method in the multi-granularity block process (each cross-channel time block can be considered as an in-packet example), so that each block simultaneously contains information from multiple sensor channels. This enables the fusion of multi-channel motion signals such as triaxial acceleration and triaxial angular velocity at the block level, enhancing the ability to model the cooperative changes between channels.

[0052] The multi-granularity attention module obtains the fused medical time series representation based on the block embedding representation and the corresponding routing tags. Thus, intra-granularity attention models the relationships between different block examples at the same granularity, and the routing tags aggregate example-level symptom evidence at that granularity; subsequently, inter-granularity attention fuses routing tags of different granularities to form a packet-level medical time series representation.

[0053] The classification output unit outputs classification labels based on the fused medical time series representation.

[0054] Therefore, this method employs a multi-instance learning modeling approach. First, in the time-aware module, the input medical time series undergoes multi-period two-dimensional expansion, two-dimensional feature extraction, one-dimensional reconstruction, and adaptive aggregation to generate an enhanced time series representation. Subsequently, in the multi-granularity block embedding module, the enhanced time series representation is treated as a package and divided into cross-channel time blocks according to multiple time granularities, with each cross-channel time block serving as an in-package instance. Further, in the multi-granularity attention module, the relationship between in-package instances at the same time granularity is modeled through intra-granularity attention, and instance-level symptom evidence at that granularity is aggregated through routing labels. Then, inter-granularity attention fuses routing labels from different time granularities to obtain a package-level medical time series representation, which is used for classification output, thereby achieving automatic assessment of Parkinson's disease.

[0055] Specifically, such as Figure 3 As shown, this time-aware module extends the one-dimensional time variation in medical time series into a multi-period two-dimensional representation, thus explicitly expressing intra-period variations and inter-period variations. Each time point simultaneously presents two types of time variations: intra-period variations related to its local neighborhood, and inter-period variations aligned with the same phase position in different periods. However, the original one-dimensional organization of time series mainly reflects the variations between adjacent time points, leaving cross-period and phase-aligned variations implicit. To overcome this limitation, this application extends time variations into a two-dimensional representation, thereby explicitly decoupling intra-period variations and inter-period variations. After feature extraction, the two-dimensional representation is restored to one-dimensional as an enhanced time series representation, specifically including: First, for a length of ,Include A multivariate time series with variables, expressed in one-dimensional form as: To capture periodic variations, the channel average amplitude is first calculated using FFT. Subsequently, at the positive frequency Within the scope, top- The index corresponding to the peak value is selected as The dominant frequency, namely and each frequency Mapped to period length :

[0056] in, Represents the Fast Fourier Transform. Used to calculate the amplitude spectrum. Given Calculate the amplitude at each frequency, and in The average is calculated across the dimensions of each variable to obtain... . No. element Measure the average amplitude of the input medical time series at the j-th frequency component and the corresponding period length. By leveraging the sparsity of the frequency spectrum and to suppress pseudo-high-frequency noise, the search range is limited to positive frequencies. Inside, and select top- peak and its corresponding unnormalized amplitude ,in These frequencies are hyperparameters. These frequencies are further induced to yield the corresponding set of periods. For the sake of brevity, this process will be referred to as... Its return , and .

[0057] Given the selected dominant frequency and its corresponding period length , one-dimensional sequence Convert to a set of multi-period two-dimensional tensors:

[0058] in Zero-filling along the time dimension, Able to compatible, and Specify the number of rows and columns of the reshaped tensor, respectively. The resulting... Indicates by the first A two-dimensional representation defined by a period: the column direction corresponds to the representation within the period, i.e., the change within the period. The row direction, however, varies within the period length. The adjacent periods are aligned downwards, representing the changes during the period. From this, the time-aware module obtains a set of multi-period two-dimensional representations. This multi-period two-dimensional representation can simultaneously introduce two types of complementary localities: first, it models adjacent time points along the column direction to express local changes within the period; second, it models adjacent periods along the row direction to express changes during the period. Through this transformation, the periodic structure originally implicit in the one-dimensional sequence can be explicitly processed by the two-dimensional feature extraction structure.

[0059] Subsequently, the time-aware module performs two-dimensional feature extraction on the multiple two-dimensional periodic representations. In one specific implementation, a shared Inception module can be used to extract multi-scale two-dimensional features. Since the two-dimensional representations corresponding to different dominant periods share the same set of feature extraction parameters, the model size does not increase linearly with the increase of the number of dominant periods k. After completing the two-dimensional feature extraction, each two-dimensional feature is remapped back to a one-dimensional time series form, resulting in... Aligned one-dimensional features Finally, based on the magnitude of the amplitude corresponding to each dominant frequency, the above k one-dimensional periodic enhancement features are weighted and summed using a normalized weighting method through a softmax operation to form the output of the time-aware module, i.e., the time series enhancement representation.

[0060] In one embodiment, the present invention stacks M layers of time-aware modules in a residual manner, where M can be set according to the actual task. Each layer of time-aware modules takes the time series augmentation representation output by the previous layer as input and outputs a new time series augmentation representation, which is then used as input to the next layer.

[0061] Specifically, such as Figure 4 As shown, the multi-granularity block embedding module is used to divide the time series augmentation representation into blocks according to multiple time granularities and generate block embedding representations at different time scales.

[0062] Set the block length set as .in, Indicates the number of particles. Indicates the first The block length corresponding to each granularity. For each granularity First, the time series augmentation representation is padded along the time axis to make the time length... able to be Divisible. Subsequently, the time series is divided into... Non-overlapping windows: .

[0063] Each window contains all Data from multiple channels. In other words, this invention does not simply divide data into blocks within a single channel, but rather uses a cross-channel block method, so that each block simultaneously contains information from multiple sensor channels within that time window.

[0064] For the At a granularity, each cross-channel block is flattened into a vector and transformed into a block embedding representation of uniform dimension through a linear mapping. This process can be represented as: .in, Indicates the first Cross-channel block representation at each granularity This represents the corresponding embedding mapping parameters. This represents the initial block embedding result.

[0065] To distinguish between different time locations and different granularities, the multi-granularity block embedding module further adds location identifier information and granularity identifier information to the initial block embedding representation, resulting in the... Embedding representation at each granularity:

[0066] in, Indicates location identification information; Indicates the first Granularity identification information for each granularity; Indicates the first Cross-channel initial block embedding at each granularity; Indicates the first The number of blocks at each granularity, where D represents the dimension of the embedded token in each block.

[0067] Meanwhile, to reduce the complexity of subsequent attention calculations, this invention constructs a routing tag for each granularity:

[0068] in, .here, It is not location encoding. It is used to provide granularity for routing labels. The number of blocks below, and This provides granular identity information, making different route tags distinguishable. Ultimately, this yields the set corresponding to the chunk length. of and , Indicates the first Each granularity corresponds to a routing tag. This routing tag is used to summarize global information at the current granularity and participates in the modeling of inter-granularity relationships in subsequent multi-granularity attention modules.

[0069] By employing a multi-granularity block embedding module, this invention can obtain medical time series representations at different time scales. Smaller block lengths are advantageous for capturing short-term local changes, such as tremor fluctuations, while larger block lengths are advantageous for capturing changes in movement rhythm over longer time periods, such as bradykinesia.

[0070] Specifically, such as Figure 5 As shown, this multi-granularity attention module is used to model relationships between multi-granularity block embeddings. The module comprises two stages: intra-granularity attention and inter-granularity attention. Intra-granularity attention learns the relationships between blocks at the same temporal granularity, while inter-granularity attention learns the associations between representations at different temporal granularities, thereby enhancing the model's ability to express cross-granularity symptom relationships.

[0071] Directly embedding and concatenating all blocks at all granularities before performing global attention calculation results in high computational complexity and can easily obfuscate feature relationships across different time granularities. Therefore, a routing label is introduced to decompose self-attention modeling into two stages: intra-granularity attention and inter-granularity attention.

[0072] Intra-granular attention is used to model the relationships between blocks within the same granularity at the same time, including: For granularity That is, block length The insertion location identifier and the block embedding representation after the granularity identifier are added at this granularity. With the corresponding route tag To splice together, form in," " indicates a splicing operation.

[0073] Subsequently, Perform intragranular attention computation while updating the block embedding representation and route tags:

[0074]

[0075] Intra-granular attention can be implemented using scaled dot product attention. and Joint updates ensure that route tags remain a consistent summary at the current granularity and prepare for subsequent inter-granularity attention. This intra-granularity stage focuses on dependencies within a single scale to capture local temporal patterns and same-scale correlations.

[0076] Through this process, information can be exchanged between different blocks at the same granularity, and routing tags can aggregate global information at that granularity. In the Parkinson's disease assessment scenario, intragranular attention can capture the correlation between different time segments at the same time scale, such as the relationship between motion changes at different stages of the same drinking action, or the correlation between multiple local abnormal segments at the same time scale.

[0077] Inter-granularity attention is used to model the relationships between different time granularities, including: After completing the intra-granularity attention, the route tag corresponding to each granularity already contains global information for that granularity. The route tags corresponding to all granularities are collected as follows:

[0078] Then, each route tag is made to pay attention to all route tags, and inter-granular attention calculation is performed:

[0079] Through this process, information at different time granularities can be exchanged and fused. Because each Granularity was summarized after intragranular attention. Information, therefore for Executional attention enables information exchange and dependency modeling between different granularities. In Parkinson's disease assessment scenarios, different granularities can correspond to symptom manifestations at different time scales, such as short-term tremor, moderate-term bradykinesia, and long-term rhythmic abnormalities. Granularity-based attention can model cross-scale associations between these different symptom manifestations. Specifically, the classification output unit can include a fully connected layer and a normalized classification function to output probability values ​​for each category. Finally, the category with the highest probability value is output as the classification label.

[0080] The classification labels can be set to binary or multi-class classification depending on the specific task. For example, in a Parkinson's disease screening task, the classification labels may include healthy controls and Parkinson's disease; in a Parkinson's disease severity assessment task, the classification labels may include healthy controls, mild Parkinson's disease, moderate Parkinson's disease, and severe Parkinson's disease.

[0081] The implementation of the present invention will be described below with reference to specific embodiments. It should be understood that the following embodiments are only for illustrating the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Any scheme that, based on the technical concept of the present invention, makes equivalent substitutions or conventional adjustments to the data source, sensor type, time series length, block granularity, model parameters, or classification category should fall within the scope of protection of the present invention.

[0082] Example 1: A medical time series classification model for Parkinson's disease based on a wrist-worn sensor This embodiment uses medical time series data collected by a wrist-worn sensor as input to construct a binary classification model for Parkinson's disease medical time series data, used to distinguish between Parkinson's disease subjects and healthy subjects. Specifically, the binary classification task in this embodiment is a Parkinson's disease identification task, with classification labels including healthy controls and Parkinson's disease. The model output is used to determine whether the subject corresponding to the input medical time series is a Parkinson's disease subject, including the following steps: 1. Data Acquisition: Wrist motion signals were collected from the subjects during the drinking activity. The wrist motion signals included triaxial acceleration and triaxial angular velocity signals, totaling 6 channels.

[0083] In one specific implementation, a research-grade wrist-worn sensor was used to collect data on water intake activity. The sensor was worn on the subject's right wrist at a sampling rate of 200 Hz. The data included healthy controls and participants with Parkinson's disease. Each time-series sample was segmented into 4000-byte sequences.

[0084] In another specific implementation, a consumer-grade smartwatch was used to collect data. The device was worn on the subject's right wrist, with a sampling rate of 100 Hz. The time series corresponding to each drinking activity was 976 seconds long and also included six channels for triaxial acceleration and triaxial angular velocity.

[0085] Therefore, each medical time series sample can be represented as: X ∈ ℝ T×C Where T represents the time series length and C represents the number of channels. In this embodiment, C=6.

[0086] In the experimental verification of this embodiment, two publicly available wrist-worn medical time-series datasets were used to evaluate the model. The first is the Parkinson's Disease Smartwatch dataset, also known as the PADS dataset, which was collected by a consumer-grade wrist-worn smartwatch and contains triaxial acceleration and triaxial angular velocity signals during subjects' daily activities. The second is the Shimmer drinking activity dataset, which was collected by a research-grade wrist-worn Shimmer sensor and contains triaxial acceleration and triaxial angular velocity signals during subjects' drinking activities.

[0087] In this embodiment, both the PADS and Shimmer datasets are used for model evaluation using a binary classification task between Parkinson's disease subjects and healthy subjects. The classification labels include healthy controls and Parkinson's disease. The model input is a six-channel wrist movement time series during the subject's drinking activity, and the model output is the corresponding binary classification prediction label for the subject. The above datasets are only examples for experimental verification of this invention and are not intended to limit the scope of protection of this invention.

[0088] 2. Data Preprocessing The acquired raw medical time series data undergoes preprocessing to improve the stability of subsequent classification. The preprocessing steps include: Step one involves filtering the original triaxial acceleration and triaxial angular velocity signals. In one specific implementation, a third-order Butterworth bandpass filter with a frequency range of 0.3 Hz to 20 Hz is used to remove noise.

[0089] Step two involves standardizing the six channels for each subject. In one specific implementation, a normalization method based on the subject's z-score is used to ensure that the data from different channels for each subject have a relatively uniform numerical distribution.

[0090] Step 3: Perform fixed-length segmentation on the continuous time series. For research-grade sensor data, each drinking activity time series is segmented into a sequence segment of length 4000; for consumer-grade smartwatch data, each drinking activity time series is standardized into a sequence segment of length 976.

[0091] Step four: For sequences that are too short, zero-padding or boundary padding can be used to pad them. For sequences that are too long, truncation or sliding window methods can be used to obtain multiple time series samples. After the above processing, a standardized time series input is obtained.

[0092] 3. Model Setup 3.1 Time-aware module The preprocessed medical time series is input into the time-aware module. This module first performs frequency domain analysis on the input time series to extract several dominant periods. In one specific implementation, the number of dominant periods k is set to an integer between 2 and 20, and the M-layer stack is set to 1 or 2 layers.

[0093] For each dominant cycle, the time-aware module reconstructs the one-dimensional time series into a corresponding two-dimensional representation. One direction of this two-dimensional representation is used to represent changes within the cycle, and the other direction is used to represent changes during the cycle.

[0094] Subsequently, two-dimensional features are extracted from the two-dimensional representations corresponding to multiple periods, and the extracted features are remapped back to a one-dimensional time series form. Finally, weighted aggregation is performed based on the frequency intensity corresponding to each dominant period to obtain the time series enhanced representation.

[0095] Through this processing, the model can explicitly express the periodic motion changes in daily activities such as drinking and walking.

[0096] 3.2 Multi-granularity block embedding module The time-series augmented representation is input into a multi-granularity block embedding module. In one specific embodiment, multiple block lengths are set: {2, 4, 8, 16, 32, 64, 128, 256, 512}, each block length corresponding to a time granularity. For each time granularity, the time-series augmented representation is non-overlappingly divided along the time dimension. Each block simultaneously contains data from all six sensor channels. Subsequently, this cross-channel block is expanded into a vector and mapped to a block embedding representation of a unified dimension.

[0097] To distinguish between different time locations and different time granularities, location identification information and granularity identification information can be added to the block embedding representation.

[0098] Simultaneously, a corresponding route tag is generated for each time granularity. The route tag is used to summarize the overall information at that time granularity and for subsequent modeling of relationships between granularities.

[0099] 3.3 Multi-granularity attention module The multi-granularity block embedding representations output by the multi-granularity block embedding module are input to the multi-granularity attention module. The multi-granularity attention module includes intra-granularity attention and inter-granularity attention.

[0100] In the intra-granularity attention stage, for each temporal granularity, multiple blocks within that granularity are embedded and their corresponding routing tags are input into the attention computation unit. Through this process, information can be exchanged between different temporal blocks within the same granularity, and the routing tags can aggregate global information at that granularity.

[0101] In the intergranularity attention stage, routing tags corresponding to all granularities are combined, and attention calculations are performed between different routing tags. Through this process, information between different temporal granularities can interact, thereby modeling the relationships between symptom features at different time scales, such as short-term tremor, mid-term bradykinesia, and long-term rhythm abnormalities.

[0102] After processing by the multi-granularity attention module, a medical time series representation is obtained that integrates periodic information, multi-granularity features, cross-channel relationships, and cross-granularity relationships.

[0103] 3.4 Category Label Output The medical time series representation output by the multi-granularity attention module is input into the classification output unit.

[0104] In one specific implementation, the classification output unit includes a fully connected layer and a normalized classification function, used to output the probability value of each category. The category with the highest probability value is output as the final classification label.

[0105] In a binary classification task, the classification labels can include healthy controls and Parkinson's disease. In a multiclass classification task, the classification labels can include healthy controls, mild Parkinson's disease, moderate Parkinson's disease, and severe Parkinson's disease.

[0106] 4. Model Training: In one specific implementation, the model is trained and evaluated using participant-independent five-fold cross-validation. The specific steps are as follows: Step 1: Perform stratified balanced sampling on all subjects according to their participant numbers and divide them into five non-overlapping subsets to ensure that the data of the same subject does not appear in the training set, validation set, and test set at the same time, and try to keep the proportion of different categories of samples in each subset consistent.

[0107] Step 2: In the five-fold cross-validation, subjects from one subset are selected sequentially as the test set, and subjects from the remaining four subsets are selected as training candidate sets.

[0108] Step 3: Divide the subjects in the training candidate set into 20% by subject level as the validation set, and the remaining subjects as the training set. The validation set is used for early stopping judgment and hyperparameter selection of the model.

[0109] Step four: Train the model using the cross-entropy loss function, with the optimization objective being to make the predicted classification label consistent with the true classification label.

[0110] Step 5: Update the parameters using the Adam optimizer. In one specific implementation, the learning rate is set to 5 × 10⁻³.

[0111] Step six: The model is trained for a maximum of 300 training epochs, and the macro-average F1 score on the validation set is used as the basis for early stopping. In one specific implementation, the early stopping patience value is set to 50, that is, training stops when the macro-average F1 score on the validation set has not improved for 50 consecutive training epochs.

[0112] Step 7: Save the model parameters with the best macro-average F1 score on the validation set, and perform a final evaluation on the test set.

[0113] In one specific implementation, the model encoder layer is set to 1. For consumer-grade smartwatch data, the batch size is set to 64; for research-grade wrist-worn sensor data, the batch size is set to 16. A fixed random seed of 42 is used to improve the reproducibility of experimental results.

[0114] 5. Model Evaluation This embodiment uses the following metrics to evaluate the performance of medical time series classification: precision, macro-average precision, macro-average recall, and macro-average F1 score. Precision represents the overall percentage of correct classifications; macro-average precision represents the average precision of each class; macro-average recall represents the average recall of each class; and macro-average F1 score is used to comprehensively measure precision and recall. In five-fold cross-validation, the above evaluation metrics are calculated on the test set for each fold, and the average of the five-fold test results is reported.

[0115] For each dataset, report the average of the five-fold cross-validation results.

[0116] 6. Operating Environment In one implementation, the model is implemented using a deep learning framework, such as PyTorch. Training and testing can be performed on a server, workstation, or computing device equipped with a graphics processing unit (GPU).

[0117] A specific operating environment includes: an Intel Xeon series CPU; an NVIDIA GeForce RTX 4090 GPU; a PyTorch deep learning framework; an Adam optimizer; a maximum of 300 training epochs; and an early stopping patience value of 50.

[0118] The above operating environment is only one example. This invention can also be deployed in other servers, edge devices, mobile terminals or cloud platforms with corresponding computing capabilities.

[0119] Example 2: Home-based Parkinson's Disease Screening Method Based on Consumer Smartwatch Step 1: Subjects wore consumer-grade smartwatches (with built-in accelerometer and gyroscope sensors) to collect data on their drinking activities in a home environment.

[0120] Step two: The smartwatch collects triaxial acceleration and triaxial angular velocity signals to form a multi-channel medical time series.

[0121] Step 3: The mobile terminal or server filters, standardizes, segments, and fills the collected medical time series data to form a time series input that meets the model input requirements.

[0122] Step 4: Send the time series input to the model trained in Example 1.

[0123] Step 5: The model sequentially performs feature extraction and relationship modeling through the time-aware module, the multi-granularity block embedding module, and the multi-granularity attention module.

[0124] Step six: The classification output unit outputs classification labels, such as healthy control, suspected Parkinson's disease, mild Parkinson's disease, moderate Parkinson's disease, or severe Parkinson's disease.

[0125] Step 7: Display the classification results on mobile terminals, doctor workstations, or remote health management platforms for doctors or nurses to refer to.

[0126] Through this implementation method, the present invention can achieve medical time series classification related to Parkinson's disease based on a low-cost wrist-worn sensor without relying on large medical equipment, and is suitable for home screening, long-term follow-up and auxiliary assessment.

[0127] Example 3: A method for assessing the severity of Parkinson's disease based on research-grade sensors Step 1: Data on the subject's right wrist drinking activity was collected using a research-grade wrist-worn Shimmer device (with built-in accelerometer and gyroscope sensors) at a sampling rate of 200 Hz.

[0128] Step 2: Collect triaxial acceleration and triaxial angular velocity signals of the subject when completing the drinking task.

[0129] Step 3: Divide the acquired continuous time series into sequence segments of length 4000.

[0130] Step four: Perform third-order Butterworth bandpass filtering and z-score normalization on the data of each subject.

[0131] Step 5: Input the processed time series into the medical time series classification model trained in Example 1.

[0132] Step six: The model extracts the dominant period and generates an enhanced time series representation through the time-aware module.

[0133] Step 7: The model obtains cross-channel block embedding representations at multiple time granularities through the multi-granularity block embedding module.

[0134] Step 8: The model completes the modeling of intra-granular and inter-granular relationships through the multi-granularity attention module.

[0135] Step 9: The classification output unit outputs classification labels for the severity of Parkinson's disease.

[0136] Through this embodiment, the present invention can utilize high-precision wrist-worn sensor data to assess the severity of Parkinson's disease, and can be used for clinical research, model validation, and disease follow-up analysis.

[0137] The above embodiments demonstrate that the present invention explicitly models the periodic changes in medical time series through a time-aware module, extracts cross-channel block features at different time scales through a multi-granularity block embedding module, and models intra-granular and inter-granular relationships through a multi-granularity attention module, ultimately achieving medical time series classification.

[0138] This invention can be applied to both research-grade wrist-worn sensors and consumer-grade smartwatches; it can be used for binary classification between Parkinson's disease patients and healthy controls, and can also be extended to multi-classification tasks involving mild, moderate, and severe Parkinson's disease. Therefore, this invention has good feasibility, scalability, and practical application value.

[0139] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A medical time series classification method based on time-aware multi-instance learning, characterized in that, include: Acquire medical time series data collected by wearable sensors; The medical time series is input into a medical time series classification model, which includes a time-aware module, a multi-granularity block embedding module, a multi-granularity attention module, and a classification output unit. The time-aware module generates an enhanced time-series representation based on the input medical time series; The multi-granularity block embedding module divides and embeds the time series enhanced representation across channels according to multiple time granularities to obtain block embedding representations and corresponding routing tags at multiple granularities. The multi-granularity attention module obtains the fused medical time series representation based on the block embedding representation and the corresponding routing tags; The classification output unit outputs classification labels based on the fused medical time series representation.

2. The medical time series classification method based on time-aware multi-instance learning as described in claim 1, characterized in that, The time-aware module generates an enhanced time-series representation based on the input medical time series, including: The time-aware module extends the one-dimensional time changes in the medical time series into a multi-period two-dimensional representation, and performs two-dimensional feature extraction, one-dimensional reconstruction and adaptive aggregation on the multi-period two-dimensional representation to generate an enhanced time series representation.

3. The medical time series classification method based on time-aware multi-instance learning as described in claim 1, characterized in that, Before inputting the medical time series data into the medical time series classification model, the following steps are also included: The medical time series is then filtered, normalized, segmented, and padded.

4. The medical time series classification method based on time-aware multiple instance learning as described in claim 1 or 2, characterized in that, The time-aware module is configured with M layers, and the value of M is set according to the actual task. Each layer of the time-aware module takes the time series enhancement representation output by the previous layer as input and outputs a new time series enhancement representation, which is then used as the input of the next layer.

5. The medical time series classification method based on time-aware multiple instance learning as described in claim 1, characterized in that, The step of dividing and embedding the time series augmentation representation across channels according to multiple time granularities to obtain block-embedded representations at multiple granularities includes: Set the block length set as ;in, Indicates the number of particles. Indicates the first The block length corresponding to each granularity; for each granularity The time series augmentation representation is padded along the time axis to increase the time length. able to be Divisible; dividing the time series into Non-overlapping windows: ; Each window contains all Data from each channel; Indicates the number of channels; For the At each granularity level, each cross-channel block is flattened into a vector and converted into a block embedding representation of uniform dimension through a linear mapping. Position and granularity identifiers are then added to the block embedding representation to obtain the _th_ granularity. The insertion location identifier and the block embedding representation after the granularity identifier are as follows: ； in, Indicates location identification information; Indicates the first Granularity identification information for each granularity; Indicates the first Cross-channel initial block embedding at each granularity; Indicates the first The number of blocks at each granularity, where D represents the dimension of the embedded token in each block.

6. The medical time series classification method based on time-aware multiple instance learning as described in claim 5, characterized in that, Construct a routing tag for each granularity: ； in, Indicates the first Each granularity corresponds to a routing tag. Used to provide granularity for routing tags The number of blocks below, Provide granular identity information to make different route tags distinguishable; corresponding to the set of block lengths are obtained and .

7. The medical time series classification method based on time-aware multiple instance learning according to claim 6, wherein, The multi-granularity attention module includes intra-granularity attention and inter-granularity attention; The intra-granularity attention is used to model the relationships between blocks within the same granularity at the same time. The intergranularity attention is used to model the relationship between different time granularities to obtain a fused medical time series representation.

8. The medical time series classification method based on time-aware multiple instance learning according to claim 7, wherein, The intra-granularity attention is used to model the relationships between blocks within the same granularity at the same time, including: For granularity That is, block length , granularity The block embedding representation after adding position identifiers and granularity identifiers With the corresponding route tag To splice together, form ; wherein, represents the combined sequence after block embedding and routing marker; [] represents the result of concatenation as a whole; represents concatenation along the token number dimension; To Perform intra-granularity attention computation while updating chunked embedding representation and routing markers.

9. The medical time series classification method based on time-aware multiple instance learning according to claim 8, wherein, The inter-granularity attention is used to model the relationships between different time granularities, including: After completing the intra-granularity attention calculation, the route tag corresponding to each granularity already contains the global information for that granularity; collect the route tags corresponding to all granularities as follows: ； Wherein, U represents a routing matrix formed by splicing routing labels of all granularities; represents a routing label of the 1st granularity; represents a routing label of the n th granularity; Each routing tag is made to pay attention to all routing tags, and inter-granular attention calculation is performed to obtain the fused medical time series representation.

10. The medical time series classification method based on time-aware multiple instance learning of claim 1, wherein, The classification output unit includes a fully connected layer and a normalized classification function, which is used to output the probability values ​​of each category, and output the category with the highest probability value as the classification label.

Images