Construction method and application of cross-domain Parkinson tremor evaluation model

Abstract

The invention discloses a construction method and application of a cross-domain Parkinson tremor evaluation model, and belongs to the technical field of artificial intelligence. By minimizing total tremor classification loss and maximizing total domain classification loss, training a feature extraction module, a tremor classifier, a domain classifier and a weight distributor at the same time; after the training is completed, constructing a cross-domain Parkinson tremor evaluation model comprising a feature extraction module and a tremor classifier which are cascaded; wherein the total domain classification loss is the sum of the domain classification loss of the source domain and the domain classification loss of the target domain, and when the domain classification loss of different domains is calculated, after weights are distributed to training samples in each domain through a weight distributor, the domain classification loss, obtained based on a domain classifier, of each training sample is weighted on the sample level, and the domain classification loss of each training sample is calculated. Different data of Parkinson tremor sensors are fully utilized, alignment of feature spaces of a source domain and a target domain is promoted, and the constructed model can accurately evaluate Parkinson tremor.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and more specifically, relates to a construction method and application of a cross-domain Parkinson's tremor assessment model. Background Art

[0002] Parkinson's disease is a common progressive neurodegenerative disease. Tremor is one of the common symptoms caused by Parkinson's disease, and its severity usually indicates the progression of the disease. The clinical evaluation method for Parkinson's tremor is usually based on the MDS Unified Parkinson's Disease Rating Scale (MDS-UPDRS), which divides its integers into 0-4 levels (0 for normal and 4 for severe) according to the manifestation of the disease. However, this evaluation method has disadvantages such as subjectivity, time-consuming and lack of comprehensiveness, resulting in limited reliability of the evaluation results.

[0003] In recent years, traditional machine learning or deep learning methods have been widely used in Parkinson's tremor assessment. Regardless of the above methods, if user-related data partitioning is adopted, that is, the data of each subject is divided into a training set and a test set, a relatively good evaluation result can be obtained; if user-independent data partitioning is adopted, that is, the data of the same subject will not appear in the training set and the test set at the same time, the accuracy will be significantly reduced. The main reason for this phenomenon is that traditional machine learning or deep learning methods require the training set and the test set to obey the independent and identically distributed assumption. The former's partitioning method ignores the influence of data distribution differences of different users. In fact, Parkinson's tremor data is affected by various factors, such as age, gender, and differences in data collection environment. The training set and the test set often cannot meet the independent and identically distributed assumption, resulting in poor model generalization ability. On the other hand, these works are all based on supervised learning methods, which require the collection and labeling of sufficient data for model training. However, in the field of Parkinson's tremor research, the problem of large-scale, high-precision data label acquisition itself is a great challenge. There are two main reasons. First, tremor data needs to be annotated by specialists with rich clinical experience, and the annotation process is time-consuming, labor-intensive and subjective. Second, experimental data contains patient privacy. Due to the existence of privacy protection regulations, it is very difficult for different research centers to share experimental data. To solve the above problems, domain adaptation methods have been proposed. It creates domain-invariant feature representations by transferring knowledge learned from the source domain to the target domain. The current mainstream methods can be divided into two categories. One is the domain adaptation method based on distance metrics, such as Maximum Mean Discrepancy (MMD) and KL divergence; the other is the domain adaptation method based on adversarial, such as DANN (Domain-Adversarial Neural Network) and CDAN (Conditional Adversarial Domain Adaptation). However, at this stage, these methods are mainly applied to image datasets and time series classification tasks, and there are relatively few studies in the field of Parkinson's tremor assessment. In addition, these methods often align samples from different domains indiscriminately during the adaptation stage without fully considering the differences between samples. In Parkinson's tremor sensor data, even at the same tremor score level, the differences between different users are still significant. If a user's tremor data contains significantly more personalized information than the tremor characteristics themselves, then the user's data may have a negative impact on the adaptation process. Based on this, existing domain adaptation methods have limitations in accurately assessing Parkinson's tremor. Summary of the invention

[0004] In view of the above defects or improvement needs of the prior art, the present invention provides a method for constructing a cross-domain Parkinson's tremor assessment model and its application, so as to solve the technical problem that the prior art cannot accurately assess Parkinson's tremor.

[0005] In order to achieve the above objectives, in a first aspect, the present invention provides a method for constructing a cross-domain Parkinson's tremor assessment model, comprising:

[0006] The Parkinson's tremor sensor data collected in two different environments are used as source domain training samples and target domain training samples respectively, and then a source domain training sample set with tremor score level labels and a target domain training sample set without tremor score level labels are constructed;

[0007] Input multiple pairs of training samples for training in each training batch:

[0008] Each time, a pair of source domain training samples x s and target domain training samples x T Input into the feature extraction module and extract x s 、x T The sample features F s 、F T ; F s and F T Input into the tremor classifier respectively to get x s 、x T The tremor score prediction results are calculated and x s 、x T Tremor classification loss; x s The tremor classification loss is used to measure x s The difference between the predicted tremor score level and the corresponding tremor score level label; x T The tremor classification loss is used to measure x T The difference between the predicted tremor score level and the corresponding pseudo label of the tremor score level;

[0009] F s and F T Input into the domain classifier respectively to get x s and x T The domain prediction result of , and calculate x s 、x T The domain classification loss is used to measure the difference between the predicted result of the domain and the corresponding true result; respectively for x s and x T , concatenate its domain classification loss and tremor classification loss to obtain the corresponding concatenation loss;

[0010] For each domain, the concatenation loss of all the training samples of the corresponding domain input in the current training batch is input into the weight distributor. After obtaining the weight value of each training sample of the corresponding domain, the domain classification loss of each training sample of the corresponding domain is weighted and summed to obtain the domain classification loss of the corresponding domain.

[0011] Calculate the sum of the domain classification losses of the source domain and the target domain to get the total domain classification loss;

[0012] Calculate the sum of the tremor classification losses of all input source domain training samples and target domain training samples to obtain the total tremor classification loss;

[0013] The feature extraction module, tremor classifier, domain classifier, and weight allocator are trained simultaneously by minimizing the total tremor classification loss and maximizing the total domain classification loss;

[0014] After training, a cross-domain Parkinson's tremor assessment model including a cascaded feature extraction module and a tremor classifier was constructed.

[0015] More preferably, x s The tremor classification loss is: s The difference loss between the predicted probability vector and the corresponding true probability vector;

[0016] Among them, x s The predicted probability vector is obtained by s Input to the tremor classifier to obtain; x s The i-th probability value in the predicted probability vector is x s The probability of being predicted as the i-th tremor score level; i = 1, 2, ..., n; n is the number of tremor score levels; x s The predicted result of tremor score is x s The tremor score level corresponding to the maximum probability value in the predicted probability vector;

[0017] The true probability vector is a vector composed of n probability values, in which the idx-th probability value is 1 and the rest are 0; idx is x s The index of the tremor rating grade label.

[0018] More preferably, when x T The predicted probability vector P T When the maximum probability value in is greater than or equal to the preset threshold ρ, x T The tremor classification loss is: P T The difference loss with the corresponding pseudo probability vector; where P T By adding F T Input to the tremor classifier to obtain; P T The i-th probability value in is x TThe probability of being predicted as the i-th tremor score level; x T The tremor score prediction result is P T The tremor score level corresponding to the maximum probability value in ; the pseudo probability vector is obtained by T The maximum value in is set to 1, and the rest are set to 0;

[0019] When P T When the maximum probability value in is less than the preset threshold ρ, x T The tremor classification loss is 0.

[0020] Further preferably, the feature extraction module comprises: a time domain feature extractor, a frequency domain feature extractor and a fusion module;

[0021] The time domain feature extractor is used to extract the time domain features of the input samples;

[0022] The frequency domain feature extractor is used to extract the frequency domain features of the frequency domain signal after the input sample has been time-frequency converted;

[0023] The fusion module is used to fuse the time domain features and frequency domain features to obtain sample features.

[0024] Further preferably, both the time domain feature extractor and the frequency domain feature extractor are CNN models.

[0025] Further preferably, the time domain feature extractor and the frequency domain feature extractor both include: a cascaded convolution module and a global average pooling layer;

[0026] The convolution module includes: a cascaded one-dimensional convolution layer, a batch normalization layer, and an activation function layer.

[0027] In a second aspect, the present invention provides a cross-domain Parkinson's tremor assessment method, comprising: inputting the collected tremor sensor data into a cross-domain Parkinson's tremor assessment model to obtain a corresponding tremor scoring level;

[0028] The cross-domain Parkinson's tremor assessment model is constructed using the method for constructing a cross-domain Parkinson's tremor assessment model provided in the first aspect of the present invention.

[0029] In a third aspect, the present invention further provides a computer-readable storage medium, comprising a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the method for constructing a cross-domain Parkinson's tremor assessment model provided in the first aspect of the present invention or the cross-domain Parkinson's tremor assessment method provided in the second aspect of the present invention.

[0030] In a fourth aspect, the invention further provides a computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, it implements the method for constructing a cross-domain Parkinson's tremor assessment model provided in the first aspect of the invention or the cross-domain Parkinson's tremor assessment method provided in the second aspect of the invention.

[0031] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0032] 1. The present invention provides a method for constructing a cross-domain Parkinson's tremor assessment model, by minimizing the total tremor classification loss and maximizing the total domain classification loss, while training a feature extraction module, a tremor classifier, a domain classifier and a weight allocator; after the training is completed, a cross-domain Parkinson's tremor assessment model including a cascaded feature extraction module and a tremor classifier is constructed; wherein the total domain classification loss is the sum of the domain classification losses of the source domain and the target domain, and when calculating the domain classification losses of different domains, after assigning weights to each training sample under each domain through the weight allocator, the domain classification loss of each training sample obtained based on the domain classifier is weighted at the sample level, thereby making full use of the differences between Parkinson's tremor sensor data, promoting the alignment of the feature spaces of the source domain and the target domain, so that the constructed model can accurately evaluate Parkinson's tremor.

[0033] 2. Furthermore, in order to make the model pay more attention to reliable predictions during the training process, the method for constructing a cross-domain Parkinson's tremor assessment model provided by the present invention sets a threshold ρ for the classification confidence of the target domain training samples, and selects target domain training samples above the threshold ρ to calculate the classification loss, thereby further improving the accuracy of Parkinson's tremor assessment.

[0034] 3. Furthermore, the method for constructing a cross-domain Parkinson's tremor assessment model provided by the present invention extracts features in both the time domain and frequency domain modes for Parkinson's tremor sensor data, and then constructs corresponding multimodal features, which not only makes full use of the correlation in the time domain, but also makes full use of the information in the frequency domain, further improving the accuracy of Parkinson's tremor assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of a construction process of a cross-domain Parkinson's tremor assessment model provided by an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of the structure of a feature extraction module provided in an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of feature splicing of time domain features and frequency domain features in the column direction provided by an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of the structure of a tremor classifier provided by an embodiment of the present invention;

[0039] Figure 5 A schematic diagram of the structure of a weight allocator provided in an embodiment of the present invention;

[0040] Figure 6 A schematic diagram of the structure of a domain classifier provided by an embodiment of the present invention;

[0041] Figure 7 A framework diagram of a cross-domain Parkinson's tremor assessment model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0043] In order to achieve the above objectives, in a first aspect, the present invention provides a method for constructing a cross-domain Parkinson's tremor assessment model, comprising:

[0044] S1. The Parkinson's tremor sensor data collected in two different environments are used as source domain training samples and target domain training samples respectively, and then a source domain training sample set with tremor rating level labels and a target domain training sample set without tremor rating level labels are constructed; the number of samples in the source domain training sample set is less than the number of samples in the target domain training sample set;

[0045] S2. Input multiple pairs of training samples for training in each training batch:

[0046] S21, each time a pair of source domain training samples x s and target domain training samples x T Input into the feature extraction module and extract x s 、x T The sample features F s 、F T ; F s and F T Input into the tremor classifier respectively to get x s 、x T The tremor score prediction results are calculated and x s 、x T Tremor classification loss; x s The tremor classification loss is used to measure x sThe difference between the predicted tremor score level and the corresponding tremor score level label; x T The tremor classification loss is used to measure x T The difference between the predicted tremor score level and the corresponding pseudo label of the tremor score level;

[0047] It should be noted that the feature extraction module can be CNN, RNN, LSTM, Transformer, etc., which is not limited here, and a fully connected layer is preferably used.

[0048] Preferably, in an optional implementation manner, the feature extraction module includes: a time domain feature extractor, a frequency domain feature extractor and a fusion module;

[0049] The time domain feature extractor is used to extract the time domain features of the input samples;

[0050] The frequency domain feature extractor is used to extract the frequency domain features of the frequency domain signal after the input sample has been time-frequency converted;

[0051] The fusion module is used to fuse the time domain features and the frequency domain features to obtain the sample features. It should be noted that there are many ways to fuse features, such as splicing in the column direction, directly adding corresponding elements, weighted merging according to feature relevance or importance, principal component analysis, etc.

[0052] It should be noted that the time domain feature extractor and the frequency domain feature extractor are independent of each other and do not share weight parameters.

[0053] In an optional implementation, both the time domain feature extractor and the frequency domain feature extractor are CNN models.

[0054] In an optional implementation, the time domain feature extractor and the frequency domain feature extractor both include: a cascaded convolution module and a global average pooling layer;

[0055] The convolution module includes: a cascaded one-dimensional convolution layer, a batch normalization layer, and an activation function layer.

[0056] In one embodiment, the tremor classifier is used to obtain the probability that the training sample is predicted to be of different tremor rating levels, and the tremor rating level corresponding to the maximum probability is used as the tremor rating level prediction result of the training sample. The tremor rating level pseudo label is obtained based on the prediction probability of the tremor classifier. Specifically, in one optional implementation manner, x s The tremor classification loss is: s The difference loss between the predicted probability vector and the corresponding true probability vector;

[0057] Among them, x s The predicted probability vector is obtained by sInput to the tremor classifier to obtain; x s The i-th probability value in the predicted probability vector is x s The probability of being predicted as the i-th tremor score level; i = 1, 2, ..., n; n is the number of tremor score levels; x s The predicted result of tremor score is x s The tremor score level corresponding to the maximum probability value in the predicted probability vector;

[0058] The true probability vector is a vector composed of n probability values, in which the idx-th probability value is 1 and the rest are 0; idx is x s The index of the tremor rating grade label.

[0059] In an alternative implementation, when x T The predicted probability vector P T When the maximum probability value in is greater than or equal to the preset threshold ρ, x T The tremor classification loss is: P T The difference loss with the corresponding pseudo probability vector; where P T By adding F T Input to the tremor classifier to obtain; P T The i-th probability value in is x T The probability of being predicted as the i-th tremor score level; x T The tremor score prediction result is P T The tremor score level corresponding to the maximum probability value in ; the pseudo probability vector is obtained by T The maximum value in is set to 1, and the rest are set to 0;

[0060] When P T When the maximum probability value in is less than the preset threshold ρ, x T The tremor classification loss is 0.

[0061] In another embodiment, the tremor rating pseudo-label of the target domain training sample is obtained by: classifying the source domain training sample according to the tremor rating label, and calculating the average value of the features extracted by the feature extraction module of the source domain training sample at different tremor rating levels as the reference feature at the corresponding tremor rating level. The features of the target domain training sample are obtained by the feature extraction module, and the similarity between the features of the target domain training sample and the reference features at different tremor rating levels is calculated respectively, and the tremor rating corresponding to the reference feature with the smallest similarity is used as the tremor rating pseudo-label of the target domain training sample.

[0062] It should be noted that the tremor classifier can be a traditional machine learning model (such as SVM, logistic regression, decision tree, etc.) and a deep learning model (such as fully connected neural network, CNN, LSTM, etc.), etc., which is not limited here, and a fully connected neural network is preferably used.

[0063] S22, F s and F T Input into the domain classifier respectively to get x s and x T The domain prediction result of , and calculate x s 、x T The domain classification loss is used to measure the difference between the predicted result of the domain and the corresponding true result; respectively for x s and x T , concatenate its domain classification loss and tremor classification loss to obtain the corresponding concatenation loss;

[0064] It should be noted that the domain classifier can be a traditional machine learning model (such as SVM, logistic regression, decision tree, etc.) and a deep learning model (such as fully connected neural network, CNN, LSTM, etc.), etc., which is not limited here, and a fully connected neural network is preferably used.

[0065] S23, for each domain, respectively, input the concatenation loss of all the training samples of the corresponding domain input in the current training batch into the weight distributor, obtain the weight value of each training sample of the corresponding domain, and then perform weighted summation on the domain classification loss of each training sample of the corresponding domain to obtain the domain classification loss of the corresponding domain; specifically, input the concatenation loss of all the source domain training samples in the current training batch into the weight distributor, obtain the weight value of each source domain training sample in the current training batch, and then perform weighted summation on the domain classification loss of each source domain training sample to obtain the domain classification loss of the source domain; input the concatenation loss of all the target domain training samples in the current training batch into the weight distributor, obtain the weight value of each target domain training sample in the current training batch, and then perform weighted summation on the domain classification loss of each target domain training sample to obtain the domain classification loss of the target domain; calculate the sum of the total classification losses of the source domain and the target domain to obtain the total domain classification loss;

[0066] Calculate the sum of the tremor classification losses of all input source domain training samples and target domain training samples to obtain the total tremor classification loss;

[0067] It should be noted that the weight distributor can be a fully connected neural network, CNN, RNN, LSTM, attention network, etc., which is not limited here. A fully connected neural network is preferably used.

[0068] S24, training the feature extraction module, the tremor classifier, the domain classifier, and the weight allocator simultaneously by minimizing the total tremor classification loss and maximizing the total domain classification loss;

[0069] S3. After training, a cross-domain Parkinson's tremor assessment model including a cascaded feature extraction module and a tremor classifier is constructed.

[0070] In order to further illustrate the method for constructing the cross-domain Parkinson's tremor assessment model provided by the present invention, a specific embodiment is described in detail below:

[0071] This embodiment is described by taking the cross-domain Parkinson's tremor assessment for wearable sensors as an example. The signals used for Parkinson's tremor collection based on wearable sensors are generally acceleration signals and gyroscope signals. These sensors will be fixed on various body parts of the subject, such as wrists, calves, feet, etc. to detect the tremor of the subject. Parkinson's tremor scoring is currently mainly based on the MDS-UPDRS scale standard in clinical practice. According to the manifestation of the disease during outpatient examination, the integer is divided into 0-4 levels (0 represents normal and 4 represents severe). The sub-items related to tremor in the scale are 3.15 to 3.18, which are the postural tremor of the hand, the action tremor of the hand, the amplitude of the rest tremor and the persistence of the rest tremor. Postural tremor, action tremor and rest tremor all belong to Parkinson's tremor, and the characteristics of different tremor types are also different, but the invention is applicable to all analyses involving tremor behavior itself. As a specific embodiment, the upper limb data of rest tremor collected in a controlled environment is taken as an example. For the stationary tremor task, the subjects were asked to sit in a chair and put their arms on the armrests, and then the measurement device was connected to each subject's wrist. Since tremors do not persist when they occur, the label reviewer carefully checked the records with the help of synchronized video and recorded the timestamps corresponding to the start and end of each patient's visible tremor. The embodiment of the present invention has no restrictions on the type of signal, so commonly used acceleration and gyroscope signals and other related multimodal signals can be used as inputs to the network.

[0072] like Figure 1 As shown in Figure 1, the construction method of the cross-domain Parkinson's tremor assessment model includes:

[0073] 1) Data acquisition and preprocessing. After obtaining the informed consent of the subjects to participate in the experiment, the tremor sensor data of the subjects and their corresponding MDS-UPDRS tremor subscores were collected according to a certain activity protocol. The tremor data were preprocessed by filling missing values ​​and filtering; the tremor signal X t Perform time-frequency mode conversion to obtain the frequency domain signal X f According to the cross-domain scenario set in the experiment, the acquired data is divided into labeled source domains and the unlabeled target domain Further sliding window segmentation to construct source domain training samples Target domain training training samples Target domain test samples

[0074] In the step of preprocessing the signal, the timestamps of each sensor data are first aligned and downsampled to the same frequency, with the sampling frequency set to 100Hz; then the signal is checked for duplicate rows and missing values, duplicate rows are deleted, and missing values ​​are filled using linear interpolation; then the signal is subjected to Butterworth bandpass filtering to extract the typical frequency of the corresponding tremor type, such as 3-7Hz for resting tremor, and low-frequency and high-frequency noise and autonomous movement are filtered out, retaining only the tremor component as much as possible. In order to simultaneously learn the time domain information and frequency domain information of the tremor signal, a one-dimensional multiscale discrete wavelet transform is used to convert the time domain signal into a frequency domain signal.

[0075] For the division of samples, first divide the samples into source domain and target domain according to the cross-domain scenario set in the experiment. The cross-domain scenario can be any factor that causes inconsistent data distribution between the source domain and the target domain of Parkinson's tremor, such as differences in gender, age, and data collection environment. If the cross-domain scenario of the experiment is set to different test centers, the subject data from test center 1 is used as the source domain data, and the subject data from test center 2 is used as the target domain data. All data in the source domain are used as source domain training samples, and the target domain data are divided into target domain training samples and target domain test samples in a 1:1 ratio. The actual ratio can be adjusted according to the number of samples.

[0076] 2) The source domain training samples and target domain training samples The time domain signal is input to the time domain feature extractor F t In the process, we can obtain the time domain feature fea T ; The source domain training samples and target domain training samples The frequency domain signal is input to the frequency domain feature extractor F f In the frequency domain, obtain the fea F .

[0077] In this embodiment, a parallel deep network is used to extract features from the time domain and frequency domain of the signal. Figure 2 In an optional implementation, both the time domain feature extractor and the frequency domain feature extractor use a convolutional neural network. The network structure includes three convolution blocks and a global average pooling layer, and each convolution block includes: a one-dimensional convolution layer, a batch normalization layer, and an activation function layer. The specific structure is as follows:

[0078] The first one-dimensional convolution block includes: a convolution layer with 64 convolution kernels, a size of 5 and a stride of 2, a batch normalization layer and a ReLU activation function layer;

[0079] The second one-dimensional convolution block includes: a convolution layer with 128 convolution kernels, a size of 3 and a stride of 2, a batch normalization layer and a ReLU activation function layer;

[0080] The third one-dimensional convolution block includes: a convolution layer with 64 convolution kernels, a size of 2 and a stride of 1, a batch normalization layer, and a ReLU activation function layer.

[0081] 3) The acquired time domain features fea T and frequency domain features F The multimodal features (sample features) fea are directly concatenated and then input into the tremor classifier C and the domain classifier D respectively to obtain the classification probability and the domain classification probability.

[0082] like Figure 3 As shown, the acquired time domain features and frequency domain features are concatenated in the column direction, and the concatenated features are called multimodal features.

[0083] In this embodiment, the structure of the tremor classifier is a fully connected layer and an activation function layer, and the output dimension is the number of tremor assessment levels. If the labels of the training samples cover all levels of the MDS-UPDRS tremor scoring items, the output dimension should be set to 5. When the activation function selects the softmax function, the tremor classifier outputs the classification probability That is, the probability that the sample belongs to each tremor level, n c is the number of categories. The structure of the domain classifier is fully connected layer 1, ReLU layer, fully connected layer 2, ReLU layer, fully connected layer 3, Sigmoid layer, and its output is the domain classification probability.

[0084] 4) Calculate the total tremor classification loss. The total tremor classification loss consists of two parts, the domain classification loss of the source domain and the domain classification loss of the target domain. Since the target domain training samples do not contain label information, pseudo labels are used for calculation. Taking the target domain training samples as input, the tremor level corresponding to the maximum probability in the classification probability output by the tremor classifier is the pseudo label, and the corresponding maximum probability value is the classification confidence. In order to make the model pay more attention to reliable predictions during the training process, a threshold ρ is set for the classification confidence of the target domain training samples, and the target domain training samples above the threshold ρ are selected to calculate the classification loss. In this embodiment, ρ takes a value of 0.7.

[0085] Specifically, the total tremor classification loss of the source domain is the sum of the tremor classification losses of each source domain training sample; where the source domain training sample x s The tremor classification loss is: the source domain training sample xs The difference loss between the predicted probability vector and the corresponding true probability vector;

[0086] Among them, x s The predicted probability vector is obtained by s Input to the tremor classifier to obtain; x s The i-th probability value in the predicted probability vector is x s The probability of being predicted as the i-th tremor score level; i = 1, 2, ..., n; n is the number of tremor score levels; x s The predicted result of tremor score is x s The tremor score level corresponding to the maximum probability value in the predicted probability vector;

[0087] The true probability vector is a vector composed of n probability values, in which the idx-th probability value is 1 and the rest are 0; idx is x s The index of the tremor rating grade label.

[0088] The total tremor classification loss of the target domain is the sum of the tremor classification losses of each target domain training sample; T The predicted probability vector P T When the maximum probability value in is greater than or equal to the preset threshold ρ, the target domain training sample x T The tremor classification loss is: P T The difference loss with the corresponding pseudo probability vector; where P T By adding F T Input to the tremor classifier to obtain; P T The i-th probability value in is x T The probability of being predicted as the i-th tremor score level; x T The tremor score prediction result is P T The tremor score level corresponding to the maximum probability value in ; the pseudo probability vector is obtained by T The maximum value in is set to 1, and the rest are set to 0; when P T When the maximum probability value in is less than the preset threshold ρ, x T The tremor classification loss is 0.

[0089] 5) The tremor classification loss and domain classification loss of each source domain training sample are directly spliced ​​and input into the weight distributor, and the output is the weight of each source domain training sample after standardization; each source domain training sample is multiplied by the corresponding weight and then added to obtain the domain classification loss of the source domain;

[0090] The tremor classification loss and domain classification loss of each target domain training sample are directly concatenated and input into the weight distributor, and the output is the weight of each target domain training sample after standardization; each target domain training sample is multiplied by the corresponding weight and then added to obtain the domain classification loss of the target domain;

[0091] The domain classification loss of the source domain and the domain classification loss of the target domain are added together to get the total domain classification loss.

[0092] In this embodiment, the weight distributor includes: a cascaded first fully connected layer, a ReLU layer, a second fully connected layer and a Sigmoid layer.

[0093] 6) Source domain training samples and target domain training samples This embodiment uses adversarial training and dual optimization to minimize the total tremor classification loss while maximizing the total domain classification loss, making it difficult for the domain classifier to classify features from the source domain and the target domain, thereby extracting domain-invariant features. The specific training steps are divided into the following three steps:

[0094] The first step is outer loop optimization. Use source domain training data and target domain training data Train the feature extraction module F and the tremor classifier C. The training data is output by the feature extraction module and the tremor classifier to obtain the corresponding classification probability. Since the target domain does not contain true labels, pseudo labels are used to calculate the classification loss. The loss in this step is the total tremor classification loss.

[0095] The second step is inner loop optimization. Use source domain training samples and target domain training samples To train the weight distributor W, first create a copy of the feature extraction module in the first step. After the training data is extracted with multimodal features by the copy feature extraction module, it enters the tremor classifier and the domain classifier respectively and calculates the tremor classification loss and domain classification loss of each training sample. After that, the tremor classification loss and domain classification loss of each source domain training sample are concatenated to obtain the corresponding concatenation loss; the tremor classification loss and domain classification loss of each target domain training sample are concatenated to obtain the corresponding concatenation loss. The splicing loss of each source domain training sample is input into the weight distributor to obtain the sample weight, and the domain classification loss of each source domain training sample is weighted summed to obtain the total domain classification loss of the source domain; the splicing loss of each target domain training sample is input into the weight distributor to obtain the sample weight, and the domain classification loss of each target domain training sample is weighted summed to obtain the total domain classification loss of the target domain; the sum of the total domain classification loss of the source domain and the total domain classification loss of the target domain is calculated to obtain the total domain classification loss; the parameters of the feature extraction module copy are updated by maximizing the total domain classification loss; on this basis, the tremor classification loss of each target domain training sample is calculated and summed to obtain the total tremor classification loss under the target domain; the loss is optimized to update the parameters of the weight distributor.

[0096] In this embodiment, maximizing the total domain classification loss is achieved through a gradient reversal layer. The gradient reversal layer can achieve an identical transformation of the gradient in the forward propagation and automatically negate it in the backward propagation process.

[0097]

[0098] Where λ is a constant and I is the identity matrix.

[0099] The third step is outer loop optimization. Use source domain training samples and target domain training samples Train the feature extraction module and domain classifier, calculate the total domain classification loss, and optimize the loss to update the parameters of the feature extraction module and domain classifier.

[0100] In this embodiment, the tremor classification loss is calculated based on the cross entropy loss function, and the domain classification loss is calculated based on the binary cross entropy loss function.

[0101] In this embodiment, the structure of the tremor classifier is a fully connected layer with 5 output neurons and a Sigmoid layer, such as Figure 4 The structure of the weight distributor is a linear layer with 128 output neurons, a ReLU layer, a linear layer with 2 output neurons, and a Sigmoid layer, as shown in Figure 5 The structure of the domain classifier is linear layer 1, batch normalization layer, ReLU layer, linear layer 2, batch normalization layer, ReLU layer, linear layer 3 and Sigmoid layer, as shown in Figure 6 As shown in Figure 2, the number of output neurons in the three linear layers are 256, 256, and 2 respectively.

[0102] In this embodiment, the Adam optimizer is used to train the network, the initial learning rate of all sub-networks is set to 0.001, and the learning rate adjustment strategy uses the cosine annealing algorithm. In training and testing, the batch size batch_size=128 and the maximum number of iterations n_epoch=10 are set. When the number of training iterations reaches the set number, the training stops, thereby obtaining the optimal tremor classifier. The optimization of the inner loop is achieved by the differentiable optimizer provided by higher.

[0103] Considering the imbalance of the tremor dataset, in addition to the three commonly used evaluation indicators, accuracy, sensitivity and F1 index, the present embodiment also adds the Kappa coefficient and MCC indicators. The Kappa coefficient provides a more realistic evaluation than accuracy because it can penalize the "bias" of the model. MCC can avoid classifier bias caused by uneven sample distribution and is a comprehensive classification performance evaluation indicator.

[0104] Finally, the target domain test samples are classified. After the above steps, the network has completed the transfer of target domain knowledge. The target domain test sample data is input, and the trained multimodal feature extractor and tremor classifier are used to obtain the final corresponding tremor score level prediction results to achieve model classification performance evaluation.

[0105] The entire training process is as follows Figure 7 shown.

[0106] In a second aspect, the present invention provides a cross-domain Parkinson's tremor assessment method, comprising: inputting the collected tremor sensor data into a cross-domain Parkinson's tremor assessment model to obtain a corresponding tremor scoring level;

[0107] The cross-domain Parkinson's tremor assessment model is constructed by using the construction method of the cross-domain Parkinson's tremor assessment model provided by the first aspect of the present invention;

[0108] Specifically, the collected tremor sensor data is input into the cross-domain Parkinson's tremor assessment model. After the tremor sensor data is subjected to feature extraction by a feature extraction module, it is input into a tremor classifier to obtain a corresponding tremor score level.

[0109] The related technical solution is the same as the method for constructing a cross-domain Parkinson's tremor assessment model provided by the first aspect of the present invention, and will not be described in detail here.

[0110] In a third aspect, the present invention further provides a computer-readable storage medium, comprising a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the method for constructing a cross-domain Parkinson's tremor assessment model provided in the first aspect of the present invention or the cross-domain Parkinson's tremor assessment method provided in the second aspect of the present invention.

[0111] The related technical solutions are the same as the method for constructing a cross-domain Parkinson's tremor assessment model provided by the first aspect of the present invention and the cross-domain Parkinson's tremor assessment method provided by the second aspect of the present invention, and will not be described in detail here.

[0112] In a fourth aspect, the invention further provides a computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, it implements the method for constructing a cross-domain Parkinson's tremor assessment model provided in the first aspect of the invention or the cross-domain Parkinson's tremor assessment method provided in the second aspect of the invention.

[0113] The related technical solutions are the same as the method for constructing a cross-domain Parkinson's tremor assessment model provided by the first aspect of the present invention and the cross-domain Parkinson's tremor assessment method provided by the second aspect of the present invention, and will not be described in detail here.

[0114] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a cross-domain Parkinson's tremor assessment model, characterized in that: include: The Parkinson's tremor sensor data collected in two different environments are used as source domain training samples and target domain training samples respectively, and then a source domain training sample set with tremor score level labels and a target domain training sample set without tremor score level labels are constructed; Input multiple pairs of training samples for training in each training batch: Each time, a pair of source domain training samples x s and target domain training samples x T Input into the feature extraction module and extract x s 、x T The sample features F s 、F T ; F s and F T Input into the tremor classifier respectively to get x s 、x T The tremor score prediction results are calculated and x s 、x T Tremor classification loss; x s The tremor classification loss is used to measure x s The difference between the predicted tremor score level and the corresponding tremor score level label; x T The tremor classification loss is used to measure x T The difference between the predicted tremor score level and the corresponding pseudo label of the tremor score level; F s and F T Input into the domain classifier respectively to get x s and x T The domain prediction result of , and calculate x s 、x T Domain classification loss; Domain classification loss is used to measure the difference between the predicted result of the domain and the corresponding true result; s and x T , concatenate its domain classification loss and tremor classification loss to obtain the corresponding concatenation loss; For each domain, the concatenation loss of all the training samples of the corresponding domain input in the current training batch is input into the weight distributor. After obtaining the weight value of each training sample of the corresponding domain, the domain classification loss of each training sample of the corresponding domain is weighted and summed to obtain the domain classification loss of the corresponding domain. Calculate the sum of the domain classification losses of the source domain and the target domain to get the total domain classification loss; Calculate the sum of the tremor classification losses of all input source domain training samples and target domain training samples to obtain the total tremor classification loss; The feature extraction module, tremor classifier, domain classifier, and weight allocator are trained simultaneously by minimizing the total tremor classification loss and maximizing the total domain classification loss; After training, a cross-domain Parkinson's tremor assessment model including a cascaded feature extraction module and a tremor classifier was constructed.

2. The method for constructing a cross-domain Parkinson's tremor assessment model according to claim 1, characterized in that: The x s The tremor classification loss is: s The difference loss between the predicted probability vector and the corresponding true probability vector; Among them, the x s The predicted probability vector is obtained by s Input to the tremor classifier to obtain; the x s The i-th probability value in the predicted probability vector is x s The probability of being predicted as the i-th tremor score level; i = 1, 2, ..., n; n is the number of tremor score levels; x s The predicted result of tremor score is x s The tremor score level corresponding to the maximum probability value in the predicted probability vector; The true probability vector is a vector composed of n probability values, in which the idx-th probability value is 1 and the rest are 0; idx is x s The index of the tremor rating grade label.

3. The method for constructing a cross-domain Parkinson's tremor assessment model according to claim 1, characterized in that: When x T The predicted probability vector P T When the maximum probability value in is greater than or equal to the preset threshold value ρ, the x T The tremor classification loss is: P T The difference loss with the corresponding pseudo probability vector; where P T By adding F T Input to the tremor classifier to obtain; P T The i-th probability value in is x T The probability of being predicted as the i-th tremor score level; x T The tremor score prediction result is P T The tremor score level corresponding to the maximum probability value in T The maximum value in is set to 1, and the rest are set to 0; When P T When the maximum probability value in is less than the preset threshold value ρ, the x T The tremor classification loss is 0.

4. The method for constructing a cross-domain Parkinson's tremor assessment model according to any one of claims 1 to 3, characterized in that: The feature extraction module includes: a time domain feature extractor, a frequency domain feature extractor and a fusion module; The time domain feature extractor is used to extract the time domain features of the input samples; The frequency domain feature extractor is used to extract the frequency domain features of the frequency domain signal after the input sample is time-frequency converted; The fusion module is used to fuse the time domain features and the frequency domain features to obtain sample features.

5. The method for constructing a cross-domain Parkinson's tremor assessment model according to claim 4, characterized in that: The time domain feature extractor and the frequency domain feature extractor are both CNN models.

6. The method for constructing a cross-domain Parkinson's tremor assessment model according to claim 5, characterized in that: The time domain feature extractor and the frequency domain feature extractor both include: a cascaded convolution module and a global average pooling layer; The convolution module includes: a cascaded one-dimensional convolution layer, a batch normalization layer and an activation function layer.

7. A cross-domain Parkinson's tremor assessment method, characterized in that: include: The collected tremor sensor data is input into the cross-domain Parkinson's tremor assessment model to obtain the corresponding tremor score level; The cross-domain Parkinson's tremor assessment model is constructed by using the method for constructing a cross-domain Parkinson's tremor assessment model according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images