Tremor Symptom Detection Method and Device Based on Electronic Handwriting

Abstract

This application proposes a tremor symptom assessment model based on electronic handwriting, comprising: an input embedding layer, a position encoding layer, a Transformer encoder, a feature fusion and classification decision layer; the input embedding layer receives input from a digital ink sequence of handwritten handwriting; the position encoding layer uses a sinusoidal function-based encoding method to obtain sequential information that can replace the time series; the Transformer encoder is a model architecture composed of a multi-head self-attention mechanism and a feedforward neural network; the embedded first deep feature is superimposed with the sequential information and input into the Transformer encoder to obtain a second deep feature with deep semantics; the feature fusion and classification decision layer includes a feature fusion module and a classification decision module; the feature fusion layer fuses the second deep feature with kinematic features through a concatenation operator to form a third deep feature; the third deep feature is input into the classification decision layer, which outputs the corresponding predicted probability of tremor severity.

Description

Technical Field

[0001] This application relates to pattern recognition, and more particularly to a method, assessment model, and assessment device for assessing tremor symptoms based on electronic handwriting. Background Technology

[0002] Essential tremor (ET) is a neurodegenerative disease affecting approximately 6.38 million to 7.63 million people in the United States, nearly 2.2% of the population. ET typically presents as postural and intention tremors involving the upper limbs, with a frequency of 8–12 Hz. Patients with ET have a more than fourfold increased risk of developing Parkinson's disease (PD). Family history and laboratory tests are currently the only methods for diagnosing this type of movement disorder. Neurologists observe patients performing specific tasks visually and score each task according to the Clinical Tremor Scoring Scale (CRST). However, this approach relies heavily on the physician's clinical experience and expertise, as well as a comprehensive review of the patient; subjective biases lead to unsatisfactory assessments of movement disorders. Furthermore, tremor symptoms can develop and change over time, and manually tracking disease progression often consumes significant medical resources. Additionally, assessing the disease remains challenging for older adults, those with cognitive impairment, or those with limited mobility. Therefore, early diagnosis tailored to the individual needs of patients with movement disorders, and accurate assessment of symptom severity, are crucial for prognosis and treatment. For this purpose, there has been increasing interest in computer-aided diagnosis over the years. There is an urgent need to provide doctors with objective, multimodal, and quantitative evaluation criteria to offer rich information to aid decision-making.

[0003] Neurological disorders such as ET and PD cause brain changes, including neuronal loss, synaptic dysfunction, and brain atrophy, which can lead to dysfunction of the motor system and its components. From this perspective, in the context of symptom assessment, the unique role of handwriting can be confidently hypothesized. Handwriting is a complex activity encompassing both motor perception and cognitive components, and its changes can be considered a promising biomarker for disease assessment. Indeed, a growing body of research provides evidence that simple and easy-to-perform writing tasks can quantify tremor symptoms in ET or differentiate PD. However, currently, no studies have collected sufficient handwriting data from ET patients and obtained reliable expert scores to support the development of an automated tremor quantification assessment system. Developing a handwriting-based decision support tool is desirable because it can provide a non-invasive, real-time, and low-cost solution to support standard clinical assessments performed by human experts.

[0004] In this research direction, an online (dynamic) system combining a digitized tablet computer with a high-precision sensor stylus can be employed. Such a device can capture not only the temporal and spatial variables of the writing process but also measurements of the pressure exerted by the pen tip on the writing surface, tilt direction, and even rotation angle. In contrast to the offline (static) morphological features of handwriting, dynamic handwriting analysis deals with the dynamic characteristics of the writing process, making it more suitable for disease diagnosis. Therefore, selecting appropriate features is crucial for designing automated diagnostic systems. Classical statistical feature classifiers focus on the raw time series and are prone to high-dimensionality burdens, leading to overfitting. For this reason, some dynamic features are derived from the raw data, representing high-level information in the handwriting, such as entropy features or signal-to-noise ratio features characterizing sequence complexity. While such high-level features help the model acquire global information about the handwriting, they also lose local details because arbitrarily long sequences are compressed into single-valued features.

[0005] Another approach to acquiring tremor information in electronic handwriting is to utilize deep learning models to automatically perceive differences in information. Recent work based on convolutional neural networks (CNNs) has leveraged automatically extracted morphological features from two-dimensional static images to achieve tremor recognition. While this method has the potential to replace manually designed features, it only provides a holistic view of the handwriting pattern; dynamic changes over time could also contribute to improving learning effectiveness. Kamran et al. constructed a recurrent neural network (RNN) model to learn dynamic features of electronic handwriting sequences, but they also neglected the exploration of global information, which could potentially provide meaningful supplementary information to the network. Summary of the Invention

[0006] To avoid the shortcomings of the existing technology, this application aims to propose a tremor symptom assessment model based on electronic handwriting to determine the probability of a subject's tremor level.

[0007] The tremor symptom assessment model based on electronic handwriting in this application is based on the Transformer model and includes: an input embedding layer, a position encoding layer, a Transformer encoder, a feature fusion and classification decision layer;

[0008] The input embedding layer takes the digital ink sequence from the handwritten handwriting as input in the form of a multi-channel sequence. After passing through multiple layers of MSSE-ResNet, the input sensor sequence is transformed into the first deep feature of the embedding.

[0009] The positional coding layer, based on a sinusoidal function coding method, uses positional coding to obtain sequential information that can replace time series.

[0010] The Transformer encoder, with its model architecture consisting of a multi-head self-attention mechanism and a feedforward neural network, is able to map the global dependencies between inputs and outputs;

[0011] The embedded first deep feature is superimposed with the sequential information and then input into the Transformer encoder to obtain the second deep feature with deep semantics.

[0012] The feature fusion and classification decision layer includes a feature fusion module and a classification decision module; the feature fusion layer is located before the classification decision layer; the feature fusion layer fuses the second-depth features with the kinematic features through a concatenation operator to form the third-depth features; the third-depth features are input to the classification decision layer, which consists of linear layers, and output the predicted probability of the corresponding tremor severity.

[0013] Preferably, the multi-layer MSSE-ResNet includes multiple MSSE-ResNet blocks; each block consists of multi-scale convolutions and SE-ResNet; wherein, the multi-scale convolution kernel size is selected as 3, 5, or 7, and each convolutional layer consists of a convolution-batch normalization-ReLU activation function; the input of the SE module is multi-scale features, and the output is the deep features with channel attention weights obtained through the residual structure.

[0014] Preferably, the basic function of the multi-head self-attention mechanism in the Transformer encoder is the scaled dot product attention function.

[0015] Preferably, the digital ink sequence includes raw ink position features derived from the sensor data stream and calculated ink dynamic features.

[0016] Preferably, the digital ink sequence includes the following features:

[0017] Ink position, pressure, signal vector amplitude of electronic ink, tilt, displacement, velocity, acceleration, jerk, radius variability, angle variability, spectral density, and kinematic characteristics.

[0018] Preferably, the kinematic features are obtained from a digital ink sequence, which includes:

[0019] The sample entropy of the signal vector amplitude (SVM_SampEn), the average rectified value of the signal vector amplitude (SVM_Arv), the variance of the signal vector amplitude (SVM_Var), the kurtosis of the signal vector amplitude (SVM_Kurt), the waveform factor of the signal vector amplitude (SVM_SF), the peak factor of the signal vector amplitude (SVM_CF), and the impulse factor of the signal vector amplitude (SVM_IF);

[0020] Peak power ratio of radius variability (RV_PwrP_R), main peak power of radius variability (RV_PrinP), fuzzy entropy of radius variability (RV_FuzEn), mean of radius variability (RV_Mean), average rectified value of radius variability (RV_Arv), peak factor of radius variability (RV_CF).

[0021] Peak power of angle variability (AV_PwrP), peak power ratio of angle variability (AV_PwrP_R), variance of angle variability (AV_Var), root mean square of angle variability (AV_RMS), waveform factor of angle variability (AV_SF), mean of angle variability (AV_RMS), and maximum value of angle variability (AV_Max).

[0022] This application also proposes a method for assessing tremor symptoms based on electronic handwriting, which uses the aforementioned tremor symptom assessment model based on electronic handwriting for evaluation.

[0023] This application also provides a tremor symptom assessment device based on electronic handwriting, which includes a computing unit for running the above-described tremor symptom assessment model based on electronic handwriting.

[0024] The main contributions of this application include: (Compared with the prior art, the advantages and positive effects of this invention are as follows:)

[0025] 1) This invention relies on rigorous patient screening experiments (biochemistry, imaging, physical examination, medical history), and through independent scoring by a neurology expert committee, the comprehensive consensus score is used for supervised learning, and the first essential tremor (ET) patient database composed of CRST drawing task and tremor examination task is established.

[0026] 2) The method of this invention explores the Transformer model of sequence learning for the first time, and combines it with a one-dimensional CNN model for an automatic quantification system of movement disorders.

[0027] 3) This invention utilizes pen tip dynamics characteristics (such as root mean square (RMS), kurtosis, waveform factor, spectral peak value, peak power, peak ratio, approximate entropy, sample entropy, and fuzzy entropy) to effectively reveal the regularity and synchronicity of tremors in patients with essential tremor, thereby characterizing the patient's tremor amplitude, tremor frequency, and other characteristics, and establishing a model that can accurately quantify and classify the patient's tremor symptoms.

[0028] 4) The present invention designed a complete verification experiment, which proved that the proposed method is the best performing method in both ET tremor assessment and early PD diagnosis.

[0029] The model presented in this application demonstrates that deep networks based on sequence learning, combined with simplified feature and algorithmic definitions, can be widely applied to various scenarios and performance requirements. However, low-dimensional raw data acquisition may limit the expected upgrades of the system technology. The need for higher diagnostic accuracy can benefit from more standardized data acquisition, higher sensor resolution, and multimodal signal channels. Therefore, tremor signal acquisition systems applied to laboratory diagnostics should ideally use high-precision sensors to provide stereoscopic pen tip dynamics beyond simple and intuitive planar rendering. This will increase the available data and help improve the quantity and quality of information acquired. Furthermore, a system that integrates global dynamic features of electronic handwriting can achieve fully automated tremor quantification assessment, avoiding interference from local morphology. Moreover, the effectiveness of the labels used to train the supervised model also determines the decision-making effectiveness of the medical assistance system. Therefore, the method of this invention establishes a rigorous posture tremor database, provides the most objective labels for supervising model learning, and effectively demonstrates the effectiveness of the proposed automated quantification system. Attached Figure Description

[0030] Figure 1 This application relates to a computer-aided diagnostic system based on digital handwriting.

[0031] Figure 2 The overall architecture and structural details of the Transformer (DIT, digital inkTransformer) model used for sequence learning of digital ink sequence features.

[0032] Figure 3 Digital ink features obtained by performing small-scale Archimedean spirals on three randomly selected ET patients of different severities.

[0033] Figure 4 The power spectral density function is calculated based on the vector amplitude of the acceleration signal (a truncated 15s signal segment).

[0034] Figure 5 This is the receiver operating characteristic (ROC) curve for the classifier. Detailed Implementation

[0035] This invention utilizes a self-developed automatic tremor symptom assessment system based on electronic handwriting to provide an automatic tremor assessment method for multiple writing tasks in movement disorder categories. The method offers two different application scenarios: 1) ET tremor quantification, which can simultaneously obtain scores for three different writing tasks, including Archimedean spirals and straight lines at different scales; and 2) PD differential diagnosis, which can automatically distinguish PD patients from healthy individuals through electronic handwriting.

[0036] The automatic tremor symptom assessment system of the present invention includes 1) a data acquisition unit; 2) a data preprocessing unit; 3) a multi-sensor channel pen tip dynamics feature acquisition unit and 4) a model construction and feedback unit.

[0037] The specific steps of the method of the present invention are as follows:

[0038] (1) An electronic digitizing tablet is designed to simultaneously record the patient's paper and electronic handwriting. A high-precision sensing digitizing pen is used to transmit the automatically rendered electronic handwriting to the local computer, while the raw data stream is transmitted to the cloud to ensure data portability. Specifically, in this embodiment, the device used for data acquisition is the Intuos Pro Paper Edition (430×287×8mm), which has 8,192 pen pressure levels and supports drawing on test paper using a Wacom Finetip Pen. Simultaneously, the rendered electronic handwriting is saved on the computer, and the raw data stream from the cloud can be converted into image formats such as jpg and npg. The Wacom Ink Layer Language (WILL) format includes the rendering result, collected sensor data, and related metadata streams. The method of this invention designs software specifically for exporting sequence information, facilitating the extraction of relevant information describing tremor.

[0039] (2) Data collection. Patients can use a digital pen to simultaneously record handwriting information on paper and a digital tablet. Specifically, a high-precision digital tablet is used to collect hand tremor data of patients under evoked movements during multiple handwriting tremor tasks; at the same time, the CRST scale is used to score the completion of the designated movements by the patients; for later follow-up and assessment of changes in the condition and for blind review and scoring by multiple neurological experts, professional neurologists record the patient's movements throughout the laboratory examination.

[0040] The entire laboratory examination was guided by a neurologist specializing in movement disorders, and video data (CMOS camera, 48MP, 1920*1080 HD, 60 frames / second) was recorded to support independent scoring by three neurologists (mutually blinded). During the laboratory examination, the patient completed drawing and writing tasks on a digital tablet following prompts, with digital signals wirelessly transmitted to the host computer via Bluetooth. The host computer displayed the rendering effect of the electronic handwriting in real time and stored it as a rendered image on the computer's hard drive. To ensure the acquisition of valid tremor signals, the sampling frequency of the digital pen was set to 100Hz.

[0041] Two experts jointly score the patients' action tremors. In cases of discrepancies, a third expert reviews the video recordings to make the final determination. The specific scoring process is as follows: 1) Two neurologists score the severity of the patient's action tremor by reviewing video footage and observing the remaining handwriting, obtaining two mutually blinded rating scales. 2) A data analysis engineer tallies the consistent scores from the two experts. For actions with inconsistent scores, another experienced neurologist makes the final decision based on the video footage and remaining handwriting until a reliable score is obtained. This experimental design also avoids training label errors caused by systematic bias, making the machine learning model more reliable.

[0042] Specifically, the data acquisition unit includes an electronic digitizing tablet, a high-precision digital pen, a computer, a camera, a wireless local area network, and data cables. An electronic handwriting database can be established based on this data acquisition unit, including paper handwriting, electronic handwriting, video recordings, and patient information from doctor consultations and examinations (including demographic information, past medical history, and scale tests).

[0043] The assigned writing tasks include Archimedean spirals and straight-line tasks of varying scales. The tremor symptoms induced by these tasks fall under the category of action tremor. Based on Part A of the CRST standard scale, the writing task requires patients to sit comfortably in a chair, maintain appropriate posture while writing, and ensure the pen tip remains on the paper as much as possible, with the handwriting within the boundaries, and flowing continuously from the starting point to the end point.

[0044] The CRST scale is used to score a patient's performance on a designated action. This involves a physician assessing the patient's resting tremor level based on their performance. The tremor is graded on a scale of 0, 1, 2, 3, and 4. The CRST scale indicates that the five grades of postural tremor are: 1) Normal handwriting; 2) Mild, with handwriting occasionally touching the boundaries; 3) Moderate amplitude, with handwriting frequently touching the boundaries; 4) Large amplitude, making the task very difficult and potentially leading to errors; 5) Large amplitude, unable to complete the writing task.

[0045] (3) Figure 1This invention demonstrates a computer-aided diagnostic system based on digital handwriting. Raw pen tip motion data can be obtained through a digital tablet, from which multimodal digital ink features are derived. Traditional subjective evaluation is based on judgments of images rendered with relative coordinates, while the system based on deep learning algorithms obtains objective evaluation references by learning high-dimensional dynamic information. The deep learning algorithm efficiently assesses tremor by learning different pen tip dynamic features: 1) The raw sensor data stream derived from the electronic handwriting and the high-level sequence features proposed in this invention are fused into multi-sensor channel sequence data for sequence learning; 2) Global statistical features, involving time domain, frequency domain, and nonlinear features, are derived from the electronic handwriting and fused with parameters representing the overall handwriting dynamics before the decision layer of the deep learning algorithm, thereby obtaining a more comprehensive and efficient feature representation.

[0046] (4) Furthermore, in combination Figure 2 The proposed deep learning-based automatic tremor symptom assessment system can be divided into four layers based on the Transformer model structure: 1) an input embedding layer; 2) a position encoding layer; 3) a Transformer encoder composed of a multi-head self-attention mechanism; and 4) a feature fusion and classification decision layer. The overall architecture and structural details of the Transformer (DIT, digital ink Transformer) model for sequence learning of digital ink sequence features are also presented. Raw data from handwritten handwriting is transformed into feature sequences. The embedding layer of the DIT model consists of MSSE-ResNet, fusing multi-scale convolutional blocks and SE modules. The basic function of the multi-head self-attention mechanism in the Transformer encoder is scaled dot product attention. Specifically, the Transformer is a model architecture composed solely of a self-attention mechanism and a feed-forward neural network, capable of effectively mapping the global dependencies between input and output. This model can handle clinical problems beyond machine translation, and is not limited to images, text, or physiological signals.

[0047] 1) Input Embedding Layer

[0048] The embedding layer performs a series of convolution operations on the digital ink sequence features to capture the representation of the latent spatial information of the tremor.

[0049] The first step is the input embedding layer, which receives digital sensor data from handwritten handwriting as a multi-channel sequence. This sequence passes through multiple layers of multi-scale squeeze-and-excitation-ResNet (MSSE-ResNet) to transform the sensor sequence into depth features. These depth features are then combined with the position encoding layer before being input into the Transformer encoding layer. Specifically, the main structure of the input embedding layer consists of MSSE-ResNet blocks, each composed of multi-scale convolutions and SE-ResNet layers, such as... Figure 2 As shown, the multi-scale selection of convolutional kernel sizes is 3, 5, and 7, and each convolutional layer consists of a convolution-batch normalization-ReLU activation function. The input of the SE module is multi-scale features, and the output is the depth feature with channel attention weights obtained through the residual structure.

[0050] The main body of the embedding layer consists of an end-to-end multi-scale squeeze-and-excitation-ResNet (MSSE-ResNet). Figure 2 The overall architecture of this deep network is illustrated in the figure. The input is multimodal digital ink sequence features, and the output is the embedded representation of the sequence. The proposed MSSE-ResNet network consists of three main parts: 1) a backbone network learns shared low-level features; 2) multi-branch subnetworks perceive multi-scale temporal information through convolutional kernels of different scales; 3) the SE module is used to fuse feature maps between different channels. This lightweight attention mechanism embedded in ResNet can efficiently learn spatial features in tremor signals. Here, we define the embedding dimension as d. model =128.

[0051] 2) Location coding layer

[0052] The positional encoding layer is a common sine function-based encoding method in Transformer. It uses positional encoding to obtain sequential information that can replace the time series and superimposes it with the embedded deep features obtained in the previous step. Together, they are input into the Transformer encoder, which is composed of a multi-head self-attention mechanism, to finely adjust the important weights of the deep features.

[0053] Unlike other sequence models, the Transformer model abandons the use of recurrent neural networks (RNNs) as the basic model for sequence learning. Instead, it uses positional encoding to obtain sequential information that can replace the time series, and then superimposes it on the embedding obtained in the previous step. The sinusoidal function-based encoding method can obtain positional vectors, rather than numerical codes, which is beneficial for handling sample sequences of different lengths than those used during training. Given an input sequence of length n, Represents the position vector at a certain time step. It generates position vectors. The sine function is defined as follows:

[0054]

[0055] Among them, frequency As can be seen from the function definition, the frequency decreases along the vector dimension, thus the wavelength forms a geometric series from 2π to 10000·2π. Simultaneously, a linear transpose exists. Such that for any valid position pos∈{1,…,nk} in the sequence, any offset k∈{1,…,n} can be expressed as the following linear function:

[0056] T (k) E pos,: =E pos+k,: (2)

[0057] Therefore, for the embedding vector ψ(ω) corresponding to each time step pos The input vector of the model is obtained as follows:

[0058] 3) Attention mechanism

[0059] The Transformer encoder, composed solely of a multi-head self-attention mechanism and a feedforward neural network, is a model architecture that effectively maps the global dependencies between input and output. Specifically, the basic function of the multi-head self-attention mechanism in the Transformer encoder is a scaled dot product attention function, and its structure consists of… Figure 2 As shown.

[0060] The self-attention mechanism consists of multi-head attention and a feedforward neural network, which are interconnected by residual connections and normalization layers. As shown in Equation (3), the method of this invention selects a multi-stacked scaled dot product attention function to represent the embedding sequence. (Scaling factor) This avoids the risk of gradient vanishing when the dot product of vectors becomes too large. Typically, an attention function consists of a query term and key-value pairs, and can be viewed as an addressing process. Specifically, given a query vector, the similarity to the key-value pairs is calculated. The softmax function is then used to normalize the weights and append them to the value vectors to calculate the attention weights (Equation 3). Therefore, the attention mechanism alleviates the complexity of neural networks, as it eliminates the need to feed all input information into the network for computation; instead, it only requires selecting task-related feature vectors based on the weighting coefficients.

[0061]

[0062] Attention(Q,K,V)=Concat(Attention(q1,K,V),...,Attention(q n ,K,V))(4)

[0063] To obtain different representation spaces at different locations, the method of this invention combines n attention functions to utilize query terms. Multiple pieces of information are selected in parallel. In this way, each attention (often called a "head") focuses on a different part of the input information, which is then concatenated (Equation 4). It is worth noting that we first extract further information from the query terms and key-value pairs through different linear mappings before inputting it into the multi-head attention mechanism. For example... Figure 2 As shown, after multi-head attention, the method of this invention uses a one-dimensional convolutional layer as a feedforward neural network.

[0064] 4) Feature fusion and classification decision layer

[0065] Finally, there is the feature fusion and classification decision layer. The raw sensor data passes through an input embedding layer, a positional encoding layer, and a Transformer encoder with a multi-head self-attention mechanism to obtain high-level deep features with deep semantics. These high-level features, weighted by attention, are strongly correlated with tremor severity. Meanwhile, manually extracted kinematic features contain rich expert experience and can represent abstract information about dynamic handwriting. Therefore, we construct a feature fusion module before the decision layer, fusing the deep features with the manually extracted kinematic features through a concatenation operator. This introduces prior knowledge into the deep network, enabling the model to obtain more discriminative information in classification decisions. Finally, the high-level features, fused from the deep and kinematic features, are input into the classification decision layer, which consists of linear layers, and outputs the predicted probability of the corresponding tremor severity.

[0066] Using only morphological depth features may overlook global tremor information. Therefore, this invention integrates digital ink features representing dynamic strokes before the classification layer of the network. Empirically, patients with more severe tremors take longer to complete writing tasks, exhibit greater stroke tremor, and occasionally experience broken handwriting. These features may help quantify the severity of tremor. Therefore, this invention uses a one-dimensional convolutional layer and a softmax layer to obtain the final decision probability.

[0067] (5) Furthermore, the method of the present invention designs a digital ink sequence feature with multiple sensing channels (examples such as...). Figure 3 As shown in the figure, the tremor pattern of the patient during writing is characterized. The multi-sensor channel digital ink sequence features include 1) digital ink sequence; 2) handwriting dynamic features. Figure 3 In the diagram, (ac) represent electronic handwriting renderings of patients with normal, mild, and severe tremor symptoms, respectively. (dg) represent common dynamic features derived from the electronic handwriting: pen tip pressure, signal vector amplitude, velocity, and acceleration. Neurologists typically only have access to handwritten notes on paper; however, dynamic handwriting features reveal much more information.

[0068] 1) Digital Ink Sequence

[0069] The ink geometry pipeline and rendering are handled by the Wacom platform's rendering engine, which converts the sensor metadata collected by the digital tablet into a universal digital ink representation and saves it as the standard Wacom InkLayer Language (WILL). The WILL file stores the sequence information of the electronic handwriting, including the ink rendering results, the collected sensor data, and related metadata streams. We designed dedicated software for exporting the sequence information, facilitating the extraction of information describing tremors. The digital ink sequence includes the raw ink position features exported from the sensor data stream and the calculated ink dynamic features, summarized in Table 1.

[0070] Table 1. Characteristics of digital ink and its performance in handwriting.

[0071]

[0072] 1.1) Original ink location characteristics

[0073] In a three-dimensional Cartesian coordinate system, each point in electronic handwriting can be represented by three orthogonal directions: its absolute position on the paper (x-axis and y-axis) and the pressure perpendicular to the paper surface (z-axis). A digital ink sequence s of length n can be viewed as a set of discrete points within a longitudinal time interval, which can be represented in three-dimensional space as s = {(x...}i ,y i ,p i ), i = 1, 2, ..., n}, which is determined by the set of x-axis coordinates x = {x1, x2, ..., xn}. n}, y-axis coordinates combined with y = {y1, y2, ..., y n} and the set of z-axis coordinates representing the magnitude of pressure, p = {p1, p2, ..., p} n}constitute.

[0074] For the Wacom Finetip digital gel pen used in the experiment, additional features of the pen tip dynamics can be obtained, such as 1) tilt angle, which represents the angle between the pen tube and the vertical direction; 2) rotation, which is the rotation of the pen tube when signing; and 3) direction, which is the straight direction of the pen when viewed from the pen tip.

[0075] Preferably, the method of the present invention selects the tilt angle from the dynamic features of the pen tip. The specific reasons are as follows: Experimental observations in this study have found that information regarding pen tip rotation and orientation may not be important for clinical handwriting diagnosis; in fact, these spatial variables may provide some noise information, interfering with the classification algorithm, because they are expressed as a single-valued feature rather than a time-varying vector. Conversely, previous studies have demonstrated that the temporal-spatial features of the pen tip, including the tilt angle, are helpful in distinguishing between PD and healthy individuals; therefore, we primarily focus on this information.

[0076] 1.2) Dynamic characteristics of ink

[0077] After obtaining the original ink position features, the method of this invention further calculates some more specific sequence features for characterizing the dynamic handwriting process, namely, ink dynamic features. In this study, ink dynamic features include displacement, velocity, acceleration, and jerk during the digital ink drawing process.

[0078] Displacement refers to the straight-line distance between two consecutive sampling points. For a digital ink feature sequence s of length n, the displacement d = {d1, d2, ..., dn} can be obtained. n}, where the displacement of the i-th point can be calculated by formula (5),

[0079]

[0080] Furthermore, the velocity can be directly calculated from the first derivative of the measured value. The instantaneous velocity of the i-th sample point can be calculated as follows:

[0081]

[0082] Similarly, acceleration and jerk are the first and second derivatives of velocity, respectively.

[0083] The above features describe the changes in local sample points and are sensitive to the transient response variables of tremors. We also designed dynamic features for relative states, called the rates of change of radius and angle, to describe the influence of tremors on the overall shape of the spiral. They represent the degree of change of radius and angle during the drawing of the spiral, respectively, measuring the contour features of the drawn spiral and characterizing the relative spatial position of the sample point relative to the center of the spiral. The radius and angle of each sample point in the handwriting can be calculated by formula (7-8), and their first derivatives are the rates of change.

[0084]

[0085]

[0086] In addition, we also calculated the signal magnitude vector (SVM) in the spatial coordinate system of each sample point, which has been proven by multiple studies to be simple and effective in state recognition scenarios. The calculation of SVM is shown in Equation (9), where the changing amplitude represents the intensity of tremor in the writing task.

[0087]

[0088] Unlike SVM, which describes the amplitude variation of tremors in the time domain, calculating the power spectral density (PSD) P S (f) can describe the changes in flutter in the frequency domain, as shown in formula (10):

[0089]

[0090] Where S dft (f) represents the discrete Fourier transform of the SVM sequence:

[0091]

[0092] 2) Dynamic characteristics of handwriting

[0093] To facilitate deep network computation, each stroke of the handwritten handwriting is segmented into multiple equal-length time series. Although this incorporates rich temporal-space details describing pen tip movement, it is still necessary to design dynamic parameters characterizing the global features of the electronic handwriting to provide macroscopic dynamic data for the classifier. This invention calculates 60 features of the SVM sequence across various time-domain, frequency-domain, and nonlinear domains, including statistics, waveform factors, and entropy features, as shown in Table 2. Finally, an initial random forest model is trained experimentally to investigate the importance of features, and the top 20 recognized features are selected.

[0094] Table 2. Feature matrices and definitions of sensing signals from each channel during the experimental phase.

[0095]

[0096]

[0097] (6) Since the early and late stages of data acquisition are easily affected by the transition between test preparation and test stop states, the data before and after the filtered data are removed according to the time axis, and the data from the center of the time axis to 1-C% are retained. This is to extract the stable signal component from the filtered data containing attitude and jitter components output in step (2). Preferably, the value of C% is 5%.

[0098] (7) For the feature sequence, perform equal-length sliding window data amplification to keep the tremor data the same length, and then calculate the characteristics such as the time domain amplitude change, frequency domain peak power change and nonlinear entropy change of the tremor data; preferably, in order to continuously describe the tremor changes, the sliding window movement step size is set to 2s, the window duration is 5s, and the data overlap rate is 40%.

[0099] (8) For all patients after filtering, a set of features of action tremor is specified and a sample training set and a sample test set are randomly constructed in an appropriate proportion; using the expert scores obtained in step (1), a label of tremor severity is set for each data segment.

[0100] Preferably, to ensure the generalization of the trained model, a 4:1 five-fold cross-validation with a data split ratio is selected.

[0101] (9) To verify the effectiveness of the proposed system, we conducted extensive experiments: 1) proof of the effectiveness of the feature sequence; 2) ablation study; 3) model comparison and 4) robustness test.

[0102] The specific implementation principle of the method of this invention is as follows:

[0103] Specifically, in step (5), to obtain the dynamic features of the handwriting in step 2), feature preprocessing is first performed: to achieve simple and effective state recognition while reducing dependence on wearing position and movement process, signal vector amplitude (SVM) is calculated for each sensor's three-axis sequence. SVM can also reduce the complexity of vector operations on each IMU signal-sensitive axis from a macroscopic perspective. Specifically, three-axis acceleration data is used to illustrate the data analysis and feature extraction process. svm (i) represents the set acceleration of the i-th sampling point, and its calculation formula is:

[0104]

[0105] Where a x (i), a y (i) and az (i) represent the accelerations along the x, y, and z axes at the i-th sampling point, respectively. For a sampling data segment of length N, the final SVM acceleration sequence is a svm ={a svm (0),a svm (1),...,a svm (N-1)}.

[0106] Furthermore, using the SVM sequence obtained after feature preprocessing, time-domain, frequency-domain, and nonlinear features of the attitude tremor state of resting tremor are extracted from accelerometer, gyroscope, and magnetometer signals to describe the severity of the tremor, as detailed below:

[0107] Temporal characteristics: Tremors in movement disorders exhibit significant amplitude variations at rest. The method of this invention extracts 20 interpretable features from the SVM sequence of three-channel sensor signals (horizontal and vertical relative coordinates and vertical pressure value), including temporal, frequency, and nonlinear features. Specific descriptions of the feature parameters are shown in Table 1. The temporal parameters are all morphological features of the signal. Skewness and kurtosis are the third and fourth normalized moments of the signal, respectively, and their specific formulas are shown in equations (13-14).

[0108]

[0109]

[0110] Among them, a mean Represents SVM sequence a svm The sample mean.

[0111] Frequency Domain Characteristics: Frequency domain analysis in signal processing provides more intuitive parametric characteristics than time domain analysis. In the invented method, the frequency distribution of the signal is obtained from an energy perspective primarily by using spectral estimation based on short signal lengths. Errors caused by coupling are almost negligible, thus improving the signal-to-noise ratio. Power spectral density (PSD) is widely defined as the signal power per unit frequency band, reflecting the signal power distribution in the frequency domain. In the frequency domain, the power spectrum is first calculated, and then the main jitter frequency, peak power, and jitter stability index are extracted, all of which are characteristics of the jitter frequency. Furthermore, in signal processing, frequency domain analysis provides more intuitive parametric characteristics than time domain analysis. The peak power P within the signal band... m (f p ) is defined as the area under the PSD curve within the main frequency range and is calculated using formula (15).

[0112]

[0113] Frequency domain analysis in signal processing provides more intuitive parametric characteristics than time domain analysis. In the invented method, the frequency distribution of the signal is obtained from an energy perspective primarily through spectral estimation based on short signal lengths. Errors caused by coupling are negligible, thus improving the signal-to-noise ratio. Power spectral density (PSD) is widely defined as the signal power per unit frequency band, reflecting the signal power distribution in the frequency domain. The acceleration signal a represents flutter. svm Power spectral density P S (f) is calculated using formula (16-17).

[0114] Where f p ±f th This represents the bandwidth of the peak power at the dominant frequency. Multiple studies suggest that the peak power is superior to the one-sided power spectrum of the sensor signal over a period of 15 seconds within ±0.5 Hz of the dominant frequency. It is S dft The complex conjugate of (f) represents the power signal a. svm The discrete Fourier transform of can be calculated by formula (16).

[0115]

[0116] The power spectral density (PSD) P can be further calculated from formula (17). S (f):

[0117]

[0118] Preferably, the method of the present invention selects the Welch method, that is, for the entire acceleration signal a svm After segmentation, each small signal sequence is preprocessed and a Blackman window is added. Spectral estimation is performed using a piecewise average periodogram method, thereby reducing spectral leakage.

[0119] Furthermore, the method of this invention calculates the ratio of peak power to total power to represent the proportion of tremor occurrence to total recording time. The percentage of peak power in the complete power estimate should be more significant than 85%, thereby enabling determination of whether the patient is in a tremor state. The peak power of the tremor signal with respect to the PSD estimate is as follows: Figure 4 As shown, the method based on the power spectral tolerance span was used to find the peak power, and the shaded area shows the peak power of the flutter signal.

[0120] Nonlinear characteristics:

[0121] This invention utilizes multiple entropy values ​​to measure the complexity of tremor data. Approximate entropy (ApEn) is a technique for quantifying the irregularity and unpredictability of fluctuations in time series data. By comparing the contributions of frequency, RMS value, and ApEn to quantifying tremor, it can be found that ApEn has the best discriminative ability. Preferably, the embedding dimension is chosen to be m=2, the similarity tolerance is r=0.1×SD (SD is the standard deviation of the sequence), and ApEn is defined as...

[0122]

[0123] Where a svm This represents a continuous 15-second sequence segment. The average similarity rate of all m-length sub-segments in the entire sequence under the similarity criterion r can be calculated as follows:

[0124]

[0125] Sample entropy (SampEn) does not involve comparing vectors when calculating the probability of sequence self-similarity, so it is not limited by data length. In contrast, fuzzy entropy proposes an ambiguous membership function, improving the similarity measurement of binary processes. This fuzzy boundary measure enhances the complexity of the signal through fuzzy entropy, making the entropy changes more continuous and stable. Entropy features characterizing sequence complexity can significantly improve the performance of tremor quantization models, so the method of this invention calculates these nonlinear features.

[0126] Furthermore, a deep network DIT model is constructed, and the corresponding hyperparameters are set as shown in Table 3.

[0127] Table 3 Hyperparameter settings for the DIT model

[0128]

[0129]

[0130] Furthermore, the method of the present invention defines verification conditions to ensure the generalization of the classification model.

[0131] Five-fold cross-validation is used during training to reduce the bias in classification results. On a finite training set, five-fold cross-validation is the most suitable validation method, capable of training all classes without overfitting. This invention defines the absolute error for CRST classification to evaluate the performance of the automatic scoring system. For deep models... Its classification error e test The calculation is as follows:

[0132]

[0133] Where I(·) represents the index function, y i This is the consensus score of the CRST scale for the i-th ET patient by three neurologists. This is the tremor level determined by the classifier. In addition to the minimum classification error, this study also calculated the classification indices of the confusion matrix and the AUC values ​​for multiple categories.

[0134] Furthermore, the present invention defines a classification model performance evaluation method to comprehensively assess the classification performance of tremor severity. Preferably, the present invention uses four main indicators to evaluate the performance of arrhythmia detection classification results, including accuracy (ACC), sensitivity (SED), specificity (SPEC), precision (PRE), and F1 score, defined as follows (21-25).

[0135]

[0136]

[0137]

[0138]

[0139]

[0140] TP, TN, FP, and FN represent the number of correctly labeled IMU segments, correctly identified as misidentified segments, etc. Since the F1 score has equal weights on FP and FN, it provides a smaller bias than accuracy. In contrast, the Receiver Operating Characteristic (ROC) curve considers a classification threshold that balances sensitivity and specificity. The area under the curve (AUC) is often used as an evaluation metric for unevenly distributed databases.

[0141] The specific experimental results verifying the method of this invention are as follows:

[0142] (1) Validation of the effectiveness of digital ink features

[0143] Table 4 summarizes the average accuracy values ​​of different feature subsets designed in this study in the proposed automated tremor symptom assessment system. The system incorporating all dynamic features from all sensor channels exhibits the most superior performance. Furthermore, ink dynamic features calculated from the original ink sequence, as a more advanced characterization of transient changes, achieve an average accuracy of over 90% when input alone into the DIT model. It can be seen that the latter contributes to most channels of the entire dynamic feature set, providing additional information beyond morphology. This useful information includes temporal characteristics during writing; for example, patients with more severe tremor symptoms typically write slower, with more pronounced shaking manifested as frequent changes in acceleration, while healthy subjects are able to maintain a constant writing speed.

[0144] In contrast, classification performance using only single-channel sensing features such as pressure and tilt angle was the lowest. Considering only the pressure changes during writing makes it difficult to differentiate symptom severity. Rosenblum et al. demonstrated that patients often applied relatively low pressure and significant numerical fluctuations during writing, usually due to muscle difficulty. However, pen pressure is generally controversial in the literature, and providing only pressure information does risk performance loss. Especially from a signature verification perspective, pressure often leads to conflicting results. Similar to pressure, pen tilt angle's ability to discriminate tremors is also limited. Diaz et al. also suggested that tilt angle performs slightly worse than pressure in the spiral task, but in handwriting, the former yields better classification results. Most studies do not provide information on tilt angle during writing; based on our experimental results, its contribution is similar to that of single-channel pressure, and it may also have the potential for rich feature representation.

[0145] Furthermore, the relative positions of the sample points in the handwriting can greatly assist in tremor classification, with average accuracy exceeding 80% for each task. This may be attributed to the powerful information extraction capabilities of convolutional embedding layers, which can depict the morphological features of the handwriting by rendering the horizontal and vertical coordinates of the sample points. Previous image-based studies have shown that learning two-dimensional representations of handwriting can also achieve good identification results, which aligns with our findings.

[0146] In summary, the system proposed in this invention demonstrates superior recognition capabilities for small spirals. Specifically, the specificity and accuracy for severity levels reached 96.96% and 96.16%, respectively, indicating impaired fine motor control in circular movements in ET patients. Indeed, ET patients are more prone to irregular, uncontrolled movements and frequent miniaturization in challenging writing tasks. While Drottár et al. previously suggested that Archimedean spirals had no significant impact on classification, some studies have found that spirals exhibit high classification accuracy across almost all feature subsets. One reason for this may be that spirals require continuous drawing in various directions on the paper surface, better capturing the dynamic changes in handwriting from multiple angles. Furthermore, small spirals not only require longer drawing times, but their narrower boundaries also more readily induce intention tremor in subjects.

[0147] Furthermore, the straight-line drawing task is also crucial in assessing tremor symptoms. In fact, continuously writing long straight lines within varying boundaries may require a greater cognitive load, particularly high-intensity simultaneous processing, involving the coordinated participation of attention, memory, and control. Participants need to ensure the written lines remain straight while avoiding exceeding the boundaries; compared to the spiral task, the former requires greater control over wrist and even elbow translation. Hesitation or pauses between three lines can highlight the need for replanning the writing activity; when participants find the boundaries narrowing, they need more precise control over writing speed and pen tip direction. Fluent writing can indicate that participants have pre-planned movement, but ET patients often experience greater tremor as writing time increases. These findings reinforce the clinical validity of this task, as clinicians often use it to screen for early signs of ET and even PD.

[0148] The method of this invention selected the optimal feature set, namely all kinematic features. The quantitative results of tremor severity in terms of accuracy, sensitivity, specificity, precision, and F1 score are summarized in Table 5. Generally, all indicators and all tasks achieved high values, confirming the usability of the proposed method. Sensitivity and specificity usually have a trade-off; the method of this invention appears to be slightly biased towards specificity. This suggests that the screening test based on the proposed method will better classify subjects with tremor symptoms. To further validate the proposed method, we also describe (…). Figure 5 (ac) The ROC curves of the DIT with optimal parameter settings represent the classification results for the large spiral, small spiral, and straight line drawing tasks. (df) The ROC curves of the top 4 deep learning models with the best classification results. ) ROC plots for each of the three tasks, where the highest true positive rate and lowest false positive rate validate the strong discriminative ability of the proposed model, which pays more attention to patients with more severe tremors and greatly avoids the possibility of false screening.

[0149] Table 4. Contribution of Key Design Subsets of Digital Ink Features to Each Drawing Task

[0150]

[0151]

[0152] Table 5 shows the classification performance of the best-performing feature set (all kinematic features) on other evaluation metrics.

[0153]

[0154] (2) Ablation Research

[0155] It is generally believed that in traditional Transformers, cascaded self-attention modules can capture long-range feature dependencies but neglect local feature details. Therefore, the DIT model adopts a hybrid network structure, fusing convolutional operations that excel at extracting local features and concatenating manually extracted global features at the decision layer to enhance feature representation learning. Specifically, we report the results of an ablation study conducted on the selection of key design modules for the method of this invention (Table 6). These results were obtained by providing the model with all kinematic features. It was observed that using one-dimensional convolutional modules with different structures in the input embedding layer caused different performance losses. Among them, the model using only multi-layer stacked convolutional blocks performed the worst. Convolutional blocks that can fuse multi-scale information of digital ink features seem to have a greater impact than SE modules, but both can significantly improve classification performance on various tasks. Notably, our experimental results on evaluating handwriting dynamics features show that these hand-designed features have a greater impact on the performance of tremor quantization for spiral tasks than on straight-line tasks. This result is similar to previous conjectures that spiral-based handwriting tasks are more concerned with the dynamic changes of handwriting from multiple angles and have a greater dependence on global morphological features. In straight line tasks, handwriting usually exhibits a regular, stretched shape, and the information provided by global features is relatively limited.

[0156] Table 6. Contributions of Key Design Modules for Each Drawing Task

[0157]

[0158] (3) Comparison with other models

[0159] To further confirm the effectiveness of the proposed method, Table 7 lists classic sequence learning networks for comparative analysis. These networks have all achieved great success in the field of computer vision and serve as important cornerstones of deep learning, offering valuable references. These models include: 1) classic networks, such as RegNet (the optimal structure found in the network design space), ResNet with residual structures, and DenseNet with dense connections; 2) cascaded networks, such as U-Net (an encoder-decoder structure based on ResNet as a basic module) and CNN-LSTM (integrating temporal and spatial information); and 3) the latest novel model with further improvements, ResNet incorporating channel attention mechanisms. We transform it into a one-dimensional convolutional form to make it more suitable for time series classification tasks. It should be noted that the same evaluation metrics and validation scheme (five-fold cross-validation) were used during the comparison process. The algorithm complexity was measured by the total number of model parameters and floating-point operations (FLOPs).

[0160] Table 7 shows that the DIT proposed in this invention outperforms other peers on all evaluation metrics. Compared to the simple CNN-LSTM network with the fewest parameters, our proposed method improves the F1 score by 20.18%-22.79% on three tasks, but only increases the total number of parameters by 0.49M and FLOPs by only 0.12Gmac. Compared to the SEMkResNet model, which ranks second in classification performance, DIT has only 1.24% of the total number of parameters and only 34.21% of the FLOPs. It is worth noting that, unlike other models, DenseNet and SEMkResNet may be relatively better at classifying line-based tasks, and improvements based on such architectures may have the potential to further improve the classification performance of line-based tasks.

[0161] Table 7 Performance comparison with classic sequence learning models

[0162]

[0163]

[0164] (4) Robustness test

[0165] To verify the generalization ability of the method of this invention, we conducted further validation experiments on the PaHaw dataset. The Parkinson's disease handwriting database (PaHaW) collected handwriting data from 37 Parkinson's disease patients and 38 age- and sex-matched healthy controls (HC). Participants registered at the Department of Neurology I of Masaryk University and the Sánchez-Anne University Hospital in Brno, Czech Republic. All participants were asked to complete eight handwriting tasks according to a pre-filled template. In this experiment, we only selected to draw Archimedean spirals, and after deriving the original sequence features, we calculated the corresponding digital ink features according to the previous experimental steps. This dataset was not seen in our system configuration and is slightly different from the previous task of quantifying tremor. Therefore, the optimal configuration found was used here, and the softmax layer of the network was replaced with binary units to facilitate the differentiation between PD and HC. The experimental results of the method of this invention are compared with the current state-of-the-art research results, summarized in Table 8.

[0166] Drota et al. proposed using a feature subset fused from traditional, handcrafted dynamics and pressure features to build a machine learning model. This work additionally computed kinematic parameters in the horizontal and vertical directions, resulting in a large, high-dimensional feature set, thus requiring a feature selection strategy based on the overall dataset. Impedov et al. focused on supplementing the velocity features of electronic handwriting, enriching the traditional feature set through methods such as the Sigma-Lognormal model, Maxwell–Boltzmann distribution, and spectral transformation. Notably, these studies have the potential for data leakage, leading to overly optimistic results. Machine learning models typically require feature selection algorithms to filter highly relevant features, but this can introduce bias. Angelillo et al. further supplemented the model with validation work on features such as Shannon entropy and Rényi entropy, signal-to-noise ratio, and empirical mode decomposition. They used nested cross-validation, making the results relatively reliable and avoiding the adverse effects of feature selection bias.

[0167] Shallow machine learning has several limitations in identifying tremors: 1) Model performance largely depends on feature engineering, often making it difficult to extract completely effective information. 2) It is prone to the curse of dimensionality; pen-tip dynamics often have signals from many modalities, resulting in a massive amount of traditional features, and validating these useful features often requires extensive prior knowledge. 3) The model's learning ability is limited; simple mapping seems insufficient to explain the complex pathological features of movement disorders. In recent years, deep learning methods have gradually been introduced into research on the early diagnosis of PD. Due to the powerful fitting ability of deep networks and the avoidance of manually defined features, they appear to have great potential.

[0168] Moetesum et al. obtained median filter residual data and data with edge detail features from the original images, extracted depth features from them, and then fed them into a voting layer composed of multiple machine learning classifiers for final decision-making. Further, Kamran et al. used an end-to-end deep network instead and discussed in detail the impact of different image augmentation methods on the acquisition of depth features by 2D convolutional networks. However, since images are usually static, well-connected sample points struggle to reveal the dynamic temporal features of handwriting, resulting in less than ideal classification accuracy. Although transfer learning and contrast enhancement methods significantly improved classification accuracy after fusing larger datasets, this may also be a potential contribution of the sample size.

[0169] To avoid deep networks being limited by static morphological features, Diaz et al. proposed a method of directly plotting sample points to obtain temporal and velocity information, achieving a classification accuracy of 75% on a spiral task. This method is called Dynamically Enhanced Static Handwriting Representation, and the authors also demonstrate in detail its advantages over using only static image features in their paper.

[0170] Image analysis may focus more on morphological features, while directly fitting multi-channel digital ink sequences can improve the dynamic features in the time domain. Similar to our research approach, Diaz et al. used a sequence learning network cascaded with one-dimensional convolutions and RNNs to achieve optimal classification results based on dynamic features derived from the original sequences. Unfortunately, they did not discuss in detail the improvement in model performance from fusing more feature channels. However, our experimental results suggest that fusing more channels seems to have the potential to enhance the learning ability of sequence learning models. In conclusion, the system proposed in this invention outperforms all current SOFA studies, achieving better results not only in ET tremor severity classification but also showing great potential in early PD diagnosis. This confirms the effectiveness of the proposed method and makes it a viable candidate solution for practical clinical use.

[0171] Table 8 compares the performance of state-of-the-art methods on the PaHaW dataset (identification of PD patients).

[0172]

[0173]

[0174] Computer-based electronic handwriting analysis holds broad application prospects; for example, handwriting measurement can capture an individual's identity and cognitive characteristics. Patients with movement disorders often experience difficulty writing, so electronic handwriting analysis holds promise for aiding early clinical diagnosis and even automatically quantifying symptom severity. Domain experts can access these easy-to-use, user-friendly tools in their routine checks without requiring any specific computer expertise. Therefore, automated quantification systems based on electronic handwriting have significant clinical value because they not only provide professionals with automated decision-making support but also store patient-related metadata to support retrospective research and analysis. The decision-making system proposed in this invention can provide additional objective evidence to support the final clinical assessment.

[0175] This invention collects real electronic handwriting samples from clinical settings and uses rigorous data labeling established by an expert committee to create a usable handwritten database. To aid clinical decision-making and increase diagnostic efficiency, this invention proposes a novel DIT model based on multimodal digital ink features. Computer-aided diagnostic systems can automatically learn features significant for differentiating tremor symptoms using dynamic feature sets from different perspectives. The combination of CNN and Transformer effectively learns dynamic spatiotemporal information in electronic handwriting sequences, and by fully integrating manually extracted global dynamic features, it shows great potential for both ET tremor quantification and early PD diagnosis. Our experiments, including feature subset effectiveness evaluation, ablation studies, model comparison, and robustness testing, demonstrate the effectiveness of the proposed system. This research shows better application prospects compared to state-of-the-art technologies. The proposed method maintains a good AUC when predicting multiple classes, exhibiting best performance for current automatic ET symptom identification and early PD diagnosis. These results indicate that the proposed method is suitable for applying standardized laboratory tests to help clinicians automatically score complex or early ET cases, aiding decision-making and improving disease management efficiency.

[0176] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An electronic handwriting-based tremor symptom assessment method, which utilizes a tremor symptom assessment model for assessment; The tremor symptom assessment model is based on a Transformer model, comprising: an input embedding layer, a position encoding layer, a Transformer encoder, a feature fusion and classification decision layer; an input embedding layer, which inputs a digital ink sequence from a handwriting trajectory in the form of a multi-dimensional channel sequence, converts the input sensor sequence into embedded first deep features after passing through multiple layers of MSSE-ResNet; a position encoding layer, which uses position encoding based on the encoding method of the sine function to obtain sequential information that can replace the time sequence; a Transformer encoder, which is a model architecture composed of multi-head self-attention mechanism and feedforward neural network, and can draw global dependency between input and output; the embedded first deep features and sequential information are input into the Transformer encoder after superposition to obtain second deep features with deep semantics; a feature fusion and classification decision layer includes a feature fusion module and a classification decision module; the feature fusion layer is located before the classification decision layer; the feature fusion layer fuses the second deep features with kinematic features through a concatenation operator to form third deep features; the third deep features are input into the classification decision layer composed of linear layers, and the corresponding prediction probability of tremor severity is output.

2. The electronic handwriting-based tremor symptom assessment method according to claim 1, wherein: the multi-layer MSSE-ResNet includes multiple MSSE-ResNet blocks; each block is composed of multi-scale convolution and SE-ResNet; wherein the multi-scale convolution kernel size is 3, 5, 7; each convolution layer is composed of convolution-batch normalization-ReLu activation function; the input of the SE module is multi-scale features, and the output is deep features containing channel attention weights obtained through residual structure.

3. The electronic handwriting-based tremor symptom assessment method according to claim 1, wherein: the basic function of the multi-head self-attention mechanism in the Transformer encoder is the scaled dot-product attention function.

4. The electronic handwriting-based tremor symptom assessment method according to claim 1, wherein: the digital ink sequence includes original ink position features derived from sensor data streams and calculated ink dynamic features.

5. The electronic handwriting-based tremor symptom assessment method according to claim 1, wherein: the digital ink sequence includes the following features: ink position, pressure, signal vector amplitude of electronic ink, inclination, displacement, velocity, acceleration, jerk, radius variability, angle variability, spectral density, kinematic features.

6. The electronic handwriting-based tremor symptom assessment method according to claim 5, wherein: the kinematic features are obtained from the digital ink sequence, which includes: Sample entropy of signal vector amplitude (SVM_SampEn), average rectification of signal vector amplitude (SVM_Arv), variance of signal vector amplitude (SVM_Var), kurtosis of signal vector amplitude (SVM_Kurt), waveform factor of signal vector amplitude (SVM_SF), peak factor of signal vector amplitude (SVM_CF), impulse factor of signal vector amplitude (SVM_IF); Peak power ratio of radius variability (RV_PwrP_R), principal peak of radius variability (RV_PrinP), fuzzy entropy of radius variability (RV_FuzEn), mean of radius variability (RV_Mean), average rectification of radius variability (RV_Arv), peak factor of radius variability (RV_CF); Peak power of angle variability (AV_PwrP), peak power ratio of angle variability (AV_PwrP_R), variance of angle variability (AV_Var), root mean square of angle variability (AV_RMS), waveform factor of angle variability (AV_SF), mean of angle variability (AV_RMS), maximum of angle variability (AV_Max).

7. An electronic handwriting-based tremor symptom assessment device, comprising a computing unit configured to execute a tremor symptom assessment model; The tremor symptom assessment model is based on a Transformer model, including: an input embedding layer, a position encoding layer, a Transformer encoder, a feature fusion and classification decision layer; the input embedding layer inputs a digital ink sequence from the handwriting, in the form of a multi-dimensional channel sequence, and converts the input sensor sequence into embedded first deep features after passing through a multi-layer MSSE-ResNet; the position encoding layer uses position encoding based on a sinusoidal function coding method to obtain sequential information that can replace the time sequence; the Transformer encoder is a model architecture composed of a multi-head self-attention mechanism and a feed-forward neural network, which can draw global dependencies between input and output; the embedded first deep features and the sequential information are input into the Transformer encoder to obtain second deep features with deep semantics; the feature fusion and classification decision layer includes a feature fusion module and a classification decision module; the feature fusion layer is located before the classification decision layer; the feature fusion layer fuses the second deep features with kinematic features through a concatenation operator to form third deep features; the third deep features are input into the classification decision layer composed of linear layers to output the predicted probability of the corresponding tremor severity.

8. The electronic handwriting-based tremor symptom assessment device according to claim 7, wherein: the multi-layer MSSE-ResNet includes a plurality of MSSE-ResNet blocks; each block is composed of multi-scale convolution and SE-ResNet; wherein the multi-scale convolution kernel size is 3, 5, and 7; each convolution layer is composed of convolution-batch normalization-ReLu activation function; the input of the SE module is multi-scale features, and the output is deep features containing channel attention weights obtained through the residual structure.

9. The electronic penmanship based tremor symptom assessment device of claim 7, wherein: The basic function of multi-head self-attention mechanism in Transformer encoder is scaled dot-product attention function.

10. The electronic penmanship based tremor symptom assessment device of claim 7, wherein: The digital ink sequence includes original ink position features derived from the sensor data stream and calculated ink dynamic features.

11. The electronic penmanship based tremor symptom assessment device of claim 7, wherein: The digital ink sequence includes the following features: Ink position, pressure, signal vector amplitude of electronic ink, skew, displacement, velocity, acceleration, jerk, radius variability, angle variability, spectral density, kinematic features.

12. The electronic penmanship based tremor symptom assessment device of claim 11, wherein: The kinematic features are obtained from the digital ink sequence, which includes: Sample entropy of signal vector amplitude (SVM_SampEn), average rectified value of signal vector amplitude (SVM_Arv), variance of signal vector amplitude (SVM_Var), kurtosis of signal vector amplitude (SVM_Kurt), waveform factor of signal vector amplitude (SVM_SF), peak factor of signal vector amplitude (SVM_CF), impulse factor of signal vector amplitude (SVM_IF); peak power proportion of radius variability (RV_PwrP_R), principal peak power of radius variability (RV_PrinP), fuzzy entropy of radius variability (RV_FuzEn), mean of radius variability (RV_Mean), average rectified value of radius variability (RV_Arv), peak factor of radius variability (RV_CF); peak power of angle variability (AV_PwrP), peak power proportion of angle variability (AV_PwrP_R), variance of angle variability (AV_Var), root mean square of angle variability (AV_RMS), waveform factor of angle variability (AV_SF), mean of angle variability (AV_RMS), maximum of angle variability (AV_Max).

Images