Hand tremor recognition system based on improved mobilevit

Abstract

The application discloses a hand tremor recognition system based on an improved MobileViT, which comprises a data acquisition module, an Archimedes spiral hand drawing drawn by a tester on a collection table; a data preprocessing module, which performs data preprocessing on the Archimedes spiral hand drawing collected by the data acquisition module; and a recognition module, which classifies the image processed by the data preprocessing module by using a trained and tested improved MobileViT lightweight network PD-Net. The application can obtain rich and important global and local information, including tremor and spacing features, and the information is effectively captured by applying an attention mechanism. The classification network can better complete the classification task and obtain accurate classification results, thereby improving the performance of the entire deep attention network model.

Description

Technical Field

[0001] This invention belongs to the field of recognition technology and relates to a hand tremor recognition system based on an improved MobileViT lightweight network. Background Technology

[0002] Hand tremor is a common movement disorder, especially prevalent among the elderly. Hand tremor manifests as involuntary shaking or trembling of the hands and may be a precursor to Parkinson's disease. Monitoring hand tremors can predict the presence of Parkinson's disease. Currently, clinical diagnosis of Parkinson's disease by observing hand tremors typically requires doctors to visually assess the amplitude of the tremors, a method demanding extensive clinical experience for accurate judgment. To address this issue, computer-aided diagnostic techniques have been introduced, analyzing the characteristics of hand tremors to diagnose the severity of tremors and thus predict the likelihood of Parkinson's disease. However, misdiagnosis still occurs due to factors such as insufficient doctor experience. In recent years, some researchers have proposed using machine learning or deep learning techniques to identify hand-drawn spirals to determine if a patient has severe hand tremors. However, current methods require large acquisition devices and their accuracy is not very high. Therefore, further research and improvement of these methods are needed to enhance the accuracy and reliability of identification.

[0003] Therefore, this invention proposes an improved lightweight neural network MobileViT hand tremor recognition system, which uses hand-drawn spiral lines to achieve rapid and convenient recognition of hand tremors.

[0004] Although research on hand tremor recognition based on hand-drawn spirals has achieved some results and established various handwritten datasets, two problems still exist:

[0005] ① Low recognition accuracy. Most current spiral feature classification methods employ fixed polar coordinate expansion algorithms, which limits the feasibility of experiments as most handwritten datasets can only be drawn on prescribed writing templates. However, medical research shows that Parkinson's disease patients experience visual impairments, such as decreased spatial contrast sensitivity, after basal ganglia nerve damage. Using fixed templates interferes with the patient's writing behavior and obscures the original disease characteristics. Traditional CNNs cannot effectively extract key features such as tremors and spacing from handwriting; they treat handwriting as a type of image recognition, failing to highlight key features and resulting in low recognition accuracy. Therefore, new methods are needed to address these issues to better detect hand tremors and predict the possibility of Parkinson's disease.

[0006] ② The collection of handwriting data requires specialized equipment. The collection of dynamic handwriting characteristics, such as pressure and acceleration, can only be accomplished with specialized data acquisition equipment, which is inconvenient for Parkinson's disease patients to perform early self-diagnosis.

[0007] Therefore, there is an urgent need for a recognition system that can eliminate the need for specialized large-scale data acquisition equipment and effectively extract features such as tremors and spacing in handwriting. Summary of the Invention

[0008] The purpose of this invention is to solve the above-mentioned technical problems and provide a hand tremor recognition system based on an improved MobileViT lightweight network.

[0009] In a first aspect, the present invention provides a hand tremor recognition system based on an improved MobileViT lightweight network, comprising:

[0010] The data acquisition module obtains the Archimedean spiral hand-drawn by the tester on the data acquisition form;

[0011] The data preprocessing module performs data preprocessing on the hand-drawn Archimedean spiral diagram acquired by the data acquisition module;

[0012] The recognition module uses the trained and tested improved MobileViT lightweight network PD-Net to classify the images processed by the data preprocessing module.

[0013] The improved MobileViT lightweight network PD-Net inserts a CBAM attention mechanism module after the first MobileViT block module of the original MobileViT network, a feature extraction module FR after the second MobileViT block module, and an SE attention mechanism module after the last MobileViT block module and before the last convolutional layer in the MV2 module.

[0014] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the system.

[0015] Thirdly, the present invention provides a computing device, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the system.

[0016] The present invention has the following advantages:

[0017] The two attention mechanisms introduced in this invention, SE (Squeeze-and-Excitation) and CBAM (Convolutional Block Attention Module), significantly improve the feature extraction process of the model. These attention mechanisms allow the model to focus its attention on regions related to the spiral, thereby ignoring surrounding blank areas. This enables the model to acquire more distinctive features and further improves its robustness to various spiral shapes and scales.

[0018] To enhance the model's ability to learn substantial variations in spiral features (such as spacing and tremors), this invention introduces a Feature Refinement (FR) module. The FR module first performs two consecutive convolutions on the input feature map. Pooling layers are omitted to prevent them from filtering out key features such as tremors and spacing. The convolutionally processed feature map is further weighted using a channel attention mechanism (CA) to optimize channel weights. Then, the map is split into two paths: one path continues through convolutional layers to extract higher-dimensional features, resulting in channel weights containing high-dimensional features; the other path is superimposed on the original feature map to increase the weights of key positions with important features in the original map. Multiplying the results of these two paths—that is, applying the channel weights to the feature map—uncovers deeper feature information, effectively utilizing the multi-scale and multi-level information of the spiral features. This module effectively learns and captures substantial variations in spiral features while suppressing features unrelated to these variations. For example, noise around the curve and line thickness are suppressed, thus enhancing the network's sensitivity to variations.

[0019] Through these improvements, the model of this invention exhibits better performance when handling helical-related tasks. It can more accurately extract and identify various helical shapes and scales, and has higher sensitivity to substantial changes in helical features. The PD-Net model applied in this invention performs well in helical recognition, achieving an accuracy of 98.75% on the dataset. Attached Figure Description

[0020] Figure 1 This invention improves the model architecture of the MobileViT lightweight network;

[0021] Figure 2 This is a schematic diagram of the SE module;

[0022] Figure 3 This is a structural diagram of the MV2 module;

[0023] Figure 4This is a schematic diagram of the channel attention mechanism module in CBAM;

[0024] Figure 5 This is a schematic diagram of the structure of the multilayer perceptron network in the channel attention mechanism module.

[0025] Figure 6 This is a schematic diagram of the spatial attention mechanism module in CBAM;

[0026] Figure 7 This is a schematic diagram of the CBAM attention mechanism module.

[0027] Figure 8 This is a schematic diagram of the FR module. Detailed Implementation

[0028] The present invention will be further analyzed below with reference to specific embodiments.

[0029] A hand tremor recognition system based on an improved MobileViT lightweight network, comprising:

[0030] 1. Data acquisition module: acquires the Archimedean spiral hand-drawn by the tester on the data acquisition form;

[0031] Based on observations of the dataset, hand-drawn spiral images exhibit characteristics such as line tremors, irregular shapes, and varying spacing between lines on opposite sides. Compared to healthy individuals, hand-drawn images from Parkinson's disease (PD) patients are more complex. Specifically, the lines in these images show localized tremors, and the shape is no longer a regular Archimedean spiral. This necessitates careful consideration of feature extraction during the design of the classification network. While pooling layers, a common layer structure in convolutional neural networks, can improve accuracy in general image recognition tasks, they can lead to the loss of key features in classifying spirals from Parkinson's patients. In fact, when an image passes through a pooling layer, key features such as localized tremors may be filtered out, resulting in unsatisfactory recognition results. Therefore, when designing a classification network for spirals from Parkinson's patients, the use of pooling layers must be carefully chosen. Simultaneously, the application of attention mechanisms allows the model to focus more on the main pixels, i.e., the spiral portion, ensuring accurate feature extraction and recognition results.

[0032] This invention establishes a dataset based on hand-drawn Archimedes' spiral. This dataset has no template reference and no application scenario limitations, and can accurately reflect the tremor, shape, and spacing characteristics of images. The dataset contains 144 hand-drawn images of Parkinson's patients and 50 hand-drawn images of healthy controls. These data came from Zhejiang Hospital in China and were approved by the hospital's ethics committee. The Archimedes' spiral pattern was chosen for data collection for two main reasons: firstly, the spiral performs best in classification experiments; secondly, the spiral drawing task is not limited by language or educational background, making it more practical. During the collection of handwritten spiral data, each participant used a ballpoint pen to draw the image on a designated data collection form. The drawing process was conducted without interfering with the participant's movements, and no drawing template was used. After a series of preprocessing steps, the images retain a range of features related to Parkinson's disease, including tremor.

[0033] 2. A data preprocessing module performs data preprocessing on the hand-drawn Archimedean spiral image acquired by the data acquisition module; specifically, the data preprocessing involves adjusting the original image to a fixed size and expanding the dataset through a series of data augmentation operations.

[0034] Cropping out an area of ​​appropriate size from the boundaries of the spiral in the hand-drawn diagram; then adjusting all images to 224*224 size; then, in order to prevent overfitting and enhance the generalization ability of the deep attention network model, data augmentation is required on the image data in the training set, including one or more methods: rotation, mirroring, and adding random Gaussian noise.

[0035] This invention utilizes a hand-drawn Archimedes spiral dataset. After digitization scanning, the images are randomly rotated, randomly mirrored, and have random Gaussian noise added to increase the number of images. The augmented images are then used as input to a classification network. During data acquisition, all testers used ordinary ballpoint pens and A4 paper to draw the spiral. The resulting hand-drawn image dataset accurately reflects the tremors, shape, and spacing characteristics of the spiral images, and image augmentation improves the performance of the image classification network. The original scanned images are first resized to 224*224 pixels, then randomly rotated clockwise by 90, 180, and 270 degrees, followed by random mirroring and the addition of random Gaussian noise to obtain the final augmented image dataset. The distribution of the augmented dataset is as follows: the training set contains 1152 hand-drawn images from PD patients and 456 from healthy individuals; the test set contains 576 hand-drawn images from PD patients and 144 from healthy individuals.

[0036] 3. Recognition module: The improved MobileViT lightweight network, which has been trained and tested, is used to classify the images processed by the data preprocessing module;

[0037] Hand-drawn images are a common method for assisting in the diagnosis of hand tremors. The basic process involves extracting various features from hand-drawn images, training the dataset using various algorithms, and ultimately predicting the severity of hand tremors. To better extract features from hand-drawn images, this invention uses MobileViT, a lightweight neural network structure combining traditional CNNs and VisionTransformers, as the baseline network. Furthermore, SE attention mechanism modules, CBAM attention mechanism modules, and FR feature extraction modules are added to the baseline network. Traditional CNN networks may lose some key features during local feature extraction, especially during pooling operations. While current Transformer networks can consider information from the entire image simultaneously and achieve global processing through self-attention mechanisms, pure Transformers are insensitive to spatial location information and lack spatial inductive bias. However, location information is crucial in visual applications. To address this issue, positional encoding is introduced, leading to the Vision Transformer. The Vision Transformer is sensitive to the input image size; modifying the input image size requires fine-tuning the absolute positional bias. MobileVit is a variant of VisionTransformer, combining CNN and Transformer. The CNN architecture itself has spatial inductive bias, which solves the positional bias problem while effectively showcasing the advantages of the Transformer model. Furthermore, adding CNN accelerates network convergence and makes the network more stable. MobileVit alone already performs well for image classification tasks, but adding an attention mechanism to the original MobileVit network can further enhance its performance.

[0038] Image processing method (formally feeding images into the network for training). Analysis of original hand-drawn sketches from PD patients revealed features including line tremors, variations in line spacing, and irregular shapes. Therefore, this invention employs the PD-Net network to extract these features for automatic classification of PD hand-drawn sketches. The self-made dataset used in this invention is divided into a PD patient group and a healthy control group. The augmented dataset is then randomly divided into training and test sets. The training set is fed into the neural network for training, while the test set is used to verify whether the model is overfitting. The model uses the AdamW gradient descent algorithm with a cross-entropy loss function. The batch size is set to 16, the initial learning rate is 0.0002, and the iterations are performed for 200 epochs. The model converges well.

[0039] The improved MobileViT lightweight network (hereinafter referred to as "PD-Net") adds a CBAM attention mechanism and a feature refinement (FR) module after the first and second MobileViTblock modules of the original MobileViT network, respectively. It also adds an SE attention mechanism module after the final MobileViTblock module and before the last convolutional layer in the MV2 module. By adding attention mechanisms, the output feature maps with rich semantics are enriched, thereby improving feature extraction and recognition accuracy. According to the dataset used in this invention, the PD-Net network input image size is 224x224, and the output is divided into two categories: normal individuals and PD patients.

[0040] The PD-Net (see appendix) Figure 1 Specifically, it consists of a 2D convolutional layer with a kernel size of 3*3 and a stride of 2, an MV2 module with a stride of 1, an MV2 module with a stride of 2, two MV2 modules with a stride of 1, an MV2 module with a stride of 2, a MobileViTblock (N=2) module (N represents the number of transformer blocks in the MobileViTblock module, the same below), a CBAM module, an MV2 module with a stride of 2, a MobileViTblock (N=4) module, an FR module, an MV2 module with a stride of 2, a MobileViTblock (N=3) module, an SE module, a 2D convolutional layer with a kernel size of 1*1 and a stride of 1, a global average pooling layer, and a fully connected layer;

[0041] SE module (see appendix) Figure 2 This is used to process the input feature map in channels through a global average pooling layer, compressing the two-dimensional feature (h*w) of each channel into a single real number. Then, it is passed through two fully connected layers to reduce the dimensionality and then increase it. The two fully connected layers are used to build the correlation between channels. The number of output weights is the same as the number of channels in the input feature map. Then, it is passed through a sigmoid activation function to obtain normalized weights. Finally, the normalized weights are multiplied with the original map channel by channel to generate a weighted feature map.

[0042] MV2 module (see attachment) Figure 3The proposed approach involves inserting an SE attention mechanism module before the last convolutional layer in the existing MV2 module. The existing MV2 module is an inverse residual structure from MobileNetV2. This structure first increases the dimensionality through a 1*1 convolution, followed by a batch normalization layer and a ReLU6 activation function, then through a 3*3 depthwise separable convolutional layer, followed by another batch normalization layer and a ReLU6 activation function, and finally through a 1*1 convolutional layer for dimensionality reduction.

[0043] The CBAM module combines Channel Attention (CA) and Spatial Attention (SA) mechanisms. The Channel Attention mechanism performs global max pooling and global average pooling on the original image, resulting in two feature maps. These two feature maps are then passed through the same multilayer perceptron network. This network consists of one fully connected layer for dimensionality reduction, one ReLU activation layer, and one fully connected layer for dimensionality increase. The two feature vectors are then superimposed, and a Sigmoid activation layer normalizes the channel weights to between 0 and 1, outputting the channel attention vector. The spatial attention mechanism processes the output feature map of the channel attention mechanism in the spatial domain. It takes the improved feature values ​​from the channel attention mechanism as input, performs max pooling and average pooling in the channel dimension, stacks the two feature maps in the channel dimension after pooling, fuses the channel information through a convolutional layer, and normalizes the spatial weights using the Sigmoid activation function. Finally, it multiplies the input feature map with the weights. The CBAM module is placed after the fourth MV2 module because when the network is deep, the attention mechanism can capture high-level features, making the network expression more diverse, while avoiding the loss of more image detail features in higher layers, thus obtaining feature maps with important features.

[0044] The channel attention mechanism (CA) module in CBAM (see appendix) Figure 4 First, the input feature maps are subjected to global max pooling and global average pooling respectively, compressing the feature maps based on two dimensions to obtain two feature descriptions with different dimensions; the pooled feature maps share a single multilayer perceptron network (see appendix). Figure 5 First, the number of channels is reduced by a fully connected layer, followed by a ReLU activation function layer, and then the number of channels is restored by another fully connected layer. The two feature maps are stacked in the channel dimension, and the weights of each channel of the feature map are normalized to between 0 and 1 by the Sigmoid activation function. The normalized weights are then multiplied by the input feature map.

[0045] The spatial attention mechanism (SA) module in CBAM (see appendix) Figure 6The output feature map of the channel attention mechanism is processed in the spatial domain. First, max pooling and average pooling are performed on the input feature map in the channel dimension, and the two pooled feature maps are stacked in the channel dimension. Then, a 7*7 convolutional kernel is used to fuse the channel information, and the feature map of size [h,w,c] is transformed from [h,w,2] to [h,w,1], where h, w, and c represent the height, width, and number of channels of the feature map, respectively. Finally, the spatial weights of the feature map are normalized by the sigmoid function after the convolution result is applied, and then the input feature map is multiplied by the weights.

[0046] The overall structure of the CBAM (see appendix) Figure 7 The input feature map first passes through the channel attention mechanism, multiplying the channel weights with the input feature map, and then feeds it into the spatial attention mechanism. The normalized spatial weights are multiplied with the input feature map of the spatial attention mechanism to obtain the final weighted feature map.

[0047] The CBAM attention mechanism module includes a channel attention mechanism and a spatial attention mechanism. CBAM starts from the two domains of channel and space, and introduces two analysis dimensions of spatial attention and channel attention to realize a sequential attention structure from channel to space. Spatial attention can make the neural network pay more attention to the pixel regions in the image that play a decisive role in classification and ignore irrelevant regions. Channel attention is used to process the allocation relationship of feature map channels. At the same time, the attention allocation of the two dimensions enhances the effect of the attention mechanism on improving model performance.

[0048] The FR module uses the channel attention mechanism CA module in CBAM. The original image first undergoes two convolutions to initially extract features. Then, the CA channel attention mechanism is used to obtain appropriate channel weights for the feature map in the channel dimension. The obtained feature map is divided into two paths. One path is superimposed on the original image, and the other path is multiplied with the superimposed result. Finally, a 1*1 convolutional layer is used to adjust the output image channels to be consistent with the input image.

[0049] The overall structure of the FR module (see appendix) Figure 8 The input feature map is denoted by f. Feature map f is fed into a 3x3 convolutional layer and a ReLU activation function layer, then again through another 3x3 convolutional layer and a ReLU activation function layer, followed by a CA channel attention mechanism. The resulting feature map is called f1, which can be expressed by the following formula:

[0050] f1=CA(Conv3(ReLU(Conv3(ReLU(f)))))

[0051] Conv3 represents a 3x3 convolutional kernel, ReLU represents the ReLU activation function, and CA represents the CA channel attention mechanism module. After performing the above operations, more useful feature information is obtained, and then this feature information is processed again using a 3x3 convolutional layer and the ReLU activation function. This process can be represented as follows:

[0052] f2 = Conv3(ReLU(f1))

[0053] f2 represents the processed feature map. Then, feature map f1 is added to the original map f, and the result is multiplied by feature map f2 to obtain feature map f3. This process is described as follows:

[0054]

[0055] Represents element-wise multiplication. This represents element-wise addition.

[0056] Finally, f3 is used to adjust the channels of the output image to match the original image by passing through a convolutional layer with a kernel size of 1*1.

[0057] The channel attention SE module specifically includes: a global average pooling layer, two fully connected layers, a Swish layer, and a Sigmoid layer. The feature map of size [h, w, c] is first subjected to global average pooling, where h, w, and c represent the height, width, and number of channels of the feature map, respectively. After compression by global average pooling, a feature vector of size [1, 1, c] is obtained. This vector then passes through a fully connected layer, reducing the channel dimension of the feature map vector to 1 / r of the original, i.e., [1, 1, c * 1 / r], where r represents the compression ratio. It then undergoes a non-linear transformation operation using the Swish activation function. Next, another fully connected layer elevates the feature map vector back to its original size [1, 1, c]. Finally, a Sigmoid activation function transforms it into a normalized weight vector of c feature maps between 0 and 1. The normalized weights are then multiplied channel-by-channel by the original input feature map to generate a weighted feature map with tremor information.

[0058] After configuring the appropriate hyperparameters, the data is fed into PD-Net for training, and the accuracy is calculated for both the training and test sets. Four metrics—Accuracy, Precision, Recall, and F1 Score—are used to measure the recognition performance. Accuracy represents the proportion of correctly predicted results out of the total observations; Precision is the proportion of correctly predicted positive examples; Recall is the proportion of correctly predicted positive examples; and the F1 Score is the harmonic mean of Precision and Recall, ranging from 0 to 1, with higher values ​​indicating more accurate output. The calculation method is as follows:

[0059]

[0060]

[0061]

[0062]

[0063] Where TP represents the number of positive samples that were correctly classified; FP represents the number of negative samples that were incorrectly identified as positive samples; FN represents the number of positive samples that were incorrectly identified as negative samples; and TN represents the number of negative samples that were correctly classified.

[0064] This invention evaluates and compares the original MobileViT network with the PD-Net network model described in this invention on a test dataset. To demonstrate the effectiveness of the CBAM, FR, and SE modules, a series of ablation experiments were conducted. The experimental results are shown in Table 1.

[0065] Table 1. Classification results of the method of the present invention and its ablation experiments

[0066] Methods / Evaluation Indicators accuracy Accuracy Recall rate F1 score Baseline network 97.36％ 91.39％ 95.83％ 93.56％ Baseline Network + CBAM 98.33％ 95.21％ 96.53％ 95.86％ Baseline network + FR 98.47％ 94.04％ 98.61％ 96.27％ Baseline network + SE 98.19％ 93.96％ 97.22％ 95.56％ PD-Net 98.75％ 95.92％ 97.92％ 96.91％

[0067] The original MobileViT is represented by the "baseline network". "Baseline network + CBAM" means adding a CBAM module to the original MobileViT, "Baseline network + FR" means adding a FR module to the original MobileViT, and "Baseline network + SE" means adding an SE module to the original MobileViT. PD-Net is a deep attention network formed by superimposing CBAM, FR, and SE modules on the baseline network. It can be seen that the classification accuracy of the original MobileViT is 97.36%, and the classification accuracy of the improved PD-Net used in this invention can reach 98.75%. The recall and F1 score of PD-Net are 97.92% and 96.91%, respectively, which are 2.09% and 3.35% higher than the original MobileViT method. As shown in Table 1, adding CBAM, FR, or SE modules in this invention all result in higher classification accuracy than the original MobileViT, and the classification accuracy of PD-Net, which integrates CBAM, FR, and SE modules, is further improved.

[0068] This invention combines image preprocessing, the construction and training of a deep attention network model, and testing, which greatly assists in subsequent research on hand tremors, such as the early diagnosis of Parkinson's disease and essential tremor.

Claims

1. A hand tremor recognition system based on an improved MobileViT lightweight network, characterized in that... include: The data acquisition module obtains the Archimedean spiral hand-drawn by the tester on the data acquisition form; The data preprocessing module performs data preprocessing on the hand-drawn Archimedean spiral diagram acquired by the data acquisition module; The recognition module uses the trained and tested improved MobileViT lightweight network PD-Net to classify the images processed by the data preprocessing module. The improved MobileViT lightweight network PD-Net is achieved by inserting a CBAM attention mechanism module after the first MobileViT block module of the original MobileViT network, a feature extraction module FR after the second MobileViT block module, and an SE attention mechanism module after the last MobileViT block module. The input feature map of the feature extraction module FR is denoted by f. The feature map f is input into a 3×3 convolutional layer and a ReLU activation function layer, and then similarly passed through another 3×3 convolutional layer and a ReLU activation function layer. Finally, it passes through the channel attention mechanism part of the CBAM attention mechanism module to obtain the feature map. It can be expressed by the following formula: Equation (1) Where Conv3 represents a 3×3 convolution kernel, ReLU represents the ReLU activation function, and CA represents the channel attention mechanism; Then the feature map This feature information is processed using a 3×3 convolutional layer and the ReLU activation function. This process is represented as follows: Equation (2) in The representative represents the processed feature map, and then the feature map is... Add the result to the input feature map f, and then add the result to the feature map f. Multiply to obtain the feature map The process is described as follows: Equation (3) in Represents element-wise multiplication. This represents element-wise addition. Finally, the feature map The output image channels are adjusted to match the original image by using a convolutional layer with a kernel size of 1×1.

2. The system according to claim 1, characterized in that... The improved MobileViT lightweight network PD-Net includes, in sequence, a first convolutional layer, a first MV2 module, a second MV2 module, a third MV2 module, a fourth MV2 module, a first MobileViTblock module, a CBAM attention mechanism module, a fifth MV2 module, a second MobileViT block module, a feature extraction module FR, a sixth MV2 module, a third MobileViT block module, an SE attention mechanism module, a second convolutional layer, a global average pooling layer, and a fully connected layer.

3. The system according to claim 2, characterized in that, The kernel size of the first convolutional layer is 3×3, and the stride is 2; The step size of the first MV2 module and the third MV2 module is 1; The step size of the second MV2 module, the fourth MV2 module, the fifth MV2 module, and the sixth MV2 module is 2; The first MobileViT block module includes two transformer blocks; The second MobileViT block module includes four transformer blocks; The third MobileViT block module includes three transformer blocks; The kernel size of the second convolutional layer is 3×3, and the stride is 1.

4. The system according to claim 1, characterized in that... The SE attention mechanism module is used to process the input feature map in channels through a global average pooling layer, compressing the two-dimensional feature of each channel into a single real number. Then, it is passed through two fully connected layers to reduce the dimensionality and then increase it. The two fully connected layers are used to construct the correlation between channels. The number of output weight values ​​is the same as the number of channels in the input feature map. Then, the normalized weights are obtained by passing through the Sigmoid activation function. Finally, the normalized weights are multiplied by the original map channel by channel to generate a weighted feature map.

5. The system according to claim 2, characterized in that... The first MV2 module, the second MV2 module, the third MV2 module, the fourth MV2 module, the fifth MV2 module, and the sixth MV2 module all insert an SE attention mechanism module before the last convolutional layer in the existing MV2 module.

6. The system according to claim 1, characterized in that... The CBAM attention mechanism module combines channel attention and spatial attention mechanisms. The input feature map first passes through the channel attention mechanism, multiplies the channel weights with the input feature map, and then feeds it into the spatial attention mechanism. The normalized spatial weights are multiplied with the input feature map of the spatial attention mechanism to obtain the final weighted feature map. The channel attention mechanism first performs global max pooling and global average pooling on the input feature map, compressing the feature map along two dimensions to obtain two feature descriptions with different dimensions. The pooled feature maps share a multilayer perceptron network, first passing through a fully connected layer to decrease the number of channels, then through a ReLU activation function layer, and finally through another fully connected layer to restore the number of channels. The two feature maps are stacked along the channel dimension, and the weights of each channel of the feature map are normalized to between 0 and 1 by the Sigmoid activation function. The normalized weights are then multiplied by the input feature map. The spatial attention mechanism described above processes the output feature map of the channel attention mechanism in the spatial domain. First, the input feature map is subjected to max pooling and average pooling in the channel dimension, and the two pooled feature maps are stacked in the channel dimension. Then, a 7×7 convolutional kernel is used to fuse the channel information, and the feature map of size [h,w,c] is transformed from [h,w,2] to [h,w,1], where h, w, and c represent the height, width, and number of channels of the feature map, respectively. Finally, the spatial weights of the feature map are normalized by the sigmoid function after the convolution result is applied, and the input feature map is multiplied by the weights.

7. The system according to claim 1, characterized in that... The SE attention mechanism module consists of a global average pooling layer, two fully connected layers, a Swish layer, and a Sigmoid layer. The feature map of size [h, w, c] is first processed by global average pooling, where h, w, and c represent the height, width, and number of channels of the feature map, respectively. After compression by global average pooling, a feature vector of size [1, 1, c] is obtained. This vector then passes through a fully connected layer, reducing the channel dimension of the feature map vector to 1 / r of its original size, i.e., [1, 1, c × 1 / r], where r represents the compression ratio. The vector then undergoes a non-linear transformation using the Swish activation function. Next, another fully connected layer elevates the feature map vector back to its original size [1, 1, c]. Finally, the Sigmoid activation function transforms it into a normalized weight vector of c feature maps between 0 and 1. The normalized weights are then multiplied channel-by-channel by the original input feature map to generate a weighted feature map with tremor-related information.

8. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the system of any one of claims 1-7.

9. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the system of any one of claims 1-7.

Images