A handwritten letter recognition method based on piezoresistive signal detection

Abstract

This invention belongs to the field of character recognition technology and provides a handwritten letter recognition method based on piezoresistive signal detection, including the following steps: collecting the temporal piezoresistive signal during the letter writing process as raw data; preprocessing the raw data to obtain training data; constructing a recognition model, inputting the training data into the recognition model to extract temporal feature vectors, and performing classification training; using the trained recognition model for forward inference to confirm the letter category; this invention accurately captures changes in writing force and temporal logic based on piezoresistive electrical signals, significantly improving the accuracy of handwritten letter recognition.

Description

Technical Field

[0001] This invention relates to the field of character recognition technology, and more specifically to a method for recognizing handwritten letters based on piezoresistive signal detection. Background Technology

[0002] In recent years, the widespread adoption of portable smart devices, wearable terminals, and implicit interaction technologies has enabled handwritten letter recognition to be widely used in fields such as input interaction, educational assessment, security authentication, and rehabilitation monitoring, making it a key research area of ​​interest both domestically and internationally.

[0003] Traditional handwritten letter recognition methods largely rely on image data, using convolutional neural networks for spatial feature extraction. However, traditional static image recognition methods primarily depend on the two-dimensional spatial features of the final handwriting, lacking dynamic information representation of the writing process and easily affected by ambient lighting, image noise, and individual letter shape differences. Especially when processing letters with similar shapes, the lack of crucial information such as pressure variations and time-series features limits recognition accuracy and robustness.

[0004] Existing methods generally suffer from the following shortcomings: First, traditional temporal networks are prone to gradient vanishing or attention distraction when processing long temporal signals, making it difficult to stably model key writing stages; second, they lack effective feature fusion mechanisms and cannot uniformly represent multi-scale changes caused by different writing speeds and writing pressures; third, deep models are prone to overfitting in small sample cases, are sensitive to writing habits and individual differences, and have insufficient generalization performance.

[0005] Therefore, how to improve the recognition accuracy and robustness of dynamic letter signals is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of the above problems, the present invention is proposed to provide a handwritten letter recognition method based on piezoresistive signal detection that overcomes or at least partially solves the above problems.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A handwritten letter recognition method based on piezoresistive signal detection includes the following steps: The timing piezoresistive signal during the letter writing process is collected as the raw data; The original data is preprocessed to obtain training data; Construct a recognition model, input the training data into the recognition model to extract temporal feature vectors, and perform classification training; The trained recognition model is used for forward reasoning to confirm the letter category.

[0009] Preferably, the recognition model includes a bidirectional long short-term memory network and an attention pooling layer; The bidirectional long short-term memory network is used to capture the forward and backward context dependencies of the writing process; The attention pooling layer is used to assign corresponding weights to each time step based on the context dependency.

[0010] Preferably, the processing steps of the bidirectional long short-term memory network and the attention pooling layer include: The bidirectional long short-term memory network calculates the forward hidden state at each time step. and backward hidden state The time-series feature vector sequence is obtained by concatenating the vectors. The attention pooling layer performs a weighted summation of the temporal feature vector sequence to obtain a deep attention feature vector. The formula is as follows:

[0011]

[0012]

[0013] in, This represents the temporal characteristics corresponding to time t. To score attention, This is the weight matrix. For parameter vectors, For normalized weights.

[0014] Preferably, the recognition model further includes a multi-scale residual convolution module, which is used to capture stroke transition features in the training data through multi-size convolution kernels and to transmit feature information through residual connections.

[0015] Preferably, the processing procedure of the multi-scale residual convolution module includes:

[0016] Where Y is the output of the residual block, k is the kernel size, and n is the number of kernels. The size of the nth convolutional kernel. For batch normalization layer; The kernel size is One-dimensional convolution; The residual block is followed by a max pooling layer to output a depth spatial feature sequence.

[0017] Preferably, the steps further include: Set a manual feature vector, concatenate the manual feature vector with the temporal feature vector, and input the concatenated vector into the classifier for classification.

[0018] Preferably, the preprocessing step includes calculating the resistance change rate as a training sample.

[0019] Preferably, the processing steps include standardization:

[0020] in, This is the sample mean; The standard deviation is the sample standard deviation. This is the standardized resistance value.

[0021] Preferably, the preprocessing step further includes: The sequence lengths of all training samples are standardized. and / or Use a moving average algorithm to smooth out transient noise; and / or The input samples are processed using random amplitude scaling and random time shifting.

[0022] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a handwritten letter recognition method based on piezoresistive signal detection. Unlike traditional static image recognition methods that rely on cameras, the present invention uses dynamic timing resistance signals collected by a flexible pressure sensor for recognition. This method directly characterizes the changes in force and timing logic during the writing process, and is unaffected by changes in ambient light, writing angle deviations, and image background noise, thus possessing stronger anti-interference capabilities.

[0023] This invention constructs a multi-scale residual convolution module. By employing convolution kernels of different sizes in parallel, it can simultaneously capture minute instantaneous jitters in high-frequency details and macroscopic stroke transitions in low-frequency trends in handwritten signals, overcoming the limitation of fixed receptive fields in traditional single-scale convolutional networks. Combined with a residual connection mechanism, it effectively alleviates the gradient vanishing and model degradation phenomena that may occur as the network depth increases, significantly improving the feature extraction capability of deep networks for dynamic temporal signals.

[0024] This invention introduces a bidirectional long short-term memory network and an attention pooling layer, utilizing Bi-LSTM to simultaneously capture the forward and backward contextual dependencies of the writing process, thus solving the problem that traditional unidirectional networks cannot utilize future information. Combined with the attention mechanism, the model can automatically learn and assign higher weights to key time steps, effectively suppressing redundant information and noise interference, and significantly improving the model's discriminative power when processing long-sequence signals.

[0025] This invention employs a fusion strategy combining deep learning features with handcrafted statistical features. It combines nonlinear abstract features automatically extracted by neural networks with handcrafted features possessing clear physical meaning, achieving a complementary advantage. This fusion mechanism enriches the expressive power of the feature space and compensates for the shortcomings of purely data-driven models in feature extraction under small sample or extreme conditions, thereby improving the model's generalization ability when faced with individuals with different writing habits.

[0026] This invention effectively solves the problem of overfitting with small samples and significantly improves the generalization performance of the model by performing data augmentation processing on input samples through random amplitude scaling and random time shifting, combined with sample standardization and moving average smoothing preprocessing. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of a handwritten letter recognition method based on piezoresistive signal detection provided in an embodiment of the present invention; Figure 2 This is a structural diagram of the multi-scale RESCNN-LSTM algorithm in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the character classification prediction accuracy in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the overall effect analysis of character classification in an embodiment of the present invention; Figure 5 This is a bar chart showing the results of 5-fold cross-validation in an embodiment of the present invention; Figure 6 This is a bar chart comparing the recognition accuracy of different models in the embodiments of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] like Figure 1 This invention discloses a handwritten letter recognition method based on piezoresistive signal detection, comprising the following steps: S1: Collect the timing piezoresistive signal during the letter writing process as the raw data; S2: Preprocess the original data to obtain training data; S3: Construct a recognition model, input the training data into the recognition model to extract temporal feature vectors, and perform classification training; S4: Use the trained recognition model to perform forward reasoning to confirm the letter category.

[0031] The steps in this embodiment will now be explained in detail.

[0032] S1 represents the data acquisition process. In one embodiment, the prepared flexible pressure sensor is laid flat and fixed on a flat, insulated experimental platform. Based on the piezoresistive effect, the sensor can convert the dynamic pressure changes generated during writing into a continuous analog resistance signal in real time. Subsequently, a standard writing pen is used as the interaction medium to perform handwriting operations on the surface of the flexible pressure sensor, representing the 26 uppercase English letters (AZ).

[0033] During the data acquisition process, a digital multimeter was electrically connected to the electrodes of a flexible pressure sensor. The sampling frequency was set to 50 Hz, and the timing resistance signals corresponding to the letter writing process were recorded and saved in real time. To construct a statistically significant dataset, each letter was repeatedly sampled according to the standard stroke order. Specifically, for the 26 uppercase English letters, each letter was independently sampled 100 times. After removing obvious erroneous data, a raw resistance timing data set containing 2600 samples was finally established.

[0034] S2 is the preprocessing process: For the raw resistance time-series signal acquired in step 1, in order to eliminate environmental noise interference, standardize data specifications, and adapt to the input requirements of the deep learning model, this embodiment performs a series of preprocessing operations on the raw data. The preprocessing specifically includes: resistance change rate calculation, sequence length standardization, moving average filtering for noise reduction, standardization, and data augmentation. After the above processing, a preprocessed sample set with high signal-to-noise ratio, significant features, and consistent distribution is constructed. The processed data results include time-series signals consisting of 26 uppercase English letters. Specific operations are as follows: Because flexible sensors may exhibit slight differences in baseline resistance between different batches or under different initial pressure conditions, directly using the absolute resistance value would introduce bias. Therefore, this embodiment calculates the relative resistance change rate to eliminate the influence of the initial resistance value and highlight the dynamic characteristics during the pressure process. The calculation formula is as follows:

[0035] in, The rate of change of resistance, For the current resistance, The initial resistance; To ensure the consistency of the input tensor shape for subsequent deep learning models and avoid feature alignment difficulties caused by sequence length variations due to different writing speeds, this invention uniformly processes all collected samples into standardized sequences of fixed length. Specifically, a standard sequence length is set. For raw data exceeding 300 points in length, the middle segment or main feature segment is truncated, retaining 300 points; for raw data shorter than 300 points, edge padding is used to complete it to 300 points. This step ensures the uniformity of the input data in the time dimension.

[0036] Since high-frequency electromagnetic noise or transient spikes caused by contact instability are inevitably introduced during data acquisition, this embodiment uses a moving average algorithm to smooth the resistance change rate signal. This algorithm effectively filters out random high-frequency noise while preserving the main trend of the waveform. The calculation formula is as follows:

[0037] in The window length is set as follows in this invention. =10; i is the current sampling point number; The sequence is the result of the moving average; k is the index variable of each original sampling point participating in the averaging within the window; To accelerate the convergence speed of the neural network and improve training stability, the denoised handwritten letter data is standardized to conform to a standard normal distribution with a mean of 0 and a standard deviation of 1. The calculation formula is as follows:

[0038] in This is the sample mean; The standard deviation is the sample standard deviation. This is the normalized resistance value; Random amplitude scaling and random time shifting techniques are used to proportionally adjust the overall amplitude of different input samples, and the sampling position of the time-series signal is shifted to enhance the robustness of the model. This embodiment performs data augmentation on the standardized samples: random amplitude scaling simulates different writing pressures. The amplitude of the entire signal is proportionally adjusted by multiplying by a random factor. Random time shifting simulates the randomness of the start time of the writing action. The time-series signal is randomly shifted left and right on the time axis and correspondingly padded or clipped to enhance the model's tolerance to time misalignment.

[0039] S3 is the model training process, where the training data obtained in S2 is proportionally divided into training and test sets. This embodiment uses a multi-scale ResCNN-Bi-LSTM-Attention ensemble model as the framework for the recognition model. This model aims to extract local time-frequency features through multi-scale convolution, capture global temporal dependencies using Bi-LSTM, focus on key handwritten segments through an attention mechanism, and finally combine handcrafted features to achieve high-precision handwritten letter classification. The specific model construction process and network structure layers are detailed below: like Figure 2 The preprocessed standardized time-series sample data is input into the multi-scale residual convolution module, which aims to extract local features across different time spans in parallel, so as to take into account both the small-scale features of minute instantaneous jitter in the handwritten signal and the large-scale features of the overall stroke transition.

[0040] Specifically, the multi-scale parallel convolution module contains several cascaded residual blocks. Each residual block contains three parallel convolutional branches with kernel sizes set to (3, 5, 7). Data passes through these three branches in parallel, capturing time-frequency features within different receptive fields. The output features of each branch are then fused and added together, and finally added to the input path through the convolution via residual connections to alleviate the gradient vanishing problem in deep networks. The output calculation formula for the residual block is as follows:

[0041] in, For batch normalization layer; The kernel size is The one-dimensional convolution; X represents the input feature, and Y represents the output of the residual block.

[0042] The features processed by convolution and activation are further downsampled through max pooling layers. This operation halves the temporal dimension while preserving the most salient feature responses and progressively increases the number of feature channels. Furthermore, dropout layers are introduced to prevent overfitting. The final output is a deep space feature sequence. ;in The length of the time window; For dimensions; The feature sequences extracted by the CNN are input into a bidirectional long short-term memory (Bi-LSTM) network to extract global temporal dependencies, and then processed using the bidirectional LSTM. It can simultaneously utilize the contextual dependencies of both the forward past information and the backward future information of the handwritten signal. The bidirectional LSTM simultaneously outputs the forward LSTM at each time step to compute the hidden state. And backward LSTM to compute hidden state Then, the states before and after each time step are concatenated, resulting in a feature dimension of 256, thus obtaining a vector sequence containing complete temporal features. ,in ; To enable the model to automatically distinguish and focus on key time segments in the writing process of different letters, an attention pooling layer is introduced after the Bi-LSTM. This attention mechanism allows the model to automatically learn to assign higher weights to important time points, thereby performing weighted aggregation. The specific computation process first involves a fully connected layer... and The activation function calculates the score at each time step, and then the weights are obtained through Softmax normalization. Finally, the outputs of the LSTM are weighted and summed to obtain the final deep attention feature vector. The formula is as follows:

[0043]

[0044]

[0045] in, Score for attention; The weight matrix is ​​for learning. The vector to be learned; For normalized weights; This is a deep attention feature vector; This embodiment employs a dual-path fusion strategy of deep features and handcrafted features. Handcrafted features excel at capturing global features with clear physical meaning, while deep learning can capture abstract, non-linear, and deeper features. Combining the two can cover a more comprehensive amount of information, therefore, the deep attention vector is used... Compared with standardized handmade feature vectors Fuse into feature vectors along the channel dimension , Then will The input is fed into a fully connected layer containing batch normalization and dropout rate to prevent overfitting. Finally, the output is mapped to the probability distribution of 26 letter categories through the Softmax function. The category with the highest probability is selected as the final recognition result, thereby achieving accurate classification of handwritten English letters.

[0046] During model training and parameter optimization, a loss function incorporating class weighting and regularization mechanisms is constructed. A decoupled weight decay optimization algorithm, coupled with an adaptive learning rate scheduling strategy, is used to iteratively train the constructed ensemble model. The specific implementation steps are as follows: To address the potential class imbalance issue in the training samples, the frequency distribution of samples from each class is statistically analyzed, and inverse frequency weights are calculated and incorporated into the cross-entropy loss function to dynamically adjust the contribution ratio of different classes in gradient updates. Simultaneously, a label smoothing regularization mechanism is integrated into the loss calculation to suppress model overfitting by softening the target probability distribution. The weighted classification cross-entropy loss function is defined as follows:

[0047] in, It is the number of categories; It is the weight of category c; It is a sample Indicator of whether it belongs to category c; It is a sample The probability of belonging to category c; It is the total number of samples; The AdamW optimizer is used for network parameter updates. This algorithm decouples the weight decay term from the gradient update process, directly applying L2 regularization constraints in the parameter update step to improve the model's generalization and convergence performance. Simultaneously, a gradient pruning strategy is implemented during backpropagation. Before parameter updates, the L2 norm of the gradient vectors of all layers is calculated; if this norm exceeds a preset threshold, the gradient is scaled proportionally. This operation limits the maximum magnitude of the gradient, effectively avoiding the potential gradient explosion risk in Long Short-Term Memory (LSTM) network training and ensuring numerical stability during training.

[0048] An adaptive learning rate scheduling strategy based on accuracy monitoring is implemented. After each training epoch, the model's accuracy on the test set is evaluated. If this metric does not show a significant improvement within a set tolerance epoch, a decay mechanism is triggered, multiplying the current learning rate by a decay factor (set to 0.5 in this embodiment). This strategy achieves rapid convergence with large step sizes in the early stages of training and fine-tuning with small step sizes in the later stages.

[0049] Furthermore, a global search space is constructed for the model training parameters and network structure parameters. A probabilistic surrogate model is built using the Bayesian optimization algorithm, and the optimal parameter combination is efficiently locked by iteratively approximating the optimal solution of the objective function. The hyperparameter search space specifically includes the optimizer configuration parameters: Learning Rate = 0.000373, Batch Size = 128, Epochs = 100; Dropout Rate = 0.3, CNN convolutional channel number is [40, 80, 224], LSTM layer number is 1, and LSTM hidden layer node dimension is 224.

[0050] The optimized ensemble model was tested for letter recognition and classification using test set data. First, the test set data was organized and then input into the ensemble model. Subsequently, the model output the recognition results for the 26 uppercase English letters, and the overall recognition accuracy and the recognition accuracy for each letter were calculated. The results are as follows: Figure 3 and Figure 4 As shown in the figure. Experimental results show that the model has an accuracy of over 0.8 for each letter, with an overall accuracy of 0.9365, precision of 0.9383, recall of 0.9365, and F1 score of 0.9363. This indicates that the model has high accuracy and stability in letter recognition tasks.

[0051] To further verify the stability and reliability of the model performance, the complete dataset was randomly divided into five non-overlapping subsets. A five-fold cross-validation method was employed, iteratively selecting one subset as the test set and the remaining four subsets as the training sets, completing five independent rounds of model training and inference. This validation mechanism ensures that each set of samples in the dataset is used as test data at least once, effectively reducing evaluation bias caused by the randomness of the dataset partitioning. The results are as follows: Figure 5 As shown in the figure. Experimental results show that the accuracy rate is 0.9231 for the first fold, 0.8962 for the second fold, 0.9192 for the third fold, 0.9272 for the fourth fold, and 0.9115 for the fifth fold. The average accuracy rate of these five folds is 0.9150 ± 0.0105, indicating that the model maintains good recognition performance and stability under different data partitioning conditions.

[0052] A test sample set containing physical variation features was constructed to verify the model's generalization and anti-interference capabilities under complex real-world writing scenarios. These variable samples were generated by simulating changes in physical conditions such as different writing speeds and pressure intensities, thus forming test data with high variability. The variable samples were input into the model for inference, and the corresponding performance indicators were calculated based on the model's output recognition results. Experimental results show that the recognition accuracy of this test sample set was 0.9157, precision was 0.9198, recall was 0.9157, and F1 score was 0.9157, indicating that the model of this invention still possesses strong generalization and anti-interference capabilities when facing various changes in writing conditions.

[0053] To verify the performance advantage of the proposed model in the handwritten signal classification task, under the same experimental settings, the original dataset was input into several mainstream benchmark models for training and testing. These benchmark models included deep learning algorithms and traditional machine learning algorithms, and the results are as follows: Figure 6As shown in the figure, by comparing the experimental results of various models in terms of recognition accuracy, it was found that the recognition accuracy of the Transformer Encoder model is 0.8692, the XGBoost model is 0.8125, the Recurrent Neural Network (RNN) model is 0.7942, the Deep Neural Network (DNN) model is 0.8981, the Random Forest (RF) model is 0.8192, and the Support Vector Machine (SVM) model is 0.7740, all of which are lower than the recognition accuracy of the model proposed in this invention, namely the Residual Convolutional Long Short-Term Memory Network. Therefore, the model proposed in this invention has significant performance advantages in the handwritten signal letter recognition task.

[0054] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0055] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A handwritten letter recognition method based on piezoresistive signal detection, characterized in that, Includes the following steps: The timing piezoresistive signal during the letter writing process is collected as the raw data; The original data is preprocessed to obtain training data; Construct a recognition model, input the training data into the recognition model to extract temporal feature vectors, and perform classification training; The trained recognition model is used for forward reasoning to confirm the letter category.

2. The handwritten letter recognition method based on piezoresistive signal detection according to claim 1, characterized in that, The recognition model includes a bidirectional long short-term memory network and an attention pooling layer; The bidirectional long short-term memory network is used to capture the forward and backward context dependencies of the writing process; The attention pooling layer is used to assign corresponding weights to each time step based on the context dependency.

3. The handwritten letter recognition method based on piezoresistive signal detection according to claim 2, characterized in that, The processing steps of the bidirectional long short-term memory network and the attention pooling layer include: The bidirectional long short-term memory network calculates the forward hidden state at each time step. and backward hidden state The time-series feature vector sequence is obtained by concatenating the vectors. The attention pooling layer performs a weighted summation of the temporal feature vector sequence to obtain a deep attention feature vector. The formula is as follows: in, This represents the temporal characteristics corresponding to time t. To score attention, This is the weight matrix. For parameter vectors, For normalized weights.

4. The handwritten letter recognition method based on piezoresistive signal detection according to claim 2, characterized in that, The recognition model also includes a multi-scale residual convolution module, which is used to capture stroke transition features in the training data through multi-size convolution kernels and to pass feature information through residual connections.

5. The handwritten letter recognition method based on piezoresistive signal detection according to claim 4, characterized in that, The processing steps of the multi-scale residual convolution module include: Where Y is the output of the residual block, k is the kernel size, and n is the number of kernels. The size of the nth convolutional kernel. For batch normalization layer; The kernel size is A one-dimensional convolution; X is the input feature of the residual block; The residual block is followed by a max pooling layer to output a depth spatial feature sequence.

6. The handwritten letter recognition method based on piezoresistive signal detection according to claim 1, characterized in that, The steps also include: Set a manual feature vector, concatenate the manual feature vector with the temporal feature vector, and input the concatenated vector into the classifier for classification.

7. The handwritten letter recognition method based on piezoresistive signal detection according to claim 1, characterized in that, The preprocessing steps include calculating the rate of change of resistance as training samples.

8. A handwritten letter recognition method based on piezoresistive signal detection according to claim 1 or 7, characterized in that, The preprocessing steps include standardization: in, This is the sample mean; The standard deviation is the sample standard deviation. This is the standardized resistance value.

9. The handwritten letter recognition method based on piezoresistive signal detection according to claim 8, characterized in that, The preprocessing steps also include: The sequence lengths of all training samples are standardized. and / or Use a moving average algorithm to smooth out transient noise; and / or The input samples are processed using random amplitude scaling and random time shifting.

Images