Real-time handwriting recognition method based on deep neural networks

Abstract

The application is a real-time handwriting recognition method based on deep neural network, aiming to solve the problems of insufficient accuracy, poor real-time performance and poor adaptability to complex writing style in electronic whiteboard handwriting character recognition. The method first collects handwriting trajectory sample data through electronic whiteboard sensors and infrared cameras, and after missing value processing, outlier processing and format conversion, the trajectory coordinates are denoised and normalized. Secondly, enhanced samples are generated by pixel shift, 10-dimensional features such as direction angle, speed and normalized pen tip pressure are extracted, and particle swarm optimization algorithm is used to select the optimal feature subset and splice it into time sequence feature vector. Finally, a 3-layer LSTM network is used to learn the time sequence dependence of the feature vector, and a full connection layer and a softmax function are used to output the probability of 62 classes of characters, and the classification recognition is completed. The method realizes efficient and accurate recognition of electronic whiteboard handwriting characters, improves the real-time performance and reliability of handwriting interaction, and has strong engineering application value.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of character recognition, in particular to a real-time handwriting recognition method based on a deep neural network. BACKGROUND

[0002] With the rapid development of intelligent interaction technology, as an efficient handwriting interaction tool in education, office and other scenarios, the real-time and accurate recognition of handwritten characters on the electronic whiteboard has become a core requirement to improve user experience. Handwritten characters include 0-9 digits, A-Z uppercase letters and a-z lowercase letters, and other multi-class symbols. During the writing process, there are complex situations such as large handwriting differences, messy connected writing, and speed changes, which put high requirements on the robustness and real-time performance of the recognition technology.

[0003] However, the existing electronic whiteboard handwriting recognition technology has obvious limitations. The traditional feature extraction method relies on manual design and is difficult to fully capture the dynamic features of the handwriting trajectory (such as direction angle, speed, and pen tip pressure), resulting in poor adaptability to different writing styles. Feature selection lacks intelligent means, and redundant features increase the computational burden of the model, affecting the real-time performance of recognition. The time series modeling capability is insufficient, and it is difficult to effectively mine the dependence relationship of the handwriting trajectory over time, resulting in low recognition accuracy when dealing with connected writing and variable speed writing. These problems make the electronic whiteboard handwriting recognition often appear misjudgment, delay and other situations, which restricts its application effect in real-time interaction scenarios.

[0004] Based on this, a real-time handwriting recognition method based on a deep neural network is proposed, which aims to solve the above problems through systematic feature extraction, intelligent selection and time series modeling. SUMMARY

[0005] In order to solve the technical problems mentioned in the background art, the present application proposes a real-time handwriting recognition method based on a deep neural network.

[0006] To this end, the technical solution adopted by the present application is as follows: a real-time handwriting recognition method based on a deep neural network, characterized in that the method comprises the following steps:

[0007] Step S1: Through the electromagnetic sensor and infrared camera of the electronic whiteboard, the motion trajectory and stroke form of the pen tip during the writing process of the single character sample are collected in real time as sample data; according to the character category, the sample is divided into 62 categories according to the single character category; the sample data is processed for missing values, outliers and data format conversion as first data;

[0008] Step S2: The trajectory in the first data is preprocessed for trajectory coordinates as second data;

[0009] Step S3: performing a slight random shift on the second data as an enhanced sample; performing feature extraction on the enhanced sample to obtain an initial feature vector; taking the initial feature vector as an input of a particle swarm optimization algorithm to output a final feature vector;

[0010] Step S4: taking the final feature vector as an input, performing time sequence dependent learning on the final feature vector by using an LSTM algorithm to obtain a class probability, and taking the class with the maximum probability as the final output result of electronic whiteboard handwriting recognition.

[0011] Further, the sample data is expressed as:

[0012]

[0013] wherein t represents a time step, represents the total number of pen tip movement track steps of the single character sample in the writing process; represents a track coordinate of the movement track at the time step t point, represents an x-axis coordinate in the track coordinate at the time step t point, represents a y-axis coordinate in the track coordinate at the time step t point; represents a pen tip pressure of the movement track at the time step t point; represents a stroke width of the movement track at the time step t point; represents a stroke curvature of the movement track at the time step t point;

[0014] The 62 classes are respectively: 0-9 digits, A-Z uppercase letters and a-z lowercase letters.

[0015] Further, the sample data is processed by using a linear interpolation method for missing values, and the process is as follows:

[0016] For the missing point of the x-axis coordinate at the missing time step q , the linear interpolation formula is as follows:

[0017]

[0018] wherein, represents a time point corresponding to the missing time step q, respectively represent a time point of a previous time step and a time point of a next time step of the missing time step q point; represents a time point corresponding to the x-axis coordinate, represents a time point corresponding to the x-axis coordinate;

[0019] the missing point of the x-axis in the track coordinate is processed by using the linear interpolation formula The same missing value processing method is used to process missing points of the y-axis coordinates, missing points of the pen tip pressure, missing points of the stroke width, and missing points of the stroke curvature.

[0020] Further, the trajectory coordinate preprocessing of step S2 includes the following steps:

[0021] The first step is discrete wavelet transform; the trajectory coordinates in the first data are decomposed into high-frequency detail components and low-frequency trend components, and are expressed as:

[0022]

[0023] wherein, φ(j, t) represents a wavelet function, j represents a scale parameter, and t represents a translation parameter. m The high-frequency detail component corresponds to local detail information after decomposition. The low-frequency trend component corresponds to global trend information after decomposition. The low-frequency base function is expressed as:

[0024] The second step is to suppress noise of the high-frequency detail component by soft threshold processing, and the formula is:

[0025]

[0026] wherein, sgn(x) represents a sign function.

[0027] wherein, T represents the total number of steps of the pen tip trajectory during writing of the single character sample; and σ represents a noise standard deviation, and the formula is: The third step is to recombine the denoised high-frequency detail component and the original low-frequency trend component to obtain denoised trajectory coordinates

[0028]

[0029]

[0030] The fourth step is to normalize the denoised trajectory coordinates to the range of [-1, 1], and the formula is:

[0031] ​​​​​​​​​​​​

[0032] wherein, represents the minimum and maximum values of the x-axis coordinates in the denoised trajectory coordinates in all time steps; represents the minimum and maximum values of the y-axis coordinates in the denoised trajectory coordinates in all time steps.

[0033] The pre-processed trajectory coordinates, the original pen tip pressure, stroke width, and stroke curvature in the first data are merged as the second data.

[0034] Further, the step S3 comprises the following steps:

[0035] Step S301: input the second data;

[0036] Step S302: only the trajectory coordinates in the second data are subjected to a slight random shift to generate an enhanced sample, and the calculation formula is:

[0037]

[0038] wherein, represents the maximum shift amplitude; represents a uniformly distributed random number in the range of (-1, 1); represents a clipping function to ensure that the trajectory coordinates in the enhanced sample are within the area ;

[0039] the direction angle of time step t , the speed of time step t , the pen tip pressure normalization value of time step t , the curvature of time step t , the stroke area of time step t , the pressure-area correlation , the stroke shape change rate of time step t , the stroke width of time step t , the number of strokes , and the writing duration ;

[0040] A 10-dimensional feature vector of time step t is constructed as the initial feature vector , denoted as:

[0041]

[0042] Step S304: the final feature vector is obtained using a particle swarm optimization algorithm, and the steps are as follows:

[0043] S3041: Particle coding is performed on the initial feature vector;

[0044] Each particle is a binary vector corresponding to the initial feature vector A 10-dimensional feature vector is included;

[0045] Definition represents selecting the h-th dimension feature; represents discarding the h-th dimension feature; each of the particles represents a feature selection scheme;

[0046] S3042: A fitness function is defined to minimize the intra-class sample difference and maximize the inter-class sample distinction, and the expression is:

[0047]

[0048] wherein N represents the total number of classes, and the value is 62; represents the Euclidean distance mean of the sample feature vectors in class n; represents the Euclidean distance mean of the sample feature vectors in class n; and the Euclidean distance mean of the sample feature vectors in class n; measures the separability of different class sample features; q and represents the weight, and the value range is ;

[0049] S3043: The particles are updated; the formula for updating the particle speed is:

[0050]

[0051] wherein represents the inertia weight; represents the learning factor; represents a random number in the range of (-1, 1); represents the individual optimal position of particle i in the k-th generation; represents the current position of particle i in the k-th generation; represents the global optimal position of the population in the k-th generation; represents the search speed of particle i in the k-th generation, to control the direction and step length of the next search of the particle;

[0052] The particle position is updated, and the formula is:

[0053]

[0054] wherein represents mapping the continuous speed to [0, 1];

[0055] S3044: The iterative update is stopped when any of the following conditions is met:

[0056] Condition one: stop after the number of iterations reaches the preset maximum number of iterations;

[0057] Condition two: if the fitness changes less than a threshold for 10 consecutive rounds, determine convergence, stop updating, and the threshold is determined by a human;

[0058] After the iteration is terminated, output the optimal particle Map the optimal particle to the initial feature vector to obtain and output the optimal feature vector ;

[0059] Step S305: concatenate all the optimal feature vectors of the time steps in chronological order to obtain a time sequence feature vector, and obtain a final feature vector F , which is expressed as:

[0060]

[0061] wherein, represents the optimal feature vector of the t-th time step, and the feature dimensions of all the optimal feature vectors are consistent.

[0062] Further, the step S4 comprises the following steps:

[0063] Step S401: construct an LSTM network structure, adopt a 3-layer LSTM network with 128 units in each layer, input the final feature vector F, and output 62 class probabilities, each unit comprising the following gates: forget gate, input gate, cell state update, and output gate;

[0064] Step S402: input the hidden state output by the LSTM into a fully connected layer to map to 62 classes:

[0065]

[0066] wherein, represents a fully connected layer weight matrix; represents a bias term; represents a fully connected layer output logits;

[0067] Generate class probabilities through a softmax function, and the formula is as follows:

[0068]

[0069] wherein, represents the probability that the sample belongs to class n; is an index for traversing 62 classes; represents the logits value of the n-th class output by the fully connected layer. represents summing the logits values for the 62 classes of the fully connected layer output;

[0070] Taking the class with the maximum probability as the result of electronic whiteboard handwriting recognition.

[0071] Compared with the prior art, the present application has the advantages that:

[0072] 1. The present application generates augmented samples by pixel shift, simulates natural writing deviation, and extracts 10-dimensional features of direction angle, speed, pen tip pressure normalized value, curvature, stroke area, pressure-area correlation, stroke shape change rate, stroke width, stroke number, and writing duration, thereby solving the problems of single feature dimension and poor adaptability to complex writing styles in the prior art and providing rich feature support for accurate recognition.

[0073] 2. The present application uses a particle swarm optimization (PSO) algorithm to screen an optimal feature subset, constructs an adaptability function with the minimum intra-class distance and the maximum inter-class distance as the target, realizes intelligent screening of features, solves the problems of manual dependence and redundant features affecting recognition efficiency in the prior art, and improves the discriminability of features and the lightweight degree of the model.

[0074] 3. The present application learns the timing feature dependency based on a 3-layer LSTM network, combines a fully connected layer and a softmax function to output class probability, and solves the problems of insufficient timing modeling capability and poor real-time performance in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0075] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0076] Figure 1 is a flowchart of the method of the present application;

[0077] Figure 2 is a flowchart of step S3 of the present application. DETAILED DESCRIPTION

[0078] To achieve the above object, the present application is implemented by the following technical solutions. The present application provides a real-time handwriting recognition method based on a deep neural network, and the method flowchart is shown in Figure 1 , and the specific implementation process includes:

[0079] Step S1, electronic whiteboard handwriting data acquisition

[0080] Data collection: Through the electromagnetic sensor and infrared camera of the electronic whiteboard, the sampling frequency is 10 ms, 100 users aged 18-60 years old are collected, writing habits include but are not limited to regular script, cursive script and continuous writing, each person writes 62 types of characters, and each type of single character is written 20 times, the movement trajectory and stroke form of the single character in the writing process are recorded in real time, the collected single character writing movement trajectory and stroke form are defined as sample data, a total of 124000 samples are collected; the 62 types include: 0-9 digits, A-Z uppercase letters and a-z lowercase letters;

[0081] The infrared camera and the electromagnetic sensor of the electronic whiteboard have been spatially calibrated, and the default multiplexing pen tip trajectory coordinates are used As a spatial reference for infrared form analysis, the sample data is represented as:

[0082]

[0083] Wherein, t represents a time step, represents the total number of pen tip movement trajectories of the single character sample in the writing process; represents the trajectory coordinates of the movement trajectory at time step t, represents the x-axis coordinate of the trajectory coordinates at time step t, represents the y-axis coordinate of the trajectory coordinates at time step t; represents the pen tip pressure of the movement trajectory at time step t; represents the stroke width of the movement trajectory at time step t; represents the stroke curvature of the movement trajectory at time step t;

[0084] Data processing: missing value processing, outlier processing and data format conversion are performed on the sample data;

[0085] The missing value processing adopts a linear interpolation method, and for the missing point of the x-axis coordinate at the missing time step q , the linear interpolation formula is as follows:

[0086]

[0087] Wherein, represents the time point corresponding to the missing time step q, respectively represent the time point of the previous time step and the time point of the next time step of the missing time step q; represents the x-axis coordinate corresponding to the time point ; represents the x-axis coordinate corresponding to the time point ;

[0088] Adopting the missing point of x-axis in the trajectory coordinates The same missing value processing method is adopted to process the missing points of y-axis coordinates, missing points of pen tip pressure, missing points of stroke width, and missing points of stroke curvature.

[0089] The abnormal value processing adopts Kalman filter algorithm, which is a prior art and will not be described in detail;

[0090] The data format conversion converts the sample data after missing value processing and abnormal value processing into a time sequence, and takes the converted time sequence as first data, which is represented as

[0091]

[0092] Step S2, trajectory coordinate preprocessing

[0093] The trajectory coordinate preprocessing is to process the trajectory coordinates in the first data, including trajectory denoising and trajectory normalization, and the specific process is as follows:

[0094] The first step is discrete wavelet transform; the trajectory coordinates in the first data are decomposed into high-frequency detail components and low-frequency trend components, represented as:

[0095]

[0096] wherein, φ (x) represents a wavelet function, j represents a scale parameter, and b represents a translation parameter. m represents a high-frequency detail component, corresponding to the local detail information after decomposition; represents a low-frequency base function; represents a low-frequency trend component, corresponding to the global trend information after decomposition;

[0097] The second step is to suppress noise through soft threshold processing for the high-frequency detail component , and the formula is represented as:

[0098]

[0099] wherein, sgn (x) represents a sign function.

[0100] represents a noise threshold, and the calculation formula is ; T represents the total number of steps of the pen tip trajectory in the writing process of the single character sample. represents a noise standard deviation, and the formula is:​​​​​​​ , denotes a median operation;

[0101] wherein: denotes a median operation;

[0102] Thirdly, the denoised high-frequency detail component is recombined with the original low-frequency trend component to obtain a denoised trajectory coordinate , and the formula is:

[0103]

[0104] Fourthly, the denoised trajectory coordinate is standardized to the range of [-1, 1], and the formula is:

[0105]

[0106] wherein, denotes the minimum and maximum values of the x-axis coordinates in the denoised trajectory coordinate in all time steps; denotes the minimum and maximum values of the y-axis coordinates in the denoised trajectory coordinate in all time steps.

[0107] The preprocessed trajectory coordinate, the original pen tip pressure, stroke width and stroke curvature in the first data are combined as the second data.

[0108] Step S3, robust feature extraction based on particle swarm optimization algorithm

[0109] The second data is subjected to robust feature extraction, and the specific process is as shown in Figure 2 , and the specific steps are as follows:

[0110] Step S301: input the second data;

[0111] Step S302: only the trajectory coordinate in the second data is subjected to a slight random shift to generate an enhanced sample, and the formula is:

[0112]

[0113] wherein, denotes the maximum shift amplitude; denotes a random number uniformly distributed in the range of (-1, 1); denotes a clipping function to ensure that the trajectory coordinate in the enhanced sample is in the range of ;

[0114] It should be noted that the enhanced sample, the second data and the first data are completely consistent in the pen tip pressure, stroke width and stroke curvature.

[0115] Step S303: Multi-dimensional feature extraction is performed on the enhanced sample, including:

[0116] the direction angle of the time step t the speed of the time step t the pen tip pressure normalization value of the time step t the curvature of the time step t the stroke area of the time step t the pressure-area correlation the stroke shape change rate of the time step t the stroke width of the time step t the number of strokes and the writing duration .

[0117] A 10-dimensional feature vector of the time step t is constructed as the initial feature vector , which is represented as:

[0118]

[0119] the direction angle of the time step t

[0120] wherein, respectively represent the y and x coordinate values of the time step t-1 in the enhanced sample;

[0121] the speed of the time step t

[0122] wherein, the sampling frequency is fixed at 10 ms;

[0123] the pen tip pressure normalization value of the time step t

[0124] wherein, represent the maximum and minimum values of the pen tip pressure in the enhanced sample; represent the pen tip pressure of the enhanced sample at the time step t;

[0125] the curvature of the time step t

[0126] wherein, represent the y coordinate value of the time step t-2 in the enhanced sample;

[0127] the stroke area of the time step t is approximately calculated as:

[0128] wherein, denotes the length of the trajectory at time step t, and is calculated as:

[0129] Pressure-area correlation:

[0130] wherein, denotes the mean of the pressure sequence, and is calculated as:

[0131]

[0132] denotes the mean of the infrared area, and is calculated as:

[0133]

[0134] Stroke shape change rate at time step t:

[0135] Stroke width at time step t:

[0136] Stroke number:

[0137] Writing duration: Time from the beginning to the end of writing;

[0138] S3041: Particle coding is performed on the initial feature vector;

[0139] Each particle is a binary vector corresponding to the initial feature vector contains a 10-dimensional feature vector; definition when the hthdimensional feature is selected; for example when all class direction angle features are retained ; when the hthdimensional feature is discarded; the particle represents a feature selection scheme;

[0140] S3042: A fitness function is defined with the optimization objective of minimizing the intra-class sample difference and maximizing the inter-class sample distinction, and the expression is:

[0141]

[0142] wherein, N represents the total number of classes, and the value is 62; denotes the Euclidean distance mean of the sample feature vector in class n; denotes the class and the class Euclidean distance mean of sample feature vectors, measuring the separability of different class sample features; q and represents the weight, the value range is ;

[0143] S3043: the particle is updated; the formula of particle velocity update is:

[0144]

[0145] wherein, represents the inertia weight; represents the learning factor; represents a random number in the range of (-1, 1); represents the individual optimal position of particle i in the kth generation; represents the current position of particle i in the kth generation; represents the global optimal position of the population in the kth generation; represents the search speed of particle i in the kth generation, to control the direction and step length of the next search of the particle;

[0146] The particle position is updated, and the formula is:

[0147]

[0148] wherein, represents mapping the continuous speed to [0, 1];

[0149] S3044: stop the iterative update when any of the following conditions is met:

[0150] Condition one: stop after the number of iterations reaches the preset maximum number of iterations;

[0151] Condition two: the fitness changes less than a threshold value for 10 consecutive rounds, determine convergence, stop updating, and the threshold value is determined by human;

[0152] After the iteration is terminated, the optimal particle is mapped to the initial feature vector to obtain and output the optimal feature vector ;

[0153] Step S305: according to the chronological order of time steps, all the optimal feature vectors of the time steps are spliced into a time sequence feature vector to obtain a final feature vector F , which is expressed as:

[0154]

[0155] wherein, represents the optimal feature vector of the tth time step, and the feature dimensions of all the optimal feature vectors are consistent.

[0156] Step S4, real-time classification recognition based on LSTM

[0157] The time sequence dependence of the final feature vector F is learned through the LSTM network, and a character category probability is outputted;

[0158] Step S401: constructing an LSTM network structure, adopting a 3-layer LSTM network, 128 units per layer, inputting the final feature vector, and outputting 62 class probabilities, each unit containing the following gates: forget gate, input gate, cell state update, and output gate;

[0159] Forget gate:

[0160] Input gate:

[0161] Cell state update:

[0162]

[0163] Output gate:

[0164] wherein, represents a weight matrix; represents the output at the previous time; represents a sigmoid activation function; represents a bias; represents the final feature vector of the input at time t; represents the cell state; represents the weight matrix and bias of the input gate; represents element-wise multiplication; represents the hidden state of the current LSTM unit; represents the output of the forget gate; represents the output of the input gate;

[0165] Step S402: inputting the hidden state of the final output of the LSTM to a fully connected layer to map to 62 categories:

[0166]

[0167] wherein, represents a fully connected layer weight matrix; represents a bias term; represents a fully connected layer output logits;

[0168] A class probability is generated through a softmax function, and the formula is as follows:

[0169]

[0170] wherein, denotes the probability that the sample belongs to class n; is an index for traversing 62 classes; denotes the logits value of the n-th class of the fully connected layer output; denotes the sum of the logits values of the 62 classes of the fully connected layer output;

[0171] The class with the largest probability is taken as the result of electronic whiteboard handwriting recognition.

[0172] The above merely provides a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be encompassed in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of real-time handwriting recognition based on a deep neural network, characterized in that, The method comprises the following steps: Step S1: Real-time collection of the motion trajectory and stroke form of the pen nib in the single-character sample writing process by the electromagnetic sensor and infrared camera of the electronic whiteboard as sample data; division of the sample data into 62 categories according to the single-character categories; missing value processing, abnormal value processing and data format conversion of the sample data as first data; Step S2: Trajectory coordinate preprocessing of the trajectories in the first data as second data; Step S3: Micro-random shift of the second data to generate an enhanced sample; feature extraction of the enhanced sample to obtain an initial feature vector; input of the initial feature vector into a particle swarm optimization algorithm to output a final feature vector; Step S4: Input of the final feature vector into an LSTM algorithm for time sequence dependent learning of the final feature vector to obtain category probabilities, and taking the category with the maximum probability as the final output result of the electronic whiteboard handwriting recognition; The step S2 comprises the following steps: The trajectory coordinates in the first data The system is decomposed into high-frequency detail components and low-frequency trend components; noise is suppressed in the high-frequency detail components through soft thresholding; the denoised high-frequency detail components are then recombined with the original low-frequency trend components to obtain the denoised trajectory coordinates; the denoised trajectory coordinates are then... Standardize to the range of [-1, 1]; combine the preprocessed trajectory coordinates, pen tip pressure, stroke width, and stroke curvature from the original first data, and use them as the second data; The step S3 comprises the following steps: Step S301: Input of the second data; Step S302: Only the trajectory coordinates in the second data A small random shift is performed to generate enhanced samples. The calculation formula is as follows: where, denotes the maximum shift magnitude; denotes a uniformly distributed random number in the range (-1, 1); denotes a clipping function to ensure that the trajectory coordinates in the augmented sample are within the region ; Step S303: Multi-dimensional feature extraction of the enhanced sample, including: direction angle at time step t velocity at time step t pen tip pressure normalized value at time step t curvature at time step t stroke area at time step t pressure-area correlation stroke shape change rate at time step t stroke width at time step t stroke number and writing duration ; constructing a 10-dimensional feature vector at time step t as the initial feature vector is represented as: Step S304: obtaining the final feature vector using a particle swarm optimization algorithm.

2. The method of claim 1, wherein, The sample data is expressed as: wherein t represents a time step, represents a total number of stroke coordinates of the stroke trajectory of the single character sample at a time step t, represents a trajectory coordinate of the stroke trajectory at a time step t, represents an x-axis coordinate of the trajectory coordinate at a time step t, represents a y-axis coordinate of the trajectory coordinate at a time step t, represents a pen tip pressure of the stroke trajectory at a time step t, represents a stroke width of the stroke trajectory at a time step t, represents a stroke curvature of the stroke trajectory at a time step t. The 62 categories are respectively: 0-9 digits, A-Z uppercase letters and a-z lowercase letters.

3. The method of claim 2, wherein, The sample data is subjected to missing value processing by a linear interpolation method, and the process is as follows: for the missing point of the x-axis coordinate at missing time step q The linear interpolation formula is as follows: wherein, denotes the time point corresponding to the missing time step q, denotes the time point of the preceding time step and the time point of the following time step of the missing time step q, respectively; denotes the time point corresponding x-axis coordinate, denotes the time point corresponding x-axis coordinate; adopting a missing point in the x-axis of the trajectory coordinates The same missing value processing method is adopted for missing points in the y-axis coordinates, missing points in the pen tip pressure, missing points in the stroke width, and missing points in the stroke curvature.

4. The method of claim 3, characterized in that, The trajectory coordinate preprocessing of the step S2 comprises the following steps: The first step is discrete wavelet transform; the trajectory coordinates in the first data are... Decomposed into high-frequency detail components and low-frequency trend components, represented as: wherein, denotes a wavelet function, j denotes a scale parameter, m denotes a translation parameter; denotes a high frequency detail component, corresponding to local detail information after decomposition; denotes a low frequency basis function; denotes a low frequency trend component, corresponding to global trend information after decomposition; In a second step, the high frequency detail component The noise is suppressed by soft thresholding, which is expressed as wherein denotes the sign function; represents a noise threshold, and the calculation formula is ; T represents the total number of steps of the pen tip movement track of the single character sample in the writing process; represents a noise standard deviation, and the formula is: , represents a median operation; In a third step, the denoised high frequency detail component is recombined with the original low frequency trend component to obtain denoised trajectory coordinates The formula is: Fourth, the denoised trajectory coordinates are normalized to the range [-1, 1] with the formula: wherein, denotes the minimum and maximum values of the x-axis coordinates in the denoised trajectory coordinates in all time steps; denotes the minimum and maximum values of the y-axis coordinates in the denoised trajectory coordinates in all time steps.

5. The method of claim 4, wherein, The step S304 comprises the following steps: S3041: Particle coding of the initial feature vector; Each particle is a binary vector corresponding to the initial feature vector contains a 10-dimensional feature vector; Definitions when h represents selecting the hthdimensional feature; when h represents discarding the hthdimensional feature; each of the particles represents a feature selection scheme, and h is an integer from 0 to 10. S3042: Definition of a fitness function with the optimization objective of minimizing the intra-class sample difference and maximizing the inter-class sample discrimination degree, and the expression is: where N represents the total number of classes, and the value is 62; represents the Euclidean distance mean of the sample feature vector in class n; represents the Euclidean distance mean of the sample feature vector in class n; and class Euclidean distance mean of the sample feature vector, which measures the separability of different class sample features; q and represents the weight, and the value range is ; S3043: Particle updating; the formula for updating the particle velocity is: wherein, represents an inertia weight; represents a learning factor; represents a random number ranging from (-1, 1); represents the individual optimal position of particle i in the kth generation; represents the current position of particle i in the kth generation; represents the global optimal position of the population in the kth generation; represents the search speed of particle i in the kth generation, to control the direction and step length of the next search of the particle; The formula for updating the particle position is: wherein, denotes mapping the continuous velocity to [0, 1]; S3044: Stopping of the iterative updating when any of the following conditions is met: Condition one: stopping after the number of iterations reaches the preset maximum number of iterations; Condition two: stopping the updating when the fitness continues to change for 10 rounds and the change is less than a threshold value, which is determined by a human being, and the convergence is determined; After termination of the iteration, output the optimal particle map the optimal particle to the initial feature vector to obtain and output an optimal feature vector ; Step S305: In chronological order, all the optimal eigenvectors of the time steps are spliced into a time sequence eigenvector to obtain a final eigenvector F is expressed as: wherein, denotes the optimal feature vector at the t-th time step, and the feature dimensions of all the optimal feature vectors are of the same type.

6. The method of claim 5, wherein, The step S4 comprises the following steps: Step S401: Construction of an LSTM network structure, adoption of a 3-layer LSTM network with 128 units in each layer, input of the final feature vector, and output of 62 category probabilities, and each unit containing the following gates: a forgetting gate, an input gate, a cell state update and an output gate; Step S402: the LSTM final output hidden state , access the fully connected layer mapping to 62 categories: where, denotes the fully connected layer weight matrix; denotes the bias term; denotes the fully connected layer output logits; Generation of the category probability by a softmax function, and the formula is as follows: where, represents the probability that the sample belongs to class n; is an index for traversing 62 classes; represents the logits value of the n-th class of the fully connected layer output; represents the sum of the logits values of the 62 classes of the fully connected layer output; Taking the category with the maximum probability as the result of the electronic whiteboard handwriting recognition.

Images