Writing handwriting real-time drawing method based on deep learning

Abstract

The invention relates to a handwriting real-time drawing method based on deep learning, and aims to improve the experience feeling of a user in a digital writing environment and reduce the hand following delay of screen display of a nonlinear handwriting input track. The core technology comprises the following steps that firstly, a system captures real-time writing actions of a user on a touch screen, tracks left by the actions are recorded in order, and data streams which are arranged according to a time sequence and are fixed in length are formed; and then, normalizing the data stream, and inputting the data stream into a trajectory prediction model constructed based on deep learning to perform trajectory prediction. The method mainly relates to a model structure design based on a multi-layer perceptron (MLP), a gated loop unit (GRU) and a Transform architecture, the accuracy of predicting writing track points is effectively improved, and the writing experience of the electronic equipment is remarkably improved.

Description

Technical Field

[0001] The present invention relates to multiple fields such as machine learning, deep learning, data processing and analysis, and realizes a handwriting prediction algorithm model based on deep learning, aiming to provide a real-time handwriting drawing method based on deep learning. Background Art

[0002] With the rapid development of information technology, digital writing has become an indispensable part of daily life. Although the popularization of these technologies has greatly promoted the convenience of information exchange and knowledge dissemination, it is accompanied by a continuous increase in demand for hardware performance, which poses a severe challenge to cost control. At present, most devices on the market are limited by hardware resources when processing real-time handwriting tracking and display. It is difficult to effectively reduce touch feedback delays and ensure the immediacy and accuracy of writing trajectories without increasing costs. This bottleneck not only affects the fluency of handwriting interaction, but also hinders the maximization of the accessibility of technology. In view of this, exploring an innovative and cost-effective handwriting prediction technology solution has become an urgent problem to be solved. Therefore, the present invention aims to achieve the goal of reducing the delay of handwriting input through advanced algorithm optimization rather than relying solely on hardware upgrades, while ensuring that the prediction accuracy meets the actual application needs. Summary of the invention

[0003] The purpose of the present invention is to provide a real-time handwriting drawing method based on deep learning in order to solve the above-mentioned problems.

[0004] The technical solution adopted by the present invention is as follows: a real-time drawing method of handwriting based on deep learning,

[0005] The steps include:

[0006] S1: The system captures the writing action of each stroke of the user on the touch screen in real time, and records the key writing parameter information in order to form a data stream F arranged in time series and with a fixed length;

[0007] S2: Normalize the collected data stream, build model training and verification data sets, and build a trajectory prediction model based on deep learning for training. The model is responsible for comprehensive analysis of historical data and predicting future trajectory point coordinates.

[0008] In a preferred embodiment, the S1 key parameter information mainly includes point coordinates (x, y) and a timestamp t of the corresponding trajectory point.

[0009] In a preferred embodiment, the S1 is a data stream arranged in time series and of fixed length. In real-time prediction, for each stroke, key parameter information is recorded from the beginning of the stroke. When the recorded key parameter group information first accumulates to the set length threshold Slength_max When the data is normalized, the data L is obtained. i , and sent to the model to predict the subsequent trajectory path. In order to achieve continuous prediction, the sequence will be dynamically updated - the earliest set of parameters entered will be removed, and the newly collected parameter values ​​will be added to the end of the sequence. This process is repeated until the current stroke is completed.

[0010] In a preferred embodiment, the S2 dataset is constructed and the data acquisition process includes a wide range of writing samples from multiple participants, covering Chinese character strokes, Arabic numerals, English letters and common graphics, and is divided into training set, validation set and test set in a ratio of 8:1:1 to promote effective verification of the model and ensure generalization ability.

[0011] In a preferred embodiment, the handwriting prediction model S2 is constructed based on deep learning, and comprises three modules: a shallow feature encoding module, a deep feature extraction module and a trajectory decoding module. The innovation of this design lies in the combination of multi-layer perceptron (MLP), gated recurrent unit (GRU) and Transformer architecture, making full use of the advantages of each technology to deeply extract sequence features, thereby improving the accuracy and real-time performance of trajectory prediction.

[0012] In a preferred embodiment, the three modules are:

[0013] S251: Shallow feature encoding module, which first converts the sequence data L through a perceptron (Multi-Layer Perceptron, MLP) containing 2 hidden layers i Convert to embedded representation:

[0014] e=f MLP (p t ,W e )

[0015] Where W e is the embedding weight matrix, f MLP It is an MLP layer, and then a single-layer GRU is used to encode the embedded data;

[0016] S252: Deep feature extraction module, which uses a 3-layer Transformer encoder to perform deep feature extraction on the historical data information of the writing trajectory. The encoder includes a multi-head attention mechanism layer, a feedforward network layer, a residual connection and layer normalization. This structure can capture long-term dependencies and improve the accuracy of prediction;

[0017] S253: Trajectory decoding module, which uses a 1-layer Transformer encoder and a fully connected layer in series for decoding. With this design, the model can effectively output predicted trajectory points and meet the real-time trajectory prediction requirements.

[0018] In a preferred embodiment, the Transformer encoder has a specific structure as follows:

[0019] S2521: First, use the multi-head attention mechanism to extract features;

[0020] S2522: Then, the residual connection is used to fuse the features and access the normalization layer;

[0021] S2523: After that, a feedforward neural network is used to improve nonlinearity and enhance the expressiveness of data features;

[0022] S2524: Finally, the residual structure and normalization layer are used again to alleviate the problem of gradient disappearance or explosion and promote the model's ability to extract features.

[0023] In a preferred embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for real-time drawing of handwriting based on deep learning as described in any one of the above is implemented.

[0024] In a preferred embodiment, an electronic whiteboard writing device includes a memory and a processor, wherein the memory is preinstalled with a specially designed computer program, which, driven by the processor, implements any one of the methods for real-time drawing of handwriting based on deep learning.

[0025] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0026] The present invention proposes a real-time drawing method for handwriting based on deep learning, aiming to reduce hardware costs through advanced algorithm optimization rather than relying solely on hardware upgrades, and to ensure that the prediction accuracy meets the actual application needs while achieving the goal of reducing handwriting input delay. The present invention is based on key deep learning technologies, and innovatively combines key technologies such as attention mechanism, GRU network, residual structure, etc., covering the entire process from acquisition of original trajectory data, preprocessing, and prediction of deep learning models. It achieves significant improvements in the real-time and accuracy of touch screen handwriting input without relying on high-end hardware, and through innovations at the algorithm level, it brings a solution that is closer to the real paper and pen writing experience to a wide range of touch screen devices, greatly promoting the popularization and application of digital writing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1Real-time trajectory prediction logic block diagram;

[0028] Figure 2 Model structure diagram. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0030] Reference Figure 1-2 ,

[0031] A method for real-time drawing of handwriting based on deep learning comprises the following steps:

[0032] S1: The system captures the writing action of each stroke of the user on the touch screen in real time, and records the key writing parameter information in order to form a data stream F arranged in time series and with a fixed length;

[0033] S2: Normalize the collected data streams, build model training and validation data sets, and build a trajectory prediction model based on deep learning for training. The model is responsible for comprehensive analysis of historical data and predicting future trajectory point coordinates.

[0034] The S1 key parameter information mainly includes point coordinates (x, y) and the timestamp t of the corresponding trajectory point.

[0035] The S1 is a data stream arranged in time series and of fixed length. In real-time prediction, for each stroke, key parameter information is recorded from the beginning of the stroke. When the recorded key parameter group information first accumulates to the set length threshold S length_max When the data is normalized, the data L is obtained. i , and sent to the model to predict the subsequent trajectory path. In order to achieve continuous prediction, the sequence will be dynamically updated - the earliest set of parameters entered will be removed, and the newly collected parameter values ​​will be added to the end of the sequence. This process is repeated until the current stroke is completed.

[0036] The S2 dataset was constructed and the data acquisition process included a wide range of writing samples from multiple participants, covering Chinese character strokes, Arabic numerals, English letters and common graphics, and was divided into training set, validation set and test set in a ratio of 8:1:1 to facilitate effective verification of the model and ensure generalization ability.

[0037] The S2 handwriting prediction model built based on deep learning includes three modules: shallow feature encoding module, deep feature extraction module and trajectory decoding module. The innovation of this design lies in the combination of multi-layer perceptron (MLP), gated recurrent unit (GRU) and Transformer architecture, which fully utilizes the advantages of each technology to deeply extract sequence features and thus improve the accuracy and real-time performance of trajectory prediction.

[0038] The three modules are:

[0039] S251: Shallow feature encoding module, which first converts the sequence data L through a perceptron (Multi-Layer Perceptron, MLP) containing 2 hidden layers i Convert to embedded representation:

[0040] e=f MLP (p t ,W e )

[0041] Where W e is the embedding weight matrix, f MLP It is an MLP layer, and then a single-layer GRU is used to encode the embedded data;

[0042] S252: Deep feature extraction module, which uses a 3-layer Transformer encoder to perform deep feature extraction on the historical data information of the writing trajectory. The encoder includes a multi-head attention mechanism layer, a feedforward network layer, a residual connection and layer normalization. This structure can capture long-term dependencies and improve the accuracy of prediction;

[0043] S253: Trajectory decoding module, which uses a 1-layer Transformer encoder and a fully connected layer in series for decoding. With this design, the model can effectively output predicted trajectory points and meet the real-time trajectory prediction requirements.

[0044] The specific structure of the Transformer encoder is as follows:

[0045] S2521: First, use the multi-head attention mechanism to extract features;

[0046] S2522: Then, the residual connection is used to fuse the features and access the normalization layer;

[0047] S2523: After that, a feedforward neural network is used to improve nonlinearity and enhance the expressiveness of data features;

[0048] S2524: Finally, the residual structure and normalization layer are used again to alleviate the problem of gradient disappearance or explosion and promote the model's ability to extract features.

[0049] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements a real-time handwriting drawing method based on deep learning as described in any one of the above.

[0050] An electronic whiteboard writing device comprises a memory and a processor, wherein the memory is preinstalled with a specially designed computer program, and the program, driven by the processor, implements any one of the methods for real-time drawing of handwriting based on deep learning.

[0051] In the present invention, a real-time drawing method of handwriting based on deep learning is proposed, aiming to reduce hardware costs through advanced algorithm optimization rather than simply relying on hardware upgrades, and to ensure that the prediction accuracy meets the actual application needs while achieving the goal of reducing handwriting input delay. The present invention is based on deep learning key technologies, and innovatively combines key technologies such as attention mechanism, GRU network, residual structure, etc., covering the entire process from acquisition of original trajectory data, preprocessing, and prediction of deep learning models. It achieves significant improvement in the real-time and accuracy of touch screen handwriting input without relying on high-end hardware, and through innovations at the algorithm level, it brings a solution that is closer to the real paper and pen writing experience to a wide range of touch screen devices, greatly promoting the popularization and application of digital writing technology.

[0052] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0053] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time handwriting drawing method based on deep learning, characterized in that: The steps include: S1: The system captures the writing action of each stroke of the user on the touch screen in real time, and records the key writing parameter information in order to form a data stream F arranged in time series and with a fixed length; S2: Normalize the collected data streams, build model training and validation data sets, and build a trajectory prediction model based on deep learning for training. The model is responsible for comprehensive analysis of historical data and predicting future trajectory point coordinates.

2. The method for real-time drawing of handwriting based on deep learning according to claim 1, characterized in that: The S1 key parameter information mainly includes point coordinates (x, y) and the timestamp t of the corresponding trajectory point.

3. The method for real-time drawing of handwriting based on deep learning according to claim 1, characterized in that: The S1 is a data stream arranged in time series and of fixed length. In real-time prediction, for each stroke, key parameter information is recorded from the beginning of the stroke. When the recorded key parameter group information first accumulates to the set length threshold S length_max When the data is normalized, the data L is obtained. i , and sent to the model to predict the subsequent trajectory path. In order to achieve continuous prediction, the sequence will be dynamically updated - the earliest set of parameters entered will be removed, and the newly collected parameter values ​​will be added to the end of the sequence. This process is repeated until the current stroke is completed.

4. The method for real-time drawing of handwriting based on deep learning according to claim 1, characterized in that: The S2 dataset was constructed and the data acquisition process included a wide range of writing samples from multiple participants, covering Chinese character strokes, Arabic numerals, English letters and common graphics, and was divided into training set, validation set and test set in a ratio of 8:1:1 to facilitate effective verification of the model and ensure generalization ability.

5. The method for real-time drawing of handwriting based on deep learning according to claim 1, characterized in that: The S2 handwriting prediction model built based on deep learning includes three modules: shallow feature encoding module, deep feature extraction module and trajectory decoding module. The innovation of this design lies in the combination of multi-layer perceptron (MLP), gated recurrent unit (GRU) and Transformer architecture, which fully utilizes the advantages of each technology to deeply extract sequence features and thus improve the accuracy and real-time performance of trajectory prediction.

6. The method for real-time drawing of handwriting based on deep learning according to claim 5, characterized in that: The three modules are: S251: Shallow feature encoding module, which first converts the sequence data L through a perceptron (Multi-Layer Perceptron, MLP) containing 2 hidden layers i Convert to embedded representation: e=f MLP (p t ,W e ) Where W e is the embedding weight matrix, f MLP It is an MLP layer, and then a single-layer GRU is used to encode the embedded data; S252: Deep feature extraction module, which uses a 3-layer Transformer encoder to perform deep feature extraction on the historical data information of the writing trajectory. The encoder includes a multi-head attention mechanism layer, a feedforward network layer, a residual connection and layer normalization. This structure can capture long-term dependencies and improve the accuracy of prediction; S253: Trajectory decoding module, which uses a 1-layer Transformer encoder and a fully connected layer in series for decoding. With this design, the model can effectively output predicted trajectory points and meet the real-time trajectory prediction requirements.

7. The method for real-time drawing of handwriting based on deep learning according to claim 6, characterized in that: The specific structure of the Transformer encoder is as follows: S2521: First, use the multi-head attention mechanism to extract features; S2522: Then, the residual connection is used to fuse the features and access the normalization layer; S2523: After that, a feedforward neural network is used to improve nonlinearity and enhance the expressiveness of data features; S2524: Finally, the residual structure and normalization layer are used again to alleviate the problem of gradient disappearance or explosion and promote the model's ability to extract features.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, a real-time handwriting drawing method based on deep learning as described in any one of claims 1-7 is implemented.

9. An electronic whiteboard writing device, characterized in that: It comprises a memory and a processor, wherein the memory is pre-installed with a specially designed computer program, and the program, driven by the processor, implements a real-time handwriting drawing method based on deep learning as described in any one of claims 1-7.

Images