Method for drawing handwritings on touch screen in real time

Abstract

The invention relates to a touch screen handwriting real-time drawing method. The key technology of the method comprises pretreatment, post-treatment and a rapid screen-on technology, and the algorithm process comprises the following steps: firstly, obtaining real writing track data generated by real-time writing of a touch screen, and performing pretreatment to obtain sequence data formed according to a time sequence; then, the preprocessed data are sent into a deep learning handwriting prediction model, writing track prediction is carried out, and coordinate prediction of future writing track points is achieved; and finally, in order to realize more accurate handwriting prediction, post-processing is performed on predicted trajectory point data, and a rapid on-screen technology is used, so that smooth transition between the real strokes and the predicted strokes and a visual effect of on-screen display are ensured. According to the method and the device, the writing track of the touch screen can be accurately predicted, the predicted writing track point can be drawn in advance, the touch track delay is reduced, and the real-time performance of the track and the hand is improved, so that the electronic equipment can more smoothly and naturally simulate the real handwriting experience.

Description

Technical Field

[0001] The present invention relates to multiple fields such as handwriting prediction algorithm based on deep learning, data processing and analysis, and aims to provide a real-time drawing method for handwriting on a touch screen. Background Art

[0002] With the rapid development of information technology, digital writing has become an indispensable part of daily life, especially in touch devices such as mobile computing devices, tablet computers, and smart whiteboards. Traditional handwriting recognition technology mainly focuses on converting existing static handwriting into electronic text, while there are still deficiencies in dynamic handwriting prediction and real-time drawing technology during the writing process, which limits the potential of electronic writing in maintaining natural fluency and personalized expression. As a branch of artificial intelligence, deep learning has demonstrated powerful data modeling and prediction capabilities in image recognition, natural language processing and other fields. However, its effective application in real-time prediction of handwriting still faces challenges, mainly including how to efficiently process continuous and high-dimensional time series data, and how to build a lightweight model structure to adapt to multi-finger touch and uneven performance of hardware platforms. Therefore, it is particularly important to develop a new method that can accurately predict writing trajectories in real time and adapt to different user writing styles. Summary of the invention

[0003] The purpose of the present invention is to provide a method for real-time drawing handwritten traces on a touch screen in order to solve the above-mentioned problem.

[0004] The technical solution adopted by the present invention is as follows: A method for real-time drawing handwritten handwriting on a touch screen comprises the following steps:

[0005] S1: Acquire the writing track point data generated by real-time writing on the touch screen and perform preprocessing;

[0006] S2: Send the preprocessed data to the deep learning handwriting prediction model to predict the writing trajectory;

[0007] S3: To ensure a smooth transition between the actual writing trajectory and the predicted handwriting, the predicted trajectory point data is post-processed and combined with fast screen display technology to improve the display effect.

[0008] In a preferred embodiment, the S1 writing track reporting data, the original writing track reporting data

[0009] The data includes point coordinates (x, y), timestamp t of the corresponding trajectory point, point pressure p, and tilt angle

[0010] In a preferred embodiment, the S1 pretreatment process mainly comprises the following steps:

[0011] S11: Subtract the timestamps t of adjacent trajectory points to get t dif , the calculation formula is as follows:

[0012] t dif =t i -t i-1

[0013] S12: t calculated based on S21 dif , calculate the average velocity v of two adjacent points avg_i , the average speed calculation formula is as follows:

[0014]

[0015] S13: t calculated based on S21 dif , calculate the angular velocity w of two adjacent points i , calculated as follows:

[0016]

[0017] S14: The track coordinates x, y, point pressure p, and tilt angle obtained by real-time writing on the touch screen are recorded. And the calculated v avg_i and w i Align and normalize to get the final data sequence When the L1 sequence length n=4, it is fed into the model for trajectory prediction, and experiments show that the larger the input sequence length is within a reasonable range, the higher the trajectory prediction accuracy.

[0018] In a preferred embodiment, the S2 deep learning trajectory prediction model uses the attention mechanism and GRU to construct a sequence feature extraction model, and the model prediction steps are as follows:

[0019] S21: input the preprocessed sequence data into the model;

[0020] S22: Extract important features from data sequences through attention mechanism;

[0021] S23: Use the GRU network to perform time series modeling on the extracted features and generate predicted trajectory points.

[0022] In a preferred embodiment, the S3 predicted trajectory post-processing method has the following specific processing flow:

[0023] S31: In order to avoid the deviation of the predicted trajectory when the handwriting stroke moves slowly or stops, the horizontal and vertical coordinates of the adjacent coordinate points obtained in real time are respectively calculated to obtain x dif and dif , the calculation formula is as follows:

[0024] x dif =x i -x i-1

[0025] y dif =y i -y i-1

[0026] S32: x calculated based on S31 dif and dif , set the threshold l dis = 0.1, when x dif and dif Both are greater than l dis When the output result is processed, the handwriting prediction trajectory points are drawn on the screen in combination with the fast screen display technology, otherwise the handwriting prediction trajectory point sequence L obtained by the handwriting prediction model is cleared. pre .

[0027] In a preferred embodiment, the S32 subsequent processing method uses a Bezier curve to smoothly connect the transition between the real point and the writing prediction trajectory point.

[0028] In a preferred embodiment, the S3 fast screen-on key technology includes the following steps:

[0029] S31: Combined with the refresh rate and rendering capabilities of the device, an incremental update strategy is adopted to give priority to drawing newly generated trajectory points, and double buffering technology is used to reduce screen flicker and delay;

[0030] S32: Dynamically adjusts the drawing frequency and granularity of the track according to the actual writing speed, ensuring accurate tracking when writing quickly while maintaining high accuracy when writing slowly or at rest;

[0031] S33: Uses hardware acceleration technology to improve drawing efficiency and ensure the smoothness and response speed of real-time display of writing tracks.

[0032] In a preferred embodiment, the S3 fast screen technology further includes the following optimization steps:

[0033] S34: Uses multi-threading technology to process multiple writing strokes simultaneously, improving real-time response speed;

[0034] S35: Uses an adaptive algorithm to adjust the stroke thickness based on touch pressure to enhance the user's writing experience.

[0035] In a preferred embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for real-time handwriting drawing on a touch screen as described in any one of the above is implemented.

[0036] In a preferred embodiment, an electronic whiteboard writing device includes a memory and a processor, wherein the memory is preinstalled with a specially designed computer program, and the program, driven by the processor, implements a touch screen handwriting real-time drawing method as described in any one of the above.

[0037] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0038] In the present invention, an innovative real-time drawing method of handwriting on a touch screen based on deep learning is proposed, aiming to overcome the limitations of traditional handwriting recognition systems in dynamic handwriting prediction and real-time drawing, and by combining the powerful learning ability of deep learning models, refined data processing strategies and fast on-screen key technologies, high-precision real-time prediction and display of writing trajectories are achieved, thereby significantly improving the user experience of electronic writing in terms of natural fluency and personalized expression. The core of the present invention is to construct a comprehensive method system, covering the entire process from the acquisition of raw trajectory data, preprocessing, prediction of deep learning models, to post-processing optimization, effectively solving the technical problem of achieving fast and accurate handwriting tracking on different touch devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Flow chart of the present invention;

[0040] Figure 2 Schematic diagram of the attention mechanism;

[0041] Figure 3 Note prediction model network structure;

[0042] Figure 4 Handwriting prediction logic block diagram. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] Reference Figure 1-4 ,

[0045] A method for real-time drawing handwriting on a touch screen comprises the following steps:

[0046] S1: Acquire the writing track point data generated by real-time writing on the touch screen and perform preprocessing;

[0047] S2: Send the preprocessed data to the deep learning handwriting prediction model to predict the writing trajectory;

[0048] S3: To ensure a smooth transition between the actual writing trajectory and the predicted handwriting, the predicted trajectory point data is post-processed and combined with fast screen display technology to improve the display effect.

[0049] The S1 writing track point reporting data, the original writing track point reporting data includes point coordinates (x, y), the timestamp t of the corresponding track point, the point pressure p, the tilt angle

[0050] The S1 preprocessing process described above mainly includes the following steps:

[0051] S11: Subtract the timestamps t of adjacent trajectory points to get t dif , the calculation formula is as follows:

[0052] t dif =t i -t i-1

[0053] S12: t calculated based on S21 dif , calculate the average velocity v of two adjacent points avg_i , the average speed calculation formula is as follows:

[0054]

[0055] S13: t calculated based on S21 dif , calculate the angular velocity w of two adjacent points i , calculated as follows:

[0056]

[0057] S14: The track coordinates x, y, point pressure p, and tilt angle obtained by real-time writing on the touch screen are recorded. And the calculated v avg_i and w i Align and normalize to get the final data sequence When the L1 sequence length n=4, it is fed into the model for trajectory prediction, and experiments show that the larger the input sequence length is within a reasonable range, the higher the trajectory prediction accuracy.

[0058] The S2 deep learning trajectory prediction model uses the attention mechanism and GRU to build a sequence feature extraction model. The model prediction steps are as follows:

[0059] S21: input the preprocessed sequence data into the model;

[0060] S22: Extract important features from data sequences through attention mechanism;

[0061] S23: Use the GRU network to perform time series modeling on the extracted features and generate predicted trajectory points.

[0062] The S3 prediction trajectory post-processing method has the following specific processing flow:

[0063] S31: In order to avoid the deviation of the predicted trajectory when the handwriting stroke moves slowly or stops, the horizontal and vertical coordinates of the adjacent coordinate points obtained in real time are respectively calculated to obtain x dif and dif , the calculation formula is as follows:

[0064] x dif =x i -x i-1

[0065] y dif =y i -y i-1

[0066] S32: x calculated based on S31 dif and dif , set the threshold l dis = 0.1, when x dif and dif Both are greater than l dis When the handwriting prediction model is used, the output result is processed and the handwriting prediction trajectory points are drawn on the screen in combination with the fast screen display technology. Otherwise, the handwriting prediction trajectory point sequence obtained by the handwriting prediction model is cleared. L .

[0067] pre

[0068] The S32 subsequent processing method uses a Bezier curve to smoothly connect the transition between the real point and the writing prediction trajectory point.

[0069] The S3 fast screen display key technology includes the following steps:

[0070] S31: Combined with the refresh rate and rendering capabilities of the device, an incremental update strategy is adopted to give priority to drawing newly generated trajectory points, and double buffering technology is used to reduce screen flicker and delay;

[0071] S32: Dynamically adjusts the drawing frequency and granularity of the track according to the actual writing speed, ensuring accurate tracking when writing quickly while maintaining high accuracy when writing slowly or at rest;

[0072] S33: Uses hardware acceleration technology to improve drawing efficiency and ensure the smoothness and response speed of real-time display of writing tracks.

[0073] The S3 fast screen technology further includes the following optimization steps:

[0074] S34: Uses multi-threading technology to process multiple writing strokes simultaneously, improving real-time response speed;

[0075] S35: Uses an adaptive algorithm to adjust the stroke thickness based on touch pressure to enhance the user's writing experience.

[0076] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a method for real-time drawing handwritten traces on a touch screen as described in any one of the above is implemented.

[0077] An electronic whiteboard writing device comprises a memory and a processor. The memory is preinstalled with a specially designed computer program. Driven by the processor, the program implements a touch screen handwriting real-time drawing method as described in any one of the items.

[0078] In the present invention, an innovative real-time drawing method of handwriting on a touch screen based on deep learning is proposed, aiming to overcome the limitations of traditional handwriting recognition systems in dynamic handwriting prediction and real-time drawing, and by combining the powerful learning ability of deep learning models, refined data processing strategies and fast on-screen key technologies, high-precision real-time prediction and display of writing trajectories are achieved, thereby significantly improving the user experience of electronic writing in terms of natural fluency and personalized expression. The core of the present invention is to construct a comprehensive method system, covering the entire process from the acquisition of raw trajectory data, preprocessing, prediction of deep learning models, to post-processing optimization, effectively solving the technical problem of achieving fast and accurate handwriting tracking on different touch devices.

[0079] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0080] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time drawing method of handwriting on a touch screen, characterized in that: The steps include: S1: Acquire the writing track point data generated by real-time writing on the touch screen and perform preprocessing; S2: Send the preprocessed data to the deep learning handwriting prediction model to predict the writing trajectory; S3: To ensure a smooth transition between the actual writing trajectory and the predicted handwriting, the predicted trajectory point data is post-processed and combined with fast screen display technology to improve the display effect.

2. A touch screen handwriting real-time drawing method according to claim 1, characterized in that: The S1 writing track point reporting data, the original writing track point reporting data includes point coordinates (x, y), the timestamp t of the corresponding track point, the point pressure p, the tilt angle 3. A touch screen handwriting real-time drawing method according to claim 1, characterized in that: The S1 preprocessing process described above mainly includes the following steps: S11: Subtract the timestamps t of adjacent trajectory points to get t dif , the calculation formula is as follows: t dif =t i -t i-1 S12: t calculated based on S21 dif , calculate the average velocity v of two adjacent points avg_i , the average speed calculation formula is as follows: S13: t calculated based on S21 dif , calculate the angular velocity w of two adjacent points i , calculated as follows: S14: The track coordinates x, y, point pressure p, and tilt angle obtained by real-time writing on the touch screen are recorded. And the calculated v avg_i and w i Align and normalize to get the final data sequence When the L1 sequence length n=4, it is fed into the model for trajectory prediction, and experiments show that the larger the input sequence length is within a reasonable range, the higher the trajectory prediction accuracy.

4. A touch screen handwriting real-time drawing method according to claim 1, characterized in that: The S2 deep learning trajectory prediction model uses the attention mechanism and GRU to build a sequence feature extraction model. The model prediction steps are as follows: S21: input the preprocessed sequence data into the model; S22: Extract important features from data sequences through attention mechanism; S23: Use the GRU network to perform time series modeling on the extracted features and generate predicted trajectory points.

5. The method for real-time drawing handwriting on a touch screen according to claim 1, characterized in that: The S3 prediction trajectory post-processing method has the following specific processing flow: S31: In order to avoid the deviation of the predicted trajectory when the handwriting stroke moves slowly or stops, the horizontal and vertical coordinates of the adjacent coordinate points obtained in real time are respectively calculated to obtain x dif and dif , the calculation formula is as follows: x dif =x i -x i-1 and dif =and i -and i-1 S32: x calculated based on S31 dif and dif , set the threshold l dis = 0.1, when x dif and dif Both are greater than l dis When the output result is processed, the handwriting prediction trajectory points are drawn on the screen in combination with the fast screen display technology, otherwise the handwriting prediction trajectory point sequence L obtained by the handwriting prediction model is cleared. pre .

6. A touch screen handwriting real-time drawing method according to claim 5, characterized in that: The S32 subsequent processing method uses a Bezier curve to smoothly connect the transition between the real point and the writing prediction trajectory point.

7. A touch screen handwriting real-time drawing method according to claim 1, characterized in that: The S3 fast screen display key technology includes the following steps: S31: Combined with the refresh rate and rendering capabilities of the device, an incremental update strategy is adopted to give priority to drawing newly generated trajectory points, and double buffering technology is used to reduce screen flicker and delay; S32: Dynamically adjusts the drawing frequency and granularity of the track according to the actual writing speed, ensuring accurate tracking when writing quickly while maintaining high accuracy when writing slowly or at rest; S33: Uses hardware acceleration technology to improve drawing efficiency and ensure the smoothness and response speed of real-time display of writing tracks.

8. A touch screen handwriting real-time drawing method according to claim 1, characterized in that: The S3 fast screen technology further includes the following optimization steps: S34: Uses multi-threading technology to process multiple writing strokes simultaneously, improving real-time response speed; S35: Uses an adaptive algorithm to adjust the stroke thickness based on touch pressure to enhance the user's writing experience.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, a method for real-time drawing handwritten handwriting on a touch screen as described in any one of claims 1-8 is implemented.

10. An electronic whiteboard writing device, characterized in that: The invention comprises a memory and a processor, wherein the memory is preinstalled with a specially designed computer program, and the program, driven by the processor, implements a real-time handwriting drawing method for a touch screen as claimed in any one of claims 1 to 8.

Images