Handwriting Display Method, Device, Electronic Device and Storage Medium

Abstract

The present invention provides a handwriting display method, apparatus, electronic device and storage medium. The method includes: determining a writing trajectory based on the nib pressure corresponding to each nib position point in the nib position point sequence; aligning the writing trajectory and the handwriting in the handwriting image collected during writing to obtain a target handwriting image corresponding to the writing trajectory; and displaying the target handwriting image. The method, apparatus, electronic device and storage medium provided by the present invention overcome the problem in the traditional solution that a single writing trajectory can only show the trajectory of the content written by the user and cannot show the characteristics of the real handwriting by aligning the writing trajectory and the handwriting in the collected handwriting image to obtain a target handwriting image and displaying it. The target handwriting image can reflect the real writing handwriting, and it is ensured that the handwriting seen by the user is also consistent with the direction, size, etc. of the handwriting written by himself, which helps to optimize the user experience.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a handwriting display method, device, electronic device, and storage medium. Background Art

[0002] Traditional writing is carried out directly on paper to complete the recording of information. With the progress of technology, there begins to appear the use of electronic devices to synchronously display and record the content written by users on paper, which is not only convenient to use, but also can realize the digital storage of the written content.

[0003] Currently, for the scheme of displaying the written content, most are to write on paper with a dot matrix code using a smart pen, and use a display device to real-time display the content written by the user on the dot matrix paper. However, the above scheme can only display the writing trajectory route, which is formed by connecting discrete points with curves, and there is a large gap from the actual writing handwriting, resulting in large distortion. Summary of the Invention

[0004] The present invention provides a handwriting display method, device, electronic device, and storage medium, so as to solve the defect in the prior art that the real writing handwriting cannot be reflected, and realize the reflection of the real writing handwriting.

[0005] The present invention provides a handwriting display method, including:

[0006] Determine a writing trajectory based on the nib pressure corresponding to each nib position point in the nib position point sequence;

[0007] Align the handwriting in the writing trajectory and the handwriting image collected during writing to obtain a target handwriting image corresponding to the writing trajectory;

[0008] Display the target handwriting image.

[0009] According to the handwriting display method provided by the present invention, the displaying of the target handwriting image includes:

[0010] Evaluate the writing trajectory based on the standard trajectory of the text corresponding to the writing trajectory to obtain an evaluation result of the writing trajectory;

[0011] Display the target handwriting image and the evaluation result of the writing trajectory.

[0012] According to the handwriting display method provided by the present invention, the evaluating the writing trajectory based on the standard trajectory of the text corresponding to the writing trajectory to obtain an evaluation result of the writing trajectory includes:

[0013] Compare the writing trajectory with the standard trajectory, and evaluate the writing trajectory based on the comparison result and evaluation metrics to obtain the evaluation result;

[0014] The evaluation metrics include at least one of literal size, stroke structure, and character center of gravity.

[0015] According to a handwriting display method provided by the present invention, the handwriting image is determined based on the following steps:

[0016] Obtain multiple frames of writing area images collected during writing;

[0017] Stitch the multiple frames of writing area images to obtain a stitched image;

[0018] Based on the stitched image, determine the handwriting image.

[0019] According to a handwriting display method provided by the present invention, the determining the handwriting image based on the stitched image includes:

[0020] Perform character recognition on the stitched image to obtain the recognition result of the stitched image;

[0021] Based on the recognition result of the stitched image, intercept the handwriting image from the stitched image.

[0022] According to a handwriting display method provided by the present invention, the determining the writing trajectory based on the nib pressure corresponding to each nib position point in the nib position point sequence includes:

[0023] Remove the nib position points with nib pressure less than or equal to the pressure threshold from the nib position point sequence, and split the nib position point sequence into multiple trajectory point sequences based on the removed nib position points;

[0024] Based on each trajectory point sequence, determine each stroke segment of the writing trajectory.

[0025] According to a handwriting display method provided by the present invention, the pressure threshold is determined based on the nib pressure corresponding to each nib position point in the user's historical nib position point sequence.

[0026] The present invention also provides a handwriting display device, including:

[0027] A trajectory determination unit, configured to determine a writing trajectory based on the nib pressure corresponding to each nib position point in the nib position point sequence;

[0028] An image determination unit, configured to align the handwriting in the writing trajectory and the handwriting image collected during writing to obtain the target handwriting image corresponding to the writing trajectory;

[0029] A handwriting display unit for displaying the target handwriting image.

[0030] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of any one of the above-mentioned handwriting display methods are implemented.

[0031] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned handwriting display methods are implemented.

[0032] The handwriting display method, device, electronic device, and storage medium provided by the present invention determine the writing trajectory through the nib pressure corresponding to each nib position point in the nib position point sequence, align the writing trajectory and the handwriting in the collected handwriting image to obtain a target handwriting image and display it. It overcomes the problem that in the traditional solution, a single writing trajectory can only show the trajectory route of the content written by the user and cannot show the characteristics of the real handwriting. The target handwriting image can reflect the real writing handwriting, and it is ensured that the handwriting seen by the user is consistent with the direction, size, etc. of the handwriting they wrote themselves, which helps to optimize the user experience. Description of the Drawings

[0033] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0034] Figure 1 is one of the flowcharts of the handwriting display method provided by the present invention;

[0035] Figure 2 is the second flowchart of the handwriting display method provided by the present invention;

[0036] Figure 3 is one of the flowcharts of the method for determining a handwriting image provided by the present invention;

[0037] Figure 4 is the second flowchart of the method for determining a handwriting image provided by the present invention;

[0038] Figure 5 is the flowchart of the method for determining a writing trajectory provided by the present invention;

[0039] Figure 6 is the structural diagram of the handwriting display device provided by the present invention;

[0040] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] At present, the scheme for displaying written content is mostly to use a smart pen to write on paper with a dot matrix code, and use a display device to display the user's writing content on the dot matrix code paper in real time. However, the above scheme can only display the writing trajectory route, and the line thickness of all writing trajectories is the same. It cannot display the thickness, pen stroke and other handwriting characteristics of real handwriting, and cannot show the beauty of calligraphy. The application is very limited. In addition, the writing trajectory route is formed by connecting discrete points with curves, which is very different from the actual handwriting and has a large distortion.

[0043] In view of the above problems, the present invention provides a handwriting display method. Figure 1 FIG. 1 is one of the flow charts of the handwriting display method provided by the present invention, such as Figure 1 As shown, the method includes:

[0044] Step 110, determining the writing trajectory based on the pen tip pressure corresponding to each pen tip position point in the pen tip position point sequence.

[0045] Specifically, the user can use a writing pen to write on a writing medium. It is understandable that different writing pens correspond to different writing media. For example, if the writing pen is a dot matrix pen, the writing medium can be a paper with a dot matrix code. For another example, if the writing pen is a touch screen pen, the writing medium can be a touch screen. In addition, the writing pen should have a writing detection function, and can collect the pen tip position point through the image acquisition device set on the pen in the writing state, and collect the pressure value of the pen tip through the pressure sensing device set at the pen tip.

[0046] When the user is writing, the pen tip position points can be continuously collected. On this basis, the pen tip position points can be arranged in the order in which they are collected, that is, the writing order, to obtain a pen tip position point sequence. Here, the pen tip position point is the position point where the pen tip is located during the writing process, which can be the position point where the pen tip passes on the writing medium, the position point where the pen tip stays on the writing medium, or the position point where the pen tip is located when it is lifted. The embodiment of the present invention does not specifically limit this.

[0047] After obtaining the sequence of pen tip position points, a writing trajectory can be generated according to the pen tip pressure corresponding to each pen tip position point in the sequence of pen tip position points. For example, each pen tip position point can be screened and filtered according to the pen tip pressure corresponding to it, and then the remaining pen tip position points can be connected by a smooth curve to obtain the writing trajectory. During the process of drawing the writing trajectory, the thickness and depth of the trajectory at the corresponding points in the writing trajectory can also be further determined according to the pen tip pressure of the remaining pen tip position points. The embodiments of the present invention do not make specific limitations on this either.

[0048] Here, the pen tip pressure is the pressure value borne by the pen tip at each pen tip position point during writing. The pen tip pressure can be realized by a pressure sensing device provided on the writing pen. For example, the pressure sensing device can be directly elastically connected to the pen tip of the writing pen. When the pen tip of the writing pen is writing, the acting force between the pen tip and the writing medium will be directly fed back to the pressure sensing device, and then the pressure sensing device can be used to detect the pen tip pressure in real time.

[0049] Step 120: Align the writing trajectory and the handwriting in the handwriting image collected during writing to obtain the target handwriting image corresponding to the writing trajectory;

[0050] Step 130: Display the target handwriting image.

[0051] Specifically, considering that the drawn writing trajectory may still have a certain gap from the actual writing handwriting and there is distortion, therefore, in the embodiments of the present invention, when the user is writing, in addition to collecting the pen tip position points, the image acquisition device provided on the pen can also collect the handwriting image, in order to reflect the real handwriting of the user through the actual image display, avoid the distortion problem, and optimize the user experience. Here, the handwriting image is an image containing the complete writing handwriting of the user. The image acquisition device can be set at the pen tip of the writing pen or at other positions on the writing pen. The embodiments of the present invention do not make specific limitations on this.

[0052] Based on this, considering that due to the influence of factors such as shooting angle and distance, the handwriting in the obtained handwriting image may have problems such as inclination and size mismatch compared with the real handwriting, which affects the user experience. Therefore, in the embodiments of the present invention, after obtaining the handwriting image, the writing trajectory obtained in step 110 and the handwriting in the handwriting image can be aligned so that the handwriting lines in the handwriting image are roughly coincident with the writing trajectory. During the alignment process, adjustment operations such as zooming and rotating the handwriting image can be performed to obtain the target handwriting image corresponding to the writing trajectory. On this basis, the target handwriting image can be displayed, so that the user can see the target handwriting image with the same direction and size as his own written handwriting, further improving the user experience.

[0053] Here, there are various ways to display the target handwriting image. For example, it can be displayed through a preset display device, or through a terminal device connected to the writing medium, etc. It can only display the target handwriting image, or while displaying the target handwriting image, according to the nib pressure of each point in the writing trajectory, mark the nib pressure for the corresponding points in the target handwriting image. It can also, while displaying the target handwriting image, mark the writing evaluation result for the handwriting in the target handwriting image. The embodiments of the present invention do not make specific limitations on this either.

[0054] It should be noted that displaying the target handwriting image enables the user to see the actual image of the written handwriting, rather than discrete points connected by curves, so as to reflect the true handwriting characteristics such as the pen tip, weight, etc. when the user writes. This usage experience cannot be shown by a single writing trajectory. The target handwriting image can reflect the true written handwriting without distortion. Moreover, the target handwriting image is obtained by aligning the writing trajectory and the handwriting in the handwriting image, which can ensure that the handwriting seen by the user is completely consistent with the handwriting they wrote in terms of direction, size, etc., further enhancing the user experience.

[0055] The method provided by the embodiments of the present invention determines the writing trajectory through the nib pressure corresponding to each nib position point in the nib position point sequence, aligns the writing trajectory and the handwriting in the collected handwriting image to obtain the target handwriting image and displays it, overcoming the problem in the traditional solution that a single writing trajectory can only show the trajectory route of the content written by the user and cannot show the true handwriting characteristics. The target handwriting image can reflect the true written handwriting and ensure that the handwriting seen by the user is also consistent with the handwriting they wrote in terms of direction, size, etc., which helps to optimize the user experience.

[0056] Based on the above embodiments, Figure 2 is the second flowchart of the handwriting display method provided by the present invention, as Figure 2 shown, step 130 includes:

[0057] Step 131, evaluate the writing trajectory based on the standard trajectory corresponding to the text of the writing trajectory to obtain the evaluation result of the writing trajectory;

[0058] Step 132, display the target handwriting image and the evaluation result of the writing trajectory.

[0059] Specifically, after obtaining the writing trajectory, the writing trajectory can be evaluated from evaluation dimensions such as stroke order, strokes, pen tip, and structure according to the standard trajectory of the text corresponding to the writing trajectory, so as to obtain the evaluation result of the writing trajectory. Then, while displaying the target handwriting image, the evaluation result of the writing trajectory can also be displayed. The display form here can be, for example, text, image, etc., so that users can understand their own writing situation and adjust their own writing habits according to the evaluation results.

[0060] Furthermore, the method for determining the standard trajectory can be to first use OCR (Optical Character Recognition) technology to obtain the text represented by the writing trajectory, and then search the database based on the text to obtain the standard trajectory of the text. In order to further facilitate users to view their own writing conditions and improve user experience, the evaluation results of the writing trajectory can be directly displayed on the target handwriting image. For example, if the text written by the user is "八", the "八" in the target handwriting image can be directly annotated according to the evaluation results of the writing trajectory, such as annotating that the stroke "丿" is too long, or that the stroke " The angle is wrong.

[0061] Considering that the existing writing trajectory lacks the pen tip information in the real handwriting, which will affect the accuracy of the evaluation of the pen tip dimension based on the writing trajectory, in order to solve this problem, in the process of drawing the writing trajectory, the thickness and depth of the trajectory at the corresponding point in the writing trajectory can be determined according to the pen tip pressure and speed information of the pen tip position point, so that the writing trajectory can reflect the pen tip information in the real handwriting, thereby ensuring the accuracy of the evaluation of the pen tip dimension based on the writing trajectory. Here, the speed information can be determined according to the length of time the pen tip position point stays.

[0062] It should be noted that, considering that the target handwriting image is static and does not contain the time information of each point on the handwriting, it is impossible to directly evaluate the handwriting in the target handwriting image from evaluation dimensions such as stroke order and writing speed. However, the timestamp information of each pen tip position point on the writing trajectory can be recorded when it is collected. Therefore, compared with the method of writing evaluation based on the target handwriting image, writing evaluation based on the writing trajectory can have a more comprehensive evaluation dimension, and the evaluation result obtained will be more accurate. In addition, since the amount of data transmitted by the image is larger than the amount of data transmitted by the trajectory, the method of writing evaluation based on the writing trajectory will also have higher real-time performance.

[0063] The method provided in the embodiment of the present invention evaluates the text written by the user based on the writing trajectory obtained by connection, thereby retaining the accuracy and easy implementation characteristics of the traditional writing trajectory line evaluation, and simultaneously displaying the target handwriting image and the evaluation result of the writing trajectory, which not only allows the user to see the real handwriting of the written text, but also facilitates the user to adjust his or her own writing habits according to the evaluation results.

[0064] Based on any of the above embodiments, step 131 includes:

[0065] Comparing the writing trajectory with the standard trajectory, and evaluating the writing trajectory based on the comparison result and the evaluation index to obtain an evaluation result;

[0066] The evaluation index includes at least one of the font size, the structure of the font and the center of gravity of the font.

[0067] Specifically, the evaluation result of the writing trajectory can be obtained in the following manner: the writing trajectory obtained in step 110 is compared with the standard trajectory of the corresponding text, and the writing trajectory is evaluated based on the comparison result and the evaluation index, so as to obtain the evaluation result of the writing trajectory. Considering that the font size, spacing structure and glyph center of gravity are strongly correlated with the aesthetics of the written text, in this regard, the evaluation index in the embodiment of the present invention may include at least one evaluation dimension of font size, spacing structure and glyph center of gravity, wherein the font size is the size of the entire text, the spacing structure is the form and rules of the strokes matching, arranging and combining into a character, and the glyph center of gravity is the glyph center of gravity, and the font size, spacing structure and glyph center of gravity of the text corresponding to the writing trajectory can be obtained specifically through text recognition technology.

[0068] Furthermore, the evaluation indicators may also include evaluation dimensions such as character position, number of strokes, stroke order, stroke length, and writing speed. One or more evaluation dimensions may be selected for evaluation according to user needs, or the number of selected evaluation dimensions may be determined according to the complexity of the writing trajectory. For example, if the character written by the user is "eight", the corresponding writing trajectory has a lower complexity, then fewer evaluation dimensions may be selected to evaluate the writing trajectory.

[0069] Among them, the writing speed can be determined according to the writing time of the writing trajectory. During the writing process, the time of each pen tip position point collected can be recorded, thereby determining the time of the first trajectory point and the last trajectory point in the writing trajectory, and then obtaining the writing time based on the time interval between the two.

[0070] The specific method for determining the evaluation result can be, for each evaluation dimension selected by the user, calculating the percentage difference between the writing trajectory and the standard trajectory, converting it into a 100-point system as the score for each evaluation dimension, and then calculating the average value of the scores of these evaluation dimensions as the evaluation result of the writing trajectory.

[0071] Based on any of the above embodiments, Figure 3 is one of the flow schematic diagrams of the method for determining the handwriting image provided by the present invention. As Figure 3 shown, the handwriting image is determined based on the following steps:

[0072] Step 310, obtain multiple frames of writing area images collected during writing;

[0073] Step 320, splice the multiple frames of writing area images to obtain a spliced image;

[0074] Step 330, determine the handwriting image based on the spliced image.

[0075] Specifically, considering interferences such as the occlusion of the pen tip, it is impossible to obtain a complete image of the written Chinese characters in the image collected at one time. To address this problem, in the embodiments of the present invention, when the user writes in the writing area on the writing medium, the image acquisition device provided on the pen is used to collect images of the writing area multiple times to obtain multiple frames of writing area images. On this basis, the processing unit for processing the handwriting image can obtain the multiple frames of writing area images and splice the multiple frames of writing area images, thereby obtaining an image containing the user's complete writing handwriting, that is, a spliced image. Immediately afterwards, the handwriting image containing the user's complete writing handwriting can be determined according to the spliced image.

[0076] Here, the specific method for splicing the multiple frames of writing area images can be to splice the frames of writing area images pairwise in the order of acquisition time until all the writing area images are spliced. For example, if the writing area images obtained by two consecutive acquisitions are P1 and P2, P1 and P2 can be compared first, and the part of P2 that does not overlap with P1 can be separated, and then the part of P2 that does not overlap with P1 can be spliced into P1. Alternatively, the overlapping parts of P1 and P2 can be registered first, and then the overlapping parts can be fused to obtain the image after splicing P1 and P2. Splicing is continuously performed in this way, and finally, a spliced image containing the complete handwriting can be obtained.

[0077] The specific method for determining the handwriting image can be to directly use the spliced image as the handwriting image, or to perform processing such as denoising and cropping on the spliced image to obtain the handwriting image. The embodiments of the present invention do not make specific limitations in this regard.

[0078] Based on any of the above embodiments, Figure 4This is the second schematic flowchart of the method for determining a handwriting image provided by the present invention. As Figure 4 shown, step 330 includes:

[0079] Step 331: Perform character recognition on the spliced image to obtain the recognition result of the spliced image;

[0080] Step 332: Based on the recognition result of the spliced image, intercept the handwriting image from the spliced image.

[0081] Specifically, after obtaining the spliced image, character recognition can be performed on the spliced image to obtain the recognition result of the spliced image. The recognition result here may include information such as the type, position of the characters in the spliced image, and whether the characters are complete, etc. Immediately, according to the recognition result of the spliced image, the image of the text part is intercepted from the spliced image, and thus the handwriting image can be obtained, thereby avoiding the display of irrelevant information and further improving the user experience.

[0082] Furthermore, if the recognition result of the spliced image indicates that the characters in the spliced image are incomplete, the image acquisition device can be controlled to continue to acquire the handwriting area image, splice the obtained new handwriting area image with the original spliced image to obtain a new spliced image, and then perform character recognition on the new spliced image until the recognition result indicates that the characters are complete, and then the acquisition of the handwriting area image can be stopped.

[0083] Based on any of the above embodiments, Figure 5 This is the schematic flowchart of the method for determining a writing trajectory provided by the present invention. As Figure 5 shown, step 110 includes:

[0084] Step 111: Exclude the pen tip position points with a pen tip pressure less than or equal to the pressure threshold from the pen tip position point sequence, and split the pen tip position point sequence into multiple trajectory point sequences based on the excluded pen tip position points;

[0085] Step 112: Based on each trajectory point sequence, determine each stroke segment of the writing trajectory.

[0086] Specifically, considering that during the process of a user writing Chinese characters, there may be operations such as lifting the pen or gently pausing the pen on the paper between strokes. Therefore, in the pen tip position point sequence obtained by acquisition, there may be some pen tip position points that are not on the user's real writing trajectory. And in chronological order, these pen tip position points are very likely to be invalid points between the pen tip position points of one stroke and the pen tip position points of the next stroke.

[0087] In this regard, in the embodiments of the present invention, the nib pressures of the respective nib position points in the nib position point sequence are used to screen and filter the nib position points in the nib position point sequence, so as to filter out the nib position points that do not belong to the true writing trajectory. Specifically, the nib pressures of the respective nib position points in the nib position point sequence can be compared with a preset pressure threshold. If there are nib position points with nib pressures less than or equal to the pressure threshold, i.e., invalid points, then such invalid points are removed from the nib position point sequence, so as to obtain the nib position point sequence after removal that can reflect the true writing trajectory.

[0088] Subsequently, the nib position point sequence can be split according to the removed invalid points, so as to obtain a plurality of trajectory point sequences. It can be understood that each trajectory point sequence only contains the nib position points with nib pressures greater than the pressure threshold, i.e., trajectory points, and each trajectory point sequence can reflect each stroke of the writing trajectory. On this basis, for each trajectory point sequence, the respective trajectory points in the trajectory point sequence can be connected by a smooth curve, so as to obtain each stroke segment of the writing trajectory. For example, if the user writes the Chinese character "Lin", and the nib position point sequence collected during writing is {P1, P2, P3, P4, P5, P6...}, where P2, P3, P4, P6 are trajectory points and P1, P5 are invalid points, then P1 and P5 can be removed, and the nib position point sequence after removal can be split into {P2, P3, P4} and {P6...} according to P1 and P5. P2, P3, P4 can form a trajectory point sequence corresponding to the stroke "-", and P6 belongs to the trajectory point sequence corresponding to the next stroke "|".

[0089] Here, the pressure threshold for screening the nib position points can be a manually set threshold or can be obtained by analyzing the nib pressures generated and recorded during the user's historical handwriting process. The embodiments of the present invention do not make specific limitations in this regard.

[0090] Based on any of the above embodiments, the pressure threshold is determined based on the nib pressures corresponding to the respective nib position points in the user's historical nib position point sequence.

[0091] Specifically, since the pen forces of different users when writing with a writing pen are mostly different, and the detected nib pressures will also be different, the pressure threshold for implementing the screening of nib position points and the construction of the trajectory point sequence can also be customized according to the historical writing habits of different users.

[0092] Further, for any user, the nib pressure of each nib position point in the historical nib position point sequence collected during the user's previous writing can be obtained. On this basis, based on the nib pressure of each nib position point in the historical nib position point sequence, the nib pressure can be classified into effective writing and ineffective writing through a clustering method, so as to obtain the pressure threshold for distinguishing effective writing and ineffective writing for this user.

[0093] The method provided by the embodiment of the present invention customizes the pressure threshold according to the user's historical writing habits, further ensuring the authenticity of the construction of each stroke segment of the writing trajectory.

[0094] Based on any of the above embodiments, the embodiment of the present invention provides a writing evaluation display and writing handwriting display method, and the specific process of this method is as follows:

[0095] S1. Collect the image of the writing area on the writing medium, identify the coordinate information of the nib position point, record the nib pressure corresponding to the nib position point, and record the time corresponding to the nib position point:

[0096] 1) Continuously collect the image of the writing area through an image acquisition device of the camera type installed at the pen tip;

[0097] 2) The writing medium is pre-set with dot matrix codes. From the collected image, the dot matrix code of the nib position point can be identified and parsed into coordinate information. When the nib is lifted, the nib position point can also be identified from the collected image;

[0098] 3) Through the pressure sensing device elastically connected to the nib, obtain the nib pressure corresponding to each nib position point;

[0099] 4) Record the time when each nib position point is obtained.

[0100] S2. Form a writing trajectory by connecting the trajectory points according to the nib pressure, and evaluate the writing trajectory:

[0101] First, among the collected nib position points, the nib position points with nib pressure greater than the pressure threshold are used as trajectory points, and each trajectory point sequence is obtained. Connect each trajectory point sequence with a smooth curve to form each stroke of the writing trajectory. Then, compare the writing trajectory with the standard trajectory of the corresponding text, and based on the comparison result and evaluation indicators in dimensions such as literal size, literal position, number of strokes, stroke order, stroke length, frame structure, glyph center of gravity, and writing speed, evaluate the writing trajectory, so as to obtain the evaluation result of the writing trajectory.

[0102] S3. Stitch the collected images of the writing area:

[0103] The images of each writing area frame are spliced pairwise in the order of collection time until the splicing of all writing area images is completed, thereby obtaining a spliced image, and a handwriting image is determined according to the spliced image.

[0104] S4. Perform operations such as scaling and rotation on the handwriting image to make the lines therein coincide as much as possible with the writing trajectories outlined in S2, thereby obtaining a target handwriting image.

[0105] S5. Display the evaluation result of the writing trajectory on the target handwriting image.

[0106] The method provided by the embodiment of the present invention evaluates the written text based on the writing trajectory, retains the accuracy and easy implementation characteristics of the traditional writing trajectory line evaluation, and at the same time displays the target handwriting image, so that what the user sees is the actual image of the written handwriting, rather than discrete points connected by curves. The actual image can reflect the pen tip, weight, etc. when the user writes, and this user experience cannot be shown by a single writing trajectory.

[0107] Next, the handwriting display device provided by the present invention will be described. The handwriting display device described below can be correspondingly referred to the handwriting display method described above.

[0108] Based on any of the above embodiments, the present invention provides a handwriting display device. Figure 6 is a schematic structural diagram of the handwriting display device provided by the present invention, as Figure 6 shown, the device includes:

[0109] A trajectory determination unit 610, configured to determine a writing trajectory based on the pen tip pressure corresponding to each pen tip position point in the pen tip position point sequence;

[0110] An image determination unit 620, configured to align the handwriting in the writing trajectory and the handwriting in the handwriting image collected during writing, to obtain a target handwriting image corresponding to the writing trajectory;

[0111] A handwriting display unit 630, configured to display the target handwriting image.

[0112] The device provided by the embodiment of the present invention determines a writing trajectory through the pen tip pressure corresponding to each pen tip position point in the pen tip position point sequence, aligns the writing trajectory and the handwriting in the collected handwriting image to obtain a target handwriting image and display it, overcoming the problem that the single writing trajectory in the traditional solution can only show the trajectory line of the content written by the user and cannot show the characteristics of the real handwriting. The target handwriting image can reflect the real written handwriting, and it is ensured that the handwriting seen by the user is consistent with the direction, size, etc. of the handwriting they wrote, which helps to optimize the user experience.

[0113] Based on any of the above embodiments, the handwriting display unit 630 is configured to:

[0114] Evaluate the writing trajectory based on the standard trajectory of the text corresponding to the writing trajectory, and obtain an evaluation result of the writing trajectory;

[0115] Display the target handwriting image and the evaluation result of the writing trajectory.

[0116] Based on any of the above embodiments, the evaluating the writing trajectory based on the standard trajectory of the text corresponding to the writing trajectory and obtaining an evaluation result of the writing trajectory includes:

[0117] Compare the writing trajectory with the standard trajectory, and evaluate the writing trajectory based on the comparison result and evaluation metrics to obtain the evaluation result;

[0118] The evaluation metrics include at least one of literal size, structure of Chinese characters, and center of character shape.

[0119] Based on any of the above embodiments, the handwriting image is determined based on the following steps:

[0120] Obtain multiple frames of writing area images collected during writing;

[0121] Stitch the multiple frames of writing area images to obtain a stitched image;

[0122] Determine the handwriting image based on the stitched image.

[0123] Based on any of the above embodiments, the determining the handwriting image based on the stitched image includes:

[0124] Perform character recognition on the stitched image to obtain a recognition result of the stitched image;

[0125] Intercept the handwriting image from the stitched image based on the recognition result of the stitched image.

[0126] Based on any of the above embodiments, the trajectory determination unit 610 is configured to:

[0127] Remove the pen tip position points with pen tip pressure less than or equal to the pressure threshold from the pen tip position point sequence, and split the pen tip position point sequence into multiple trajectory point sequences based on the removed pen tip position points;

[0128] Determine each stroke segment of the writing trajectory based on each trajectory point sequence.

[0129] Based on any of the above embodiments, the pressure threshold is determined based on the pen tip pressure corresponding to each pen tip position point in the user's historical pen tip position point sequence.

[0130] Figure 7 Illustrates a schematic diagram of the physical structure of an electronic device, as Figure 7 shown. The electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communications interface 720, and the memory 730 complete mutual communication through the communication bus 740. The processor 710 can call the logical instructions in the memory 730 to execute a handwriting display method, which includes: determining a writing trajectory based on the nib pressure corresponding to each nib position point in the nib position point sequence; aligning the writing trajectory and the handwriting in the handwriting image collected during writing to obtain a target handwriting image corresponding to the writing trajectory; and displaying the target handwriting image.

[0131] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0132] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the handwriting display method provided by the above-mentioned various methods. The method includes: determining a writing trajectory based on the nib pressure corresponding to each nib position point in the nib position point sequence; aligning the writing trajectory and the handwriting in the handwriting image collected during writing to obtain a target handwriting image corresponding to the writing trajectory; and displaying the target handwriting image.

[0133] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a handwriting display method provided by the above-mentioned various methods. The method includes: determining a writing trajectory based on the nib pressure corresponding to each nib position point in the nib position point sequence; aligning the writing trajectory and the handwriting in the handwriting image collected during writing to obtain a target handwriting image corresponding to the writing trajectory; and displaying the target handwriting image.

[0134] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0135] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course also by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0136] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A handwriting display method, It is characterized in that include: Determine the writing trajectory based on the pen tip pressure corresponding to each pen tip position point in the pen tip position point sequence; Aligning the writing trajectory and the handwriting in the handwriting image collected during writing to obtain a target handwriting image corresponding to the writing trajectory, wherein the handwriting image is an image containing the complete handwriting of the user; Displaying the target handwriting image; The step of aligning the writing trajectory with the handwriting in the handwriting image collected during writing to obtain a target handwriting image corresponding to the writing trajectory includes: Based on the writing trajectory, the handwriting image is adjusted so that the handwriting lines in the handwriting image are aligned with the writing trajectory, and based on the adjusted handwriting image, the target handwriting image is obtained, and the target handwriting image is used to represent the real handwriting characteristics of the user when writing; The handwriting image is determined based on the following steps: Acquire multiple frames of writing area images collected during writing; splicing the multiple frames of writing area images to obtain a spliced ​​image; Based on the spliced ​​image, the handwriting image is determined.

2. The handwriting display method according to claim 1, It is characterized in that The displaying of the target handwriting image comprises: Based on the standard trajectory of the text corresponding to the writing trajectory, the writing trajectory is evaluated to obtain an evaluation result of the writing trajectory; The target handwriting image and the evaluation result of the writing trajectory are displayed.

3. The handwriting display method according to claim 2, It is characterized in that The step of evaluating the writing trajectory based on the standard trajectory of the text corresponding to the writing trajectory to obtain the evaluation result of the writing trajectory includes: Comparing the writing trajectory with the standard trajectory, and evaluating the writing trajectory based on the comparison result and the evaluation index to obtain the evaluation result; The evaluation index includes at least one of the font size, the structure of the font and the center of gravity of the font.

4. The handwriting display method according to claim 1, It is characterized in that The step of determining the handwriting image based on the spliced ​​image includes: Performing text recognition on the spliced ​​image to obtain a recognition result of the spliced ​​image; Based on the recognition result of the spliced ​​image, the handwriting image is intercepted from the spliced ​​image.

5. The handwriting display method according to any one of claims 1 to 4, It is characterized in that The step of determining the writing trajectory based on the pen tip pressure corresponding to each pen tip position point in the pen tip position point sequence includes: Eliminating pen tip position points whose pen tip pressure is less than or equal to a pressure threshold from the pen tip position point sequence, and splitting the pen tip position point sequence into multiple trajectory point sequences based on the eliminated pen tip position points; Based on each trajectory point sequence, each stroke segment of the writing trajectory is determined.

6. The handwriting display method according to claim 5, It is characterized in that The pressure threshold is determined based on the pen tip pressure corresponding to each pen tip position point in the user's historical pen tip position point sequence.

7. A handwriting display device, It is characterized in that include: A trajectory determination unit, configured to determine a writing trajectory based on the nib pressure corresponding to each nib position point in the nib position point sequence; An image determination unit, configured to align the writing trajectory and the handwriting in the handwriting image collected during writing, to obtain a target handwriting image corresponding to the writing trajectory, where the handwriting image is an image containing the complete handwriting of the user; A handwriting display unit, configured to display the target handwriting image; The image determination unit is specifically configured to: Based on the writing trajectory, adjust the handwriting image, so that the handwriting lines in the handwriting image are aligned with the writing trajectory, and based on the adjusted handwriting image, obtain the target handwriting image, where the target handwriting image is used to characterize the true handwriting characteristics of the user during writing; The handwriting image is determined based on the following steps: Obtain multiple frames of writing area images collected during writing; Stitch the multiple frames of writing area images to obtain a stitched image; Based on the stitched image, determine the handwriting image.

8. An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, When the processor executes the program, the steps of the handwriting display method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein, When the computer program is executed by a processor, the steps of the handwriting display method according to any one of claims 1 to 6 are implemented.

Images