Handwriting real-time digital conversion method and system

By performing frame-by-frame processing and inter-frame differential extraction of handwriting image sequences, combined with pressure and speed data correction, efficient and accurate real-time handwriting digital conversion is achieved, solving the problem of conversion delay and unstable recognition.

CN120375393AActive Publication Date: 2025-07-25GUANGZHOU DAZZLE VIEW INTELLIGENT TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202510851650.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-24
Publication Date
2025-07-25
Estimated Expiration
2045-06-24

AI Technical Summary

Technical Problem

The prior art is difficult to efficiently and accurately generate digitized text of real-time handwriting, resulting in conversion delays and identification instability.

Method used

By collecting handwriting related data in the handwriting process in real time, the suspected handwriting pixel data in the handwriting image sequence is extracted, inter-frame differential extraction is performed, the initial handwriting is constructed, and the pressure and speed data are used to correct it, and finally converted into digital text.

Benefits of technology

It significantly improves the digitization accuracy and real-timeness of handwriting, reduces conversion delays, and ensures the consistency and accuracy of handwriting.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120375393A_ABST
    Figure CN120375393A_ABST
Patent Text Reader

Abstract

The invention provides a handwriting real-time digital conversion method and system, and relates to the technical field of handwriting data processing, and the method comprises the steps: collecting handwriting related data in a handwriting process in real time; extracting a handwriting image sequence from the handwriting related data, performing handwriting pixel extraction on each handwriting image in the handwriting image sequence to obtain suspected handwriting pixel data of each handwriting image, and converting each piece of suspected handwriting pixel data into a corresponding suspected handwriting image; performing inter-frame difference extraction on all the suspected handwriting images to obtain a difference handwriting pixel set in the handwriting process, and constructing initial handwriting in the handwriting process through the difference handwriting pixel set; and converting the initial handwriting into a digital text of the handwriting according to pressure data and speed data in the handwriting related data. According to the method, the digital text of the real-time handwriting can be efficiently and accurately generated, so that the conversion delay of the handwriting is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the technical field of handwriting data processing. More specifically, this application relates to a method and system for real-time digital conversion of handwritten handwriting. Background Art

[0002] Real-time digital conversion of handwritten handwriting is a technology that instantaneously converts handwritten input into editable digital text, and is widely used in fields such as education, office work, and intelligent device interaction. This technology combines multiple technologies such as smart pens, touch screens, computer vision, deep learning, and natural language processing to achieve efficient recognition and conversion of handwritten content. Its core process generally includes links such as data acquisition, feature extraction, handwriting recognition, text optimization, and user interaction. In recent years, with the development of artificial intelligence, handwriting recognition algorithms based on deep learning have made remarkable progress, greatly improving the accuracy and real-time performance of this technology.

[0003] However, the dynamics of the handwriting during the writing process increase the complexity of real-time recognition. Traditional OCR processes static images, while real-time handwriting recognition requires dynamically tracking the handwriting trajectory and predicting possible characters while the user is writing. This not only requires the system to have extremely high computing speed, but also requires optimized algorithms to reduce latency and ensure a smooth writing experience. In addition, during the user's writing process, there may be stroke overlaps, broken strokes, or scribbles, which will affect the stability of the recognition model, resulting in misrecognition or incomplete recognition. Therefore, how to efficiently and accurately generate digital text of real-time handwritten handwriting to reduce the conversion latency of handwritten handwriting is a difficult problem faced by the industry. Summary of the Invention

[0004] This application provides a method and system for real-time digital conversion of handwritten handwriting, which can efficiently and accurately generate digital text of real-time handwritten handwriting to reduce the conversion latency of handwritten handwriting.

[0005] In a first aspect, this application provides a method for real-time digital conversion of handwritten handwriting. The conversion method includes the following steps: Real-time collect handwriting-related data during the handwriting process; Extract a sequence of handwritten handwriting images from the handwriting-related data, extract handwriting pixels from each handwritten handwriting image in the sequence of handwritten handwriting images to obtain suspected handwriting pixel data for each handwritten handwriting image, and convert each piece of suspected handwriting pixel data into a corresponding suspected handwritten handwriting image; Extract the inter-frame differences of all suspected handwritten handwriting images, and then obtain a set of differential handwriting pixels during the handwriting process. Construct an initial handwritten handwriting during the handwriting process through the set of differential handwriting pixels; Convert the initial handwritten handwriting into a digital text of the handwritten handwriting based on the pressure data and speed data in the handwriting-related data.

[0006] In this embodiment, the handwriting-related data during the handwriting process is collected in real time through a pressure sensor, a gyroscope, and a camera.

[0007] In this embodiment, the handwriting-related data includes a sequence of handwritten handwriting images, pressure data, and speed data during the writing process.

[0008] In this embodiment, extracting the handwriting pixels from each handwritten handwriting image in the sequence of handwritten handwriting images to obtain the suspected handwriting pixel data of each handwritten handwriting image specifically includes: For each handwritten handwriting image in the sequence of handwritten handwriting images, determine the image blocks corresponding to each pixel point in the handwritten handwriting image; Determine the suspected coefficient of the handwriting pixels of each pixel point in the handwritten handwriting image through the corresponding image block; Take the pixel points with the suspected coefficient of the handwriting pixels greater than the set suspected threshold as the suspected handwriting pixel points, and then obtain a plurality of suspected handwriting pixel points; Take the data set composed of all suspected handwriting pixel points as the suspected handwriting pixel data of the handwritten handwriting image, and then obtain the suspected handwriting pixel data of each handwritten handwriting image.

[0009] In this embodiment, converting each suspected handwriting pixel data into a corresponding suspected handwritten handwriting image is to perform information mapping on each suspected handwriting pixel data respectively, and then obtain the suspected handwritten handwriting image corresponding to each suspected handwriting pixel data.

[0010] In this embodiment, extracting the inter-frame differences of all suspected handwritten handwriting images to obtain the set of differential handwriting pixels during the handwriting process specifically includes: Select a suspected handwritten handwriting image from all the suspected handwritten handwriting images, and then obtain the suspected handwritten handwriting image of the previous frame of the selected suspected handwritten handwriting image; Perform differential extraction on the selected suspected handwritten handwriting image and the suspected handwritten handwriting image of the previous frame, and then obtain the differential handwriting pixel group corresponding to the selected suspected handwritten handwriting image, and then obtain the differential handwriting pixel group corresponding to each suspected handwritten handwriting image; Construct a set of differential handwriting pixels during the handwriting process according to all the differential handwriting pixel groups.

[0011] In this embodiment, obtaining the differential handwriting pixel group corresponding to the selected suspected handwritten handwriting image further includes: performing clustering analysis on all the differential handwriting pixel points in the differential handwriting pixel group, and then removing the outlier pixel points in the differential handwriting pixel group according to the clustering analysis result.

[0012] In this embodiment, constructing the initial handwritten stroke during the handwriting process through the set of differential handwriting pixels specifically includes: Obtain the coordinates of the initial handwritten stroke; Using the coordinates of the initial handwritten stroke as the starting point, connect each differential handwriting pixel point in the set of differential handwriting pixels in the corresponding order, thereby obtaining the initial handwritten stroke during the handwriting process.

[0013] In this embodiment, converting the initial handwritten stroke into a digital text of the handwritten stroke based on the pressure data and speed data in the handwriting-related data is to correct the initial handwritten stroke using the pressure data and speed data in the handwriting-related data, thereby obtaining the digital text of the handwritten stroke.

[0014] In a second aspect, the present application provides a real-time digital conversion system for handwritten strokes, which is used to execute a real-time digital conversion method for handwritten strokes. The conversion system includes: A data acquisition module for real-time collecting handwriting-related data during the handwriting process; An image processing module for extracting a sequence of handwritten stroke images from the handwriting-related data, performing handwriting pixel extraction on each handwritten stroke image in the sequence of handwritten stroke images to obtain suspected handwriting pixel data for each handwritten stroke image, and converting each piece of suspected handwriting pixel data into a corresponding suspected handwritten stroke image; An initial stroke construction module for performing inter-frame difference extraction on all suspected handwritten stroke images, thereby obtaining a set of differential handwriting pixels during the handwriting process, and constructing the initial handwritten stroke during the handwriting process through the set of differential handwriting pixels; A handwriting conversion module for converting the initial handwritten stroke into a digital text of the handwritten stroke based on the pressure data and speed data in the handwriting-related data.

[0015] The technical solutions provided by the disclosed embodiments of the present application have the following beneficial effects: By real-time collecting handwriting-related data during the handwriting process; extracting a sequence of handwritten stroke images from the handwriting-related data, performing handwriting pixel extraction on each handwritten stroke image in the sequence of handwritten stroke images to obtain suspected handwriting pixel data for each handwritten stroke image, and converting each piece of suspected handwriting pixel data into a corresponding suspected handwritten stroke image; performing inter-frame difference extraction on all suspected handwritten stroke images, thereby obtaining a set of differential handwriting pixels during the handwriting process, and constructing the initial handwritten stroke during the handwriting process through the set of differential handwriting pixels; converting the initial handwritten stroke into a digital text of the handwritten stroke based on the pressure data and speed data in the handwriting-related data.

[0016] It can be seen that in this application, first, by processing each frame of the handwritten stroke image sequence, extracting suspected stroke pixel data, and converting it into a suspected handwritten stroke image, the accuracy and real-time performance of handwritten stroke digitization can be significantly improved; then, by extracting the inter-frame differences of all suspected handwritten stroke images, a set of differential stroke pixels during the handwriting process can be obtained, which can significantly improve the digital conversion efficiency of real-time handwritten strokes. By focusing on the changing parts between each frame rather than the entire image, the computational amount can be greatly reduced, and only the newly added or changed stroke pixels are processed. Such a processing method reduces the interference of repeated calculations and irrelevant information and improves the real-time performance of the conversion; finally, by converting the initial handwritten stroke based on the pressure data and speed data in the stroke-related data, the accuracy and real-time performance of converting handwritten strokes into digital text can be significantly improved, so as to reduce the conversion delay of handwritten strokes.

[0017] In summary, the technical solution adopted in this application can efficiently and accurately generate digital text of real-time handwritten strokes to reduce the conversion delay of handwritten strokes. Brief Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 is a flowchart of a method for real-time digital conversion of handwritten strokes provided by the present application; Figure 2 is a schematic flow diagram for determining suspected stroke pixel data of each handwritten stroke image provided by the present application; Figure 3 is a schematic flow diagram for determining a set of differential stroke pixels during the handwriting process provided by the present application; Figure 4 is a module structure diagram of a system for real-time digital conversion of handwritten strokes provided by the present application. Detailed Embodiments

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0021] An embodiment of the present application provides a method and system for real-time digital conversion of handwritten strokes. The core is to collect real-time stroke-related data during the handwriting process; extract a sequence of handwritten stroke images from the stroke-related data, extract stroke pixels from each handwritten stroke image in the sequence of handwritten stroke images to obtain suspected stroke pixel data for each handwritten stroke image, and convert each piece of suspected stroke pixel data into a corresponding suspected handwritten stroke image; perform inter-frame difference extraction on all the suspected handwritten stroke images to obtain a set of differential stroke pixels during the handwriting process, and construct an initial handwritten stroke during the handwriting process through the set of differential stroke pixels; convert the initial handwritten stroke into a digital text of the handwritten stroke according to the pressure data and speed data in the stroke-related data. By adopting the above solution, the digital text of real-time handwritten strokes can be generated efficiently and accurately to reduce the conversion delay of handwritten strokes.

[0022] Embodiment 1. To better understand the above technical solution, the above technical solution will be described in detail below in combination with the accompanying drawings of the specification and specific implementation manners. Refer to Figure 1 As shown, this figure is an exemplary flowchart of a method for real-time digital conversion of handwritten strokes according to this embodiment of the present application. The conversion method includes the following steps: In step S1, real-time collect stroke-related data during the handwriting process.

[0023] Specifically, the stroke-related data during the handwriting process can be collected in real time through a pressure sensor, a gyroscope, and a camera; it should be noted that in this application, the stroke-related data includes a sequence of handwritten stroke images, the pressure data and speed data of the writing process, and the high-precision pressure sensor, gyroscope, and high-resolution camera used can all be integrated in a smart pen, such as a dot matrix pen. The dot matrix pen mainly consists of a pressure sensor (pressure sensitivity level of 1024), a high-speed camera, a processor, a battery, a data memory, and a Bluetooth or USB communication module. When writing with a dot matrix pen, when the pen tip touches the paper and is pressed down, the pressure sensor will be triggered, and the built-in high-speed camera near the pen tip will be activated to take pictures of the positions passed by the pen tip at a speed of 100 pictures per second, and then transmitted to the built-in processor to obtain information such as image data, pressure data, and speed data, and finally output outward through the Bluetooth or USB communication unit.

[0024] In step S2, extract a sequence of handwritten stroke images from the stroke-related data, extract stroke pixels from each handwritten stroke image in the sequence of handwritten stroke images to obtain suspected stroke pixel data for each handwritten stroke image, and convert each piece of suspected stroke pixel data into a corresponding suspected handwritten stroke image.

[0025] In specific implementation, after the handwriting-related data in the handwriting process is collected in real time, the handwriting-related data can be transmitted to the edge computing device through the communication unit for subsequent processing, so that a sequence of handwritten handwriting images can be extracted from the handwriting-related data, and the sequence of handwritten handwriting images contains multiple handwritten handwriting images arranged in chronological order.

[0026] Preferably, in this embodiment, for each handwritten handwriting image in the sequence of handwritten handwriting images, handwriting pixel extraction is performed to obtain the suspected handwriting pixel data of each handwritten handwriting image. Refer to Figure 2 As shown, this figure is a schematic flowchart of determining the suspected handwriting pixel data of each handwritten handwriting image in some embodiments of the present application. The suspected handwriting pixel data of each handwritten handwriting image in this embodiment can be implemented by the following steps: In step S21, for each handwritten handwriting image in the sequence of handwritten handwriting images, an image block corresponding to each pixel point in the handwritten handwriting image is determined; In step S22, the suspected coefficient of the handwriting pixel of each pixel point in the handwritten handwriting image is determined through the corresponding image block; In step S23, the pixel points with the suspected coefficient of the handwriting pixel greater than the set suspected threshold are used as the suspected handwriting pixel points, and then a plurality of suspected handwriting pixel points are obtained; In step S24, the data set composed of all the suspected handwriting pixel points is used as the suspected handwriting pixel data of the handwritten handwriting image, and then the suspected handwriting pixel data of each handwritten handwriting image is obtained.

[0027] In specific implementation, first, for each handwritten handwriting image in the sequence of handwritten handwriting images, an image block corresponding to each pixel point in the handwritten handwriting image can be determined, that is, for each pixel point in the handwritten handwriting image, an n×n image block is taken with the pixel point as the center, and this image block is used as the image block corresponding to the pixel point, where n represents the side length of the image block. In the present application, the value of n is 6. In actual implementation, n can also take other values, which is not limited here. Through the above method, the image blocks corresponding to each pixel point in the handwritten handwriting image can be obtained; then, the suspected coefficient of the handwriting pixel of each pixel point in the handwritten handwriting image can be determined through the corresponding image block. Among them, the suspected coefficient of the handwriting pixel represents the suspected degree that the corresponding pixel point is a handwriting pixel point. In actual implementation, for each pixel point in the handwritten handwriting image, the sum of the squares of the differences between the gray values of each pixel point in the image block corresponding to the pixel point and the gray value of this pixel point can be used as the suspected coefficient of the handwriting pixel of this pixel point. Through the above method, the suspected coefficient of the handwriting pixel of each pixel point in the handwritten handwriting image can be obtained.

[0028] In addition, in specific implementation, pixels with a suspected coefficient of handwriting pixels greater than a set suspected threshold can be used as suspected handwriting pixels, so as to obtain a plurality of suspected handwriting pixels. Here, the suspected threshold can be set through historical experiments and data analysis, which will not be elaborated here. Then, the data set composed of all suspected handwriting pixels can be used as the suspected handwriting pixel data of the handwritten handwriting image. Through the above method, the suspected handwriting pixel data of each handwritten handwriting image can be obtained.

[0029] In this embodiment, converting each piece of suspected handwriting pixel data into a corresponding suspected handwritten handwriting image is to perform information mapping on each piece of suspected handwriting pixel data respectively, so as to obtain the suspected handwritten handwriting image corresponding to each piece of suspected handwriting pixel data.

[0030] In specific implementation, first, Lagrange interpolation can be used to sparsely complete the suspected handwriting pixels in each piece of suspected handwriting pixel data to make the finally obtained handwritten handwriting smoother. Then, information mapping can be performed on each piece of suspected handwriting pixel data respectively. That is, for each piece of suspected handwriting pixel data, position mapping can be performed on each suspected handwriting pixel in the suspected handwriting pixel data, and pixel points can be drawn according to the coordinates of each suspected handwriting pixel, so as to use the drawing result as the suspected handwritten handwriting image corresponding to the suspected handwriting pixel data. Through the above method, the suspected handwritten handwriting images corresponding to each piece of suspected handwriting pixel data can be obtained. It should be noted that all the obtained suspected handwritten handwriting images can be arranged in chronological order.

[0031] It should be noted that by processing each frame of the handwritten handwriting image sequence, extracting the suspected handwriting pixel data, and converting it into a suspected handwritten handwriting image, the accuracy and real-time performance of handwritten handwriting digitization can be greatly improved. First, extracting the handwriting pixel data frame by frame can effectively separate background noise, reduce misidentification, and improve the integrity of the handwriting. Second, through the intelligent screening and information mapping of suspected handwriting pixels, the original handwritten trajectory can be accurately reconstructed, making the converted digital handwriting more realistic and natural.

[0032] In step S3, inter-frame differences are extracted from all the suspected handwritten handwriting images, so as to obtain a set of differential handwriting pixels during the handwriting process, and an initial handwritten handwriting during the handwriting process is constructed through the set of differential handwriting pixels.

[0033] Preferably, in this embodiment, inter-frame differences are extracted from all the suspected handwritten handwriting images, so as to obtain a set of differential handwriting pixels during the handwriting process. Refer to Figure 3 As shown in the figure, this figure is a schematic flow chart of determining the set of differential handwriting pixels during the handwriting process in some embodiments of the present application. The set of differential handwriting pixels during the handwriting process in this embodiment can be implemented by the following steps: In step S31, select a suspected handwritten stroke image from all the suspected handwritten stroke images, and then obtain the suspected handwritten stroke image of the previous frame of the selected suspected handwritten stroke image; In step S32, perform difference extraction on the selected suspected handwritten stroke image and the suspected handwritten stroke image of the previous frame, and then obtain the difference stroke pixel group corresponding to the selected suspected handwritten stroke image, and then obtain the difference stroke pixel group corresponding to each suspected handwritten stroke image; In step S33, construct a difference stroke pixel set during the handwriting process according to all the difference stroke pixel groups.

[0034] When specifically implemented, first, a suspected handwritten stroke image can be selected from all the suspected handwritten stroke images, and the suspected handwritten stroke image of the previous frame of the selected suspected handwritten stroke image can be obtained; then, difference extraction can be performed on the selected suspected handwritten stroke image and the suspected handwritten stroke image of the previous frame, that is, pixel-level difference calculation can be performed on the selected suspected handwritten stroke image and the suspected handwritten stroke image of the previous frame, and the different parts between the selected suspected handwritten stroke image and the suspected handwritten stroke image of the previous frame can be extracted, and the suspected stroke pixels that generate difference changes in the selected suspected handwritten stroke image are used as difference stroke pixels, so that multiple difference stroke pixels can be obtained, and the set composed of all the difference stroke pixels is used as the difference stroke pixel group corresponding to the selected suspected handwritten stroke image. Through the above method, the difference stroke pixel group corresponding to each suspected handwritten stroke image can be obtained; finally, a difference stroke pixel set during the handwriting process can be constructed according to all the difference stroke pixel groups, that is, all the difference stroke pixel groups are combined in chronological order, and the combined data set is used as the difference stroke pixel set during the handwriting process.

[0035] In this embodiment, obtaining the difference stroke pixel group corresponding to the selected suspected handwritten stroke image further includes: performing clustering analysis on all the difference stroke pixels in the difference stroke pixel group, and then removing the outlier pixels in the difference stroke pixel group according to the clustering analysis result. When specifically implemented, the DBSCAN clustering algorithm can be used to perform clustering analysis on all the difference stroke pixels in the difference stroke pixel group, so that the clustering analysis result can be obtained, and the outlier pixels in the difference stroke pixel group can be removed according to the clustering analysis result. That is, after the clustering analysis, some difference stroke pixels do not belong to any effective stroke clusters, but exist as noise, which are called outlier pixels. In order to improve the accuracy of the handwritten stroke and the efficiency of subsequent processing, these outlier points need to be removed.

[0036] In this embodiment, the initial handwritten stroke during the handwriting process can be specifically constructed by the difference stroke pixel set in the following way, that is: Obtain the coordinates of the initial handwritten stroke; Taking the initial handwritten stroke coordinates as the starting point, connect each differential handwritten stroke pixel point in the differential handwritten stroke pixel set in the corresponding order, so as to obtain the initial handwritten stroke during the handwriting process.

[0037] In specific implementation, first, the initial handwritten stroke coordinates can be obtained. The initial handwritten stroke coordinates are usually the coordinates of the first change point in the time series. The coordinates of the first detected differential handwritten stroke pixel point in the differential handwritten stroke pixel set can be selected as the initial handwritten stroke coordinates. Then, taking the initial handwritten stroke coordinates as the starting point, connect each differential handwritten stroke pixel point in the differential handwritten stroke pixel set in the corresponding order, that is, connect them in sequence according to the timestamps of each differential handwritten stroke pixel point in the differential handwritten stroke pixel set to ensure that the writing trajectory is connected in chronological order, and a stroke trajectory can be obtained, that is, the initial handwritten stroke during the handwriting process, thus realizing the conversion from a single differential handwritten stroke pixel point to a complete handwritten trajectory.

[0038] It should be noted that by performing inter-frame difference extraction on all suspected handwritten stroke images to obtain the differential handwritten stroke pixel set during the handwriting process, the digital conversion efficiency of real-time handwritten strokes can be significantly improved. By focusing on the changing part between each frame rather than the whole image, the amount of calculation can be greatly reduced, and only the newly added or changed handwritten stroke pixels are processed. Such a processing method reduces the interference of repeated calculations and irrelevant information and improves the real-time performance of the conversion. At the same time, the construction of the differential handwritten stroke pixel set makes the subsequent handwritten stroke reconstruction more accurate, avoids errors caused by noise or irrelevant parts, and ensures the coherence and accuracy of the handwritten content. The advantage of this method is that it can quickly respond to the user's writing actions, reduce latency at the same time, make the digital process of real-time handwritten strokes more efficient, accurate, and adaptable, and is especially suitable for application scenarios with high requirements for real-time performance and accuracy.

[0039] In step S4, convert the initial handwritten stroke into a digital text of the handwritten stroke according to the pressure data and speed data in the handwritten stroke related data.

[0040] In this embodiment, converting the initial handwritten stroke into a digital text of the handwritten stroke according to the pressure data and speed data in the handwritten stroke related data is to use the pressure data and speed data in the handwritten stroke related data to correct the initial handwritten stroke, and then obtain the digital text of the handwritten stroke.

[0041] It should be noted that the pressure data in the handwritten stroke related data reflects the strength of the stroke during writing, and the speed data reflects the smoothness during writing. The pressure data and speed data can help analyze the detailed changes of the handwritten stroke and judge whether there are inconsistent or incorrect phenomena in certain areas of the handwritten stroke, so as to guide the correction process.

[0042] In specific implementation, first, pressure data can be used to correct the starting and ending strokes of the initial handwritten handwriting. For example, if the pressure suddenly weakens in some strokes on the handwriting, it can be automatically inferred that these places may be writing pauses or error areas. When writing continuous strokes, the pressure data can be used to infer the intensity distribution of the strokes, especially at the junctions of letters or numbers, helping to correct the breakpoints of the initial handwritten handwriting and ensuring the coherence of the handwriting. Then, the speed data can help determine whether there are abrupt pauses or areas with overly fast strokes during writing. Under normal circumstances, the writing of letters should have a smooth speed change. A sudden speed change may indicate unnatural strokes or errors. Through the speed data, the direction and curvature of the strokes can be adjusted to make them more in line with the natural fluency of writing. For places where the speed data shows abnormal fluctuations (such as sudden acceleration or deceleration), it can be analyzed according to the context whether there are problems with stroke incoherence or writing methods, and the initial handwritten handwriting can be smoothed and repaired according to the speed change trend to make the overall writing smoother and more natural. Finally, feature extraction can be performed on the corrected initial handwritten handwriting, and character recognition algorithms (such as deep learning models, OCR technology, etc.) can be used for recognition to map each stroke to the corresponding character. The initial handwritten handwriting after correction and optimization is converted into digital text through the character recognition algorithm, completing the conversion from handwritten handwriting to text.

[0043] It should be noted that converting the initial handwritten handwriting based on the pressure data and speed data in the handwriting-related data can significantly improve the accuracy and real-time performance of converting handwritten handwriting into digital text.

[0044] Thus, in this application, first, by processing each frame of the handwritten handwriting image sequence, extracting suspected handwriting pixel data, and converting it into a suspected handwritten handwriting image, the accuracy and real-time performance of handwritten handwriting digitization can be greatly improved. Then, by extracting the inter-frame differences of all suspected handwritten handwriting images, a set of differential handwriting pixels during the writing process can be obtained, which can significantly improve the digital conversion efficiency of real-time handwritten handwriting. By focusing on the changing parts between each frame rather than the whole image, the computational amount can be greatly reduced, and only the newly added or changed handwriting pixels are processed. Such a processing method reduces the interference of repeated calculations and irrelevant information and improves the real-time performance of the conversion. Finally, converting the initial handwritten handwriting based on the pressure data and speed data in the handwriting-related data can significantly improve the accuracy and real-time performance of converting handwritten handwriting into digital text to reduce the conversion delay of handwritten handwriting.

[0045] In summary, the technical solution adopted in this application can efficiently and accurately generate digital text of real-time handwritten handwriting to reduce the conversion delay of handwritten handwriting.

[0046] Embodiment 2. The present application provides a real-time digital conversion system for handwritten handwriting. Refer to Figure 4 As shown in the figure, which is a schematic diagram of the real-time digital conversion system for handwritten handwriting according to this embodiment of the present application. The conversion system includes: A data acquisition module 100 for real-time collecting handwriting-related data during the handwriting process; An image processing module 200 for extracting a sequence of handwritten handwriting images from the handwriting-related data, extracting handwriting pixels from each handwritten handwriting image in the sequence of handwritten handwriting images to obtain suspected handwriting pixel data for each handwritten handwriting image, and converting each piece of suspected handwriting pixel data into a corresponding suspected handwritten handwriting image; An initial handwriting construction module 300 for extracting inter-frame differences from all the suspected handwritten handwriting images, thereby obtaining a set of differential handwriting pixels during the handwriting process, and constructing an initial handwritten handwriting during the handwriting process through the set of differential handwriting pixels; A handwriting conversion module 400 for converting the initial handwritten handwriting into a digital text of the handwritten handwriting according to the pressure data and speed data in the handwriting-related data.

[0047] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowcharts and / or block diagrams can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0048] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0049] It should also be noted that the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent in such a process, method, commodity or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, commodity or device including the element.

Claims

1. A real-time digital conversion method for handwritten handwriting, characterized in that, The conversion method includes the following steps: Collect handwriting-related data during the real-time handwriting process; Extract a sequence of handwritten handwriting images from the handwriting-related data, extract handwriting pixels from each handwritten handwriting image in the sequence of handwritten handwriting images to obtain suspected handwriting pixel data for each handwritten handwriting image, and convert each piece of suspected handwriting pixel data into a corresponding suspected handwritten handwriting image; Extract inter-frame differences from all the suspected handwritten handwriting images to obtain a set of differential handwriting pixels during the handwriting process, and construct an initial handwritten handwriting during the handwriting process through the set of differential handwriting pixels; Convert the initial handwritten handwriting into a digital text of the handwritten handwriting according to the pressure data and speed data in the handwriting-related data.

2. The real-time digital conversion method for handwritten handwriting according to claim 1, wherein, Collect handwriting-related data during the real-time handwriting process through a pressure sensor, a gyroscope, and a camera.

3. A real-time digital conversion method for handwritten handwriting as described in claim 1, characterized in that The handwriting-related data includes a sequence of handwritten handwriting images, pressure data, and speed data during the writing process.

4. A real-time digital conversion method for handwritten handwriting according to claim 1, characterized in that, Extracting handwriting pixels from each handwritten handwriting image in the sequence of handwritten handwriting images to obtain suspected handwriting pixel data for each handwritten handwriting image specifically includes: For each handwritten handwriting image in the sequence of handwritten handwriting images, determine the image blocks corresponding to each pixel point in the handwritten handwriting image; Determine the suspected coefficient of handwriting pixels for each pixel point in the handwritten handwriting image through the corresponding image block; Take the pixel points with the suspected coefficient of handwriting pixels greater than the set suspected threshold as suspected handwriting pixel points to obtain a plurality of suspected handwriting pixel points; Take the data set composed of all suspected handwriting pixel points as the suspected handwriting pixel data of the handwritten handwriting image to obtain the suspected handwriting pixel data of each handwritten handwriting image.

5. A real-time digital conversion method for handwritten handwriting as described in claim 1, characterized in that, Converting each piece of suspected handwriting pixel data into a corresponding suspected handwritten handwriting image is to perform information mapping on each piece of suspected handwriting pixel data to obtain the suspected handwritten handwriting image corresponding to each piece of suspected handwriting pixel data.

6. A real-time digital conversion method for handwritten handwriting according to claim 1, characterized in that Extracting inter-frame differences from all the suspected handwritten handwriting images to obtain a set of differential handwriting pixels during the handwriting process specifically includes: Select a suspected handwritten handwriting image from all the suspected handwritten handwriting images, and obtain the suspected handwritten handwriting image of the previous frame of the selected suspected handwritten handwriting image; Extract differences between the selected suspected handwritten handwriting image and the suspected handwritten handwriting image of the previous frame to obtain a set of differential handwriting pixels corresponding to the selected suspected handwritten handwriting image, and obtain a set of differential handwriting pixels corresponding to each suspected handwritten handwriting image; Construct a set of differential handwriting pixels during the handwriting process according to all the sets of differential handwriting pixels.

7. A real-time digital conversion method for handwritten handwriting according to claim 6, characterized in that, Obtaining the set of differential handwriting pixels corresponding to the selected suspected handwritten handwriting image further includes: performing clustering analysis on all the differential handwriting pixel points in the set of differential handwriting pixels, and removing the outlier pixel points in the set of differential handwriting pixels according to the clustering analysis result.

8. The real-time digital conversion method for handwritten handwriting according to claim 1, characterized in that Constructing an initial handwritten handwriting during the handwriting process through the set of differential handwriting pixels specifically includes: Obtain the coordinates of the initial handwritten handwriting; Take the coordinates of the initial handwritten handwriting as the starting point, and connect each differential handwriting pixel point in the set of differential handwriting pixels in the corresponding order to obtain the initial handwritten handwriting during the handwriting process.

9. A real-time digital conversion method for handwritten handwriting according to claim 1, characterized in that Converting the initial handwritten handwriting into a digital text of the handwritten handwriting based on the pressure data and speed data in the handwriting-related data is to correct the initial handwritten handwriting using the pressure data and speed data in the handwriting-related data to obtain a digital text of the handwritten handwriting.

10. A real-time digital conversion system for handwritten handwriting, which is used to execute a real-time digital conversion method for handwritten handwriting according to any one of claims 1 to 9, characterized in that, The conversion system includes: A data acquisition module for real-time acquisition of handwriting-related data during the handwriting process; An image processing module for extracting a sequence of handwritten handwriting images from the handwriting-related data, extracting handwriting pixels from each handwritten handwriting image in the sequence of handwritten handwriting images to obtain suspected handwriting pixel data for each handwritten handwriting image, and converting each piece of suspected handwriting pixel data into a corresponding suspected handwritten handwriting image; An initial handwriting construction module for extracting inter-frame differences from all the suspected handwritten handwriting images to obtain a set of differential handwriting pixels during the handwriting process, and constructing an initial handwritten handwriting during the handwriting process through the set of differential handwriting pixels; A handwriting conversion module for converting the initial handwritten handwriting into a digital text of the handwritten handwriting based on the pressure data and speed data in the handwriting-related data.

Citation Information

Patent Citations

  • Handwritten information recognition method and device

    CN105988567A

  • Handwriting track extraction method and device based on image frame, storage medium and equipment

    CN114648768A

  • Handwriting analysis method and device

    CN117854077A

  • Digital application method and system based on dot matrix intelligent pen

    CN118015636A

  • Device, method and graphical user interface for handwritten interaction

    US20180095653A1