Handwritten stroke order identification method and device, and storage medium

Abstract

The invention discloses a handwritten stroke order identification method, which comprises the following steps of: acquiring handwritten electronic information sequence data, constructing a distance matrix and an alignment path of a handwritten and a corresponding printing form in a character library, identifying dislocation point positions and dislocation strokes in the alignment path according to a point position alignment relationship between the handwritten sequence data and the printing form sequence data, and identifying the handwritten stroke order according to the dislocation point positions and the dislocation strokes. And replacing the printed form sequence data according to the handwritten form handwriting point positions and strokes in the distance matrix until an optimal alignment path is obtained, and outputting the current writing stroke order according to the point position pairs and strokes determined according to the optimal alignment path. The handwriting is segmented into standard strokes, DTW alignment can be used, the standard strokes can be used for segmentation, the printed stroke order in the character library is adjusted to be consistent with the handwriting stroke order, the standard strokes are determined, the stroke order is defined, and handwriting recognition of an auxiliary identification system is facilitated.

Description

Technical Field

[0001] The invention relates to the technical field of artificial intelligence and computer information processing, in particular to a method for recognizing the stroke order of handwritten electronic information. Background Art

[0002] With the popularity of mobile information tools such as smart phones and PDAs, handwriting recognition technology has also entered the era of large-scale application. Handwriting recognition allows users to input text in the most natural and convenient way. It is easy to learn and use, and can replace keyboards or mice. There are many types of devices for handwriting input, such as electromagnetic induction handwriting tablets, pressure-sensitive handwriting tablets, touch screens, trackpads, ultrasonic pens, etc.

[0003] Handwriting recognition belongs to the category of text recognition and pattern recognition. What we often call handwriting recognition refers to online handwriting recognition.

[0004] With the advancement of digital life and the increasing demand for writing speed and the need for electronic signatures, people are becoming more and more accustomed to writing and inputting on electronic devices in their lives. Therefore, stroke order recognition of handwritten Chinese characters written on electronic devices is of great significance. In terms of cultural communication, it can allow the writer to know at any time whether there are any abnormalities in the stroke order of the characters he / she has written and how to correct them.

[0005] In addition, the strokes and stroke order of handwriting can fully reflect the behavioral characteristics and habits of the writer, which is of great significance for handwriting forensic identification, providing auxiliary information for the appraiser, analyzing the writing characteristics of the signer, and identifying the identity of the signer.

[0006] The writing culture of Chinese characters is profound and has a long history. It is a core course for students in primary school. A large number of students and adults learn Chinese characters and calligraphy through tutorials. One of the major difficulties is the writing stroke order of Chinese characters.

[0007] Chinese character stroke order recognition refers to the recognition of the correct stroke order of handwritten Chinese characters. It is a computer vision technology and image processing technology that aims to recognize text characters by detecting the stroke order of Chinese characters. The research on stroke order detection and recognition algorithms is an important part of the field of computer vision and image processing. The recognition algorithm of Chinese characters mainly uses the characteristic information of Chinese characters to detect and recognize stroke order, and usually uses morphological analysis technology and topological structure technology to detect the stroke order of Chinese characters.

[0008] In order to realize the detection and recognition of the stroke order of Chinese characters, a corpus of Chinese characters is generally established first to collect, manage and control a comprehensive resource of the strokes and stroke order of Chinese characters, which helps to improve the detection and recognition accuracy of the stroke order of Chinese characters.

[0009] When writing on electronic devices, writers usually use non-standard fonts according to their own writing habits, often with connected strokes, and the pen or finger may accidentally touch the screen when inputting, resulting in incorrect strokes, etc., making it difficult to establish a corresponding relationship with the standard fonts and standard stroke order in the font library. It is difficult to determine the writing stroke order of handwriting. Summary of the invention

[0010] In view of this, the present invention proposes a method for automatically identifying the stroke order of electronic handwriting, which uses a dynamic time warping algorithm (DTW) to find the point alignment relationship between the electronic handwriting and the printed text in the corpus, find out the abnormal matching point pairs in the handwriting and other points that may be replaced, estimate the point replacement rate, further estimate the replacement rate of the entire stroke segment, find out the most likely replacement stroke pair in the point sequence, replace the corresponding strokes, and form a new replacement stroke order until all abnormal points are replaced, or until the replacement rate is small enough, and output the current writing stroke order.

[0011] Based on one aspect of the present invention, a method for handwriting stroke order recognition is proposed, which obtains handwriting electronic information sequence data, constructs a distance matrix and an alignment path between the handwriting and the corresponding printed text in a character library, identifies misaligned points and misaligned strokes in the alignment path according to the point alignment relationship between the handwriting sequence data and the printed text sequence data, replaces the printed text sequence data according to the handwriting handwriting points and strokes in the distance matrix until an optimal alignment path is obtained, and outputs the current writing stroke order according to the point pairs and strokes determined by the optimal alignment path.

[0012] Further preferably, a distance matrix and an alignment path are constructed by a dynamic time warping algorithm, and abnormal matching point pairs and their possible alternative point pairs and possible alternative stroke pairs are found according to the alignment relationship.

[0013] Further optimization is to calculate the similarity of the points, estimate the abnormal points according to the distance, align the abnormal points, detect possible alternative points, estimate the point replacement rate according to the possible alternative point pairs, continuously iterate the abnormal points, further estimate the replacement rate of the entire stroke, find the most likely alternative stroke pair in the point sequence, replace the corresponding strokes, and at the same time calculate the alignment path through the DTW algorithm, getting closer and closer to the written stroke order until the best alignment path is obtained to form the current writing stroke order.

[0014] Further preferably, the point alignment relationship is found by a dynamic time warping algorithm, specifically including constructing a distance matrix D between the handwritten electronic information sequence data S and the corresponding printed sequence data T in the character library, calculating the distance between each pen-falling point of the sequence data S and the corresponding pen-falling point of the sequence data T, and constructing the distance matrix D: D = {d kk'|k=1,2,…,n;k'=1,2,…,n'}, determine the alignment path W of sequence data S and sequence data T through dtw algorithm according to the difference,

[0015] W={(k r ,k' r )|r=1,2,…,l},

[0016] Among them, (k r ,k' r ) represents the point where the sequence S is written in the rth pair of matching points in the alignment path The point where the sequence T ends is the matching point pair, and l is the length of the alignment path.

[0017] Further preferably, the difference is the Euclidean distance between the pen-falling points, which is calculated by the bi-norm |.|2 method according to the formula: d kk' =|s k -t k' |2, calculate the distance d between the point where the sequence data S is written and the corresponding point where the sequence data T is written kk' .

[0018] Further preferably, if the handwriting point corresponding to the standard printed text is not on the alignment path w, the corresponding point is a matching abnormal point pair. For each matching abnormal point pair (k r ,k' r ), and retrieve whether there is a potential matching point pair (k r' ,k' r' ) can replace the abnormal matching point pair. The obtaining of the optimal alignment path includes, in the horizontal and vertical directions of each abnormal matching point pair, finding the point with the smallest distance and not on the path as a possible replacement point, determining the optimal path through the DTW distance matrix image, and connecting the points on the optimal path into strokes.

[0019] Further preferably, the exponential function is used to calculate the substitution rate using the distance matrix to estimate the abnormal matching point pairs, specifically, according to the formula:

[0020]

[0021] Calculate the distance matrix for the substitution rate p rr' , convert point replacement into stroke replacement, according to the point replacement rate p rr' Calculate the replacement rate p of the points in the stroke ij,i'j' , determine the replacement strokes, for each standard stroke T in the information sequence T i , according to the stroke T i Point t ij Stroke T i' Point ti'j' The probability of substitution p ij,i'j' , according to the formula:

[0022]

[0023] Calculate stroke T i By another stroke T i' The probability of substitution P ii' , determine the matching point of any point in the stroke segment alignment path in the handwritten information sequence T according to the replacement probability, obtain the best matching stroke pair, and determine the replacement stroke.

[0024] Further preferably, according to the replacement probability P ii' Calling formula: Calculate and obtain the best matching stroke pair (i * ,i' * ), according to the formula: The replacement probability is calculated, and the strokes with replacement probability greater than the threshold are selected as replacement strokes. The replacement stroke data are reinserted into the abnormal stroke position in the printed data sequence to obtain the current standard stroke data sequence.

[0025] Based on the second aspect of the present invention, an electronic device is proposed, comprising: a processor; and a memory storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the handwriting stroke order recognition method described above.

[0026] Based on the third aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is proposed, wherein the computer instructions are used to enable the computer to execute the handwriting stroke order recognition method described above.

[0027] The present invention proposes a stroke order recognition method for handwritten Chinese character sequence data written on an electronic device, by continuously aligning the point sequence of handwritten data and printed data, finding out the strokes with inconsistent stroke order and reorganizing the strokes until the stroke order of the printed data is consistent with the handwritten data. The method does not rely on the annotation of any training data, does not rely on any prior information of Chinese characters or stroke types and the adjustment of too many parameters, can achieve high stroke order recognition accuracy and robustness, can accurately and efficiently recognize the stroke order of handwritten Chinese characters of various styles written on electronic devices, helps to assist the appraiser in recognizing and confirming the signed handwriting during the handwriting identification process, and is conducive to learning Chinese character writing, calligraphy teaching, etc.

[0028] The present invention divides handwriting into standard strokes, can use DTW to align, can use standard strokes to divide, adjusts the stroke order of printed text in the character library to be consistent with the stroke order of handwriting, determines standard strokes, and defines the stroke order, which is helpful to assist the handwriting recognition of the identification system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the handwriting information stroke order recognition process in this exemplary embodiment;

[0030] Figure 2 A schematic diagram of identifying the stroke order by finding the alignment relationship between handwritten and printed text points using the method of the present invention is shown;

[0031] Figure 3 is a structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0033] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0034] like Figure 1 Shown is a schematic diagram of the stroke order recognition process of handwritten information in this exemplary embodiment.

[0035] Obtain electronic handwriting information data, obtain electronic information time series data, find the printed text corresponding to the same written content information in the corpus character library, and extract the printed text sequence data; construct the distance matrix and alignment path between the handwriting and the corresponding printed text through the dynamic time warping algorithm (DTW), find the point alignment relationship between the handwriting information and the printed text information, and determine whether there is point misalignment and stroke misalignment in the alignment path? If there is misalignment, adjust the printed electronic information sequence according to the handwriting points and strokes, determine the new printed electronic information stroke order, until all points and strokes are aligned, and further calculate the distance matrix and alignment path through the DTW algorithm, until there are no misaligned points and strokes in the alignment path, and output the stroke order of the printed text sequence.

[0036] Specifically include:

[0037] The input unit obtains handwritten electronic information data, such as obtaining electronic information sequence data S for a handwritten electronic information with content w. When the writer is writing with content w, the electronic device collects the electronic data of the pen-falling point information sequence, and obtains a series of electronic information sequence data S.

[0038] S={s k |k=1,2,…,n}

[0039] Among them, s k represents the kth pen point information in the data S, and n represents the total number of points collected.

[0040] Search the handwritten information sequence data T with content w in the corpus printed word library, obtain the stroke segment electronic data sequence, determine the standard stroke number,

[0041] T={T i |i=1,2,…,m}

[0042] Among them, T i represents the i-th stroke in the corresponding printed stroke data T in the collected corpus, and m represents the total number of strokes of the printed text.

[0043] Embodiment 1, construct a character library, the character library is written in a standardized manner according to the standard strokes and standard stroke order of a fixed script (such as Kaiti or Songti) through an electronic writing device, and written according to the standard printed format to ensure the standard strokes and standard stroke order of each character, and a stroke section from the point where the pen is placed to the point where the pen is lifted. Record the starting and ending states of the printed body in the character library, and the handwriting point position information.

[0044] Since the data in the font library is printed, the stroke segments and stroke order of each printed character are written according to the standard, m must be the standard number of strokes for printing w, T i The arrangement order between them is the standard stroke order, and T i is the writing trajectory of the standard strokes. The i-th stroke of the printed standard strokes T i The writing track consists of a series of handwriting points.

[0045] T i ={t ij |j=1,2,…,n i}

[0046] Among them, t ij represents the jth handwriting point in the i-th stroke of the printed information sequence data T in the character library collected by the electronic device, n i Indicates the total number of points of the i-th stroke segment collected.

[0047] The sequence data T of the printed handwritten information in the font library is represented by points:

[0048] T={t k' |k'=1,2,…,n'}

[0049] Among them, t k' represents the k'th pen point in the data T collected by the electronic device, and n' represents the total number of points collected, that is, the sum of the number of points in each stroke.

[0050]

[0051] In Example 2, other methods may be used, such as obtaining a certain font library in a dictionary or system character library as the printed body, obtaining a binary image, determining the starting point and the ending point of the stroke as the pen-dropping point and the pen-lifting point according to the principle of from left to right and from top to bottom, and dividing the points in each stroke segment with equal distances to distinguish and determine the number of strokes and points of the printed body.

[0052] Calculate the similarity of points, estimate abnormal points based on distance, align abnormal points, detect possible alternative points, continuously iterate abnormal points, and calculate the alignment path through the DTW algorithm, getting closer and closer to the written stroke order until the best alignment path is obtained, and output the current stroke order as the handwritten stroke order.

[0053] The dynamic time warping algorithm (DTW) is used to find the point alignment relationship between the electronic handwriting and the printed text in the corpus. Specifically,

[0054] Construct the distance matrix D between the handwritten electronic information sequence data S and the corresponding printed sequence data T in the font library, and calculate the pen point s of the sequence data S k The corresponding point t of the sequence data T k' The distance between kk' .

[0055] For example, in this embodiment, a more universal distance measurement method can be used, which is the Euclidean distance between the stroke points, that is, through the two-norm |.|2 method, according to the formula:

[0056] d kk' =|s k -t k' |2

[0057] Calculate the distance between the point where the sequence data S is written and the corresponding point where the sequence data T is written. Further, construct a distance matrix D, specifically:

[0058] D={d kk' |k=1,2,…,n;k'=1,2,…,n'}

[0059] Among them, d kk'is the distance between the handwritten electronic information sequence data and the corresponding pen point in the printed sequence data, that is, the difference between the two points. The alignment path W of the sequence data S and the sequence data T is determined by the dtw algorithm according to the difference.

[0060] W={(k r ,k' r )||r=1,2,…,l}

[0061] Among them, (k r ,k' r ) represents the point where the sequence S is written in the rth pair of matching points in the alignment path The point where the sequence T ends is the matching point pair, and l is the length of the alignment path.

[0062] Detect the point pairs with abnormal matching in the alignment path W. If the handwriting point corresponding to the standard printed text is not on the alignment path w, this point is an abnormal matching point. r ,k' r ), to check whether there is a potential matching point pair (k r' ,k' r' ) can be used for alternative matching.

[0063] The alignment path is found through the DTW algorithm, and abnormal point pairs and replacement point pairs are found based on the points on the alignment path. The replacement points are points on the non-aligned path in the horizontal and vertical directions, and the distance between the point pairs is less than the distance between the abnormal point pairs. The horizontal and vertical methods are used for searching. In the horizontal and vertical directions of each abnormal matching point pair, the previous abnormal matching point pair can be replaced, and the following points may also be replaced. The points with the smallest distance and not on the path are found as possible replacement points. By constructing the DTW distance matrix image, the optimal alignment path is determined and the points on the path are connected into strokes.

[0064] Figure 2 A schematic diagram of identifying the stroke order by finding the point alignment relationship between handwritten electronic information and printed electronic information using the method described in the present invention is shown.

[0065] Taking the writing of the character "燕" as an example to illustrate finding alternative strokes to complete the stroke order adjustment, it can be seen that for the printed form, the strokes in the middle part are written with the middle "口" first and then the left and right parts respectively, while for the handwritten form, the middle part is written from left to right. A distance matrix is constructed, including the printed form sequence number, the handwritten form sequence number, and the alternative alignment benchmark. From the matrix diagram, it can be seen that there are points and stroke segments in the original distance matrix of the data sequence pairs that are not on the alignment path (there are obvious stroke points and stroke segments not on the alignment path in the middle part of the matrix diagram). According to the alignment path, abnormal point pairs and point pairs that can be used for replacement are found, and then replaceable stroke pairs are obtained. From Figures b and c, it can be seen that the abnormal point pairs are gradually replaced, and the standard stroke order is gradually replaced with the handwritten stroke order. The alignment path basically becomes a straight line, obtaining the optimal alignment path, realizing the replacement of stroke segments, and obtaining the correct writing stroke order.

[0066] Through three figures ( Figure 2 a, 2b, 2c) the iterative algorithm process for the stroke order recognition of the handwritten character "燕" is shown, which are the initial state of iteration, the first iteration, and the last iteration respectively. In Figure a, the left figure is the printed form, the middle figure is the handwritten form, and the right figure is the matrix diagram. Among them, the left figure is the sequence trajectory of the printed form, the middle figure is the sequence trajectory of the handwritten form, and the right figure is the distance matrix, alignment path between sequence data, and the detected alternative point pairs. The positive sequence alignment path, reverse sequence alignment path, and alternative alignment path changes are shown in the sample sequence and the original point distance matrix diagram. It can be seen that with each iteration, the alignment path becomes more and more regular, and the alternative point pairs become fewer and fewer until finally the printed form is adjusted to the same stroke order as the handwritten form, and the alignment path becomes basically anomaly-free.

[0067] The difference degree is determined by the distance matrix. In this exemplary embodiment, the substitution rate of the distance matrix can be used to estimate abnormal matching point pairs through an exponential function. For example, if there are two groups of potential matching point pairs (k r , k' r' ) and (k r' , k' r ) that can be substituted respectively. Specifically, according to the formula:

[0068]

[0069] the substitution rate p rr' of the distance matrix is calculated. Among them, α is an adjustable parameter to control the distance scale. When and only when the distance is much smaller than and , the substitution rate p rr' will approach 1.

[0070] When the substitution rate p rr'The larger the value, the smaller the difference between the point pairs, and the selection rate p rr'

[0071] Replace with a larger point pair.

[0072] Furthermore, the point replacement is converted into stroke replacement, and the replacement relationship between strokes is found in the printed text sequence data T. The point replacement rate p in the sequence data determined by the distance matrix is rr' Calculate the replacement rate p of the points in the stroke ij,i'j' , determine the replacement strokes. For each standard stroke T in the printed information sequence T i , according to the stroke T i Point t ij Stroke T i' Point t i'j' The probability of substitution p ij,i'j' , according to the formula:

[0073]

[0074] Calculate stroke T i By another stroke T i' The probability of substitution P ii' , determine the matching point of any point in the stroke segment alignment path in the handwritten information sequence T according to the replacement probability, obtain the best matching stroke pair, and determine the replacement stroke.

[0075] If p ij,i'j' =p rr' , then it means That is, the jth point of the i-th stroke in the data sequence T is the matching point of the r-th T in the alignment path, and the j'th point of the i'-th stroke in T is the matching point of the r'-th T in the alignment path.

[0076] Search for the stroke segment with the maximum replacement probability from all strokes for replacement. ii' Calling formula: Calculate and obtain the best matching stroke pair (i * ,i' * ), indicating that the best replacement stroke i' * Replace abnormal strokes i * ,

[0077] According to the formula: Calculate the replacement probability and select the strokes with replacement probability greater than the threshold for replacement. For example, an adjustable probability threshold P can be set. thr ,

[0078] like i'th * Segment stroke position i *The best matching stroke of the segment stroke, the i' * Paragraph strokes replace i * The strokes of the segment form a new T's current stroke order, and the stroke T i' Reinsert T i The position of the handwritten Chinese character is replaced, and the sequence data T is updated to obtain the standard stroke data sequence after the strokes are replaced, which is output as the stroke order of the final handwritten Chinese character sequence data S.

[0079] like This means that no suitable replacement strokes can be found, and the current stroke order of T is output as the stroke order of the final handwritten Chinese character sequence data S.

[0080] The stroke order is adjusted by replacing strokes, so that the standard stroke order in the font library becomes closer and closer to the stroke order of the target handwritten Chinese character in each iteration.

[0081] Complete the calculation of replacement points and replacement probabilities of all handwritten information sequence data, and the replacement probabilities of all strokes are less than the probability threshold P thr , complete the stroke replacement, and use the stroke sequence corresponding to the updated current sequence data as the stroke sequence of the handwriting information.

[0082] Further, the distance matrix and the alignment path are calculated by the DTW algorithm until there is no stroke misalignment in the alignment path, and the stroke order of the current printed text sequence is output as the stroke order of the handwritten electronic information.

[0083] The structured flowchart of the electronic device 300 that can be used as the server or client of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent the computer device of various forms of digital electronics, such as, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only used as examples, and are not intended to limit the realization of the present application described herein and / or required.

[0084] like Figure 3As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304. Multiple components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, an output unit 307, a storage unit 308, and a communication unit 309. The input unit 306 can be any type of device that can input information to the electronic device 300, and the input unit 306 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 307 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 308 may include, but is not limited to, a magnetic disk, an optical disk. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0085] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above. For example, in some embodiments, the reconstruction and decomposition of the muscle movement trajectory redrawn according to the original trajectory of the signature stroke, and the decomposition of its logarithmic velocity curve, etc. can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. In some embodiments, the computing unit 301 can be configured to execute the signature handwriting dynamic acquisition implementation method by any other appropriate means (e.g., by means of firmware).

[0086] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0087] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0088] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0089] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0090] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0091] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

Claims

1. A handwritten stroke order recognition method, characterized in that: Obtain handwritten electronic information sequence data, construct a distance matrix and alignment path between the handwritten and the corresponding printed text in the font library, identify the misaligned points and strokes in the alignment path according to the point alignment relationship between the handwritten sequence data and the printed text sequence data, replace the printed text sequence data according to the handwritten handwriting points and strokes in the distance matrix until the optimal alignment path is obtained, and output the current writing stroke order according to the point pairs and strokes determined by the optimal alignment path.

2. The method according to claim 1, characterized in that The distance matrix and alignment path are constructed through the dynamic time warping algorithm, and the abnormal matching point pairs and their possible alternative point pairs and possible alternative stroke pairs are found according to the alignment relationship.

3. The method according to claim 2, characterized in that Calculate the similarity of points, estimate abnormal points based on distance, align abnormal points, detect possible alternative points, estimate point replacement rate based on possible alternative point pairs, continuously iterate abnormal points, further estimate the replacement rate of the entire stroke, find the most likely alternative stroke pair in the point sequence, replace the corresponding strokes, and calculate the alignment path through the DTW algorithm to approximate the written stroke order until the best alignment path is obtained to form the current writing stroke order.

4. The method according to claim 2, characterized in that: The point alignment relationship is found by the dynamic time warping algorithm, specifically including constructing a distance matrix D between the handwritten electronic information sequence data S and the corresponding printed sequence data T, calculating the distance between each pen point of the sequence data S and each pen point of the sequence data T, and constructing the distance matrix D: D = {d kk' |k=1,2,…,n;k'=1,2,…,n'}, determine the alignment path W of sequence data S and sequence data T through dtw algorithm according to the difference, W={(k r ,k' r )|r=1,2,…,l}, Among them, (k r ,k' r ) represents the point where the sequence S is written in the rth pair of matching points in the alignment path The point where the sequence T ends is the matching point pair, and l is the length of the alignment path.

5. The method according to claim 4, characterized in that The difference is the Euclidean distance between the pen points, which is calculated by the bi-norm |.|2 method according to the formula: d kk' =|s k -t k' |2, calculate the distance d between the point where the sequence data S is written and the point where the sequence data T is written kk' ; If the handwritten point corresponding to the standard printed text is not on the alignment path w, the corresponding point is a matching abnormal point pair. For each matching abnormal point pair k r ,k' r ), and retrieve whether there is a potential matching point pair (k r' ,k' r' ) can replace abnormal matching point pairs.

6. The method according to any one of claims 1 to 5, characterized in that: The obtaining of the optimal alignment path includes searching for a point with the smallest distance in the horizontal and vertical directions of each abnormal matching point pair and not on the alignment path as a possible replacement point, replacing the previous abnormal matching point pair or the following abnormal matching point pair with the replacement point, and determining the optimal alignment path by constructing a DTW distance matrix image and connecting the points on the path into strokes, wherein the replacement point is on the non-aligned path in the horizontal and vertical directions of the abnormal point pair, and the distance between the point pairs is less than the distance between the abnormal point pairs.

7. The method according to any one of claims 1 to 5, characterized in that: The exponential function is used to call the distance matrix to calculate the substitution rate and estimate the abnormal matching point pairs according to the formula: Calculate the point-to-point substitution rate p in the distance matrix rr' ; Further, the point pair replacement is converted into stroke replacement, according to the point replacement rate p rr' Calculate the point replacement rate p in the stroke ij,i'j' , determine the replacement strokes, for each standard stroke T in the printed information sequence T i , according to the stroke T i Point t ij Stroke T i' Point t i'j' The probability of substitution p ij,i'j' , according to the formula: Calculate stroke T i Stroke T i' The probability of substitution P ii' , according to the replacement probability, determine the matching points of the abnormal points in the stroke segment alignment path in the information sequence T, obtain the best matching stroke pair, and determine the replacement stroke.

8. The method according to claim 7, characterized in that According to the replacement probability P ii' Calling formula: Calculate and obtain the best matching stroke pair (i * ,i' * ), according to the formula: The replacement probability is calculated, and the strokes with replacement probability greater than the threshold are selected as replacement strokes. The replacement stroke data are reinserted into the abnormal stroke position in the printed data sequence to obtain the current standard stroke data sequence.

9. An electronic device, characterized in that: include: processor; and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the handwriting stroke order recognition method according to any one of claims 1-8.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the handwriting stroke order recognition method according to any one of claims 1-8.

Images