Online handwriting stroke order feature construction method and system and handwriting identification method

Abstract

The invention provides an online handwriting stroke order feature construction method and system and a handwriting identification method, and the method comprises the steps: analyzing handwriting collection data (x, y, p, t, s), and segmenting the handwriting into a plurality of strokes by using a point with a speed v close to 0 as a segmentation point; constructing a stroke order feature S including a stroke order feature, a stroke angle feature, a stroke type feature, a stroke length feature, a stroke position feature and a stroke conversion feature; and processing the stroke order feature S through frequency mapping transformation, extracting a high-frequency spectrum component in the stroke order feature S by using a high-frequency oscillation primary function in the frequency mapping transformation, and converting the stroke order feature S into a representation form suitable for a deep learning model. The method not only can reflect the sequence of strokes, but also can reflect the space structure difference caused by the sequence of strokes, remarkably improves the problem that a deep learning model is insensitive to different stroke orders, and improves the performance and generalization ability of the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of online handwriting recognition, and in particular to online handwriting stroke order feature construction and handwriting identification technology. BACKGROUND

[0002] Online handwriting analysis has become increasingly important in the digital age, especially in the fields of identity verification, electronic signature verification, and handwriting recognition. Currently, the existing online handwriting analysis technologies on the market mainly include image-based handwriting feature extraction and sequence-based handwriting recognition algorithms. These technologies, although to some extent meet the needs, have obvious shortcomings in processing stroke order information in deep learning models. The following focuses on three existing technologies related to the present application and their shortcomings in solving stroke order feature problems.

[0003] Technology One: Handwriting Color Chart Representation Method Based on Electronic Signature

[0004] This technology proposes a method of obtaining stroke direction features through a SOBEL operator template and converting them into a color chart. This method can intuitively display the details of electronic signatures and improve recognition accuracy. However, it relies on image processing and does not directly adapt to the needs of sequence models, and mainly solves the problem of missing stroke direction in images, without addressing the key processing problem of stroke order information in deep learning models.

[0005] Technology Two: Handwriting Stroke Recognition Method

[0006] This technology identifies handwriting stroke order by constructing an alignment path between handwriting and printed matter, especially suitable for standard stroke recognition. Although this method has certain effectiveness in stroke order recognition, it does not provide a direct method to extract and apply stroke order features to deep learning models, especially without setting a standard stroke reference set, limiting its possibility of widespread application.

[0007] Technology Three: Stroke Order Feature Extraction and Authentication Method in Online Handwriting Authentication

[0008] This technology performs online handwriting authentication by pre-learning stroke state transition probability matrix, aiming to effectively extract stroke order features and estimate the probability of features appearing in the population. Although this method shows certain advantages in handwriting identification, it does not fully consider the spatial structure differences of strokes, and also lacks a general framework to directly utilize stroke order information in deep learning models.

[0009] The above existing technologies mainly focus on how to express or identify stroke order features through indirect methods such as images or probability matrices, but none of them can completely solve the problem of directly processing and utilizing stroke order information in deep learning models, with the following limitations:

[0010] 1. Missing stroke structure information: existing technologies have difficulty directly reflecting the writing order and spatial structure differences of strokes in deep learning models, resulting in low sensitivity of the model to different stroke orders, affecting the accuracy and efficiency of recognition.

[0011] 2. Lack of flexibility: most existing technologies rely on a predefined set of standard strokes or stroke orders, which may be too rigid in practical applications and not easily adaptable to diverse and personalized writing styles, reducing the generalization ability of recognition.

[0012] 3. Limited representation: current technologies process stroke order information through position encoding or probability matrix, which can reflect the writing order to some extent, but fail to fully capture the complex conversion relationship and subtle spatial changes between strokes, which are crucial for accurate handwriting identification.

[0013] 4. Limited application scenarios: existing methods focus more on handwriting identification or stroke recognition, and less on the role of stroke features in handwriting text line recognition, online Chinese character segmentation and other more extensive application scenarios, limiting the practicality of the technology.

[0014] In summary, the current technology in the field of online handwriting analysis has obvious flaws in the processing of stroke features in deep learning models. SUMMARY

[0015] The present invention proposes an innovative solution to address the shortcomings of existing technologies, an online handwriting stroke feature construction method, system and handwriting identification method, which not only reflects the sequence of strokes but also reflects the spatial structure differences caused by the sequence of strokes, strengthens the importance of stroke features in various application scenarios of handwriting, and significantly improves the problem of deep learning model not sensitive to different stroke orders, to improve the performance and generalization ability of the model.

[0016] The technical solution of the present invention is as follows:

[0017] One aspect of the present application provides an online handwriting stroke feature construction method, comprising the following steps:

[0018] Step 1, stroke segmentation: by analyzing handwriting collection data (x, y, p, t, s), using points with a speed v close to 0 as segmentation points, the handwriting is segmented into several strokes;

[0019] Step 2, stroke feature construction: construct stroke features S including stroke sequence features, stroke angle features, stroke type features, stroke length features, stroke position features and stroke conversion features, to fully reflect the properties and stroke information of strokes;

[0020] Step 3, stroke order feature mapping: the stroke order feature S is processed by a frequency mapping transform, which uses high-frequency oscillation basis functions to extract high-frequency spectral components in the stroke order feature S, and converts the stroke order feature S into a representation suitable for a deep learning model.

[0021] Further, the stroke sequence feature construction in step 2 is to convert the absolute writing order of strokes into a normalized ratio i / n relative to the total number of strokes, thereby generating a numerical sequence seq between 0 and 1 i , to represent the stroke order sequence feature, where i represents the i-th stroke, and n represents the maximum number of strokes.

[0022] Further, the stroke angle feature construction in step 2 is to calculate the inverse tangent value of the angle between the stroke vector and the X-axis to quantitatively represent the stroke angle feature θ of the stroke, where the stroke vector is composed of the starting point and the end point of the stroke.

[0023] Further, the stroke type feature construction in step 2 is to discretize the continuous stroke angle into D direction types, D is generally even, such as 8, and use positive and negative symbols to encode the opposite relationship of the direction, thereby converting the stroke angle feature into a symbolic category value that can clearly distinguish the positive and negative directions.

[0024] Further, the stroke length feature construction in step 2 is to accumulate the lengths of the line segments formed by the multiple points of the stroke as the length feature L of the stroke.

[0025] Further, the stroke position feature construction in step 2 selects the horizontal and vertical coordinates Mx and My of the middle point M of each stroke as the position feature to ensure the robustness of the feature.

[0026] Further, the stroke conversion feature construction in step 2 is to calculate the rotation angle and translation distance between adjacent strokes to quantitatively represent the temporal relationship of the stroke order as a spatial geometric transformation parameter.

[0027] Further, the stroke order feature s is constructed by fusing the stroke sequence, angle, type, length, position, and spatial transformation multi-dimensional features of the strokes, to form a unified feature vector that can comprehensively represent the writing dynamic process and spatial structure, where the stroke order feature of the i-th stroke is Each S i There are k=9 features, and n strokes are combined together to form the stroke order feature S of the entire signature:

[0028]

[0029] Further, in step 3, the frequency mapping transformation comprises: linearly projecting the stroke order feature S by using a randomly initialized projection matrix B to map the feature to a high-dimensional frequency space; and performing a nonlinear transformation on the projected feature by using a trigonometric function to obtain the projection intensity of the feature at different frequencies, i.e., the high-frequency spectral component.

[0030] Further, in the stroke order feature mapping, the projection matrix B is a Gaussian random matrix, and the variance δ thereof is an adjustable hyperparameter; and the number of rows m of the Gaussian random matrix is an adjustable hyperparameter much larger than the dimension k of the stroke order feature S.

[0031] In another aspect, the application also provides an online handwriting stroke order feature construction system, which implements the above method and comprises:

[0032] A stroke segmentation module is configured to segment handwriting into strokes by analyzing handwriting collection data (x, y, p, t, s) and using points with a speed v close to 0 as segmentation points.

[0033] A stroke order feature construction module is configured to construct a stroke order feature S containing stroke sequence features, stroke angle features, stroke type features, stroke length features, stroke position features and stroke transition features, which comprehensively reflect the attributes of strokes and stroke order information.

[0034] A stroke order feature mapping module is configured to process the stroke order feature S by frequency mapping transformation, and the frequency mapping transformation uses a high-frequency oscillation basis function to extract a high-frequency spectral component in the stroke order feature S and converts the stroke order feature S into a representation form suitable for a deep learning network.

[0035] In a third aspect, the application also provides a handwriting identification method based on stroke order features, which comprises:

[0036] Obtaining point features and stroke order features of handwriting, and the stroke order features are obtained by using the method in claims 1-10.

[0037] Extending the stroke order features to align the number of the stroke order features with that of the point features at the point level.

[0038] Splicing and fusing the extended stroke order features with the point features to generate merged features.

[0039] Inputting the merged features into a backbone network to extract deep features and obtain a representation vector of handwriting.

[0040] Calculating the similarity between the representation vectors of two handwritings, and identifying the handwritings according to the similarity.

[0041] Further, in the handwriting identification method, the extending the stroke-level stroke sequence features comprises: mapping the stroke sequence features Smap i The stroke sequence features Smap i ' are copied to generate a copy Smap i The number of the copy Smap

[0042] Further, in the handwriting identification method, the splicing and fusing the extended stroke sequence features and the point features comprises: splicing the point feature Pi of the i-th stroke and the corresponding stroke sequence feature copy Smap i ' along the feature dimension to obtain the merged feature Mi, wherein c is the dimension of the point feature, and 2m is the dimension of the mapped stroke sequence feature.

[0043] Further, in the handwriting identification method, the handwriting identification according to the similarity comprises:

[0044] In the model training stage, the binary cross-entropy loss function is used to supervise the similarity, and the network parameters are optimized through the back propagation algorithm;

[0045] In the model testing stage, the calculated similarity is compared with the preset threshold value, and if it is greater than the threshold value, it is determined as a positive sample from the same writer, otherwise it is determined as a negative sample.

[0046] The advantages and effects of the present application are as follows:

[0047] 1. The present application realizes the change of strokes through stroke conversion features by fine processing of handwriting data and comprehensive construction of features, so as to reflect the stroke sequence difference in stroke sequence features, which can not only capture the unique pen of the writer, but also effectively reflect the conversion relationship between strokes.

[0048] 2. The present application proposes a stroke sequence feature mapping method, which uses the constructed stroke sequence features for deep learning network to improve the accuracy of the model. The robustness of the features is enhanced, and the generalization ability of the model is improved.

[0049] 3. The stroke sequence features of the present application can be used for various tasks, such as handwriting identification, handwritten text line recognition, online segmentation of handwritten text line, etc. In the field of handwriting identification and handwritten text recognition, the method shows excellent performance, can more accurately identify and understand handwritten characters, and has wide application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0050] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0051] Figure 1 A stroke order feature construction method flow chart;

[0052] Figure 2 A handwriting identification method based on stroke order features flow chart;

[0053] Figure 3 A text line recognition network based on stroke order features;

[0054] Figure 4 A text line Chinese character segmentation network based on stroke order features. DETAILED DESCRIPTION

[0055] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should fall within the protection scope of the present application.

[0056] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover not exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to the clearly listed steps or units, but can include other steps or units that are not clearly listed or inherent to the process, method, product or device.

[0057] Referring to Figure 1 In an embodiment of the present application, an online handwriting stroke order feature construction method is provided, which mainly uses stroke order features to solve the problem that a deep learning network cannot perceive stroke order differences, can more effectively reflect the sequence of strokes and structural difference information of strokes, and can improve the performance of a model. First, more rich point features such as speed v and acceleration a are constructed for x, y, p, t and s collected for each handwriting; then the handwriting is divided into several strokes through speed; then the corresponding stroke order features are constructed for each stroke, which are composed of the structural information of the stroke itself and the sequence information of the stroke transition; then the stroke order features are mapped to make them suitable for a deep learning network. The overall process includes the following steps:

[0058] Step 1, stroke segmentation: by analyzing handwriting collection data (x, y, p, t, s), using points with speed v close to 0 as segmentation points, the handwriting is segmented into several strokes;

[0059] Step 2, stroke feature construction: construct stroke features S including stroke sequence features, stroke angle features, stroke type features, stroke length features, stroke position features and stroke transition features, which comprehensively reflect the properties and stroke information of the strokes;

[0060] Step 3, stroke feature mapping: the stroke features S are processed by frequency mapping transformation, which uses high-frequency oscillation basis function to extract high-frequency spectral components in the stroke features S, and converts the stroke features S into a representation form suitable for deep learning model.

[0061] The stroke features constructed by the above method can be used in a variety of task scenarios, such as handwriting identification, handwritten text line recognition, and Chinese character segmentation of handwritten text line.

[0062] The present embodiment effectively solves the problem that deep learning model is difficult to directly process stroke information by multi-level feature extraction and conversion of original handwriting data. Stroke segmentation and stroke feature construction follow the natural law of writing behavior, and stroke feature mapping uses the concept of frequency transformation in signal processing theory to convert stroke features into input suitable for deep learning model. Therefore, not only the time sequence characteristics of the original data are preserved, but also the sensitivity of the model to stroke differences is enhanced, significantly improving the performance of the model in handwriting identification, handwritten text line recognition and handwritten text line segmentation tasks

[0063] The following embodiments will further illustrate the specific implementation of each step:

[0064] Step 1, stroke segmentation

[0065] Stroke features reflect the differences in stroke order through strokes, so stroke segmentation is needed. In order to facilitate the processing of neural networks, the strokes here are not the standard stroke segmentation in Chinese characters, but the strokes that represent horizontal and vertical strokes, so only the turning points that can be used for segmentation in the sequence need to be found, and the implementation is as follows:

[0066] In one embodiment, first, the collected handwriting time series data is obtained, each point contains horizontal coordinate x, vertical coordinate y, pressure p, time t, writing state s and other collection data, then the data is resampled by time t to ensure the time difference between two points is consistent, then the speed dx of point m in x direction and the speed dy of point m in y direction are calculated by five-point method, and then the speed of point p is obtained The point with speed v close to 0 is the segmentation point of the stroke.

[0067] According to the law of human kinematics, it is known that a person is not a uniform motion in the process of writing strokes, and the speed is close to 0 at the beginning and end of the stroke and at the turning point, so the above segmentation method can well correspond to a stroke. In addition, acceleration and other methods can also be used for stroke segmentation, but such segmentation method will cause the stroke to be too short, the extracted features are not obvious enough, and there will be differences from the stroke we understand, such as the speed change of the stroke horizontal is often from 0 to the maximum and then back to 0, at this time the acceleration in the middle will become 0, if acceleration is used for segmentation, it may cause a stroke horizontal to be segmented into two stroke horizontals.

[0068] Step 2, stroke sequence feature construction: this step is to construct a stroke sequence feature S containing stroke sequence feature, stroke angle feature, stroke type feature, stroke length feature, stroke position feature and stroke transition feature, which is used to comprehensively reflect the properties and stroke sequence information of the stroke.

[0069] (1) Construction of stroke sequence feature:

[0070] The construction of the stroke sequence feature is to convert the absolute writing sequence of the stroke into a normalized ratio i / n relative to the total number of strokes, thereby generating a numerical sequence seq between 0 and 1 i , which is used to represent the stroke sequence feature, where i represents the i-th stroke, and n represents the maximum number of strokes.

[0071] In an embodiment, first, according to the stroke segmentation, each stroke is numbered from 1, the maximum number of strokes n, and the stroke sequence feature seq of the i-th stroke i is i / n. Normalizing the maximum number of strokes normalizes the value to the range of 0-1 to represent the writing order of different strokes: seq i = i / n.

[0072] (2) Stroke angle feature:

[0073] Here, the construction of the stroke angle feature is to calculate the inverse tangent value of the angle between the stroke vector and the X-axis to quantitatively represent the stroke angle feature θ of the stroke, and the stroke vector is composed of the starting point and the end point of the stroke.

[0074] In an embodiment, first, a vector is constructed according to the starting point and the end point of each stroke after segmentation, which is called stroke vector, and the angle between the vector and the x-axis constitutes the stroke angle feature θ, which is calculated as follows:

[0075]

[0076] Where x1 and x2 represent the horizontal coordinates of the starting point and the end point of the stroke, and y1 and y2 are the corresponding vertical coordinates.

[0077] The calculation of the above stroke angle feature is based on the basic trigonometric function principle and can accurately reflect the direction information of the stroke. The extraction of the angle feature helps the model understand the spatial form of the stroke, which has a significant impact on the accuracy of the subsequent model.

[0078] (3) Constructing stroke type features:

[0079] Here, the stroke type feature is constructed by discretizing the continuous stroke angle into D direction types and using positive and negative signs to encode the opposite direction relationship, thereby converting the stroke angle feature into a symbolic category value that can clearly distinguish between forward and reverse directions.

[0080] In one embodiment, specifically, the specific stroke type C is obtained according to the stroke angle feature. Generally, 360 degrees are evenly divided into D parts, and the value of D is usually set to 8 by default. Starting from -22.5 degrees, every 45 degrees is a division of stroke type, and there are 8 different directions. Therefore, from -22.5 degrees to 22.5 degrees is stroke type 1, from 22.5 degrees to 67.5 degrees is stroke type 2, and so on, totaling 8 types. The difference between type 1 and type 5 lies in the inconsistent direction. In order for the network to learn this information, we use opposite numbers to represent the direction, i.e., types 5, 6, 7, and 8 are the opposite direction strokes of types 1, 2, 3, and 4. The calculation method is as follows, using radian system for calculation,

[0081] b = ((θ i + π / 8) % 2π) / / (π / 4) + 1

[0082]

[0083] Where b represents the stroke type value without taking the opposite number, taking values from 1 to 8, and Ci is the stroke type value taking the opposite number, taking values from -4 to -1 and 1 to 4. When the b value is greater than 4 (i.e., the stroke type is 5, 6, 7, and 8), subtract 4 from b to get the opposite number of stroke type -1, -2, -3, and -4. The main purpose is to use negative numbers to represent the relationship that type 1 and type 5 are only opposite in direction, and type 2 and type 6 are also opposite in direction.

[0084] This embodiment realizes clear classification of stroke types by mapping the stroke angle feature θ to D direction categories. The construction of stroke type features is based on the understanding of writing habits, and by converting the angle feature into discrete categories, the model can more easily identify and learn the stroke type. The introduction of stroke type features, especially through opposite numbers to distinguish forward and reverse strokes, greatly enriches the input features of the model and improves the model's ability to distinguish different stroke structures.

[0085] (4) Stroke length feature:

[0086] Stroke length feature construction is very simple, which is to accumulate the line segment lengths of the points of the stroke to obtain the length feature L of the stroke.

[0087] (5) Stroke position feature:

[0088] The stroke position feature is constructed by selecting the horizontal and vertical coordinates Mx and My of the middle point M of each stroke to ensure the robustness of the feature. The reason for selecting the middle point is that there is already length information and angle information, so the position information of the stroke can be represented without the coordinates of the starting point and the end point.

[0089] (6) Stroke transition feature:

[0090] The stroke transition feature is constructed by calculating the rotation angle and translation distance between adjacent strokes to quantify the time sequence relationship of the stroke order into spatial geometric transformation parameters.

[0091] In one embodiment, specifically, since the stroke order is reflected in the time sequence of stroke writing, the rotation and translation operations can be used to transform the previous stroke to the same position of the next stroke to make the center point positions coincide, so the rotation angle dθ, the horizontal coordinate distance dx, and the vertical coordinate distance dy can reflect the change order of the stroke order. Since there is no stroke before the first stroke, the feature values are all 0. The calculation formula of the transition feature of the ith stroke is as follows:

[0092] dθ i =θ i -θ i-1

[0093] dx i =Mx i -Mx i-1

[0094] dy i =My i -My i-1 .

[0095] This embodiment reveals the transition rule of the stroke order by calculating the rotation and translation between adjacent strokes. The construction of the stroke transition feature is based on the geometric transformation theory, which can capture the spatial relationship between strokes. The extraction of the transition feature helps the model to understand the connection mode between strokes, which plays a key role in related applications of online handwriting.

[0096] Finally, the above constructed stroke sequence feature, stroke angle feature, stroke type feature, stroke length feature, stroke position feature, and stroke transition feature are combined together as a stroke order feature S, which can represent what type of stroke can be written to the next stroke through transformation from when it starts, and exhibit the relevant information of strokes and stroke orders, which helps the model to focus on the spatial structure of strokes and the writing sequence of strokes, and improves the performance of the deep learning model.

[0097] The final stroke order feature S of the entire handwriting is as follows, where the stroke order feature S of the i-th stroke i Each S is as follows i There are k = 9 of the above features, and n strokes are combined together to form the stroke order feature S of the entire handwriting.

[0098]

[0099]

[0100] It can be seen that the stroke order feature S is a unified feature vector that can comprehensively represent the writing dynamic process and spatial structure by fusing the multi-dimensional features of stroke sequence, angle, category, length, position, and spatial transformation,

[0101] Step 3, stroke order feature mapping:

[0102] After obtaining the above stroke order feature S, it needs to be further processed, because the stroke order feature is more a display representation of geometric transformation, contains more high-frequency components, and the feature values are severely discretized, and the interaction between features is poor. When the original stroke order feature S is used as the input of the neural network, it is difficult for the network to effectively learn and utilize the feature. Therefore, a high-frequency oscillation basis function (especially Fourier transform) is used to map the frequency space of the stroke order feature to extract its high-frequency spectral components, so as to convert the feature into a form that is easier for the neural network to learn.

[0103] Step 3 is to process the stroke order feature S through frequency mapping transformation. The frequency mapping transformation uses a high-frequency oscillation basis function to extract the high-frequency spectral components in the stroke order feature S, and converts the stroke order feature S into a representation form suitable for a deep learning model.

[0104] The frequency mapping transformation includes: using a randomly initialized projection matrix B to linearly project the stroke order feature S to map the feature to a high-dimensional frequency space; and using a trigonometric function to nonlinearly transform the projected feature to obtain its projection intensity at different frequencies, i.e., the high-frequency spectral components.

[0105] Preferably, in the stroke order feature mapping, the projection matrix B is a Gaussian random matrix, and a variance δ of the Gaussian random matrix is an adjustable hyperparameter; a number of rows m of the Gaussian random matrix is an adjustable hyperparameter much larger than a dimension k of the stroke order feature S.

[0106] In one embodiment, the following is specifically implemented:

[0107] The embodiment forces the network to learn high-frequency components by introducing high-frequency oscillation basis functions, so that the network can better learn the stroke order features. The implementation is as follows:

[0108] First, a Gaussian random matrix B is used to obtain high-frequency features for the stroke order feature S where m represents a number of sampling frequencies, each row corresponds to a frequency sampling point, and m frequency bands can be covered. m is a hyperparameter, and the value is much larger than the number k of stroke order features of a single stroke. Generally, the value is in the range of 128-1024, and the recommended value is 512. B is randomly initialized using a Gaussian distribution with a mean of 0 and a variance of δ. Generally, a larger δ value highlights the details, and a value of 10 is generally sufficient. The random matrix B can also be learned, and the parameter can be optimized through back propagation. The projection intensity at a specific frequency is obtained by using a trigonometric function for nonlinear transformation after B is used for discrete random frequency sampling of S, that is, a frequency spectrum component. Because cos(2πF), sin(2πF) are real parts and imaginary parts of a complex exponential basis function e j2πF , they correspond to frequency domain orthogonal bases. The essence is a Fourier transform. The stroke order feature is more effectively used as an input of a deep learning network through the transform, and the processing capability of the stroke order feature by the representation model is improved. Finally, the transformed features are combined to obtain the final mapped stroke order feature S map .

[0109]

[0110]

[0111]

[0112] The m value (number of frequency bands) and the δ value (variance of the Gaussian distribution) mentioned above in the stroke order feature mapping are hyperparameters. These parameters are adjusted through test results of the trained model. That is, after the experiment, the value is used. The recommended value is the best value given after the experiment. Generally, the size of the data set has little to do with the value. The value is mainly adjusted through the test result. The general rule for m is 2 raised to the power of n. The general rule for δ is linear signs such as 1, 2, 4, 6, 8, 10, and 12.

[0113] The embodiment relates to adjustment strategies of m values and delta values in stroke feature mapping, and how the parameter adjustment influences the effect of the final feature representation. By comparing the recognition rate and stability of the model under different m values and delta values through experiments, the optimal parameter configuration can be determined. For example, when the m value is set as 512 and the delta value is set as 10, the model shows the highest accuracy in processing handwriting identification tasks; and when the m value is adjusted to 1024 and the delta value is adjusted to 8, the performance of the model in the text line recognition task is best. The parameter adjustment strategy ensures that the model can adapt to the requirements of different tasks, and through experimental verification, can provide more delicate and rich feature representation for the model, and significantly improves the application effect of the model in actual scenes.

[0114] In a further embodiment, an online handwriting stroke feature construction system for implementing the method of the above embodiments is provided, comprising:

[0115] A stroke segmentation module: used for converting handwriting into simple strokes by analyzing handwriting collection data (x, y, p, t, s) and using points with a speed v close to 0 as segmentation points.

[0116] A stroke feature construction module: used for constructing stroke features S containing stroke sequence features, stroke angle features, stroke type features, stroke length features, stroke position features and stroke transition features, and comprehensively reflecting the attributes of strokes and stroke information.

[0117] A stroke feature mapping module: used for processing the stroke features S through frequency mapping transformation, the frequency mapping transformation uses high-frequency oscillation basis functions to extract high-frequency spectral components in the stroke features S, and converts the stroke features S into a representation form suitable for a deep learning network.

[0118] Based on the stroke features constructed and converted according to the above method, the stroke features are applied to various task scenarios such as handwriting identification, handwritten text line recognition and handwritten text line Chinese character segmentation, and the model performance can be improved.

[0119] These tasks all need a backbone network to be completed, which can be the same network structure or can not be the same, and the backbone network is selected through experimental results.

[0120] Main network selection: the above-mentioned merged features can be directly used for any sequence-based deep learning network structure, such as 1D resnet, transformer, ecapa_tdnn, etc. Each task can be processed by the above-mentioned deep learning network, which is generally related to the size of the data set. Small data sets are recommended to use resnet, and large data sets are recommended to use transformer, ecapa_tdnn, and the experimental results are the final selection standard. According to the current experiment, ecapa_tdnn is the best, followed by transformer, and resnet is the worst.

[0121] The following are several specific application scenarios including handwriting identification, handwritten text line recognition, and handwritten text line segmentation.

[0122] Application 1: handwriting identification method based on stroke order features, the implementation method is as shown in Figure 2

[0123] First, the point features and stroke order features of the writing handwriting are obtained, and the stroke order features can be obtained by using the method shown in the above embodiment.

[0124] Next, the stroke order features are expanded to align the number of point-level point features.

[0125] Further, the expanded stroke order features and the point features are spliced and fused to generate merged features.

[0126] Then, the merged features are input into the main network for deep feature extraction to obtain the representation vector of the handwriting.

[0127] Finally, the similarity between the representation vectors of the two handwritings is calculated, and handwriting identification is performed according to the similarity.

[0128] The following is further described in detail:

[0129] For handwriting identification tasks, the original point features are generally retained. Since the dimensions of the stroke order features and the point features do not match, the neural network cannot directly learn the two different granularity features jointly. Therefore, by expanding the stroke level features to all corresponding points, the number of point features is aligned, and then spliced and fused to form a merged feature containing point and stroke information for network learning.

[0130] Specifically, compared with the point features, one stroke order feature corresponds to multiple point features, and the stroke order feature is copied multiple times to generate multiple copies with the same number of points in the point features, that is, all points of the ith stroke are assigned to the corresponding stroke order feature such as the point features of the ith stroke​ where f is the number of points of the stroke, c is the number of features of each point, Smap i By extending, i.e. copying f times, the same number of features as the number of points, i.e.

[0131] Then merge, i.e. concatenate, the multiple copies with the point features along the feature dimension. Specifically, the extended stroke features and the point features are concatenated to obtain the merged features

[0132]

[0133] The above-mentioned merging of the extended stroke features and the point features to obtain the merged features is a simple concatenation. For example, a stroke has f = 3 points, each point has c = 2 features, i.e. P i is

[0134] [[1, 2, 3], [4, 5, 6]] is the point feature of the stroke, and Smap i has features [10, 20, 30, 40], and suppose m is 2, then the extended Smap ' i is [[10, 10, 10], [20, 20, 20], [30, 30, 30], [40, 40, 40]], and the final merged features after concatenation are Mi

[0135] [[1, 2, 3], [4, 5, 6], [10, 10, 10], [20, 20, 20], [30, 30, 30], [40, 40, 40]].

[0136] Then the merged features are input into the backbone network to obtain the fused deep features, and then the length dimension is eliminated by a corresponding pooling operation such as Selective Pooling or MaxPooling to obtain the final representation vector. The probability of belonging to a positive sample is represented by calculating the cosine similarity of the two representation vectors. The binary cross-entropy function is used to supervise the cosine similarity during training, and the weights of the network are optimized using backpropagation. During testing, if the cosine similarity is greater than a given threshold, it is judged to be a positive sample, otherwise it is a negative sample.

[0137] In the handwriting identification model, the positive and negative samples are written by oneself, and the cosine similarity output by the model is greater than a threshold to determine whether it is positive or negative. The merged features are input into the backbone network during training.

[0138] Application 2: Handwritten text line recognition

[0139] The implementation method is as follows Figure 3As shown, after inputting the stroke order feature into the backbone network, the representation vector of the stroke is obtained, and finally the writing content is learned through the ctc loss, and the ctc decoding is used for prediction.

[0140] Application 3: Handwritten text line Chinese character segmentation

[0141] The implementation method is as shown in the figure Figure 4 As shown, after inputting the stroke order feature into the backbone network, the representation vector of the stroke is obtained, and then the representation vector is output through the classification layer to output the probability that the stroke is the first stroke of the Chinese character, and if the stroke is greater than a specified threshold during prediction, it is the first stroke of the Chinese character, that is, the Chinese character segmentation is realized.

[0142] In summary, the present application proposes an innovative online handwriting stroke order feature construction method, first, by analyzing handwriting collection data, using the point with speed v close to 0 to divide the strokes, and converting the original handwriting into several strokes. Then, construct stroke order features for each stroke, including stroke sequence, angle, type, length, position and conversion features, which comprehensively reflect the stroke attribute and stroke order information. Then, use the Gaussian random matrix B to perform frequency dimensionality reduction and nonlinear transformation on the stroke order feature, and convert it into an input convenient for deep learning model processing.

[0143] The present application also proposes to use stroke order features for handwriting identification, handwritten text line recognition and handwritten text line segmentation and other tasks, by adjusting the m value and the delta value, optimizing the model performance, realizing the efficient use and accurate identification of the stroke order feature.

[0144] Through the detailed step design of the present application, the fine processing and feature extraction of online handwriting data are realized, and the accuracy of handwriting analysis and recognition is significantly improved. This method not only can capture the unique pen of the writer, but also can effectively reflect the conversion relationship between strokes, through feature mapping, enhance the robustness of the feature and the generalization ability of the model. In the whole process, the stroke order feature construction and mapping are the core links, which ensure that the deep learning model can effectively understand and use the stroke order information, so as to show better recognition and processing ability in the field of handwritten text. This method solves the technical problems of low efficiency and insufficient accuracy in the field of online handwriting for a long time, and in multiple task scenarios, including handwriting identification, handwritten text line recognition and Chinese character segmentation, this method shows excellent performance and broad application prospect, has high practical value and innovation.

[0145] It should be apparent to those skilled in the art that the modules or steps of the application described above can be implemented with general computing devices, which can be centralized on a single computing device or distributed on a network of multiple computing devices, which can be implemented with program codes executable by the computing devices, so that they can be stored in storage devices and executed by the computing devices, and in some cases, the steps shown or described can be executed in different orders than shown, or made into individual integrated circuit modules, or made into a single integrated circuit module. Thus, the application is not limited to any particular combination of hardware and software.

[0146] The preferred embodiments of the application described above are intended to be merely exemplary and those skilled in the art will recognize that many changes and modifications can be made to the application without departing from the spirit and scope of the application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the application should be included in the protection scope of the application.

Claims

1. An online handwriting stroke order feature construction method, characterized in that, The method comprises the following steps: Step 1, stroke segmentation: by analyzing handwriting collection data (x, y, p, t, s), using points with a speed v close to 0 as segmentation points, the handwriting is segmented into several strokes; Step 2, stroke feature construction: construct stroke features S including stroke sequence features, stroke angle features, stroke type features, stroke length features, stroke position features and stroke conversion features, comprehensively reflecting the attributes and stroke information of the strokes; Step 3, stroke feature mapping: the stroke features S are processed through frequency mapping transformation, the frequency mapping transformation uses high-frequency oscillation basis functions to extract high-frequency spectral components in the stroke features S, and the stroke features S are converted into a representation form suitable for a deep learning model.

2. The online handwriting stroke order feature construction method according to claim 1, characterized in that, The stroke sequence feature construction in step 2, by converting the absolute writing order of strokes into a normalized ratio i / n relative to the total number of strokes, thereby generating a sequence of values between 0 and 1 seq i , to characterize the stroke order sequence feature, where i represents the i-th stroke and n represents the maximum number of strokes. 3.The online handwriting stroke order feature construction method of claim 1, wherein, In the step 2, the stroke angle feature construction is to calculate the inverse tangent value of the angle between the stroke vector and the X axis to quantitatively represent the stroke angle feature θ of the stroke, and the stroke vector is composed of the starting point and the end point of the stroke.

4. The online handwriting stroke order feature construction method according to claim 3, characterized in that, In the step 2, the stroke type feature construction is to discretize the continuous stroke angle into D direction types, D is generally an even number, and the positive and negative signs are used to encode the opposite relationship of the directions, so as to convert the stroke angle feature into a symbolic category value that can clearly distinguish the positive and negative directions.

5. The online handwriting stroke order feature construction method according to claim 1, wherein, In the step 2, the stroke length feature construction is to accumulate the lengths of the line segments formed by the multiple points of the stroke as the length feature L of the stroke.

6. The online handwriting stroke order feature construction method according to claim 1, wherein, In the step 2, the stroke position feature construction selects the horizontal and vertical coordinates Mx and My of the middle point M of each stroke as the position feature to ensure the robustness of the feature.

7. The online handwriting stroke order feature construction method according to claim 1, wherein, In the step 2, the stroke conversion feature construction is to calculate the rotation angle and translation distance between adjacent strokes to quantitatively represent the time sequence relationship of the strokes as spatial geometric transformation parameters.

8. The online handwriting stroke order feature construction method according to any one of claims 1-7, characterized in that, The stroke order feature S is a unified feature vector that can comprehensively represent the writing dynamic process and spatial structure by fusing the multi-dimensional features of stroke order, angle, category, length, position and spatial transformation of strokes. The stroke order feature of the i-th stroke Each S i There are k = 9 features that make up the stroke order feature S that combines n strokes into the entire signature:

9. The online handwriting stroke order feature construction method according to claim 1, wherein, In step 3, the frequency mapping transformation includes: using a randomly initialized projection matrix B to linearly project the stroke features S to map the features to a high-dimensional frequency space; and using a trigonometric function to perform a nonlinear transformation on the projected features to obtain their projection intensities at different frequencies, i.e., the high-frequency spectral components.

10. The online handwriting stroke order feature construction method according to claim 9, wherein, In the stroke feature mapping, the projection matrix B is a Gaussian random matrix, and the variance δ is an adjustable hyperparameter; the number of rows m of the Gaussian random matrix is an adjustable hyperparameter much larger than the dimension k of the stroke features S.

11. An online handwriting stroke order feature construction system, implementing the method of any one of claims 1-10, characterized in that, It comprises: a stroke segmentation module for segmenting handwriting into several strokes by analyzing handwriting collection data (x, y, p, t, s) and using points with a speed v close to 0 as segmentation points; a stroke feature construction module for constructing stroke features S including stroke sequence features, stroke angle features, stroke type features, stroke length features, stroke position features and stroke conversion features, comprehensively reflecting the attributes and stroke information of the strokes; a stroke feature mapping module for processing the stroke features S through frequency mapping transformation, the frequency mapping transformation uses high-frequency oscillation basis functions to extract high-frequency spectral components in the stroke features S, and the stroke features S are converted into a representation form suitable for a deep learning network.

12. A handwriting authentication method based on stroke order features, characterized by, It comprises: obtaining point features and stroke order features of the handwriting, wherein the stroke order features are obtained using the method of claims 1-10; extending the stroke order features to match the number of the point features at the point level; splicing and fusing the extended stroke order features and the point features to generate merged features; inputting the merged features into a backbone network to extract deep features and obtain a representation vector of the handwriting; calculating the similarity between the representation vectors of two handwritings, and performing handwriting identification according to the similarity.

13. The handwriting identification method of claim 12, wherein The extending the stroke-level stroke order feature includes: mapping the stroke-level stroke order feature Smap i copying multiple times to generate a copy Smap i of the stroke-level stroke order feature Smap i The number of the copy Smap is the same as the number of points f contained in the stroke. The splicing and fusing of the extended stroke order feature and the point feature comprises: splicing and fusing the point feature Pi of the i-th stroke and the corresponding stroke order feature copy Smap i The splicing operation is performed along the feature dimension, and a merged feature Mi is obtained. c is the dimension of the point feature, and 2m is the dimension of the mapped stroke order feature.

14. The method of claim 12 or 13, wherein the handwriting identification according to the similarity comprises: in the model training stage, using a binary cross-entropy loss function to supervise the similarity, and optimizing the network parameters through a back propagation algorithm; in the model testing stage, comparing the calculated similarity with a preset threshold value, and if the similarity is greater than the threshold value, determining that the handwriting is a positive sample from the same writer, otherwise, determining that the handwriting is a negative sample.

15. The method of claim 12 or 13, wherein the backbone network selects a sequence-based deep learning network structure, including but not limited to 1D resnet, transformer, ecapa_tdnn.

Images