A method, system, device and medium for online handwriting identification based on multiple samples
By performing multiple sequence feature extraction and feature fusion methods on multiple sample retention sequences and verification sequences, the problem of insufficient identification accuracy caused by single sample retention data in the prior art is solved, and higher handwriting identification accuracy and reliability are achieved.
Patent Information
- Application Number
- CN202411971360.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-04
AI Technical Summary
The prior art relies solely on a single sample data when conducting electronic handwriting identification, resulting in insufficient identification accuracy and reliability, especially in the case of changes and diversity of handwriting styles.
The online handwriting identification method based on multiple samples is adopted. By extracting and fusion of sequence features and fusion of multiple sample sequences and verification sequences, and feature extraction and fusion are repeated multiple times, and finally, classification and judgment is performed based on all fusion features to output the identification results of the verification sequence.
It improves the accuracy and reliability of online handwriting handwriting, can more effectively deal with changes and diversity in handwriting styles, and significantly improves the recognition accuracy.
Smart Images
Figure CN119763134A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of handwriting identification, and specifically to an online handwriting identification method, system, device and medium based on multiple sample retention. Background Art
[0002] With the surge of electronic information technology, signatures on paper are gradually replaced by electronic handwriting. However, this change also makes the authenticity identification of electronic handwriting more and more important. Compared with traditional paper handwriting, electronic handwriting is like a treasure trove of information. It not only contains the horizontal and vertical coordinates of the writing, but also records the timestamp of the writing, the paragraph segmentation of the handwriting, and the pressure changes during handwriting. This rich information records the unique biological characteristics of the writer and can accurately identify the writer behind the handwriting.
[0003] However, in the real writing world, each person's style is like an unpredictable movement, with both a stable main melody and subtle improvisations. If identification is based on a single sample data, the accuracy and reliability of identification will undoubtedly be greatly weakened. Therefore, making full use of multiple sample data has become the key to improving identification capabilities. Summary of the invention
[0004] The present invention provides a method, system, device and medium for online handwriting identification based on multiple retained samples, which are used for identifying online handwriting using multiple retained sample data.
[0005] The technical solution of the present invention is:
[0006] The present application provides an online handwriting identification method based on multiple samples, and the online handwriting identification method comprises:
[0007] After extracting sequence features from the input multiple retained sample sequences and verification sequences, feature fusion is performed on the obtained multiple retained sample sequence features and verification sequence features to obtain a first fusion feature; and
[0008] Repeat the following steps multiple times: extract and fuse the sequence features of the multiple retained sample sequences and the verification sequence features obtained from the previous sequence feature extraction to obtain more fused features;
[0009] Make classification judgments based on all the fused features and output the identification results of the verification sequence.
[0010] Preferably, the step of fusing the obtained multiple retained sample sequence features and verification sequence features to obtain the first fused feature includes:
[0011] Pooling is performed on multiple retained sample sequence feature vectors and verification sequence feature vectors;
[0012] According to the characteristics of multiple retained sample sequences after pooling, the overall difference of retained samples is calculated;
[0013] According to the pooled features of multiple retained sample sequences and verification sequence features, the retained sample verification similarity score is calculated;
[0014] The retained sample sequence features after pooling are screened using the retained sample verification similarity score to obtain the screened retained sample sequence features;
[0015] According to the features of the retained sample sequence after screening and the features of the verification sequence after pooling, the difference features between the retained sample sequence and the verification sequence are obtained;
[0016] The overall difference degree of the retained sample is added to the difference feature between the retained sample sequence and the verification sequence to obtain the overall difference feature between the retained sample sequence and the verification sequence;
[0017] The overall difference features between the retained sample sequence and the verification sequence are fused and normalized to obtain the first fused feature.
[0018] Preferably, the step of calculating the overall difference of the retained samples according to the pooled features of the plurality of retained sample sequences comprises:
[0019] According to the feature vectors of the multiple retained sample sequences after pooling, the difference matrix between the two retained sample sequences is calculated;
[0020] Remove the diagonal elements of the difference matrix between the adjacent sample sequences;
[0021] According to the difference matrix with the diagonal elements removed, the average difference between the current retained sample sequence and other retained sample sequences is determined;
[0022] The average difference between the current retained sample sequence and other retained sample sequences is subjected to feature mapping and nonlinear transformation to obtain the overall difference of the retained samples.
[0023] Preferably, the step of determining the average difference between the current retained sample sequence and other retained sample sequences according to the difference matrix with the diagonal elements removed comprises:
[0024] According to the difference matrix between the adjacent features of the two retained sample sequences after removing the diagonal elements, the average is calculated on the remaining dimensions to obtain the average difference between the current retained sample sequence and other retained sample sequences.
[0025] Preferably, the step of calculating the sample verification similarity score according to the pooled multiple sample sequence features and verification sequence features includes:
[0026] According to the pooled feature vectors of the plurality of retained sample sequences and the pooled feature vectors of the verification sequence, a similarity vector between the features of the plurality of retained sample sequences and the features of the verification sequence is obtained;
[0027] Performing dimension-upgrading, normalization and nonlinear transformation on the similarity vectors between the features of multiple retained sequences and the features of the verification sequences to obtain the similarity vectors after dimension-upgrading;
[0028] The dimension of the similarity vector after dimension increase is reduced to obtain a similarity vector after dimension reduction;
[0029] The similarity vector after dimensionality reduction is mapped to the range of 0-1 to obtain the sample verification similarity score.
[0030] Preferably, the step of screening the pooled retained sample sequence features using the retained sample verification similarity score to obtain the screened retained sample sequence features comprises:
[0031] Use the retained samples to verify the similarity score and calculate the weighted ratio of each retained sample sequence feature;
[0032] Multiply each retained sample sequence feature after pooling by the corresponding weighted ratio to obtain a weighted feature vector;
[0033] For each feature dimension, the weighted feature values of multiple retained sample sequences on the feature dimension are accumulated and summed to obtain the final retained sample feature value;
[0034] Arrange all the retained sample feature values into a vector to obtain the filtered retained sample sequence features.
[0035] The present application also provides an online handwriting identification system based on multiple samples, the online handwriting identification system comprising:
[0036] The sequence feature extraction module extracts sequence features from multiple input sample sequences and verification sequences;
[0037] The multiple retained sample and verification fusion module fuses the features of multiple retained sample sequences and verification sequence features to obtain the first fusion feature;
[0038] The sequence feature extraction module and the multiple retained sample and verification fusion module are repeated multiple times: sequence feature extraction and feature fusion are performed on multiple retained sample sequence features and verification sequences obtained from the previous sequence feature extraction to obtain more fusion features;
[0039] The classification module makes classification judgments based on all the fused features and outputs the identification results of the verification sequence.
[0040] Preferably, the multiple retained samples and verification fusion module includes: a sequence feature pooling submodule, a retained sample difference submodule, and a retained sample verification similarity submodule;
[0041] The sequence feature pooling submodule performs pooling processing on multiple retained sample sequence features and verification sequence features;
[0042] The difference between retained samples submodule calculates the overall difference between retained samples based on the characteristics of multiple retained sample sequences after pooling;
[0043] The retained sample verification similarity submodule calculates the retained sample verification similarity score based on the pooled retained sample sequence features and the verification sequence features; uses the retained sample verification similarity score to screen the pooled retained sample sequence features to obtain the screened retained sample sequence features; obtains the difference features between the retained sample sequence and the verification sequence based on the screened retained sample sequence features and the pooled verification sequence features; adds the overall retained sample difference to the difference features between the retained sample sequence and the verification sequence to obtain the overall difference features between the retained sample sequence and the verification sequence; performs feature fusion and normalization on the overall difference features between the retained sample sequence and the verification sequence to obtain the first fused feature.
[0044] The present application also provides a computer-readable storage medium on which a computer program is stored. The program can be loaded and run by a processor to execute the above-mentioned online handwriting identification method based on multiple samples.
[0045] The present application also provides an electronic device, which includes: one or more processors; a memory; and one or more application programs, which are stored in the memory and configured to be loaded and run by the one or more processors to execute the above-mentioned online handwriting identification method based on multiple samples.
[0046] The beneficial effects of the present invention are:
[0047] By utilizing multiple handwritten sample sequences in different scripts at different times and in different situations, handwriting identification is performed on the verification sequence, thereby improving the accuracy and reliability of online handwriting identification. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 It is a flowchart of an online handwriting identification method based on multiple samples in an embodiment of the present application;
[0049] Figure 2 This is a schematic diagram of the sample retention used in the embodiments of the present application;
[0050] Figure 3 It is a flowchart of an online handwriting identification method based on multiple samples in an embodiment of the present application;
[0051] Figure 4 This is a flow chart of step S1 in the embodiment of the present application;
[0052] Figure 5 This is a flowchart of step S1 in the embodiment of the present application;
[0053] Figure 6 This is a flow chart of step S13 in the embodiment of the present application;
[0054] Figure 7 This is a flowchart of step S13 in the embodiment of the present application;
[0055] Figure 8 This is a flow chart of step S14 in the embodiment of the present application;
[0056] Fig. 9 This is a flowchart of step S14 in the embodiment of the present application;
[0057] Fig.10 This is a verification diagram used in the embodiments of the present application. DETAILED DESCRIPTION
[0058] The exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the appended claims. The flowcharts shown in the accompanying drawings are merely exemplary and do not necessarily include all contents and operations / steps, nor do they necessarily have to be performed in the order described. For example, some operations / steps may be further decomposed, while some operations / steps may be combined or partially combined, so the actual order of execution may change according to actual conditions.
[0059] Handwriting, as a unique biological feature, is deeply rooted in a person's writing habits. It is not only a subtle reflection of personal identity, but also contains the writer's unique style. However, it is worth noting that even for the same person, there will be subtle differences in the handwriting left each time they write. These differences may be due to changes in emotions, strength or speed when writing. In addition, handwriting will also present different styles due to the different styles of calligraphy used (such as regular script, running script, cursive script, etc.). Since handwriting is not rigid and unchanging, but will adjust with changes in time, situation and even the writer's personal state, this dynamism makes it particularly difficult to identify the style simply by relying on the handwriting at a certain moment or in a certain style.
[0060] In order to overcome this challenge, the inventors designed an advanced multi-sample model to make full use of multiple sample data to capture the stable features and change trajectories of handwriting. This multi-sample model can integrate multiple handwriting samples of users at different times, in different scripts, and in different situations. Through complex algorithms and deep learning technology, it can deeply explore the common information and individual differences in handwriting, thereby achieving accurate identification and analysis of handwriting styles, and providing strong technical support for handwriting identification, identity verification and other fields.
[0061] Reference Figure 1 The embodiment of the present application provides an online handwriting identification method based on multiple samples, comprising:
[0062] S1, after extracting sequence features from the input multiple retained sample sequences and verification sequences, perform feature fusion on the obtained multiple retained sample sequence features and verification sequence features to obtain a first fusion feature; and
[0063] S2, repeating the following steps multiple times: performing sequence feature extraction and feature fusion on multiple retained sample sequence features and verification sequence features obtained from the previous sequence feature extraction to obtain more fusion features;
[0064] S3, make classification judgments based on all the fused features and output the identification results of the verification sequence.
[0065] In the embodiment of the present application, the input multiple sample sequences (replaced by m in this embodiment) are from multiple electronic handwriting samples registered in advance by the user at different times, different scripts and different situations. These data are all the user's own style; for example: Figure 2 In the example, samples 1 and 2 are written in a more regular script, samples 3 and 4 are written in a more cursive script, and sample 5 omits more strokes. The input verification sequence is a sample that needs to be identified. If it is the handwriting of the person himself, it is a positive sample, otherwise it is a negative sample.
[0066] Among them, m sample sequences and verification sequences will be input into Figure 3 In the multi-retention sample and verification fusion feature extraction network, the multi-retention sample and verification fusion feature extraction network finally outputs multiple fusion features (represented by N fusion features in this embodiment).
[0067] Combining step S1 and step S2, it can be seen that the multi-sample and verification fusion feature extraction network mainly needs to perform two main steps: sequence feature extraction and feature fusion.
[0068] The sequence data is further extracted with richer deep features through some sequence networks. This step can make the original features richer and the information more fully utilized. The sequence networks used in this application include but are not limited to transformer, LSTM (Long Short-Term Memory), GRU (Gate Recurrent Unit), etc. The sequence features are obtained through the layers proposed in these sequence networks, and the retained sample sequence and the verification sequence are extracted through the same structure, that is, the network weights are shared, so that the network does not need to model the retained sample sequence and the verification sequence separately, reducing the difficulty of model fitting, and at the same time speeding up the training and reasoning speed of the model, allowing the network to focus more on the features themselves, rather than judging whether it is retained sample data or verification data.
[0069] Among them, in step S1, the feature vector of the sample sequence obtained by extracting the sequence features of m sample sequences is expressed as: R∈R m×L×E ; The i-th retained sample sequence feature is represented by R i ∈R L×E Indicates that L represents the length of the i-th retained sample sequence, and E is the dimension of the feature. The verification sequence feature vector obtained by extracting the sequence features is expressed as: V∈R L×E .
[0070] Reference Figure 4 and Figure 5 In the embodiment of the present application, the step of fusing the obtained multiple retained sample sequence features and verification sequence features to obtain the first fused feature includes:
[0071] S11, pooling the features of multiple retained sample sequences and verification sequence features;
[0072] S12, calculating the overall difference of the retained samples according to the characteristics of the multiple retained sample sequences after pooling;
[0073] S13, calculating the sample verification similarity score according to the pooled multiple sample sequence features and verification sequence features;
[0074] S14, using the retained sample verification similarity score to screen the pooled retained sample sequence features to obtain the screened retained sample sequence features;
[0075] S15, obtaining the difference features between the retained sample sequence and the verification sequence according to the features of the retained sample sequence after screening and the features of the verification sequence after pooling;
[0076] S16, adding the overall difference degree of the retained sample to the difference feature between the retained sample sequence and the verification sequence to obtain the overall difference feature between the retained sample sequence and the verification sequence;
[0077] S17, perform feature fusion and normalization processing on the overall difference features between the retained sample sequence and the verification sequence to obtain the first fused feature.
[0078] In the aforementioned step S11, the length dimension is reduced by the pooling layer, and only the feature dimension is retained. Pooling methods include but are not limited to average pooling, maximum pooling, statistical pooling, and selective pooling (SP). In this embodiment, SP is used as the final pooling strategy in the specific implementation. After pooling the feature vector of the retained sample sequence, the pooled feature vector of the retained sample sequence R′∈R m×E ; The feature of the i-th retained sample sequence after pooling is represented by R′ i ∈R EIndicates that its length dimension is eliminated and only the feature dimension is retained. After pooling the verification sequence feature vector, the pooled verification sequence feature vector V′∈R is obtained E .
[0079] Step S12 focuses on accurately measuring the degree of difference between samples in the retained sample set. The core purpose is to detect whether there is significant variability in the retained sample sequence in terms of style, attributes or content. Step S12 calculates a global retained sample overall difference value, which can generally reflect the difference and consistency level within the retained sample set. Figure 5 , the aforementioned step S12 specifically includes:
[0080] S121, based on the feature vectors of the multiple retained sample sequences after pooling, calculate the difference matrix D∈R between the two retained sample sequences m×m (correspond Figure 5 The difference matrix between the retained samples in the sample); the difference matrix calculation method includes but is not limited to Euclidean distance, cosine distance, Manhattan distance, Chebyshev distance, etc. In this embodiment, the cosine distance is selected to calculate the difference matrix between the two retained sample sequences.
[0081] S122, remove the diagonal elements of the difference matrix between the two retained sample sequences (corresponding to Figure 5 Remove the diagonal in the difference matrix. Since the diagonal elements of the difference matrix represent the difference between the retained sample and itself, they need to be removed; the difference matrix after removing the diagonal is expressed as D′∈R m×(m-1) .
[0082] S123, according to the difference matrix excluding the diagonal elements, determine the average difference between the current sample sequence and other sample sequences (corresponding to Figure 5 The difference matrix D′∈R after removing the diagonal line is obtained. m×(m-1) , calculate the mean value on the m-1 dimension (i.e. the remaining dimension) to obtain the average difference D between the current retained sample sequence i and other retained sample sequences mean ∈R m S124, feature mapping and nonlinear transformation are performed on the average difference between the current sample sequence and other sample sequences to obtain the overall difference of the sample. In order to comprehensively evaluate the overall difference of the sample, the calculated average difference D mean ∈R mThe input is sent to the fully connected layer 1, which outputs a value to represent the overall difference of the retained samples. This operation is called feature mapping. Since the overall difference of the retained samples must be non-negative, the output of the fully connected layer 1 is constrained by the linear rectifier function (ReLU, Rectified Linear Unit), and the value less than 0 is set to 0; this operation is called nonlinear transformation. Finally, the output value D after the nonlinear transformation is all ∈R 1 It will be used as the overall difference of the retained samples.
[0083] The core function of step S13 is to compare and analyze the similarity between the verification sequence and multiple sample sequences, aiming to screen out the sample sequences that are valuable for the handwriting identification task. By applying a series of similarity measurement techniques (such as cosine similarity, Euclidean distance, etc.), the similarity score between each sample sequence and the verification sequence can be accurately calculated. These scores are crucial for the subsequent feature screening and fusion process, and help to identify the sample sequence that is closest to the verification sequence. Figure 6 and Figure 7 , step S13 specifically includes:
[0084] S131, according to the pooled multiple sample sequence feature vectors and the pooled verification sequence feature vectors, obtain the similarity vectors between the multiple sample sequence features and the verification sequence features (corresponding to Figure 7 In this embodiment, the cosine similarity method is used to calculate the similarity vector S∈R between multiple sample sequences and the verification sequence. m .
[0085] S132, perform dimension increase, normalization and nonlinear transformation on the similarity vectors between the features of the multiple retained sequences and the features of the verification sequences to obtain the similarity vector after dimension increase. m After that, the dimension is further increased through the fully connected layer 2 to obtain the increased dimension similarity vector S with dimension k (k is 128 in the present invention) 2 ∈R k . Next, the batch normalization layer is used to normalize the output of the fully connected layer 2 to prevent the output value from being too large and causing the training process to be difficult to converge. Then, a nonlinear transformation is introduced through a linear activation layer. The activation function includes but is not limited to ReLU, ELU (Exponential Linear Units) and GELU (Gaussian Error Linear Unit), etc. GELU is specifically used as the activation function in the present invention.
[0086] S133, reduce the dimension of the similarity vector after dimensionality increase to obtain the similarity vector after dimensionality reduction. Since the number of vectors after dimensionality increase changes, it is necessary to use the fully connected layer 3 to reduce the dimension of the vector to obtain the similarity vector S after dimensionality reduction. 3 ∈R m .
[0087] S134, the similarity vector after dimensionality reduction is mapped to the range of 0-1, and the sample verification similarity score (corresponding to Figure 7 The score is mapped to the range of 0-1 by the sigmoid function to represent the similarity score S′∈R between the retained sample sequence and the validation sequence. m .
[0088] Step S14 aims to integrate the output information of steps S12 and S13, namely, the overall difference of the retained samples and the retained sample verification similarity score, to guide the effective selection and fusion of the features of the m retained sample sequences. Step S14 makes full use of the retained sample verification similarity score S′∈R m For the retained sample sequence feature R′∈R after pooling m×E Specifically, the features of the retained sample sequences with high similarity to the verification sequence will be retained and appropriately weighted; while the features of the retained sample sequences with low similarity will be suppressed. This strategy helps to deal with multi-style problems and improve the accuracy and robustness of the selection of retained sample sequence features. Figure 8 and Fig. 9 , step S14 specifically includes:
[0089] S141, using the retained sample verification similarity score, calculate the weighted ratio of each retained sample sequence feature. The weighted ratio ST of each retained sample sequence feature is calculated by the formula ST = softmax(S′) i ∈R 1 , the weighted ratio reflects the similarity relationship between the retained sequence and the verification sequence.
[0090] S142, multiply each retained sample sequence feature after pooling by the corresponding weighted ratio to obtain a weighted feature vector. Using the formula RS i =ST i *R′ i , where i∈[1,m] multiplies the original features of each retained sample sequence by the corresponding weighted ratio to obtain the weighted feature vector RS i ∈R E .
[0091] S143, for each feature dimension, the weighted feature values of multiple retained sample sequences on the feature dimension are accumulated and summed to obtain the final retained sample feature value. For each feature dimension j, the weighted feature values of m retained sample sequences on the dimension are accumulated and summed to obtain the final retained sample feature value Q j ∈R 1 , specifically through where j∈[1,E] is implemented.
[0092] S144, arrange all the retained sample feature values into a vector, that is, obtain the filtered retained sample sequence feature Q∈R E , where Q = [Q 1 Q 2 ...Q E ].
[0093] Step S15 is to filter the remaining sample sequence features Q∈R E And the pooled verification sequence feature V′∈R E This is done by subtracting corresponding elements and taking the absolute value.
[0094] In order to more comprehensively reflect the difference between the retained sample and the verification sequence, the overall difference degree D of the retained sample is calculated in step S16. all ∈R 1 Add to the difference features between the retained sample sequence and the verification sequence to obtain the overall difference feature QVD∈R E .
[0095] In order to further fuse the difference features and extract deeper information, the fully connected layer 4 is used for learning in step S17. The fully connected layer 4 can capture the complex relationship between the features and output the fused feature vector. Finally, the feature vector is normalized by the normalization layer to obtain the final fused feature, thereby obtaining the first fused feature.
[0096] For the above step S2, the same technical means as the above step S1 are used to obtain more fusion features. In this embodiment, the multiple fusion features finally obtained are used as F n ∈R E Represents, where n represents the nth fused feature obtained after the nth repetition. The fused feature vector will accurately represent the overall difference between the retained sequence and the verification sequence, providing strong support for the subsequent handwriting identification task.
[0097] After N repetitions, N fusion features F will be obtained. n ∈R E , by merging on the feature dimension to obtain the merged feature F∈R NE, and then the final discrimination result is output through the classification layer, that is, if it is greater than the threshold, it is a positive sample, and if it is less than the threshold, it is a negative sample. Through N repeated processes, N different levels of fusion features F can be gradually extracted from the original data. n ∈R E Each repetition will further extract more abstract and generalized deep features based on the previous sequence feature extraction results. After obtaining these N fusion features, by merging them in the feature dimension, the merged fusion feature F∈R NE , integrating these features from different levels into a unified feature space. The final fused feature vector contains both detailed features and global features, thus improving the expressiveness and generalization ability of the model.
[0098] The merged fusion feature vector F∈R NE It will be processed through a classification layer. The classification layer usually includes a fully connected layer, an activation function, and a regularization layer, etc., which is used to fusion feature vector F∈R NE Further transformations and combinations are performed to extract the most useful information for the classification task. Ultimately, the classification layer outputs a continuous discrimination value, which reflects the probability or score of the input data belonging to different categories. In order to obtain the final discrimination result, a threshold is set. If the discrimination value output by the classification layer is greater than this threshold, the verification data is considered to be a positive sample, otherwise it is a negative sample. The choice of this threshold is crucial to the performance of the model, and it usually needs to be adjusted and optimized according to the actual application scenario and data. Finally, the classification loss is calculated through the binary cross entropy function, and the weights of the network are optimized through back propagation.
[0099] The present invention adopts a multi-sample input strategy, which can more effectively deal with multi-style verification problems. Fig.10 As shown, for the handwriting in the left-hand side Kai style, the verification process will select Figure 2 The remaining samples 1 and 2 in the above are identified, while the influence of other remaining samples is suppressed, and the positive sample result is finally output; for the handwriting with cursive style on the right, Figure 2 The sample 3 and sample 4 in the handwriting are accurately identified. If only a single handwriting is used as the sample data, the identification effect will be greatly reduced and it will be difficult to pass the verification. Therefore, the present invention introduces a multi-sample mechanism to achieve the fusion of multiple handwriting styles, which can more naturally adapt to users with changing styles and significantly improve the recognition accuracy.
[0100] The embodiment of the present application also provides an online handwriting identification system based on multiple samples, and the online handwriting identification system includes:
[0101] The sequence feature extraction module extracts sequence features from multiple input sample sequences and verification sequences;
[0102] The multiple retained sample and verification fusion module fuses the features of multiple retained sample sequences and verification sequence features to obtain the first fusion feature;
[0103] The sequence feature extraction module and the multiple retained sample and verification fusion module are repeated multiple times: sequence feature extraction and feature fusion are performed on multiple retained sample sequence features and verification sequence features obtained from the previous sequence feature extraction to obtain more fusion features;
[0104] The classification module makes classification judgments based on all the fused features and outputs the identification results of the verification sequence.
[0105] The multi-retention sample and verification fusion module includes: sequence feature pooling submodule, inter-retention sample difference submodule, and retention sample verification similarity submodule;
[0106] The sequence feature pooling submodule performs pooling processing on multiple retained sample sequence features and verification sequence features;
[0107] The difference between retained samples submodule calculates the overall difference between retained samples based on the characteristics of multiple retained sample sequences after pooling;
[0108] The retained sample verification similarity submodule calculates the retained sample verification similarity score based on the pooled retained sample sequence features and the verification sequence features; uses the retained sample verification similarity score to screen the pooled retained sample sequence features to obtain the screened retained sample sequence features; obtains the difference features between the retained sample sequence and the verification sequence based on the screened retained sample sequence features and the pooled verification sequence features; adds the overall retained sample difference to the difference features between the retained sample sequence and the verification sequence to obtain the overall difference features between the retained sample sequence and the verification sequence; performs feature fusion and normalization on the overall difference features between the retained sample sequence and the verification sequence to obtain the first fused feature.
[0109] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0110] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0111] It should also be noted that, in this article, the orientation or position relationship indicated by the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or component referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, relational terms such as "first" and "second" are used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations, nor can they be understood as indicating or implying relative importance. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements does not include those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0112] The technical solution provided by the present invention is described in detail above. The principle and implementation mode of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the present invention, and the content of this specification should not be understood as limiting the present invention. At the same time, for those skilled in the art, according to the present invention, there will be different forms of changes in the specific implementation mode and application scope. It is not necessary and impossible to list all the implementation modes here, and the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.
Claims
1. An online handwriting identification method based on multiple samples, characterized in that: The online handwriting identification method comprises: After extracting sequence features from the input multiple retained sample sequences and verification sequences, feature fusion is performed on the obtained multiple retained sample sequence features and verification sequence features to obtain a first fusion feature; and Repeat the following steps multiple times: extract and fuse the sequence features of the multiple retained sample sequences and the verification sequence features obtained from the previous sequence feature extraction to obtain more fused features; Make classification judgments based on all the fused features and output the identification results of the verification sequence.
2. The online handwriting identification method based on multiple samples according to claim 1 is characterized in that: The steps of fusing the obtained multiple retained sample sequence features and verification sequence features to obtain the first fused feature include: Pooling is performed on multiple retained sample sequence feature vectors and verification sequence feature vectors; According to the characteristics of multiple retained sample sequences after pooling, the overall difference of retained samples is calculated; According to the pooled features of multiple retained sample sequences and verification sequence features, the retained sample verification similarity score is calculated; The retained sample sequence features after pooling are screened using the retained sample verification similarity score to obtain the screened retained sample sequence features; According to the features of the retained sample sequence after screening and the features of the verification sequence after pooling, the difference features between the retained sample sequence and the verification sequence are obtained; The overall difference degree of the retained sample is added to the difference feature between the retained sample sequence and the verification sequence to obtain the overall difference feature between the retained sample sequence and the verification sequence; The overall difference features between the retained sample sequence and the verification sequence are fused and normalized to obtain the first fused feature.
3. The online handwriting identification method based on multiple samples according to claim 2 is characterized in that: According to the characteristics of multiple retained sample sequences after pooling, the steps of calculating the overall difference of the retained samples include: According to the feature vectors of the multiple retained sample sequences after pooling, the difference matrix between the two retained sample sequences is calculated; Remove the diagonal elements of the difference matrix between the adjacent sample sequences; According to the difference matrix with the diagonal elements removed, the average difference between the current retained sample sequence and other retained sample sequences is determined; The average difference between the current retained sample sequence and other retained sample sequences is subjected to feature mapping and nonlinear transformation to obtain the overall difference of the retained samples.
4. The online handwriting identification method based on multiple samples according to claim 3 is characterized in that: The step of determining the average difference between the current retained sample sequence and other retained sample sequences according to the difference matrix with the diagonal elements removed includes: According to the difference matrix between the adjacent features of the two retained sample sequences after removing the diagonal elements, the average is calculated on the remaining dimensions to obtain the average difference between the current retained sample sequence and other retained sample sequences.
5. The online handwriting identification method based on multiple samples according to claim 2 is characterized in that: According to the pooled features of multiple retained sample sequences and verification sequence features, the steps of calculating the retained sample verification similarity score include: According to the pooled feature vectors of the plurality of retained sample sequences and the pooled feature vectors of the verification sequence, a similarity vector between the features of the plurality of retained sample sequences and the features of the verification sequence is obtained; Performing dimension-upgrading, normalization and nonlinear transformation on the similarity vectors between the features of multiple retained sequences and the features of the verification sequences to obtain the similarity vectors after dimension-upgrading; The dimension of the similarity vector after dimension increase is reduced to obtain a similarity vector after dimension reduction; The similarity vector after dimensionality reduction is mapped to the range of 0-1 to obtain the sample verification similarity score.
6. The online handwriting identification method based on multiple samples according to claim 2 is characterized in that: The steps of screening the sequence features of the pooled retained samples using the retained sample verification similarity score to obtain the screened sequence features of the retained samples include: Use the retained samples to verify the similarity score and calculate the weighted ratio of each retained sample sequence feature; Multiply each retained sample sequence feature after pooling by the corresponding weighted ratio to obtain a weighted feature vector; For each feature dimension, the weighted feature values of multiple retained sample sequences on the feature dimension are accumulated and summed to obtain the final retained sample feature value; Arrange all the retained sample feature values into a vector to obtain the filtered retained sample sequence features.
7. An online handwriting identification system based on multiple samples, characterized in that: The online handwriting identification system comprises: The sequence feature extraction module extracts sequence features from multiple retained sample sequences and verification sequences; The multiple retained sample and verification fusion module fuses the features of multiple retained sample sequences and verification sequence features to obtain the first fusion feature; The sequence feature extraction module and the multiple retained sample and verification fusion module are repeated multiple times: sequence feature extraction and feature fusion are performed on multiple retained sample sequence features and verification sequence features obtained from the previous sequence feature extraction to obtain more fusion features; The classification module makes classification judgments based on all the fused features and outputs the identification results of the verification sequence.
8. The online handwriting identification system based on multiple samples according to claim 7 is characterized in that: The multi-retention sample and verification fusion module includes: sequence feature pooling submodule, inter-retention sample difference submodule, and retention sample verification similarity submodule; The sequence feature pooling submodule performs pooling processing on multiple retained sample sequence features and verification sequence features; The difference between retained samples submodule calculates the overall difference between retained samples based on the characteristics of multiple retained sample sequences after pooling; The retained sample verification similarity submodule calculates the retained sample verification similarity score based on the pooled retained sample sequence features and the verification sequence features; uses the retained sample verification similarity score to screen the pooled retained sample sequence features to obtain the screened retained sample sequence features; obtains the difference features between the retained sample sequence and the verification sequence based on the screened retained sample sequence features and the pooled verification sequence features; adds the overall retained sample difference to the difference features between the retained sample sequence and the verification sequence to obtain the overall difference features between the retained sample sequence and the verification sequence; performs feature fusion and normalization on the overall difference features between the retained sample sequence and the verification sequence to obtain the first fused feature.
9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the program can be loaded and run by a processor to execute the online handwriting identification method based on multiple samples as described in any one of claims 1 to 6.
10. An electronic device, characterized in that: The electronic device comprises: one or more processors; a memory; and one or more application programs, which are stored in the memory and configured to be loaded and run by the one or more processors so as to execute the online handwriting identification method based on multiple samples as described in any one of claims 1 to 6.