A handwritten signature recognition method and system based on field semantic prior

Abstract

The application discloses a kind of based on field semantic priori handwritten signature recognition method and system, the method includes: obtaining the document image to be parsed, at least one target field area in document image is identified to determine corresponding field semantics, document image includes at least one field area for handwritten signature;Image feature extraction and character recognition are carried out to the target field area of field semantics determination as signature field, generate the multiple candidate recognition results and confidence distribution corresponding to signature field;According to confidence distribution, the certainty of current candidate recognition result is judged by statistical analysis, and the priori constraint of field semantics is suitable for the candidate recognition result of non certainty;The candidate recognition result based on priori constraint adjustment is weighted and rearranged, and the target recognition result of handwritten signature is output.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a handwritten signature recognition method and system based on field semantic prior. Background Technology

[0002] With the development of artificial intelligence technology, document parsing systems typically use processes such as layout analysis, character recognition, and field extraction to automate the parsing of contracts, invoices, and various structured and unstructured documents. In document parsing, handwritten signature fields are prone to recognition uncertainty by character recognition models due to variations in handwriting habits, cursive writing, character distortion, and image noise.

[0003] Existing technologies, when processing documents containing handwritten signature fields, typically employ general character recognition models to output a single recognition result, lacking semantic constraints specific to the field. This leads to high uncertainty in the recognition results, especially when the handwriting is complex, significantly reducing accuracy. Furthermore, existing signature error correction methods mainly rely on static dictionaries or simple string matching rules to correct errors, but these are not deeply integrated with the character recognition process, failing to promptly correct biases and resulting in insufficient generalization ability. They also struggle to make stable judgments when recognition results are diverse. Another major drawback is that while increasing training samples can improve model performance, the extremely discrete signature distribution and privacy concerns make it difficult to comprehensively cover diverse handwriting styles in the training data.

[0004] It is evident that existing technologies have shortcomings that urgently need to be addressed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a handwritten signature recognition method and system based on field semantic prior, which can introduce field semantic prior in the text recognition process and constrain the recognition search space, effectively reduce the uncertainty of handwritten signature recognition and improve the recognition accuracy of signature fields.

[0006] To address the aforementioned technical problems, the first aspect of this invention discloses a handwritten signature recognition method based on field semantic prior, the method comprising: Obtain a document image to be parsed, identify at least one target field region in the document image to determine the corresponding field semantics, wherein the document image includes at least one field region for handwritten signature; Image feature extraction and text recognition are performed on the target field region whose semantics are determined to be a signature field, generating multiple candidate recognition results and confidence distributions corresponding to the signature field; Based on the confidence distribution, the certainty of the current candidate identification result is determined by statistical analysis, and prior constraints of field semantics are applied to the non-deterministic candidate identification result. The candidate recognition results adjusted based on prior constraints are weighted and sorted to output the target recognition result of the handwritten signature.

[0007] As an optional implementation, in the first aspect of the present invention, acquiring a document image to be parsed and identifying at least one target field region in the document image to determine the corresponding field semantics includes: For each document image, it is input into a preset recognition model, which is a dual-stream encoder architecture, including a text stream encoder and a visual stream encoder; The text stream encoder encodes the text sequence identified from the field regions of the document image to obtain a text semantic vector; The visual flow encoder extracts features from the image blocks corresponding to the field region and the context region to obtain a visual feature vector. The visual feature vector includes the field position, field size, and layout of the relative relationships between elements in the document. The text semantic vector and visual feature vector are fused using an attention mechanism or feature concatenation mechanism. The resulting fused features are input into the classification head, and the output is the probability distribution of all fields in the target field region corresponding to the predefined semantic categories. The field semantics of the target field region are uniquely determined based on the probability distribution, and the field semantics include at least one signature field.

[0008] As an optional implementation, in the first aspect of the present invention, image feature extraction and text recognition are performed on the target field region whose field semantics are determined to be a signature field, generating multiple candidate recognition results and confidence distributions corresponding to the signature field, including; A handwritten image of the region corresponding to the signature field is collected separately and input into a preset neural network model to extract a visual feature sequence; The visual feature sequence is input into a preset long short-term memory network to model the contextual dependencies between characters. The modeled visual feature sequence is then mapped to a character probability sequence through a decoding layer connected to the long short-term memory network. The sequence generation task of the decoding layer uses a beam search algorithm to retain the top-K optimal paths. Each path corresponds to a character probability sequence and outputs a candidate recognition string, generating multiple candidate recognition results. For each candidate recognition string, the geometric mean of the predicted character probabilities at each time step along its corresponding path is calculated to obtain the confidence level of the candidate recognition string, thus generating the confidence level distribution of the candidate recognition results.

[0009] As an optional implementation, in the first aspect of the present invention, determining the certainty of the current candidate identification result through statistical analysis based on the confidence distribution includes: The first statistical analysis uses the confidence difference method to calculate the confidence difference between multiple candidate recognition results. If the confidence difference between the highest confidence and the second highest confidence is less than the preset difference threshold, it is determined that the candidate recognition result is uncertain. The second statistical analysis uses the distribution entropy method to calculate the Shannon entropy value of the confidence distribution of the candidate recognition results. If the uncertainty measure of the Shannon entropy value is greater than the preset discrete threshold, it is determined that the candidate recognition results are uncertain. The third statistical analysis uses a decoding path consistency test to analyze whether the character alignment of the decoding path corresponding to the candidate output results of the decoding layer is consistent at key positions. If multiple high-confidence paths correspond to different character hypotheses at the same time step or in the same image region, it is judged that the candidate recognition results are uncertain. If at least one candidate identification result in the first, second, and third statistical analyses is determined to be uncertain, then the identification result of the current signature field is marked as uncertain, thus triggering the prior constraint of the field semantics.

[0010] As an optional implementation, in the first aspect of the invention, the construction of prior knowledge of the prior constraints of the field semantics includes: Contextual information is extracted from the document image, and a list of historically relevant personnel is queried from the business database associated with the document image metadata. The contextual information and the list of personnel are then standardized to obtain an original set of candidate signatures. Each signature candidate, each character, and the source type from different sources in the original signature candidate set are treated as a heterogeneous graph node. Multiple edge relationships are defined to connect the heterogeneous graph nodes to create a prior heterogeneous graph. A graph network layer is used to perform multi-round message passing on the prior heterogeneous graph. In each layer, the node aggregates the feature information of the neighboring nodes, and each signature candidate node obtains a refined representation vector that integrates graph structure information and multi-source context. The visual context feature vector is obtained by encoding the handwritten image corresponding to the nondeterministic signature field. The similarity score between the visual context feature vector and the refined representation vector is calculated. Each signature candidate in the original candidate set is sorted according to the similarity score, and the top-M most relevant signature candidates are selected to obtain the prior constraint set of the signature candidates.

[0011] As an optional implementation, in the first aspect of the invention, prior constraints on field semantics are applied to nondeterministic candidate identification results, including...

[0012] For each signature candidate character in the prior constraint set, a prefix tree is converted through the character-to-numerical index mapping, and all prefix trees are integrated into the decoding process of generating multiple candidate recognition results corresponding to the signature field; Based on the decoding process, any decoding path retains or expands the corresponding candidate recognition result only when matching the character transitions allowed by the prefix tree, so as to form a priori constraints on the handwritten signature recognition result to correct and adjust the candidate recognition result.

[0013] As an optional implementation, in the first aspect of the invention, applying prior constraints on field semantics to non-deterministic candidate identification results further includes: The character probability of each signature candidate in the prior constraint set is used as the text recognition score. During the decoding process of generating multiple candidate recognition results corresponding to the signature field, the text recognition score and the visual recognition score are interpolated. Based on the interpolation calculation, the decoding process is guided to output character candidates similar to the prior constraint set, so as to form prior constraints for the handwritten signature recognition result and to correct and adjust the candidate recognition result.

[0014] As an optional implementation, in the first aspect of the present invention, the candidate recognition results adjusted based on prior constraints are weighted and ranked to output the target recognition result of the handwritten signature, including: Calculate the string edit distance between each candidate recognition result and each signature candidate in the prior constraint set to obtain the matching score corresponding to the candidate recognition result; Calculate the weighted average of the confidence level and the matching score corresponding to each candidate recognition result to obtain the comprehensive score corresponding to the candidate recognition result. The weight of the weighted average is dynamically adjusted according to the degree of certainty of the candidate recognition result corresponding to the document image context and the historical recognition accuracy. Based on the comprehensive score, the candidate recognition results are rearranged in descending order from high to low, and the first candidate signature after rearrangement is selected as the target recognition result of the handwritten signature in the document image.

[0015] A second aspect of this invention discloses a handwritten signature recognition system based on field semantic prior, the system comprising: The acquisition module is used to acquire a document image to be parsed, identify at least one target field region in the document image to determine the corresponding field semantics, and the document image includes at least one field region for handwritten signature; The recognition module is used to extract image features and recognize text in the target field region whose semantics are determined to be a signature field, and generate multiple candidate recognition results and confidence distributions corresponding to the signature field. The prior module is used to determine the certainty of the current candidate identification result through statistical analysis based on the confidence distribution, and to apply prior constraints of field semantics to the non-deterministic candidate identification result; The output module is used to weight and sort the candidate recognition results based on prior constraints, and output the target recognition result of the handwritten signature.

[0016] A third aspect of this invention discloses another handwritten signature recognition system based on field semantic prior, the system comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute some or all of the steps in the handwritten signature recognition method based on field semantic prior disclosed in the first aspect of the present invention.

[0017] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute some or all of the steps in the handwritten signature recognition method based on field semantic prior disclosed in the first aspect of the present invention.

[0018] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: This invention accurately locates handwritten signature areas in documents based on layout analysis and semantic understanding, avoiding confusion with ordinary text, reducing the false negative rate and improving field location accuracy. By generating multiple candidate recognition results and confidence distributions for the signature area, it avoids the limitations of a single recognition result and overcomes the challenge of misrecognition caused by the discreteness of handwriting style. Based on the confidence distribution, it applies field semantic priors to dynamically correct candidate results. Through deterministic judgment and semantic prior constraints, it eliminates the separation between text recognition and error correction processes. By weighted fusion to rank candidate results, it outputs the optimal recognition result to achieve high-confidence and stable recognition. Through multi-candidate recognition and field semantic prior constraints based on uncertainty, it solves the problems of high uncertainty, weak error correction ability, and insufficient generalization ability in handwritten signature field recognition in document parsing, achieving a leap from single recognition to high-confidence semantic-driven recognition. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1This is a flowchart illustrating a handwritten signature recognition method based on field semantic priors disclosed in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of a handwritten signature recognition system based on field semantic prior disclosed in an embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of another handwritten signature recognition system based on field semantic prior disclosed in an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0025] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0026] This invention discloses a handwritten signature recognition method and system based on field semantic prior. By using layout analysis and semantic understanding to accurately locate handwritten signature areas in documents, it avoids confusion with ordinary text, reducing the false negative rate and improving field location accuracy. By generating multiple candidate recognition results and confidence distributions for the signature area, it avoids the limitations of a single recognition result and overcomes the challenge of misrecognition caused by the discreteness of handwriting styles. Based on the confidence distribution, it applies field semantic prior to dynamically correct candidate results. Through deterministic judgment and semantic prior constraints, it eliminates the separation between text recognition and error correction processes. By weighted fusion and ranking of candidate results, it outputs the optimal recognition result to achieve high-confidence and stable recognition. Through multi-candidate recognition and uncertainty-based field semantic prior constraints, it solves the problems of high uncertainty, weak error correction ability, and insufficient generalization ability in handwritten signature field recognition during document parsing, achieving a leap from single recognition to high-confidence semantic-driven recognition. These are described in detail below.

[0027] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating a handwritten signature recognition method based on field semantic priors disclosed in an embodiment of the present invention. Figure 1 The described handwritten signature recognition method based on field semantic prior can be applied to data processing systems / data processing equipment / data processing servers (wherein, the server includes local processing servers or cloud processing servers). For example... Figure 1 As shown, this handwritten signature recognition method based on field semantic priors may include the following operations: 101. Obtain a document image to be parsed, identify at least one target field region in the document image to determine the corresponding field semantics, wherein the document image includes at least one field region for handwritten signature.

[0028] 102. Perform image feature extraction and text recognition on the target field region whose field semantics are determined to be a signature field, and generate multiple candidate recognition results and confidence distributions corresponding to the signature field.

[0029] 103. Based on the confidence distribution, the certainty of the current candidate identification result is determined by statistical analysis, and prior constraints of field semantics are applied to the non-deterministic candidate identification result.

[0030] Optionally, the prior constraints of field semantics can be to first construct a set of prior constraints for signature candidates from document context information and associated business data, and then convert the signature candidate strings in the prior constraint set into recognition constraints that can be used to constrain the text recognition process. The constraints can be any of the following forms: (1) For the character sequence corresponding to the signature candidate, increase the recognition probability of the same or similar character sequences in the candidate recognition results; (2) During the text recognition decoding process, restrict the output character sequence to only within the range of character combinations corresponding to the signature candidate; (3) During the decoding search process, increase the priority of the decoding path with a higher matching degree with the signature candidate.

[0031] 104. Weight the candidate recognition results adjusted based on prior constraints and sort them to output the target recognition result of the handwritten signature.

[0032] As can be seen, the above-described embodiments of the invention accurately locate the handwritten signature area in the document based on layout analysis and semantic understanding, avoiding confusion with ordinary text, reducing the false negative rate and improving the accuracy of field location. By generating multiple candidate recognition results and confidence distributions for the signature area, the limitations of a single recognition result are avoided, and the challenge of misrecognition caused by the discreteness of handwriting style is overcome. Based on the confidence distribution, the candidate results are dynamically corrected using field semantic priors. The separation between text recognition and error correction is eliminated through deterministic judgment and semantic prior constraints. The candidate results are ranked through weighted fusion, and the optimal recognition result is output to achieve high-confidence and stable recognition. Through multi-candidate recognition and field semantic prior constraints based on uncertainty, the problems of high uncertainty, weak error correction ability, and insufficient generalization ability in handwritten signature field recognition in document parsing are solved, realizing a leap from single recognition to high-confidence semantic-driven recognition.

[0033] As an optional embodiment, the steps described above, including acquiring the document image to be parsed and identifying at least one target field region in the document image to determine the corresponding field semantics, include: For each document image, it is input into a preset recognition model, which is a dual-stream encoder architecture, including a text stream encoder and a visual stream encoder; The text stream encoder encodes the text sequence identified from the field regions of the document image to obtain a text semantic vector; The visual flow encoder extracts features from the image blocks corresponding to the field region and the context region to obtain a visual feature vector. The visual feature vector includes the field position, field size, and layout of the relative relationships between elements in the document. The text semantic vector and visual feature vector are fused using an attention mechanism or feature concatenation mechanism. The resulting fused features are input into the classification head, and the output is the probability distribution of all fields in the target field region corresponding to the predefined semantic categories. The field semantics of the target field region are uniquely determined based on the probability distribution, and the field semantics include at least one signature field.

[0034] Optionally, the text stream encoder can use a pre-trained BERT or RoBERTa model to encode the text sequence recognized from the field title region by OCR to obtain a text semantic vector; this application does not impose any restrictions.

[0035] Optionally, the visual flow encoder may use a convolutional neural network (CNN, such as ResNet) or a visual transformer (ViT) to extract features from image patches of the field region and its surrounding context region to obtain a layout visual feature vector containing position, size, and relative relationship with other elements in the document. This application does not impose any limitations on this.

[0036] Optionally, feature fusion can employ an attention mechanism or a simple feature concatenation followed by a fully connected layer to construct a feature fusion module in the recognition model to fuse the text semantic vector with the visual feature vector of the layout. The classification head can be a Softmax classification layer, a Sigmoid classification layer, or a ReLU classification layer, which is not limited in this application.

[0037] Specifically, when determining the signature field, the model pays special attention to title text features such as "name", "payee" and "handler", and makes a comprehensive decision based on the layout features of the field, which is usually located in a specific area of ​​the document (such as the upper left corner of a table or the signature area).

[0038] As can be seen, through the above optional embodiments, the dual-parallel architecture of text stream encoder and visual stream encoder avoids the positioning deviation caused by relying solely on text in traditional methods, breaks through the limitations of field region positioning, reduces the interference of writing habits, cursive writing, and character deformation through in-depth feature mining of text stream and visual stream, and achieves dynamic generation of semantic constraints by fusing text semantic vectors and visual feature vectors through attention mechanism and outputting field semantic probability distribution. Through the field semantic accurate recognition mechanism of the dual-stream encoder architecture, the problems of inaccurate positioning of handwritten signature fields and high recognition uncertainty caused by layout interference in document parsing are solved, realizing field positioning from isolated text recognition to multimodal semantic fusion, and significantly improving the recognition accuracy.

[0039] As an optional embodiment, the above steps, including image feature extraction and text recognition of the target field region whose field semantics are determined to be a signature field, generating multiple candidate recognition results and confidence distributions corresponding to the signature field, include: A handwritten image of the region corresponding to the signature field is collected separately and input into a preset neural network model to extract a visual feature sequence; The visual feature sequence is input into a preset long short-term memory network to model the contextual dependencies between characters. The modeled visual feature sequence is then mapped to a character probability sequence through a decoding layer connected to the long short-term memory network. The sequence generation task of the decoding layer uses a beam search algorithm to retain the top-K optimal paths. Each path corresponds to a character probability sequence and outputs a candidate recognition string, generating multiple candidate recognition results. For each candidate recognition string, the geometric mean of the predicted character probabilities at each time step along its corresponding path is calculated to obtain the confidence level of the candidate recognition string, thus generating the confidence level distribution of the candidate recognition results.

[0040] Optionally, the recognition of handwritten signature strings corresponding to multiple candidate recognition results is implemented using a deep learning-based sequence recognition model. This sequence recognition model can be a convolutional recurrent neural network or an encoder-decoder model that introduces an attention mechanism. This application does not impose any restrictions.

[0041] Optionally, the preset neural network model can be a deep CNN network that extracts visual feature sequences from handwritten images, such as VGG or DenseNet, and this application does not impose any limitations.

[0042] Optionally, the preset long short-term memory network can be a bidirectional LSTM, a deep LSTM, or a convolutional LSTM to model the contextual dependencies between characters, and this application does not impose any limitations.

[0043] Optionally, the decoding layer can be a temporal classification decoding layer or an attention decoder used to map the feature sequence output by the LSTM into a character probability sequence; this application does not impose any limitations on this.

[0044] Specifically, beam search is a heuristic search algorithm widely used in sequence generation tasks. It achieves a good balance between search efficiency and result quality. Starting from the initial state of sequence generation, the probability distribution of the first word is generated and the top k optimal candidates are selected based on the probability values ​​through beam width control. For each selected candidate, the probability distribution of the next word is generated to expand all candidate sequences, forming a new set of candidate sequences. Finally, the k optimal results are selected from all expanded candidates.

[0045] As can be seen, through the above optional embodiments, by separately collecting handwritten images of the signature area and inputting them into the neural network, visual feature sequences are extracted to avoid global image interference. LSTM is used to model the contextual dependencies between characters, and the temporal classification decoding layer outputs character probability sequences to interpret the temporal patterns of strokes and character deformation in handwritten signatures to improve character shape adaptability. TOP-K candidate generation through beam search eliminates the limitations of a single result. The confidence distribution is generated by calculating the geometric mean of the probabilities at each time step on the candidate string path, providing accurate data support for subsequent deterministic judgments. Through multi-candidate recognition and confidence distribution quantization mechanisms, the problems of high non-determinism in handwritten signature field recognition, severe interference from stroke deformation, and weak generalization ability of a single result in document parsing are solved, thus realizing the construction of a high-precision closed loop for handwritten signature recognition.

[0046] As an optional embodiment, the step above, determining the certainty of the current candidate identification result through statistical analysis based on the confidence distribution, includes: The first statistical analysis uses the confidence difference method to calculate the confidence difference between multiple candidate recognition results. If the confidence difference between the highest confidence and the second highest confidence is less than the preset difference threshold, it is determined that the candidate recognition result is uncertain. The second statistical analysis uses the distribution entropy method to calculate the Shannon entropy value of the confidence distribution of the candidate recognition results. If the uncertainty measure of the Shannon entropy value is greater than the preset discrete threshold, it is determined that the candidate recognition results are uncertain. The third statistical analysis uses a decoding path consistency test to analyze whether the character alignment of the decoding path corresponding to the candidate output results of the decoding layer is consistent at key positions. If multiple high-confidence paths correspond to different character hypotheses at the same time step or in the same image region, it is judged that the candidate recognition results are uncertain. If at least one candidate identification result in the first, second, and third statistical analyses is determined to be uncertain, then the identification result of the current signature field is marked as uncertain, thus triggering the prior constraint of the field semantics.

[0047] As an example, in the first statistical analysis, let the candidate result list be arranged in descending order of confidence level as follows: ,in This represents the recognition result of the i-th candidate. This represents the confidence level corresponding to the i-th candidate recognition result, where the maximum value of i is K (TOP-K outputs). The confidence level difference is then calculated. If the confidence difference is less than the difference threshold, then the recognition model is considered to have achieved the optimal result. and suboptimal results Hesitation and uncertainty exist between them.

[0048] As an example, in the second statistical analysis, after normalizing the confidence scores of the Top-K candidate results to a sum of 1, a discrete probability distribution can be visualized and the Shannon entropy value of the distribution can be calculated. The higher the entropy value H, the more dispersed the confidence score and the greater the uncertainty of the identification model. By setting a discrete threshold, when the Shannon entropy value exceeds the discrete threshold, the uncertainty judgment result is triggered.

[0049] Specifically, during the cluster search decoding process, not only is the final string recorded, but also its main decoding path (such as the alignment path in CTC). By analyzing whether the character alignment of the decoding paths corresponding to the Top-K candidate results is consistent at key positions (such as the boundary between the surname and the given name), if multiple high-confidence paths correspond to completely different character hypotheses at the same time step or image region, it indicates that there is ambiguity within the recognition model, which is judged as high uncertainty. This can capture cases where the confidence is close but the strings are completely different, further improving the accuracy of uncertainty judgment.

[0050] As can be seen, through the above optional embodiments, the confidence difference method is used to accurately identify highly similar candidate scenes, the distribution entropy value method is used to quantify the distribution dispersion to eliminate fuzzy judgments, the decoding path consistency test is used to reduce the interference of structured continuous writing, and the dynamic mechanism triggered by multiple dimensions is used to ensure the reliability of nondeterministic judgments. This solves the problems of high nondeterminism in handwritten signature field recognition, easy omissions in single judgments, and inaccurate error correction triggers in document parsing, and achieves a high-precision closed loop from fuzzy threshold judgment to high-precision nondeterministic quantitative evaluation.

[0051] As an optional embodiment, the construction of prior knowledge of the prior constraints of the field semantics in the above steps includes: Contextual information is extracted from the document image, and a list of historically relevant personnel is queried from the business database associated with the document image metadata. The contextual information and the list of personnel are then standardized to obtain an original set of candidate signatures. Each signature candidate, each character, and the source type from different sources in the original signature candidate set are treated as a heterogeneous graph node. Multiple edge relationships are defined to connect the heterogeneous graph nodes to create a prior heterogeneous graph. A graph network layer is used to perform multi-round message passing on the prior heterogeneous graph. In each layer, the node aggregates the feature information of the neighboring nodes, and each signature candidate node obtains a refined representation vector that integrates graph structure information and multi-source context. The visual context feature vector is obtained by encoding the handwritten image corresponding to the nondeterministic signature field. The similarity score between the visual context feature vector and the refined representation vector is calculated. Each signature candidate in the original candidate set is sorted according to the similarity score, and the top-M most relevant signature candidates are selected to obtain the prior constraint set of the signature candidates.

[0052] As an example, a dynamic and structured prior knowledge source is constructed. First, contextual information is extracted from the current document, such as inferring potentially associated personal names from other high-confidence identification fields (e.g., company name fields like "Party A" and "Party B"); and querying historical lists of relevant personnel from the business database associated with document metadata (e.g., contract number, date). Second, the textual information from these sources (e.g., "Zhang San", "Li Si·Wang") is standardized (special symbols are removed, and delimiters are standardized) to form an original candidate list of personal names.

[0053] Optionally, heterogeneous graph nodes include signature nodes, character nodes, and source type nodes, where signature nodes can use vector representations of their strings, character nodes can use their character embeddings, and source type nodes can use learnable type embeddings.

[0054] Optionally, edge relationships include containment edges, co-occurrence edges, and same-origin edges. Containment edges are character nodes that are composed of signature nodes, co-occurrence edges are edges between name nodes that appear simultaneously in the same source or document, and same-origin edges are edges between name nodes that come from the same data source.

[0055] Optionally, the graph network layer can be a graph attention network (GAT) or a graph convolutional network (GCN) for multi-round message passing; this application does not impose any restrictions.

[0056] As an example, the process of each node aggregating the feature information of its neighboring nodes is as follows: a signature node will aggregate the semantic information of each character node it contains, as well as the information of other name nodes that have a co-occurrence relationship with it, thereby allowing the model to learn the internal structure of the signature (such as the surname-first name pattern in the signature) and the strength of the association between signatures.

[0057] As can be seen, through the above optional embodiments, by extracting context from document images and querying historical handler lists from business databases, a standardized original signature candidate set is generated to construct a high-quality original candidate pool. By defining equilateral relationships such as belonging and association based on signature candidates, characters, and source types as nodes, a heterogeneous graph-enhanced multi-source semantic relationship network is constructed. Through multiple rounds of aggregating neighbor node features, a high semantic representation vector of the fusion graph structure is generated. Finally, by calculating the similarity between the visual context feature vector of the nondeterministic signature and the refined representation, dynamic filtering is performed to achieve highly relevant candidate ranking. Through the prior knowledge construction mechanism driven by multi-source heterogeneous graphs, the problems of missing semantic constraints in handwritten signature fields, insufficient utilization of historical data, and inaccurate nondeterministic recognition correction in document parsing are solved, realizing intelligent constraints from static dictionaries to dynamic semantic associations.

[0058] As an optional embodiment, the steps described above, including applying prior constraints on field semantics to non-deterministic candidate identification results, include...

[0059] For each signature candidate character in the prior constraint set, a prefix tree is converted through the character-to-numerical index mapping, and all prefix trees are integrated into the decoding process of generating multiple candidate recognition results corresponding to the signature field; Based on the decoding process, any decoding path retains or expands the corresponding candidate recognition result only when matching the character transitions allowed by the prefix tree, so as to form a priori constraints on the handwritten signature recognition result to correct and adjust the candidate recognition result.

[0060] It should be noted that a prefix tree is a special tree-like data structure used for efficient storage and retrieval of string sets. It can convert characters into integer indices for efficient storage and access within the data structure. The prefix tree can also serve as a constraint on the candidate recognition results output by the recognition model, ensuring that the generated candidate sequences are actual signature candidates that exist in the prior constraint set. This can avoid generating unreasonable names and improve recognition accuracy.

[0061] As can be seen, through the above optional embodiments, historical signature candidates are converted into prefix trees, and illegal character transfer paths are dynamically filtered during the decoding process to eliminate the impact of signature distribution discreteness. By embedding the prefix tree into the decoding layer and retaining only the paths that match the prefix tree, the process delay from recognition to error correction in traditional methods is avoided, and the fragmentation of the error correction process is eliminated. Through the dynamic constraint mechanism of the decoding process driven by the prefix tree, the problems of nondeterministic recognition of handwritten signature fields, delayed error correction timing, and insufficient utilization of historical data in document parsing are solved, realizing intelligent error correction from static dictionary matching to real-time decoding constraints.

[0062] As an optional embodiment, the above steps of applying prior constraints on field semantics to non-deterministic candidate identification results further include: The character probability of each signature candidate in the prior constraint set is used as the text recognition score. During the decoding process of generating multiple candidate recognition results corresponding to the signature field, the text recognition score and the visual recognition score are interpolated. Based on the interpolation calculation, the decoding process is guided to output character candidates similar to the prior constraint set, so as to form prior constraints for the handwritten signature recognition result and to correct and adjust the candidate recognition result.

[0063] As an example, character n-grams break down candidate names into character-level continuous segments, such as Bigram (2-gram): the combination of "Zhang" and "San" in "Zhang San", or Trigram (3-gram): the combination of "Zhang San" and "Sanfeng" in "Zhang Sanfeng". By calculating the probability of these character sequences in natural language through a pre-trained N-gram language model, it can reflect the rationality of the occurrence of the character sequence in the language. A high probability indicates that it is more common or more reasonable. It is worth noting that this language model is independent of the recognition model and is specifically used to evaluate the rationality of language.

[0064] Furthermore, the visual recognition score comes from the matching degree of the OCR or image recognition model to the candidate string. For example, the model's matching confidence for "Zhang Sanfeng" is 0.8. The interpolation calculation is to combine the two scores with certain weights. These weights are adjustable parameters that can balance the importance between text recognition and visual recognition.

[0065] As can be seen, through the above optional embodiments, by interpolating the probability of signature candidate characters and visual feature similarity according to weights to generate a fusion score, the decoding path is dynamically guided to overcome the limitations of single-modal recognition. The candidate path weights are optimized by interpolation calculation, so that the output result highly matches the prior constraint set, thereby improving the accuracy and stability of error correction. Through the dynamic decoding constraint mechanism guided by multimodal score interpolation, the problems of nondeterministic recognition of handwritten signature fields, separation of text and visual information, and delayed error correction timing in document parsing are solved, realizing intelligent error correction from static constraints to real-time multimodal fusion.

[0066] As an optional embodiment, the step above, which involves weighting and ranking the candidate recognition results adjusted based on prior constraints to output the target recognition result of the handwritten signature, includes: Calculate the string edit distance between each candidate recognition result and each signature candidate in the prior constraint set to obtain the matching score corresponding to the candidate recognition result; Calculate the weighted average of the confidence level and the matching score corresponding to each candidate recognition result to obtain the comprehensive score corresponding to the candidate recognition result. The weight of the weighted average is dynamically adjusted according to the degree of certainty of the candidate recognition result corresponding to the document image context and the historical recognition accuracy. Based on the comprehensive score, the candidate recognition results are rearranged in descending order from high to low, and the first candidate signature after rearrangement is selected as the target recognition result of the handwritten signature in the document image.

[0067] Optionally, the matching score is obtained by calculating the string edit distance (such as Levenshtein distance) between the candidate recognition result and each signature candidate in the prior constraint set, and taking the reciprocal or negative value of the shortest distance.

[0068] Optionally, the matching score can also be used to calculate the candidate recognition result from the signature representation learned in the graph neural network stage. The cosine similarity between the string representation (averaged by character embedding) and the refined representation vector of each signature candidate in the prior constraint set P is used as the matching score, and the highest similarity is taken as the matching score.

[0069] Optionally, the weighted average of the comprehensive scores can be calculated using either a weighted geometric mean or a weighted arithmetic mean. Taking the weighted arithmetic mean as an example, the calculation process can be expressed as follows: ; In the formula, The confidence level corresponding to the candidate identification result. The adjustable parameter α is used to balance the weights of candidate recognition results and prior knowledge in visual and text recognition. It can be dynamically adjusted based on the degree of certainty of candidate recognition results corresponding to the document image context and the historical recognition accuracy.

[0070] As can be seen, through the above optional embodiments, semantic similarity is accurately quantified by string edit distance matching, multi-source information is adaptively fused through dynamic weighting mechanism, high-confidence unique identification is achieved through comprehensive score ranking and output, and the problem of non-deterministic identification of handwritten signature fields, inaccurate ranking of multiple candidate results, and insufficient fusion of semantic constraints in document parsing is solved through dynamic weighting comprehensive score ranking mechanism. It realizes the dynamic optimization from fixed threshold ranking to semantic-confidence dual-dimensional optimization, thereby achieving the goal of building a zero-error closed loop for signature recognition.

[0071] As an optional embodiment, the steps described above, before outputting the target recognition result, include: A final confidence threshold for target recognition results is predefined. The first signature candidate after rearrangement is compared with the final confidence threshold. If it is lower than the final confidence threshold, the target recognition result is marked and a manual review is requested.

[0072] As can be seen, through the above optional embodiments, the false judgment rate of handwritten signature recognition is further reduced by setting a final confidence threshold, and manual review is introduced to make a final judgment to ensure the accuracy of recognition. It is understood that the frequency of target recognition results below the final confidence threshold is extremely rare. This embodiment is only used as a safety mechanism to deal with document images that are difficult for the system to recognize.

[0073] In one specific implementation scheme, a handwritten signature recognition system based on field semantic prior is provided according to the technical solution of the present invention, which addresses the shortcomings of existing technologies, including: (1) It relies on a general character recognition model to output a single recognition result and lacks semantic constraints for specific fields; (2) After recognition is completed, post-recognition correction is performed using static dictionaries or string matching rules, which is difficult to integrate deeply with the recognition process; (3) The model's capabilities can be improved by increasing the number of training samples, but the distribution of personal names is discrete and subject to privacy restrictions, making it difficult to cover the diverse writing situations of personal names in actual business.

[0074] The above-mentioned solutions generally suffer from limited error correction capabilities and insufficient generalization. In particular, when there are multiple possibilities for the identification results and the confidence distribution is scattered, the system struggles to make stable and reliable judgments.

[0075] This recognition system provides a method for identifying and correcting handwritten signatures. By explicitly recognizing the semantics of fields during document parsing and introducing prior information about the name to constrain the recognition process when uncertainty is detected in the recognition of handwritten names, the system improves the accuracy of handwritten signature field recognition and the overall stability of the system. Furthermore, this system is independent of specific character recognition model structures and can be embedded into existing document parsing systems in a general manner, demonstrating good engineering feasibility and scalability.

[0076] The working method of this recognition system is illustrated in the following example in the scenario of recognizing the name of a contract signatory: First, during the contract document parsing process, the system uses a recognition module to identify the "signer's name" field in the document image, and then processes the handwritten text in that area. The following candidate recognition results and corresponding confidence levels are generated during the recognition process: Candidate identification result 1: "Li Ming", with a confidence level of 0.45; Candidate identification result 2: "Li Ming", with a confidence level of 0.42; Candidate identification result 3: "Li Ming", with a confidence level of 0.38.

[0077] Subsequently, the system detected that the confidence distribution was not concentrated and contained uncertainty. Given that the semantics of this field is "signer's name", the system constructs a priori candidate set of signatures from the contract's party information, which includes the known contract signer's name "Li Ming".

[0078] Finally, the system uses a constraint generation module to convert the prior candidate set of signatures into recognition constraint information. This information constrains the decoding path, increasing the probability weight of decoding paths that have a higher match with the prior signature "Li Ming". After this adjustment, the ranking of the "Li Ming" candidate recognition result rises, and it is determined as the final signature recognition result in the subsequent decision module, ensuring the consistency of the output result with the contract participants.

[0079] The above embodiments demonstrate that the present invention can effectively reduce the uncertainty of handwritten signature recognition and improve the accuracy of signature field recognition in complex document parsing scenarios through field semantic priors and recognition constraint mechanisms. Furthermore, this technical solution dynamically constructs signature prior information to adapt to different business scenarios, demonstrating strong generalization ability and technical interpretability.

[0080] In summary, the technical solutions disclosed in the embodiments of the present invention have the following advantages: 1. This invention uses a constraint mechanism that triggers signature recognition through prior semantic knowledge of fields, introducing prior information only in specific fields to avoid indiscriminate intervention on the entire text.

[0081] 2. When there is uncertainty in signature recognition, this invention reduces the false recognition rate by adjusting the decoding path or the recognition probability distribution in the recognition process.

[0082] 3. The prior constraint set of signature candidates in this invention can be dynamically constructed according to the document context and business data to adapt to different complex document parsing scenarios.

[0083] 4. This invention does not rely on a specific character recognition model structure, has good versatility and scalability, and the recognition process is highly interpretable, which facilitates the optimization and auditing process of the document parsing system.

[0084] Example 2 Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of a handwritten signature recognition system based on field semantic prior, as disclosed in an embodiment of the present invention. Figure 2 The described handwritten signature recognition system based on field semantic prior can be applied to data processing systems / data processing equipment / data processing servers (wherein, the server includes local processing servers or cloud processing servers). For example... Figure 2 As shown, the handwritten signature recognition system based on field semantic priors may include: The acquisition module 201 is used to acquire a document image to be parsed, identify at least one target field region in the document image to determine the corresponding field semantics, and the document image includes at least one field region for handwritten signature; The recognition module 202 is used to extract image features and recognize text in the target field region whose field semantics are determined to be a signature field, and generate multiple candidate recognition results and confidence distributions corresponding to the signature field. The prior module 203 is used to determine the certainty of the current candidate identification result through statistical analysis based on the confidence distribution, and to apply prior constraints of field semantics to the non-deterministic candidate identification result; The output module 204 is used to weight and sort the candidate recognition results based on prior constraints, and output the target recognition result of the handwritten signature.

[0085] As an optional implementation, acquiring a document image to be parsed and identifying at least one target field region in the document image to determine the corresponding field semantics includes: For each document image, it is input into a preset recognition model, which is a dual-stream encoder architecture, including a text stream encoder and a visual stream encoder; The text stream encoder encodes the text sequence identified from the field regions of the document image to obtain a text semantic vector; The visual flow encoder extracts features from the image blocks corresponding to the field region and the context region to obtain a visual feature vector. The visual feature vector includes the field position, field size, and layout of the relative relationships between elements in the document. The text semantic vector and visual feature vector are fused using an attention mechanism or feature concatenation mechanism. The resulting fused features are input into the classification head, and the output is the probability distribution of all fields in the target field region corresponding to the predefined semantic categories. The field semantics of the target field region are uniquely determined based on the probability distribution, and the field semantics include at least one signature field.

[0086] As an optional implementation, image feature extraction and text recognition are performed on the target field region whose semantics are determined to be a signature field, generating multiple candidate recognition results and confidence distributions corresponding to the signature field, including; A handwritten image of the region corresponding to the signature field is collected separately and input into a preset neural network model to extract a visual feature sequence; The visual feature sequence is input into a preset long short-term memory network to model the contextual dependencies between characters. The modeled visual feature sequence is then mapped to a character probability sequence through a decoding layer connected to the long short-term memory network. The sequence generation task of the decoding layer uses a beam search algorithm to retain the top-K optimal paths. Each path corresponds to a character probability sequence and outputs a candidate recognition string, generating multiple candidate recognition results. For each candidate recognition string, the geometric mean of the predicted character probabilities at each time step along its corresponding path is calculated to obtain the confidence level of the candidate recognition string, thus generating the confidence level distribution of the candidate recognition results.

[0087] As an optional implementation, the certainty of the current candidate identification result is determined through statistical analysis based on the confidence distribution, including: The first statistical analysis uses the confidence difference method to calculate the confidence difference between multiple candidate recognition results. If the confidence difference between the highest confidence and the second highest confidence is less than the preset difference threshold, it is determined that the candidate recognition result is uncertain. The second statistical analysis uses the distribution entropy method to calculate the Shannon entropy value of the confidence distribution of the candidate recognition results. If the uncertainty measure of the Shannon entropy value is greater than the preset discrete threshold, it is determined that the candidate recognition results are uncertain. The third statistical analysis uses a decoding path consistency test to analyze whether the character alignment of the decoding path corresponding to the candidate output results of the decoding layer is consistent at key positions. If multiple high-confidence paths correspond to different character hypotheses at the same time step or in the same image region, it is judged that the candidate recognition results are uncertain. If at least one candidate identification result in the first, second, and third statistical analyses is determined to be uncertain, then the identification result of the current signature field is marked as uncertain, thus triggering the prior constraint of the field semantics.

[0088] As an optional implementation, the construction of prior knowledge of the prior constraints of the field semantics includes: Contextual information is extracted from the document image, and a list of historically relevant personnel is queried from the business database associated with the document image metadata. The contextual information and the list of personnel are then standardized to obtain an original set of candidate signatures. Each signature candidate, each character, and the source type from different sources in the original signature candidate set are treated as a heterogeneous graph node. Multiple edge relationships are defined to connect the heterogeneous graph nodes to create a prior heterogeneous graph. A graph network layer is used to perform multi-round message passing on the prior heterogeneous graph. In each layer, the node aggregates the feature information of the neighboring nodes, and each signature candidate node obtains a refined representation vector that integrates graph structure information and multi-source context. The visual context feature vector is obtained by encoding the handwritten image corresponding to the nondeterministic signature field. The similarity score between the visual context feature vector and the refined representation vector is calculated. Each signature candidate in the original candidate set is sorted according to the similarity score, and the top-M most relevant signature candidates are selected to obtain the prior constraint set of the signature candidates.

[0089] As an optional implementation, prior constraints on field semantics are applied to nondeterministic candidate identification results, including...

[0090] For each signature candidate character in the prior constraint set, a prefix tree is converted through the character-to-numerical index mapping, and all prefix trees are integrated into the decoding process of generating multiple candidate recognition results corresponding to the signature field; Based on the decoding process, any decoding path retains or expands the corresponding candidate recognition result only when matching the character transitions allowed by the prefix tree, so as to form a priori constraints on the handwritten signature recognition result to correct and adjust the candidate recognition result.

[0091] As an optional implementation, applying prior constraints on field semantics to nondeterministic candidate identification results also includes: The character probability of each signature candidate in the prior constraint set is used as the text recognition score. During the decoding process of generating multiple candidate recognition results corresponding to the signature field, the text recognition score and the visual recognition score are interpolated. Based on the interpolation calculation, the decoding process is guided to output character candidates similar to the prior constraint set, so as to form prior constraints for the handwritten signature recognition result and to correct and adjust the candidate recognition result.

[0092] As an optional implementation, the candidate recognition results adjusted based on prior constraints are weighted and ranked to output the target recognition result of the handwritten signature, including: Calculate the string edit distance between each candidate recognition result and each signature candidate in the prior constraint set to obtain the matching score corresponding to the candidate recognition result; Calculate the weighted average of the confidence level and the matching score corresponding to each candidate recognition result to obtain the comprehensive score corresponding to the candidate recognition result. The weight of the weighted average is dynamically adjusted according to the degree of certainty of the candidate recognition result corresponding to the document image context and the historical recognition accuracy. Based on the comprehensive score, the candidate recognition results are rearranged in descending order from high to low, and the first candidate signature after rearrangement is selected as the target recognition result of the handwritten signature in the document image.

[0093] Example 3 Please see Figure 3 , Figure 3 This is another handwritten signature recognition system based on field semantic prior disclosed in the embodiments of the present invention. Figure 3 The described handwritten signature recognition system based on field semantic prior is applied in data processing systems / data processing equipment / data processing servers (wherein, the server includes local processing servers or cloud processing servers). Figure 3 As shown, the handwritten signature recognition system based on field semantic priors may include: Memory 301 storing executable program code; Processor 302 coupled to memory 301; The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the handwritten signature recognition method based on field semantic prior described in Embodiment 1.

[0094] Example 4 This invention discloses a computer read storage medium that stores a computer program for electronic data interchange, wherein the computer program causes a computer to execute the steps of the handwritten signature recognition method based on field semantic prior described in Embodiment 1.

[0095] Example 5 This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps of the handwritten signature recognition method based on field semantic prior described in Embodiment 1.

[0096] The foregoing has described specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily have to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0097] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0098] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0099] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0101] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0102] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0103] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0104] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0105] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0106] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0107] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0108] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0109] Finally, it should be noted that the handwritten signature recognition method and system based on field semantic prior disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A handwritten signature recognition method based on field semantic prior, characterized in that, The method includes: Obtain a document image to be parsed, identify at least one target field region in the document image to determine the corresponding field semantics, wherein the document image includes at least one field region for handwritten signature; Image feature extraction and text recognition are performed on the target field region whose semantics are determined to be a signature field, generating multiple candidate recognition results and confidence distributions corresponding to the signature field; Based on the confidence distribution, the certainty of the current candidate identification result is determined by statistical analysis, and prior constraints of field semantics are applied to the non-deterministic candidate identification result. The candidate recognition results adjusted based on prior constraints are weighted and sorted to output the target recognition result of the handwritten signature.

2. The handwritten signature recognition method based on field semantic prior as described in claim 1, characterized in that, Obtain the document image to be parsed, identify at least one target field region in the document image to determine the corresponding field semantics, including: For each document image, it is input into a preset recognition model, which is a dual-stream encoder architecture, including a text stream encoder and a visual stream encoder; The text stream encoder encodes the text sequence identified from the field regions of the document image to obtain a text semantic vector; The visual flow encoder extracts features from the image blocks corresponding to the field region and the context region to obtain a visual feature vector. The visual feature vector includes the field position, field size, and layout of the relative relationships between elements in the document. The text semantic vector and visual feature vector are fused using an attention mechanism or feature concatenation mechanism. The resulting fused features are input into the classification head, and the output is the probability distribution of all fields in the target field region corresponding to the predefined semantic categories. The field semantics of the target field region are uniquely determined based on the probability distribution, and the field semantics include at least one signature field.

3. The handwritten signature recognition method based on field semantic prior as described in claim 1, characterized in that, Image feature extraction and text recognition are performed on the target field region whose semantics are determined to be a signature field, generating multiple candidate recognition results and confidence distributions corresponding to the signature field, including: A handwritten image of the region corresponding to the signature field is collected separately and input into a preset neural network model to extract a visual feature sequence; The visual feature sequence is input into a preset long short-term memory network to model the contextual dependencies between characters. The modeled visual feature sequence is then mapped to a character probability sequence through a decoding layer connected to the long short-term memory network. The sequence generation task of the decoding layer uses a beam search algorithm to retain the top-K optimal paths. Each path corresponds to a character probability sequence and outputs a candidate recognition string, generating multiple candidate recognition results. For each candidate recognition string, the geometric mean of the predicted character probabilities at each time step along its corresponding path is calculated to obtain the confidence level of the candidate recognition string, thus generating the confidence level distribution of the candidate recognition results.

4. The handwritten signature recognition method based on field semantic prior as described in claim 1, characterized in that, Based on the confidence distribution, the certainty of the current candidate identification result is determined through statistical analysis, including: The first statistical analysis uses the confidence difference method to calculate the confidence difference between multiple candidate recognition results. If the confidence difference between the highest confidence and the second highest confidence is less than the preset difference threshold, it is determined that the candidate recognition result is uncertain. The second statistical analysis uses the distribution entropy method to calculate the Shannon entropy value of the confidence distribution of the candidate recognition results. If the uncertainty measure of the Shannon entropy value is greater than the preset discrete threshold, it is determined that the candidate recognition results are uncertain. The third statistical analysis uses a decoding path consistency test to analyze whether the character alignment of the decoding path corresponding to the candidate output results of the decoding layer is consistent at key positions. If multiple high-confidence paths correspond to different character hypotheses at the same time step or in the same image region, it is judged that the candidate recognition results are uncertain. If at least one candidate identification result in the first, second, and third statistical analyses is determined to be uncertain, then the identification result of the current signature field is marked as uncertain, thus triggering the prior constraint of the field semantics.

5. The handwritten signature recognition method based on field semantic prior as described in claim 1, characterized in that, The construction of prior knowledge of the prior constraints of the field semantics includes: Contextual information is extracted from the document image, and a list of historically relevant personnel is queried from the business database associated with the document image metadata. The contextual information and the list of personnel are then standardized to obtain an original set of candidate signatures. Each signature candidate, each character, and the source type from different sources in the original signature candidate set are treated as a heterogeneous graph node. Multiple edge relationships are defined to connect the heterogeneous graph nodes to create a prior heterogeneous graph. A graph network layer is used to perform multi-round message passing on the prior heterogeneous graph. In each layer, the node aggregates the feature information of the neighboring nodes, and each signature candidate node obtains a refined representation vector that integrates graph structure information and multi-source context. The visual context feature vector is obtained by encoding the handwritten image corresponding to the nondeterministic signature field. The similarity score between the visual context feature vector and the refined representation vector is calculated. Each signature candidate in the original candidate set is sorted according to the similarity score, and the top-M most relevant signature candidates are selected to obtain the prior constraint set of the signature candidates.

6. The handwritten signature recognition method based on field semantic prior as described in claim 5, characterized in that, Prior constraints on field semantics are applied to non-deterministic candidate identification results, including...

7. For each signature candidate character in the prior constraint set, convert it into a prefix tree through the character-to-numerical index mapping, and integrate all prefix trees into the decoding process of generating multiple candidate recognition results corresponding to the signature field; Based on the decoding process, any decoding path retains or expands the corresponding candidate recognition result only when matching the character transitions allowed by the prefix tree, so as to form a priori constraints on the handwritten signature recognition result to correct and adjust the candidate recognition result.

8. The handwritten signature recognition method based on field semantic prior as described in claim 5, characterized in that, Applying prior constraints on field semantics to nondeterministic candidate identification results also includes: The character probability of each signature candidate in the prior constraint set is used as the text recognition score. During the decoding process of generating multiple candidate recognition results corresponding to the signature field, the text recognition score and the visual recognition score are interpolated. Based on the interpolation calculation, the decoding process is guided to output character candidates similar to the prior constraint set, so as to form prior constraints for the handwritten signature recognition result and to correct and adjust the candidate recognition result.

9. The handwritten signature recognition method based on field semantic prior as described in claim 6 or 7, characterized in that, The candidate recognition results adjusted based on prior constraints are weighted and ranked to output the target recognition results of the handwritten signature, including: Calculate the string edit distance between each candidate recognition result and each signature candidate in the prior constraint set to obtain the matching score corresponding to the candidate recognition result; Calculate the weighted average of the confidence level and the matching score corresponding to each candidate recognition result to obtain the comprehensive score corresponding to the candidate recognition result. The weight of the weighted average is dynamically adjusted according to the degree of certainty of the candidate recognition result corresponding to the document image context and the historical recognition accuracy. Based on the comprehensive score, the candidate recognition results are rearranged in descending order from high to low, and the first candidate signature after rearrangement is selected as the target recognition result of the handwritten signature in the document image.

10. A handwritten signature recognition system based on field semantic prior, characterized in that, The system includes: The acquisition module is used to acquire a document image to be parsed, identify at least one target field region in the document image to determine the corresponding field semantics, and the document image includes at least one field region for handwritten signature; The recognition module is used to extract image features and recognize text in the target field region whose semantics are determined to be a signature field, and generate multiple candidate recognition results and confidence distributions corresponding to the signature field. The prior module is used to determine the certainty of the current candidate identification result through statistical analysis based on the confidence distribution, and to apply prior constraints of field semantics to the non-deterministic candidate identification result; The output module is used to weight and sort the candidate recognition results based on prior constraints, and output the target recognition result of the handwritten signature.

11. A handwritten signature recognition system based on field semantic prior, characterized in that, The system includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the handwritten signature recognition method based on field semantic prior as described in any one of claims 1-8.

Images