Handwriting recognition method, computer device and readable storage medium

Abstract

The application discloses a handwriting recognition method, a computer device and a readable storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: in response to the input of a first multi-modal feature and a historical handwriting template of a target author, a machine learning model is called to generate a recognition result of whether a to-be-recognized author is the target author; wherein the historical handwriting template comprises local stability of handwriting corresponding to a plurality of local writing units and a second multi-modal feature, and the local writing units comprise a character, a partial component or a character block; the local stability is used to generate a local attention weight corresponding to each local writing unit, and the local attention weight is used to perform weighted processing on the similarity of the first multi-modal feature and the second multi-modal feature during inference of the machine learning model. The application can improve the accuracy of recognizing whether the author is the target author.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a handwriting recognition method, computer equipment, and readable storage medium. Background Technology

[0002] With the popularization of information-based teaching and digital homework management models, the collection, submission, grading, and archiving of student assignments are gradually becoming electronic. However, in actual teaching management, phenomena such as having others write assignments for students or imitating handwriting exist, affecting the authenticity of assignment evaluation and the effectiveness of teaching management.

[0003] In traditional techniques, deep learning-based identification methods have been proposed to determine whether student assignments are written by the student themselves. However, these methods often struggle to distinguish between the natural variations in a student's handwriting and deliberate imitation by another person, resulting in a high false positive rate. Summary of the Invention

[0004] The purpose of this application is to provide a handwriting recognition method, computer device, and readable storage medium that can improve the accuracy of identifying whether an author is the target author.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a handwriting recognition method, the method comprising: In response to the input of the first multimodal feature and the target author's historical handwriting template, a machine learning model is invoked to generate an identification result as to whether the author to be identified is the target author; The historical handwriting template includes local stability and second multimodal features corresponding to multiple local writing units. The local writing units include characters, radicals, or character blocks. The local stability is used to generate local attention weights corresponding to each local writing unit. The local attention weights are used to perform weighted processing on the similarity between the first multimodal features and the second multimodal features during the inference of the machine learning model. The first multimodal features are extracted from the first local writing unit image, which is obtained by segmenting a first image. The first image includes the text written by the author to be identified, and each first local writing unit image corresponds to one local writing unit.

[0006] Secondly, embodiments of this application provide a handwriting recognition device, the handwriting recognition device comprising: The module is used to invoke a machine learning model in response to the input of the first multimodal feature and the target author's historical handwriting template to generate an identification result as to whether the author to be identified is the target author; The historical handwriting template includes local stability and second multimodal features corresponding to multiple local writing units. The local writing units include characters, radicals, or character blocks. The local stability is used to generate local attention weights corresponding to each local writing unit. The local attention weights are used to perform weighted processing on the similarity between the first multimodal features and the second multimodal features during the inference of the machine learning model. The first multimodal features are extracted from the first local writing unit image, which is obtained by segmenting a first image. The first image includes the text written by the author to be identified, and each first local writing unit image corresponds to one local writing unit.

[0007] Thirdly, embodiments of this application provide a computer device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0010] In this embodiment, by introducing historical handwriting templates into the recognition process, these templates include the local stability and second multimodal features of handwriting corresponding to multiple local writing units. This achieves modeling of the long-term stable local writing habits of the same author, rather than focusing solely on overall character shape similarity. Therefore, when determining whether a student's work is written by the student themselves, it is possible to accurately distinguish between the natural fluctuations in the student's handwriting and deliberate imitation by others, thereby improving the accuracy of identifying whether the author is the target author. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating some embodiments of the handwriting recognition method provided in this application; Figure 2 This is a structural block diagram of a handwriting recognition device provided in some embodiments of this application; Figure 3 These are internal structural diagrams of a computer device provided in some embodiments of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0014] The embodiments of this application can be applied to a variety of application scenarios, such as teaching scenarios.

[0015] In one exemplary embodiment, this application proposes a handwriting recognition method. The handwriting recognition method provided by this application will be described in detail below with reference to the accompanying drawings and specific embodiments and application scenarios.

[0016] Reference Figure 1 The method includes step 102. Wherein: Step 102: In response to the input of the first multimodal feature and the target author's historical handwriting template, a machine learning model is invoked to generate an identification result as to whether the author to be identified is the target author.

[0017] The historical handwriting template includes local stability and second multimodal features corresponding to multiple local writing units. The local writing units include characters, radicals, or character blocks. The local stability is used to generate local attention weights corresponding to each local writing unit. The local attention weights are used to perform weighted processing on the similarity between the first multimodal features and the second multimodal features during the inference of the machine learning model. The first multimodal features are extracted from the first local writing unit image, which is obtained by segmenting a first image. The first image includes the text written by the author to be identified, and each first local writing unit image corresponds to one local writing unit.

[0018] The identification result is obtained by calculating the score during the reasoning process of the machine learning model.

[0019] In some embodiments, the identification result is based on the final score calculated during machine learning model inference. The final score is generated. It includes at least a local matching score, which is calculated by weighting the local similarity between the first multimodal feature and the second multimodal feature using local attention weights.

[0020] In some embodiments, when using similarity output, the final score is... Compare with the threshold τ; when When the target author is identified, it can be determined that the author to be identified is the target author; otherwise, it can be determined that the author to be identified is not the target author. For example, in a teaching scenario, when... If the current assignment is written by the target student, it is determined that the assignment was written by the target student himself; otherwise, it is determined that it was not written by the student himself.

[0021] In some embodiments, when using the classifier output, the final score can also be... The binary classification result and its reliability are directly output by concatenating the local attention weight distribution statistics with the classification header.

[0022] In some embodiments, a historical handwriting template refers to a template formed through statistical modeling to characterize the long-term stable writing habits and natural fluctuation range of a target author.

[0023] In some embodiments, the machine learning model may be implemented using a Siamese network, a contrastive learning network, an attention network, or a combination of networks containing the above structures.

[0024] The following explains the process of generating historical handwriting templates: In some embodiments, the method further includes: Acquire multiple second images; wherein the second images include text written by the target author.

[0025] Each of the second images is divided into multiple second local writing unit images.

[0026] Based on all the second local writing unit images, the local stability and second multimodal features of the handwriting corresponding to each local writing unit are extracted respectively.

[0027] In some embodiments, the second multimodal feature includes a second macroscopic character shape feature, a second pen pressure distribution feature, and a second dynamic inversion feature.

[0028] Wherein, the second macroscopic glyph feature characterizes the character structure layout, stroke distribution, center of gravity position, overall contour and spatial form, and can be extracted by a convolutional network, a Vision Transformer or an encoding network combining the two. The second pen pressure distribution feature is obtained from a static image through a pressure estimation model, and is used to characterize the relative weight changes of different stroke regions. The pressure estimation model can output a local pressure heat map, a pressure distribution vector or a corresponding embedding feature. The second dynamic inversion feature is inferred from a static image through a dynamic inversion model, and the dynamic inversion model may adopt an encoder-decoder structure for outputting at least one of a writing speed change feature, a pressure change feature, or a stroke generation sequence feature.

[0029] In some embodiments, the second image may be a historical handwritten text image of the target author, for example, a historical handwritten homework image of a target student.

[0030] In some embodiments, the second image may include, but is not limited to: a scanned image of paper homework, a photographed image, or an image exported from electronic writing.

[0031] In some embodiments, the number of the second images may be no less than 10, or no less than 20, so as to fully reflect the individual writing habits of the target author under different times and different writing states.

[0032] In some embodiments, after acquiring a plurality of second images, the images need to be preprocessed, and the preprocessing process may include, but is not limited to: page correction, perspective correction, grayscale normalization, denoising, binarization, text region localization, and character-level, radical-component-level or character-block-level segmentation and alignment. After the character-level, radical-component-level or character-block-level segmentation and alignment processing, a second local writing unit image can be obtained.

[0033] In some embodiments, a character block refers to a text region divided in the second image according to a spatial layout or a semantic position.

[0034] In some embodiments, for the second local writing unit image, that is, the segmented character image, radical component image or character block image, a modality quality can also be generated, and the modality quality may include, but is not limited to: alignment confidence, clarity score and completeness score, which are used for subsequent modality weight generation.

[0035] It can be understood that a local writing unit refers to the same character, the same radical component or the same character block. Therefore, two different second local writing unit images may correspond to the same local writing unit, for example, both of the two second local writing unit images correspond to the character "write", but come from different second images respectively.

[0036] In some embodiments, an initial feature can be extracted from each second local writing unit image obtained after preprocessing . Herein, represents the th second image, represents a local writing unit, represents a modality. Herein, the local writing unit may be at a character level, such as the Chinese character "写"; may be at a radical and component level, such as the radical "亻"; and may be at a character block level, such as the phrase "写你名字". Herein, the modality may refer to three types: macroscopic glyph, pen pressure distribution and dynamic inversion.

[0037] after extraction from each second local writing unit image to obtain , for each local writing unit and each modality, statistical calculation is performed on all , so as to obtain a second multi-modal feature (i.e., a second mean feature) and a local stability.

[0038] In some embodiments, for a first multi-modal feature, a first image is an image to be recognized. For example, when an author to be recognized is a student to be recognized, the first image may be an image corresponding to a current assignment. The first multi-modal feature may be extracted from the first local writing unit image corresponding to any local writing unit, without performing statistical calculation.

[0039] In some embodiments, the local stability is calculated from a local fluctuation degree, and a smaller local fluctuation degree indicates a higher local stability.

[0040] In some embodiments, the local fluctuation degree refers to the degree of dispersion of a corresponding local writing unit in a certain second image among all second images, and may be represented by variance, standard deviation or other statistical quantities.

[0041] Herein, the second mean feature may be calculated by the following formula (1): (1) Herein, is the number of samples, that is, the number of second images.

[0042] Herein, the local fluctuation degree may be calculated by the following formula (2): (2) Herein, the local stability may be calculated by the following formula (3): (3) Herein, is a tiny constant for preventing the denominator from being zero.

[0043] In summary, historical handwriting templates can be composed of multiple template nodes arranged according to character order, layout position, or adjacency of writing units to form a historical template sequence or graph structure. Each template node is associated with the local stability and second multimodal features of the handwriting corresponding to a local writing unit. For example, template nodes... for: in, , , These represent three modes: macroscopic character shape, pen pressure distribution, and dynamic inversion, respectively. Representing modes In local writing units The trainable discriminative response parameters of the machine learning model described herein; Representing modes In local writing units The trainable discriminative response parameters of the machine learning model described herein; Representing modes In local writing units The trainable discriminative response parameters of the machine learning model described herein; Representing modes In local writing units The local stability of the machine learning model described in the text; Representing modes In local writing units The local stability of the machine learning model described in the text; Representing modes In local writing units The local stability of the machine learning model described in the text.

[0044] And due to It can be done Calculations show that when it is needed... In the case of, through In This can be obtained through calculation; therefore... It can also be: in, , , These represent three modes: macroscopic character shape, pen pressure distribution, and dynamic inversion, respectively. Representing modes In local writing units The trainable discriminative response parameters of the machine learning model described herein; Representing modes In local writing units The trainable discriminative response parameters of the machine learning model described herein; Representing modes In local writing units The trainable discriminative response parameters of the machine learning model described herein; Representing modes In local writing units The degree of local fluctuation in the machine learning model described above; Representing modes In local writing units The degree of local fluctuation in the machine learning model described above; Representing modes In local writing units The degree of local fluctuation in the machine learning model described in the text.

[0045] In some embodiments, It can also carry the alignment success rate or completeness rate of each local writing unit in multiple second images. ,Right now It can also be: in, , , These represent three modes: macroscopic character shape, pen pressure distribution, and dynamic inversion, respectively. Representing modes In local writing units The trainable discriminative response parameters of the machine learning model described herein; Representing modes In local writing units The trainable discriminative response parameters of the machine learning model described herein; Representing modes In local writing units The trainable discriminative response parameters of the machine learning model described herein; Representing modes In local writing units The degree of local fluctuation in the machine learning model described above; Representing modes In local writing units The degree of local fluctuation in the machine learning model described above; Representing modes In local writing units The degree of local fluctuation in the machine learning model described above; Representing a local writing unit Completeness rate in multiple second images.

[0046] In some embodiments, the It can be used as template node confidence information based on historical sample statistics. Indicates the current mode of the sample to be identified. The quality score is calculated below; when generating local attention weights, the following can be used: As prior confidence level and Used in combination, or utilized right Perform initialization or calibration.

[0047] In some embodiments, the historical handwriting template may also carry global features for calculating a global consistency score.

[0048] In some embodiments, the recognition result is generated based on a local matching score, wherein the local matching score It is calculated based on the following formula (4).

[0049] (4) Among them, the Representing a local writing unit The corresponding local attention weights; Indicates the mode in the second multimodal feature In local writing units Modal weights in; the Indicates the mode in the first multimodal feature In local writing units The first feature in; the Indicates the mode in the second multimodal feature In local writing units The second feature; This represents the similarity function.

[0050] In some embodiments, It can be a function of cosine similarity, negative Euclidean distance, or negative Mahalanobis distance.

[0051] In some embodiments, the It is calculated using the following formula (5): (5) Among them, the Indicated by An exponential function with base 0; the The index variable represents all local writing units; Representing a local writing unit Attention score; Representing a local writing unit Attention scoring. It's understandable that Formula 5 is a normalization operation.

[0052] Among them, the It is calculated using the following formula (6): (6) Among them, the , These are the query transformation matrix and the key transformation matrix, respectively, both of which are learnable parameters of the machine learning model; Represents the feature dimension; the Represents the fluctuation penalty coefficient; the This indicates that the first multimodal feature is in the local writing unit. The embedded aggregation results on; This indicates that the second multimodal feature is in the local writing unit. The embedded aggregation results on; Representing a local writing unit The local fluctuation aggregation value; It is calculated using the following formula (7): (7) Among them, the This indicates an embedded function.

[0053] The It is calculated using the following formula (8): (8) The It is calculated using the following formula (9): (9) Among them, the Indicates the mode in the second multimodal feature In local writing units The degree of local fluctuations.

[0054] It can be understood that, through the above formulas (4)-(9), the local attention weights can be adjusted. Control measures are implemented, meaning that regions with smaller fluctuations and more reliable current alignment are given a higher proportion during the comparison process. The greater the contribution.

[0055] In some embodiments, the It is calculated using the following formula (10): (10) Among them, the (Local stability weight) (Modal mass weights) and (Discriminative response weights) are the learnable parameters of the machine learning model; Indicates the mode in the second multimodal feature In local writing units Local stability in; the Representing modes In local writing units The quality score in the, The confidence level is obtained from the current sample's sharpness, segmentation completeness, alignment confidence, or feature extraction network confidence. Representing modes In local writing units The trainable discriminative response parameters of the machine learning model mentioned above; An index variable representing a modality, used to iterate through all modalities.

[0056] Formula (10) is a normalization operation. It can be understood that in Formula (10), "local fluctuation degree (local stability), modal quality, and current alignment response" are used together to generate local attention weights. Compared with using fixed weights or only using a single similarity to drive local comparison, it can highlight long-term stable and difficult-to-fake writing areas, thereby further improving the accuracy of identifying whether the author is the target author.

[0057] In some embodiments, modal weights refer to the relative contribution coefficients of different modalities, such as macroscopic character shape, pen pressure distribution, and dynamic inversion, during comparison in a certain local writing unit. Through formula (10), the contribution of different modalities in different local regions can be adaptively adjusted. Specifically, for missing, occluded, or poor-quality local units, the contribution can be adjusted using... The ranking will be lowered.

[0058] In some embodiments, the identification result is generated based on a final score, the final score Based on global consistency score The above And temporal coherence score It is calculated using the following formula (11): (11) Among them, the , and These are preset parameters or learnable parameters of the machine learning model, and .

[0059] In some embodiments, The similarity can be calculated as the global feature similarity. Specifically, a global mean feature can be extracted from multiple second images, providing the mean of the second global features for each second image; a first global feature can be extracted from the first image; the similarity of the global features is the similarity between the global mean feature and the first global feature. For example, if the image corresponds to an assignment, the global feature can be the feature of the entire page of assignment or the entire line or paragraph of writing.

[0060] In some embodiments, It can be obtained by comparing the consistency of word-by-word feature sequences, local template node sequences, or dynamic inversion sequences. Specifically, it can be implemented using Transformer, bidirectional gated recurrent unit network, or one-dimensional convolutional temporal network.

[0061] In some embodiments, the local attention weights are used to construct the loss function of the machine learning model.

[0062] Specifically, during the training phase, attention guided by local stability can be used to guide loss. This allows the locally learned attention weights to converge first towards local stability. Total loss The function expression is shown in formula (12).

[0063] (12) in, Indicates classification loss; Indicates the measurement of learning loss; This indicates a loss of attentional guidance; To measure learning loss The weighting coefficients, Loss of attention The weighting coefficients.

[0064] Among them, the It can be calculated using the following formula (13).

[0065] (13) in, Indicates a local writing unit; Representing a local writing unit The corresponding local attention weights; These are the prior attention weights calculated based on the degree of local fluctuation, used to guide model learning; (symbol) This represents the absolute value operation, used for measurement. and The differences between them.

[0066] The It can be calculated using the following formula (14).

[0067] (14) In some embodiments, by introducing historical handwriting templates into the recognition process, which include the local stability and second multimodal features of handwriting corresponding to multiple local writing units, the modeling of long-term stable local writing habits of the same author is achieved, rather than focusing solely on overall character shape similarity. This enables the accurate differentiation between natural fluctuations in a student's handwriting and deliberate imitation by others when determining whether a student's work is their own, thereby improving the accuracy of identifying whether the author is the target author.

[0068] In some embodiments, by statistically modeling multiple second images of the target author, an individualized historical handwriting template containing mean features, natural fluctuation range and template confidence information is constructed. Compared with traditional techniques that rely on only a small number of samples or overall similarity, this can more realistically depict the long-term stable writing habits of the same author.

[0069] In some embodiments, by modeling the mean and fluctuations of multiple historical instances of the same character, radical, or block, the normal writing differences of authors at different times and in different states can be included within an acceptable range, which is more adaptable to natural fluctuations and thus reduces the risk of misjudgment.

[0070] In some embodiments, not only are macroscopic character features analyzed, but pen pressure distribution features and dynamic inversion features are also integrated. Furthermore, the ability to identify cases such as disguised handwriting and substitute writing is improved through local attention comparison guided by local stability and temporal coherence comparison.

[0071] In some embodiments, by fusing global, local and temporal multi-level information and using local stability priors to constrain attention allocation, the system's discrimination accuracy and robustness under complex writing environments, sample fluctuations and imitation interference are improved.

[0072] In some embodiments, it can be applied to student daily homework identity verification, and can also be extended to exam answer sheet verification, signature-assisted verification, and other handwritten identity recognition scenarios.

[0073] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0074] Based on the same inventive concept, this application also provides a handwriting recognition device for implementing the handwriting recognition method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more handwriting recognition device embodiments provided below can be found in the limitations of the handwriting recognition method described above, and will not be repeated here.

[0075] In one exemplary embodiment, such as Figure 2 As shown, a handwriting recognition device is provided, including: a calling module 100, wherein: The module 100 is invoked in response to the input of the first multimodal features and the target author's historical handwriting template, to invoke the machine learning model and generate an identification result as to whether the author to be identified is the target author.

[0076] The historical handwriting template includes local stability and second multimodal features corresponding to multiple local writing units. The local writing units include characters, radicals, or character blocks. The local stability is used to generate local attention weights corresponding to each local writing unit. The local attention weights are used to perform weighted processing on the similarity between the first multimodal features and the second multimodal features during the inference of the machine learning model. The first multimodal features are extracted from the first local writing unit image, which is obtained by segmenting a first image. The first image includes the text written by the author to be identified, and each first local writing unit image corresponds to one local writing unit.

[0077] In one embodiment, the recognition result is generated based on a local matching score, wherein the local matching score Calculated based on the following formula: Among them, the Representing a local writing unit The corresponding local attention weights; Indicates the mode in the second multimodal feature In local writing units Modal weights in; the Indicates the mode in the first multimodal feature In local writing units The first feature in; the Indicates the mode in the second multimodal feature In local writing units The second feature; This represents the similarity function.

[0078] In one embodiment, the It is calculated using the following formula: Among them, the Indicated by An exponential function with base 0; the Represents all local writing units; the It is calculated using the following formula: Among them, the Indicated by An exponential function with base 0; the The index variable represents all local writing units; Representing a local writing unit Attention score; Representing a local writing unit Attention score; It is calculated using the following formula: Among them, the , These are the query transformation matrix and the key transformation matrix, respectively, both of which are learnable parameters of the machine learning model; Represents the feature dimension; the Represents the fluctuation penalty coefficient; the This indicates that the first multimodal feature is in the local writing unit. The embedded aggregation results on; This indicates that the second multimodal feature is in the local writing unit. The embedded aggregation results on; Representing a local writing unit The local fluctuation aggregation value; It is calculated using the following formula: Wherein, E(·) represents the embedding function; The It is calculated using the following formula: The It is calculated using the following formula: Among them, the Indicates the mode in the second multimodal feature In local writing units The degree of local fluctuations.

[0079] In one embodiment, the It is calculated using the following formula: Among them, the , and The learnable parameters of the machine learning model; Indicates the mode in the second multimodal feature In local writing units Local stability in; the Representing modes In local writing units The quality score in; Representing modes In local writing units The trainable discriminative response parameters of the machine learning model mentioned above; An index variable representing a modality, used to iterate through all modalities.

[0080] In one embodiment, the recognition result is generated based on a final score, the final score Based on global consistency score The above And temporal coherence score It is calculated using the following formula: Among them, the , and These are preset parameters or learnable parameters of the machine learning model, and .

[0081] In one embodiment, the second multimodal feature includes a second macroscopic character shape feature, a second pen pressure distribution feature, and a second dynamic inversion feature.

[0082] In one embodiment, the apparatus further includes: The acquisition module is used to acquire multiple second images; wherein the second images include text written by the target author.

[0083] The segmentation module is used to segment each of the second images into multiple second local writing unit images.

[0084] The extraction module is used to extract the local stability and second multimodal features of the handwriting corresponding to each of the second local writing units based on all the second local writing unit images.

[0085] In one embodiment, the local attention weights are used to construct the loss function of the machine learning model.

[0086] Each module in the aforementioned handwriting recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0087] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a handwriting recognition method.

[0088] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0089] In one embodiment, a computer-readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps in the above method embodiments.

[0090] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0091] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0092] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0093] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A handwriting recognition method, characterized in that, The handwriting recognition method includes: Acquire multiple second images; wherein the second images include text written by the target author; Each of the second images is divided into multiple second local writing unit images; Based on all the images of the second local writing units, the local stability and second multimodal features of the handwriting corresponding to each local writing unit are extracted respectively; wherein, the local writing unit includes characters, radicals or character blocks, and the second multimodal features include second macroscopic character shape features, second pen pressure distribution features and second dynamic inversion features, wherein the second pen pressure distribution features are obtained from the second image through a pressure estimation model; and the second dynamic inversion features are inferred from the second image through a dynamic inversion model. In response to the input of the first multimodal feature and the historical handwriting template of the target author, a machine learning model is invoked to generate an identification result as to whether the author to be identified is the target author; wherein, the historical handwriting template includes the local stability and the second multimodal feature; The local stability is used to generate local attention weights corresponding to each local writing unit. The local attention weights are used to weight the similarity between the first multimodal feature and the second multimodal feature during the inference of the machine learning model, and to construct the loss function of the machine learning model. The first multimodal feature is extracted from the first local writing unit image, which is obtained by segmenting a first image. The first image includes the text written by the author to be identified, and each first local writing unit image corresponds to one local writing unit. The loss function includes an attention-guided loss, which is used to make the local attention weights learned by the model converge to a prior attention distribution based on local stability.

2. The handwriting recognition method according to claim 1, characterized in that, The recognition result is generated based on the local matching score. Calculated based on the following formula: Among them, the Representing a local writing unit The corresponding local attention weights; Indicates the mode in the second multimodal feature In local writing units Modal weights in; the Indicates the mode in the first multimodal feature In local writing units The first feature in; the Indicates the mode in the second multimodal feature In local writing units The second feature; This represents the similarity function.

3. The handwriting recognition method according to claim 2, characterized in that, The w(r) is calculated using the following formula: Among them, the Indicated by An exponential function with base 0; the The index variable represents all local writing units; Representing a local writing unit Attention score; Representing a local writing unit Attention score; It is calculated using the following formula: Among them, the , These are the query transformation matrix and the key transformation matrix, respectively, both of which are learnable parameters of the machine learning model; Represents the feature dimension; the Represents the fluctuation penalty coefficient; the This indicates that the first multimodal feature is in the local writing unit. The embedded aggregation results on; This indicates that the second multimodal feature is in the local writing unit. The embedded aggregation results on; Representing a local writing unit The local fluctuation aggregation value; It is calculated using the following formula: Wherein, E(·) represents the embedding function; The It is calculated using the following formula: The It is calculated using the following formula: Among them, the Indicates the mode in the second multimodal feature In local writing units The degree of local fluctuations.

4. The handwriting recognition method according to claim 2, characterized in that, The It is calculated using the following formula: Among them, the , and The learnable parameters of the machine learning model; Indicates the mode in the second multimodal feature In local writing units Local stability in; the Representing modes In local writing units The quality score in; Representing modes In local writing units The trainable discriminative response parameters of the machine learning model mentioned above; An index variable representing a modality, used to iterate through all modalities.

5. The handwriting recognition method according to claim 2, characterized in that, The recognition result is generated based on the final score. Based on global consistency score The above And temporal coherence score It is calculated using the following formula: Among them, the , and These are preset parameters or learnable parameters of the machine learning model, and .

6. A computer device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the handwriting recognition method as described in any one of claims 1-5.

7. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the handwriting recognition method as described in any one of claims 1-5.

Images