A deep learning-based handwritten handwriting author verification system

Abstract

The application discloses a handwriting author verification system based on deep learning, comprising: a handwriting collection and processing module, which is used for collecting handwriting image, performing gray scale conversion, binarization, morphological denoising, size adjustment and training enhancement, generating handwriting training image set and model verification image set; a sample construction module, which is used for establishing same author sample relationship and different author sample relationship according to author identification and content identification, generating handwriting comparison sample set; a feature extraction module, which is used for extracting handwriting visual features by using ConvNeXt structure with removed classification output layer, generating identity feature vector through global feature mapping, local attention feature mapping and fusion processing; a feature optimization module, which is used for optimizing model parameters by using supervised contrast loss, semi-hard sample contrast loss and local consistency loss; a verification feature generation module, which is used for generating identity feature vectors to be verified and registered; a similarity calculation module, which is used for calculating cosine similarity of two vectors; and an author verification output module, which is used for comparing similarity and author verification threshold and outputting verification result. The application relates to the technical field of image processing and biometric recognition, realizes efficient handwriting author verification, and improves verification accuracy and adaptability to complex scenes through content debiasing sampling, double-head feature fusion and difficult sample mining.

Description

Technical Field

[0001] This invention relates to the fields of image processing and biometric recognition technology, and in particular to a deep learning-based handwriting author verification system. Background Technology

[0002] Existing handwriting author verification methods mostly employ manually designed stroke widths, tilt angles, connected regions, texture distributions, and contour structures as recognition features, and then use support vector machines, distance metrics, or neural networks to determine the author. Some methods use convolutional neural networks as classification models, outputting classification results based on pre-set author categories in the training set; others use paired samples or triplet samples to train feature extraction networks, and then complete handwriting comparison based on feature distances.

[0003] Handwriting is affected by writing speed, writing tools, character content, image quality, and writing state. Different handwriting images of the same author exhibit significant morphological variations, and local strokes from different authors often show similar structures. Models trained using classification output layers tend to rely on training author identifiers, lacking stable identity feature representations for authors not involved in training. In training batches formed by ordinary random sampling methods, easily distinguishable samples account for a high proportion, while samples from dissimilar authors with similar features and samples from the same author with significant differences are not fully involved in parameter optimization, resulting in a loose distribution of same-author features and overlapping distributions of dissimilar-author features in the author verification feature space. Within the same content area identified by page numbers and line numbers, handwriting images from different authors show highly similar structures, easily forming difficult-to-distinguish negative samples from dissimilar authors. Simultaneously, a single global feature is insufficient to distinguish local stroke details, failing to capture subtle writing habits. Existing methods also suffer from inconsistent feature measurement methods during training and similarity calculation methods during verification, and a lack of clearly defined procedures for author verification thresholds, affecting the stability of author consistency verification results. Therefore, providing a deep learning-based handwriting author verification system is a problem urgently needing to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose a deep learning-based handwriting author verification system. This invention combines ConvNeXt feature extraction, dual-head feature fusion, supervised contrastive learning, semi-difficult sample screening, and author-level difficult sample mining to perform identity feature extraction and similarity determination on paired handwriting images. It has the advantages of strong feature discrimination, high verification accuracy, and good adaptability.

[0005] A deep learning-based handwriting author verification system according to an embodiment of the present invention includes:

[0006] The handwriting acquisition and processing module is used to acquire handwritten handwriting images, perform preprocessing, and generate handwriting training image sets and model verification image sets;

[0007] The sample construction module is used to establish same-author and different-author sample relationships based on the author identifier and content identifier corresponding to the handwriting training image, and generate a handwriting comparison sample set.

[0008] The feature extraction module is used to input the handwriting comparison sample set into the handwriting author feature extraction model with the ConvNeXt structure, remove the classification output layer in the ConvNeXt structure, retain the feature extraction layer, and perform global feature mapping, local feature mapping and fusion processing on the visual features to generate handwriting identity feature vectors.

[0009] The feature optimization module is used to establish an author verification feature space based on the handwriting identity feature vector, perform feature distance constraints on handwriting sample pairs of the same author, perform feature distance separation constraints on handwriting sample pairs of different authors, optimize model parameters using supervised contrast loss, semi-hard sample contrast loss and local consistency loss, and generate an optimized handwriting author feature extraction model.

[0010] The verification feature generation module is used to input the paired handwriting images to be verified and the registered handwriting images into the optimized handwriting author feature extraction model to generate the feature vector of the identity to be verified and the feature vector of the registered identity.

[0011] The similarity calculation module is used to calculate the feature similarity between the feature vector of the identity to be verified and the feature vector of the registered identity based on the handwriting identity feature vector set;

[0012] The author verification output module is used to compare feature similarity with the author verification threshold and generate author consistency verification results based on the comparison results.

[0013] Optionally, the operation of generating the handwriting training image set and the model validation image set includes:

[0014] Acquire the original handwritten handwriting image, perform grayscale conversion on the original handwritten handwriting image, and generate a grayscale handwriting image;

[0015] Binarization is performed on the grayscale handwriting image to distinguish the handwriting area from the background area, generating a binary handwriting image;

[0016] Perform morphological denoising on the binary handwriting image to generate a denoised handwriting image;

[0017] Perform resizing on the denoised handwriting image to generate a standardized handwriting image;

[0018] The single-channel image of the standardized handwriting image is copied into a three-channel image, and pixel normalization is performed.

[0019] While the model is in training mode, pre-defined training enhancement processing is performed on the standardized handwriting images;

[0020] According to the preset data partitioning rules, the standardized handwriting images are divided into the handwriting training image set and the model validation image set, with the author identifier as the partitioning unit, while retaining the author identifier corresponding to the standardized handwriting images.

[0021] Optionally, the operation of generating the handwriting comparison sample set includes:

[0022] The handwriting training images in the handwriting training image set are grouped according to the author identifier to form a handwriting training image group corresponding to each author. The page number and line number are parsed from the path information of the handwriting training images to form the content identifier corresponding to each handwriting training image.

[0023] Select a preset number of authors from the handwriting training image groups corresponding to different authors, select a preset number of handwriting training images from the handwriting training image groups corresponding to each selected author, and combine the selected handwriting training images into a handwriting training batch.

[0024] When selecting handwriting training images, priority is given to handwriting training images corresponding to content identifiers shared by multiple authors, and handwriting training images with significant differences in page numbers or line numbers are given priority when the author identifiers are the same.

[0025] Establish same-author sample relationship between any handwriting training image in the handwriting training batch and other handwriting training images with the same author identifier, and establish different-author sample relationship between any handwriting training image with a different author identifier;

[0026] The handwriting training images that establish the relationship between samples from the same author and samples from different authors are collected into a handwriting comparison sample set.

[0027] Optionally, the operation of generating handwriting identity feature vectors using the handwriting author feature extraction model with the ConvNeXt structure includes:

[0028] The ConvNeXt structure with pre-trained weight configuration is adopted. The classification output layer after the global average pooling layer is removed, while the initial convolutional layer, four feature extraction stages and the global average pooling layer are retained.

[0029] The output of the global average pooling layer is connected to the global feature mapping process, and the output of the fourth feature extraction stage is connected to the local feature mapping process. The global feature mapping result and the local feature mapping result are concatenated and then fused to generate the identity feature vector.

[0030] At the start of training, the gradient update state of the initial convolutional layer and the parameters in the first three feature extraction stages is turned off, and the fourth feature extraction stage and the feature mapping process are set to trainable state.

[0031] At the start of the preset unfreezing round, the fixed state of the second and third feature extraction stages is released, and only the model parameters of the initial convolutional layer and the first feature extraction stage are fixed, and the optimizer is rebuilt.

[0032] Optionally, the operations of global feature mapping, local feature mapping, and fusion processing include:

[0033] The global feature mapping process connects the flattening layer, the first fully connected layer, the first one-dimensional batch normalization layer, the GELU activation layer, the random deactivation layer, the second fully connected layer, and the second one-dimensional batch normalization layer sequentially at the output of the global average pooling layer.

[0034] The local feature mapping process performs attention-weighted spatial aggregation on the feature map output from the fourth feature extraction stage, and then sequentially connects the third fully connected layer, the third one-dimensional batch normalization layer, the GELU activation layer, the random deactivation layer, the fourth fully connected layer, and the fourth one-dimensional batch normalization layer after the aggregation result.

[0035] The fusion process concatenates the global feature mapping results with the local feature mapping results, maps them to identity mapping features through the fifth fully connected layer and the fifth one-dimensional batch normalization layer, and normalizes each feature component according to the L2 norm of the identity mapping features to obtain the handwriting identity feature vector.

[0036] Optionally, the operation of establishing the author verification feature space includes:

[0037] Arrange the handwriting identity feature vectors according to the handwriting training batch, and maintain the correspondence between the handwriting identity feature vectors and the author identifier;

[0038] Establish same-author identity feature pairs based on the relationship between same-author samples, and establish different-author identity feature pairs based on the relationship between different-author samples;

[0039] Calculate the cosine similarity between two handwriting feature vectors in the same author identity feature pair and the different author identity feature pair respectively, and use the difference between 1 and the cosine similarity as the corresponding feature distance;

[0040] The author verification feature space is composed of the feature vectors of each handwriting identity feature, the feature distances corresponding to the same author identity feature pairs, and the feature distances corresponding to different author identity feature pairs.

[0041] Optionally, the operation of performing feature constraints using supervised contrastive loss includes:

[0042] The handwriting identity feature vector in the author verification feature space is determined as the anchor vector, the handwriting identity feature vector that establishes the same author identity feature pair with the anchor vector is determined as the positive sample vector, and the handwriting identity feature vector that establishes the opposite author identity feature pair with the anchor vector is determined as the negative sample vector.

[0043] Calculate the feature similarity between the anchor vector and the positive sample vector and the negative sample vector respectively, and scale each feature similarity according to a preset temperature coefficient;

[0044] The scaled feature similarity of all positive sample vectors corresponding to the same anchor vector is summarized, and the summarized result is subjected to exponential normalization and negative logarithmic processing with the scaled feature similarity of all non-self sample vectors.

[0045] Anchor vectors that correspond to at least one positive sample vector are identified as valid anchor vectors. The negative logarithm processing results corresponding to each valid anchor vector in the handwriting training batch are summarized and the average value is calculated to obtain the supervised contrast loss.

[0046] Optionally, the operation of optimizing model parameters using semi-hard sample contrast loss and local consistency loss includes:

[0047] For each valid anchor vector, the handwriting identity feature vector with the lowest feature similarity among the corresponding positive sample vectors is determined as the difficult positive sample vector;

[0048] Negative sample vectors whose feature similarity is between the difference between the feature similarity corresponding to the difficult positive sample vector and the preset interval threshold and the feature similarity corresponding to the difficult positive sample vector are determined as a semi-difficult negative sample candidate set. A preset number of negative sample vectors with the highest feature similarity are selected from the semi-difficult negative sample candidate set. When the semi-difficult negative sample candidate set is empty, all negative sample vectors are selected to obtain the difficult negative sample vector set.

[0049] The feature similarity of the corresponding features of the difficult positive sample vector is subtracted from the sum of the average feature similarity of the corresponding features of the difficult negative sample vector set and a linear correction constraint is applied to form the difficult sample contrast loss.

[0050] Based on the author-level difficult sample map, author-level difficult sample mining is performed between handwriting training batches, and local consistency constraints are applied to handwriting training samples of the same author based on the local feature mapping results, forming a local consistency loss.

[0051] Based on the preset hardness loss weight and preset local consistency loss weight, the hard sample comparison loss and local consistency loss are combined with the supervised comparison loss to form a joint comparison loss. The model parameters in the handwriting author feature extraction model that are in a trainable state are updated according to the joint comparison loss.

[0052] Optionally, the operation of generating the identity feature vector to be verified and the registration identity feature vector includes:

[0053] Receive a pair of handwriting images to be verified and registered handwriting images, generate standardized handwriting images to be verified and standardized handwriting images to be registered according to the same preprocessing process as the model training images, and do not perform training enhancement during the preprocessing process;

[0054] Load the optimized handwriting author feature extraction model and switch to the evaluation state. Turn off the gradient calculation of the model parameters. Input the standardized handwriting image to be verified and the standardized handwriting image to be registered into the handwriting author feature extraction model using the same set of model parameters to generate the feature vector of the identity to be verified and the feature vector of the registered identity.

[0055] The feature vectors of the identity to be verified and the feature vectors of the registered identity are associated according to the pairwise correspondence.

[0056] Optionally, the operation of calculating feature similarity and generating author consistency verification results includes:

[0057] Based on the pairwise correspondence, obtain the identity feature vector to be verified and the registered identity feature vector from the handwriting identity feature vector set;

[0058] Perform cosine similarity calculation on the feature vector of the identity to be verified and the feature vector of the registered identity to obtain the corresponding feature similarity;

[0059] Retrieve a pre-determined author verification threshold and compare the feature similarity with the author verification threshold;

[0060] When the feature similarity is not lower than the author verification threshold, the corresponding image pair is marked as having the same author; when the feature similarity is lower than the author verification threshold, the corresponding image pair is marked as having different authors.

[0061] Image pairs, feature similarity, and corresponding labels are associated to generate author consistency verification results.

[0062] The beneficial effects of this invention are:

[0063] (1) The present invention uses the ConvNeXt structure that removes the classification output layer to extract handwriting visual features, describes the overall writing style of the author through global feature mapping, strengthens the details of local strokes through local attention feature mapping, and generates handwriting identity feature vectors through fusion and L2 normalization, thereby enhancing the ability to express identity features under different writing content and writing conditions.

[0064] (2) This invention uses supervised contrast loss to constrain the feature distribution of samples from the same author and samples from different authors, selects semi-difficult negative samples from the training batch and performs author-level difficult sample mining, reduces the feature interval between the same author and expands the feature interval between different authors, especially improves the problem of difficulty in distinguishing samples from different authors in the same content area, and enhances the ability to distinguish between similar handwriting.

[0065] (3) In this invention, the same set of model parameters is used to perform feature extraction on the handwriting image to be verified and the registered handwriting image. The author consistency verification result is generated by cosine similarity and a clearly defined author verification threshold, so as to keep the feature measurement caliber consistent between the training stage and the verification stage and improve the accuracy and stability of the verification result. Attached Figure Description

[0066] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0067] Figure 1 This is a system architecture diagram of a deep learning-based handwriting author verification system proposed in this invention.

[0068] Figure 2 This is a flowchart of the handwriting preprocessing process for a deep learning-based handwriting author verification system proposed in this invention.

[0069] Figure 3 This is a flowchart of a handwriting feature extraction model for a deep learning-based handwriting author verification system proposed in this invention.

[0070] Figure 4 This is a flowchart of the handwriting author verification process for a deep learning-based handwriting author verification system proposed in this invention.

[0071] Figure 5 This is a flowchart illustrating the model training and optimization process of a deep learning-based handwriting author verification system proposed in this invention. Detailed Implementation

[0072] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0073] refer to Figures 1-5 A deep learning-based handwriting author verification system includes:

[0074] The handwriting acquisition and processing module is used to acquire handwritten handwriting images, perform preprocessing, and generate handwriting training image sets and model verification image sets;

[0075] The sample construction module is used to establish same-author and different-author sample relationships based on the author identifier and content identifier corresponding to the handwriting training image, and generate a handwriting comparison sample set.

[0076] The feature extraction module is used to input the handwriting comparison sample set into the handwriting author feature extraction model with the ConvNeXt structure, remove the classification output layer in the ConvNeXt structure, retain the feature extraction layer, and perform global feature mapping, local feature mapping and fusion processing on the visual features to generate handwriting identity feature vectors.

[0077] The feature optimization module is used to establish an author verification feature space based on the handwriting identity feature vector, perform feature distance constraints on handwriting sample pairs of the same author, perform feature distance separation constraints on handwriting sample pairs of different authors, optimize model parameters using supervised contrast loss, semi-hard sample contrast loss and local consistency loss, and generate an optimized handwriting author feature extraction model.

[0078] The verification feature generation module is used to input the paired handwriting images to be verified and the registered handwriting images into the optimized handwriting author feature extraction model to generate the feature vector of the identity to be verified and the feature vector of the registered identity.

[0079] The similarity calculation module is used to calculate the feature similarity between the feature vector of the identity to be verified and the feature vector of the registered identity based on the handwriting identity feature vector set;

[0080] The author verification output module is used to compare feature similarity with the author verification threshold and generate author consistency verification results based on the comparison results.

[0081] In this embodiment, the operations for generating the handwriting training image set and the model validation image set include:

[0082] Acquire the original handwritten handwriting image, perform grayscale conversion on the original handwritten handwriting image, and generate a grayscale handwriting image;

[0083] Binarization is performed on the grayscale handwriting image to distinguish the handwriting area from the background area, generating a binary handwriting image;

[0084] Perform morphological denoising on the binary handwriting image to generate a denoised handwriting image;

[0085] Perform resizing on the denoised handwriting image to generate a standardized handwriting image;

[0086] The single-channel image of the standardized handwriting image is copied into a three-channel image, and pixel normalization is performed.

[0087] While the model is in training mode, pre-defined training enhancement processing is performed on the standardized handwriting images;

[0088] According to the preset data partitioning rules, the standardized handwriting images are divided into the handwriting training image set and the model validation image set, with the author identifier as the partitioning unit, while retaining the author identifier corresponding to the standardized handwriting images.

[0089] The original handwritten handwriting images are those that have not undergone grayscale conversion, binarization, morphological denoising, or resizing. Each original handwritten handwriting image is associated with an author identifier, which remains unchanged during image preprocessing and dataset partitioning. Each handwritten handwriting image is a single-line text image. Handwriting training images and model validation images are partitioned according to the author identifier, and handwriting images corresponding to the same author identifier do not appear simultaneously in the training set, validation set, and test set.

[0090] Grayscale conversion merges multiple color channels at the same pixel location in the original handwritten image into a single-channel grayscale value, forming a grayscale handwritten image according to the original pixel arrangement. Binarization analyzes the grayscale value distribution in the grayscale handwritten image, uses the OTSU adaptive thresholding method to determine the binarization threshold, and assigns pixels on either side of the threshold as handwriting pixels and background pixels respectively, forming a binary handwritten image according to pixel location. Morphological denoising uses a 2×2 structuring element to perform opening operations on the binary handwritten image, i.e., sequentially performing erosion and dilation, removing isolated pixel regions that do not form continuous stroke structures from the handwriting region, and forming a denoised handwritten image according to the processed pixel distribution.

[0091] The resizing process scales the denoised handwriting image proportionally to its original aspect ratio, ensuring that the width and height of the scaled image do not exceed 224 pixels. The scaled image is then centered on a 224×224 pixel canvas, and background pixel values ​​are used to fill the un-scaled pixel positions, creating a standardized handwriting image. Non-uniform stretching, skew correction, and stroke width normalization are not performed during the resizing process. The single-channel image of the standardized handwriting image is then copied to a three-channel image, and pixel normalization is performed according to the mean and standard deviation of the pre-trained model.

[0092] The preset training enhancement process includes performing at least one of the following operations on the standardized handwriting image with preset probabilities: random rotation, random translation, random scaling, and random erasure. Training enhancement is only enabled during model training; it is not performed during model evaluation and author verification.

[0093] The preset data partitioning rule uses author identifiers as the partitioning unit. A portion of the author identifiers and their corresponding handwriting images are assigned to the handwriting training image set. The remaining handwriting images corresponding to the author identifiers are divided into a model validation image set and an author validation test set according to a preset ratio. Handwriting images corresponding to the same author identifier do not appear in the two image sets simultaneously, and each handwriting image continues to be associated with its original author identifier. During model training, handwriting training images are selected from the handwriting training image set to form handwriting training batches. Backpropagation is performed on the joint contrastive loss, and the model parameters in the trainable state are modified. Same-author validation image pairs and different-author validation image pairs are established from the model validation image set. Model evaluation metrics and candidate thresholds are calculated, and model parameters are not directly modified based on the evaluation results generated by the model validation image set. The author validation test set undergoes a final evaluation only once after model training is completed.

[0094] In this embodiment, the operation of generating a handwriting comparison sample set includes:

[0095] The handwriting training images in the handwriting training image set are grouped according to the author identifier to form a handwriting training image group corresponding to each author. The page number and line number are parsed from the path information of the handwriting training images to form the content identifier corresponding to each handwriting training image.

[0096] Select a preset number of authors from the handwriting training image groups corresponding to different authors, select a preset number of handwriting training images from the handwriting training image groups corresponding to each selected author, and combine the selected handwriting training images into a handwriting training batch.

[0097] When selecting handwriting training images, priority is given to handwriting training images corresponding to content identifiers shared by multiple authors, and handwriting training images with significant differences in page numbers or line numbers are given priority when the author identifiers are the same.

[0098] Establish same-author sample relationship between any handwriting training image in the handwriting training batch and other handwriting training images with the same author identifier, and establish different-author sample relationship between any handwriting training image with a different author identifier;

[0099] The handwriting training images that establish the relationship between samples from the same author and samples from different authors are collected into a handwriting comparison sample set.

[0100] The handwriting training image groups are established according to author identifiers. Each handwriting training image is checked against its corresponding author identifier. Images with the same author identifier are grouped into the same group, while images with different author identifiers are grouped into different groups, maintaining the association between each image and its corresponding author identifier. The path information of the handwriting training images includes page and line number identifiers. For example, an identifier like "-P16_L3" at the end of the filename indicates page 16, line 3. Content identifiers with the same page number and line number are comparable between authors and used to construct negative samples with the same content but different authors. Images whose path information does not contain comparable page and line numbers across authors are used only as samples within the same author group.

[0101] The preset quantities include the number of authors selected for each handwriting training batch and the number of handwriting training images selected from the handwriting training image group corresponding to each selected author. The number of handwriting training images contained in each handwriting training image group is counted, and the number of handwriting training images selected for each author is set to be no less than two, ensuring that this selection number does not exceed the minimum number of images contained in any handwriting training image group participating in the current training. Handwriting training image groups with fewer images than this selection number are removed from the current author selection range. The number of handwriting training images selected for each author and the number of authors selected for each handwriting training batch are determined before the model training begins and remain unchanged during the same round of model training.

[0102] When selecting handwriting training images, the number of authors sharing the same content identifier in the current batch of authors is counted, and candidate content identifiers with a number of shared authors not less than a preset sharing threshold are determined as the shared content identifier set. From the shared content identifier set, a preset number of target content identifiers are selected based on the distance between content identifiers, ensuring that the selected target content identifiers are as dispersed as possible. For each selected author, handwriting training images are preferentially selected from the handwriting training images corresponding to the target content identifier, and then supplemented from the remaining handwriting training images based on page number and line number diversity, so that the handwriting training images selected by the same author in the same handwriting training batch come from different page numbers or different line numbers as much as possible. The same author identifier and the same handwriting training image are selected only once in the current handwriting training batch.

[0103] Within each handwriting training batch, one handwriting training image is sequentially designated as the current handwriting training image, excluding the current handwriting training image itself. The author identifier of the current handwriting training image is then compared with the author identifiers of the remaining handwriting training images in the batch. Two handwriting training images with the same author identifier establish a same-author sample relationship, while two handwriting training images with different author identifiers establish a different-author sample relationship. After the next handwriting training batch is formed, the same-author sample relationships and different-author sample relationships are re-established according to the handwriting training images included in the next handwriting training batch. The handwriting comparison sample set includes the handwriting training images in the handwriting training batch, the author identifiers corresponding to each handwriting training image, and the same-author sample relationships and different-author sample relationships established within the current handwriting training batch. The same-author sample relationships and different-author sample relationships are incorporated into the training process of the handwriting author feature extraction model along with the corresponding handwriting training batch.

[0104] In this embodiment, the operation of generating handwriting identity feature vectors using the handwriting author feature extraction model with the ConvNeXt structure includes:

[0105] The ConvNeXt structure with pre-trained weight configuration is adopted. The classification output layer after the global average pooling layer is removed, while the initial convolutional layer, four feature extraction stages and the global average pooling layer are retained.

[0106] The output of the global average pooling layer is connected to the global feature mapping process, and the output of the fourth feature extraction stage is connected to the local feature mapping process. The global feature mapping result and the local feature mapping result are concatenated and then fused to generate the identity feature vector.

[0107] At the start of training, the gradient update state of the initial convolutional layer and the parameters in the first three feature extraction stages is turned off, and the fourth feature extraction stage and the feature mapping process are set to trainable state.

[0108] At the start of the preset unfreezing round, the fixed state of the second and third feature extraction stages is released, and only the model parameters of the initial convolutional layer and the first feature extraction stage are fixed, and the optimizer is rebuilt.

[0109] The pre-trained weights are retrieved from a pre-trained model that matches the ConvNeXt structure, and the pre-trained parameters are loaded into the initial convolutional layer, the four feature extraction stages, and the corresponding downsampling layers. The ConvNeXt structure includes an initial convolutional layer, four sequentially arranged feature extraction stages, downsampling layers placed between adjacent feature extraction stages, a global average pooling layer, and a classification output layer. The initial convolutional layer receives the handwriting training image after three-channel transformation and pixel normalization, and arranges the image pixels according to the convolution stride and number of output channels configured in the initial convolutional layer to form an initial feature map. The first feature extraction stage receives the initial feature map, and the second, third, and fourth feature extraction stages sequentially receive the output of the previous feature extraction stage; the downsampling layers between adjacent feature extraction stages reduce the spatial size of the feature map according to the corresponding sampling stride and rearrange the feature channels according to the number of channels configured in the next feature extraction stage.

[0110] When removing the classification output layer, the parameter connection between the global average pooling layer and the classification output layer is disconnected, the classification weight parameters and classification bias parameters corresponding to the classification output layer are deleted, and the output of the global average pooling layer is connected to the global feature mapping process. Following the image arrangement order in the handwriting training batch, each handwriting training image is combined into a batch image tensor. Each handwriting training image in the batch image tensor uses the same set of ConvNeXt model parameters, passing through an initial convolutional layer, four feature extraction stages, and a corresponding downsampling layer, respectively, and forming the corresponding final-level handwriting feature map at the output of the fourth feature extraction stage.

[0111] At the start of training, the initial freeze count is set to 2: Gradient update states for parameters in the initial convolutional layer, the first feature extraction stage, the second feature extraction stage, and the third feature extraction stage are disabled, and the corresponding parameters are not added to the optimizer's parameter set to be updated. Gradient update states for parameters in the fourth feature extraction stage and feature mapping process are enabled, and the corresponding parameters are added to the backbone network parameter set to be updated and the feature mapping parameter set to be updated. At the start of the preset unfreezing round, the freeze count is adjusted to 1, the fixed states of the second and third feature extraction stages are released, and only the parameters of the initial convolutional layer and the first feature extraction stage remain fixed. The optimizer is rebuilt, and the optimizer state of the existing parameters is backfilled into the new optimizer, allowing the newly unfrozen parameters to be added to training in a cold-start state.

[0112] In this embodiment, the operations of global feature mapping, local feature mapping, and fusion processing include:

[0113] The global feature mapping process connects the flattening layer, the first fully connected layer, the first one-dimensional batch normalization layer, the GELU activation layer, the random deactivation layer, the second fully connected layer, and the second one-dimensional batch normalization layer sequentially at the output of the global average pooling layer.

[0114] The local feature mapping process performs attention-weighted spatial aggregation on the feature map output from the fourth feature extraction stage, and then sequentially connects the third fully connected layer, the third one-dimensional batch normalization layer, the GELU activation layer, the random deactivation layer, the fourth fully connected layer, and the fourth one-dimensional batch normalization layer after the aggregation result.

[0115] The fusion process concatenates the global feature mapping results with the local feature mapping results, maps them to identity mapping features through the fifth fully connected layer and the fifth one-dimensional batch normalization layer, and normalizes each feature component according to the L2 norm of the identity mapping features to obtain the handwriting identity feature vector.

[0116] In this process, the global average pooling layer aggregates the feature values ​​of all spatial locations for each feature channel in each final-level handwriting feature map, calculates the average value for each feature channel, and flattens these average values ​​into a one-dimensional visual feature according to the original arrangement of the feature channels. The number of feature components contained in the one-dimensional visual feature is consistent with the number of feature channels output in the fourth feature extraction stage. Each one-dimensional visual feature is associated with its original image position in the batch image tensor, along with the corresponding handwriting training image, author identifier, same-author sample relationship, and different-author sample relationship.

[0117] The local feature mapping process uses a lightweight attention branch to estimate the importance of each spatial location in the final handwriting feature map. The attention weights are multiplied by the feature values ​​of each spatial location, and then weighted summation is performed to obtain attention-aggregated features. These are then mapped to intermediate features via a third fully connected layer. The intermediate features are then sequentially processed through a third one-dimensional batch normalization layer, a GELU activation layer, and a random deactivation layer before being mapped to local mapping features via a fourth fully connected layer. The fusion process concatenates the global and local mapping features along the feature channel direction, maps them to identity mapping features via a fifth fully connected layer, and then processes them again via a fifth one-dimensional batch normalization layer. Each feature component is scaled uniformly according to the L2 norm of the identity mapping features to form a handwriting identity feature vector. Each processing layer follows the image arrangement in the handwriting training batch, ensuring that each visual feature, intermediate feature, mapping feature, and handwriting identity feature vector is consistent with the corresponding handwriting training image and author identifier.

[0118] A preset inactivation rate is written into the random inactivation layer before the handwriting author feature extraction model begins training. The preset inactivation rate is a value greater than 0 and less than 1. When the model is in training mode, the random inactivation layer selects inactivation positions from the intermediate features after activation according to the preset inactivation rate, sets the feature components corresponding to the inactivation positions to zero, and performs scale compensation on the unselected feature components before passing them to the next layer. When the model is in validation mode, the selection of inactivation positions is turned off, while all feature components are allowed to enter the next layer.

[0119] In this embodiment, the operation of establishing the author verification feature space includes:

[0120] Arrange the handwriting identity feature vectors according to the handwriting training batch, and maintain the correspondence between the handwriting identity feature vectors and the author identifier;

[0121] Establish same-author identity feature pairs based on the relationship between same-author samples, and establish different-author identity feature pairs based on the relationship between different-author samples;

[0122] Calculate the cosine similarity between two handwriting feature vectors in the same author identity feature pair and the different author identity feature pair respectively, and use the difference between 1 and the cosine similarity as the corresponding feature distance;

[0123] The author verification feature space is composed of the feature vectors of each handwriting identity feature, the feature distances corresponding to the same author identity feature pairs, and the feature distances corresponding to different author identity feature pairs.

[0124] In this process, the author verification feature space is constructed using a single handwriting training batch as the scope. Based on the arrangement of each handwriting training image within the batch, corresponding handwriting identity feature vectors are received sequentially. The batch position, handwriting training image, and author identifier are associated with the corresponding handwriting identity feature vector, without altering the feature dimensions or the order of feature components of each vector. One handwriting identity feature vector is selected sequentially as the current handwriting identity feature vector. The vector itself is excluded, and its author identifier is compared with the author identifiers of the remaining handwriting identity feature vectors in the current training batch. If two handwriting identity feature vectors have the same author identifier, a same-author identity feature pair is established; if they have different author identifiers, a different-author identity feature pair is established. No identity feature pairs are established between the same handwriting identity feature vector and itself.

[0125] For each pair of same-author and different-author identity feature vectors, the feature components at the same dimension positions in the two L2-normalized handwriting identity feature vectors are multiplied term by term. The products at each dimension position are accumulated to obtain the corresponding cosine similarity. The difference between 1 and this cosine similarity is used as the feature distance, ensuring that the two types of identity feature vectors use the same distance calculation method. According to the batch position of the current handwriting identity feature vector, the author identifier, same-author identity feature pairs and their feature distances, and different-author identity feature pairs and their feature distances corresponding to the handwriting identity feature vector are collected respectively. After collecting all handwriting identity feature vectors in the current handwriting training batch, the author verification feature space corresponding to the handwriting training batch is formed.

[0126] In this embodiment, the operation of performing feature constraints using supervised contrastive loss includes:

[0127] The handwriting identity feature vector in the author verification feature space is determined as the anchor vector, the handwriting identity feature vector that establishes the same author identity feature pair with the anchor vector is determined as the positive sample vector, and the handwriting identity feature vector that establishes the opposite author identity feature pair with the anchor vector is determined as the negative sample vector.

[0128] Calculate the feature similarity between the anchor vector and the positive sample vector and the negative sample vector respectively, and scale each feature similarity according to a preset temperature coefficient;

[0129] The scaled feature similarity of all positive sample vectors corresponding to the same anchor vector is summarized, and the summarized result is subjected to exponential normalization and negative logarithmic processing with the scaled feature similarity of all non-self sample vectors.

[0130] Anchor vectors that correspond to at least one positive sample vector are identified as valid anchor vectors. The negative logarithm processing results corresponding to each valid anchor vector in the handwriting training batch are summarized and the average value is calculated to obtain the supervised contrast loss.

[0131] In this model, the anchor vector, positive sample vector, and negative sample vector are all taken from the author verification feature space corresponding to the same handwriting training batch. After L2 normalization, the handwriting identity feature vector is multiplied term by term by the feature components of a sample vector along the same dimension, and the products at each dimension are accumulated. The accumulated result is used as the feature similarity between the current anchor vector and the sample vector. A preset temperature coefficient is written into the calculation configuration of the supervised contrastive loss before the model training begins. It is a real number greater than 0, and all anchor vectors in the same handwriting training batch use the same preset temperature coefficient.

[0132] The scaling results of each positive sample vector corresponding to the current anchor vector are subjected to exponential operations, and the resulting exponential values ​​are summed to form the positive sample exponential sum. The scaling results of all positive and negative sample vectors corresponding to the current anchor vector are also subjected to exponential operations, and the resulting exponential values ​​are summed to form the non-self sample exponential sum; the current anchor vector itself is not included in the calculation of the non-self sample exponential sum. The positive sample exponential sum is divided by the non-self sample exponential sum to obtain the exponential normalization result corresponding to the current anchor vector. A numerical stability coefficient greater than 0 is added to the exponential normalization result, and the natural logarithm of the value after adding the numerical stability coefficient is taken, and the negative of the resulting natural logarithm value is taken to form the negative logarithm processing result corresponding to the current effective anchor vector. The negative logarithm processing results corresponding to all effective anchor vectors are summed, and the average value is calculated according to the number of effective anchor vectors. This average value is recorded as the supervised contrast loss. The supervised contrast loss is calculated based on the correspondence between handwriting identity feature vectors and their author identifiers, without reading the classification output results.

[0133] In this embodiment, the operations for optimizing model parameters using semi-hard sample contrast loss and local consistency loss include:

[0134] For each valid anchor vector, the handwriting identity feature vector with the lowest feature similarity among the corresponding positive sample vectors is determined as the difficult positive sample vector;

[0135] Negative sample vectors whose feature similarity is between the difference between the feature similarity corresponding to the difficult positive sample vector and the preset interval threshold and the feature similarity corresponding to the difficult positive sample vector are determined as a semi-difficult negative sample candidate set. A preset number of negative sample vectors with the highest feature similarity are selected from the semi-difficult negative sample candidate set. When the semi-difficult negative sample candidate set is empty, all negative sample vectors are selected to obtain the difficult negative sample vector set.

[0136] The feature similarity of the corresponding features of the difficult positive sample vector is subtracted from the sum of the average feature similarity of the corresponding features of the difficult negative sample vector set and a linear correction constraint is applied to form the difficult sample contrast loss.

[0137] Based on the author-level difficult sample map, author-level difficult sample mining is performed between handwriting training batches, and local consistency constraints are applied to handwriting training samples of the same author based on the local feature mapping results, forming a local consistency loss.

[0138] Based on the preset hardness loss weight and preset local consistency loss weight, the hard sample comparison loss and local consistency loss are combined with the supervised comparison loss to form a joint comparison loss. The model parameters in the handwriting author feature extraction model that are in a trainable state are updated according to the joint comparison loss.

[0139] Both difficult positive and difficult negative sample vectors are selected within the current handwriting training batch. After the current handwriting training batch completes the generation of handwriting identity feature vectors, the feature similarity between each valid anchor vector and its corresponding positive and negative sample vectors is recalculated. After the next handwriting training batch is formed, the selection of difficult samples is performed again based on the handwriting identity feature vectors and feature similarities generated in the next handwriting training batch. For a valid anchor vector, the valid anchor vector itself is excluded, and the feature similarity of all its corresponding positive sample vectors is checked. The positive sample vector with the lowest feature similarity is determined as the difficult positive sample vector. The difficult positive sample vector is taken from the handwriting identity feature vector with the same author identifier as the valid anchor vector, and the difficult negative sample vector is taken from the handwriting identity feature vector with a different author identifier than the valid anchor vector.

[0140] The preset interval threshold is a pre-configured positive number. For each valid anchor vector, a semi-hard negative sample candidate set is formed, consisting of negative sample vectors whose feature similarity falls between the difference between the feature similarity corresponding to the hard positive sample vector and the preset interval threshold, and the feature similarity corresponding to the hard positive sample vector. When the semi-hard negative sample candidate set is not empty, a preset number of negative sample vectors with the highest feature similarity are selected, and their average similarity is calculated as the feature similarity corresponding to the hard negative sample vector set. When the semi-hard negative sample candidate set is empty, all negative sample vectors in the current batch are selected, and their average similarity is calculated as the feature similarity corresponding to the hard negative sample vector set. The feature similarity corresponding to the hard positive sample vector is subtracted from the sum of the preset interval threshold and this average value, and a linear correction is applied to the resulting difference. The correction result is used as the hard sample contrast loss. The hard sample loss values ​​corresponding to each valid anchor vector in the current handwriting training batch are summarized, and the batch average value is calculated according to the number of valid anchor vectors. This batch average value is recorded as the hard sample contrast loss.

[0141] The operations for author-level hard sample mining include: after a preset initial mining round, a preset number of prototype handwriting images are deterministically selected for each training author; the prototype handwriting images are encoded into prototype features using the current handwriting author feature extraction model and aggregated into author prototype features; the similarity between author prototype features is calculated, and a preset number of dissimilar authors with the highest feature similarity are retained as hard neighbors for each training author, forming an author-level hard sample map; the author-level hard sample map is reconstructed every preset refresh round; in each training round, a handwriting training batch for author-level hard sample mining is determined according to a preset hard batch ratio, and authors with hard neighbor relationships are preferentially selected to form the handwriting training batch in the handwriting training batch, while authors for the remaining handwriting training batches are selected randomly. The operations for local consistency constraints include: for each author in the current handwriting training batch, the variance of the local feature mapping result corresponding to that author to the author's local feature center is calculated, the variances corresponding to all authors that meet the sample quantity requirements are summarized, and the average value is calculated as the local consistency loss.

[0142] The hard sample contrast loss and the local consistency loss are both pre-configured non-negative weights. The hard sample contrast loss and the local consistency loss are weighted according to their respective pre-configured weights. The weighted result is added to the supervised contrast loss to form the joint contrast loss, where the supervised contrast loss is the first loss component, the hard sample contrast loss is the second loss component, and the local consistency loss is the third loss component. Backpropagation is performed on the joint contrast loss to calculate the gradients corresponding to the model parameters in the trainable state within the feature mapping process and the ConvNeXt structure. The corresponding model parameters are modified based on these gradients; no parameter modification is performed on model parameters in a fixed state. Automatic mixed precision is used to calculate and perform gradient clipping during training.

[0143] In this embodiment, the operations for generating the identity feature vector to be verified and the registered identity feature vector include:

[0144] Receive a pair of handwriting images to be verified and registered handwriting images, generate standardized handwriting images to be verified and standardized handwriting images to be registered according to the same preprocessing process as the model training images, and do not perform training enhancement during the preprocessing process;

[0145] Load the optimized handwriting author feature extraction model and switch to the evaluation state. Turn off the gradient calculation of the model parameters. Input the standardized handwriting image to be verified and the standardized handwriting image to be registered into the handwriting author feature extraction model using the same set of model parameters to generate the feature vector of the identity to be verified and the feature vector of the registered identity.

[0146] The feature vectors of the identity to be verified and the feature vectors of the registered identity are associated according to the pairwise correspondence.

[0147] The handwriting images to be verified are the handwritten handwriting images whose author consistency needs to be checked in the current author verification task; the registered handwriting images are the handwritten handwriting images designated as the comparison benchmark in the current author verification task. "Registered" indicates that the image is used as the comparison benchmark in the current paired input, without limiting the pre-establishment of an author registration library, registration database, or registration image update process. The standardized handwriting images to be verified and the standardized handwriting images to be registered are inputs to the actual author verification task, but are not included in the handwriting training image set or model verification image set, and do not participate in model parameter training or author verification threshold determination. When multiple sets of paired images exist, the relationship between each set of handwriting images to be verified and the corresponding registered handwriting images is maintained, and images from different sets are not re-paired.

[0148] When loading the optimized handwriting author feature extraction model, the network structure and model parameters saved after model training are retrieved, and the saved parameters are loaded into the corresponding network layers, while the classification output layer remains in the removed state. After switching the model to the evaluation state, the random inactivation layer stops selecting inactivation positions, and each one-dimensional batch normalization layer uses the running mean, running variance, scaling parameters, and offset parameters saved during the training phase to process the input features, without recalculating the batch mean and batch variance based on the current paired images. In the evaluation state, the calculation of supervised contrast loss, semi-hard sample contrast loss, local consistency loss, and joint contrast loss is not performed, and backpropagation and parameter modification are not performed, keeping the loaded model parameters unchanged. Pairs of handwriting images to be verified and registered handwriting images are respectively input into the same handwriting author feature extraction model. The two images use the same model structure and the same set of model parameters, and the forward computation is completed in a consistent order of visual feature extraction, feature mapping, and normalization processing. The handwriting identity feature vector output after the forward computation of the handwriting image to be verified is denoted as the identity feature vector to be verified, and the handwriting identity feature vector output after the forward computation of the registered handwriting image is denoted as the identity feature vector to be registered. Both types of identity feature vectors use the same feature dimension and feature component arrangement order. The identity feature vector to be verified and the identity feature vector to be registered are associated according to the original paired image relationship, and each pair of associated identity feature vectors is compiled into a handwriting identity feature vector set; the handwriting identity feature vector set stores the identity feature vector to be verified, the identity feature vector to be registered, and their paired correspondence.

[0149] In this embodiment, the operations for calculating feature similarity and generating author consistency verification results include:

[0150] Based on the pairwise correspondence, obtain the identity feature vector to be verified and the registered identity feature vector from the handwriting identity feature vector set;

[0151] Perform cosine similarity calculation on the feature vector of the identity to be verified and the feature vector of the registered identity to obtain the corresponding feature similarity;

[0152] Retrieve a pre-determined author verification threshold and compare the feature similarity with the author verification threshold;

[0153] When the feature similarity is not lower than the author verification threshold, the corresponding image pair is marked as having the same author; when the feature similarity is lower than the author verification threshold, the corresponding image pair is marked as having different authors.

[0154] Image pairs, feature similarity, and corresponding labels are associated to generate author consistency verification results.

[0155] The cosine similarity calculation is performed according to the pairwise correspondence between the feature vector of the identity to be verified and the feature vector of the registered identity. The feature dimensions and feature component order of the two identity feature vectors are checked, and the feature components at the same dimension positions are multiplied item by item. The products of all dimensions are then accumulated, and the accumulated value is recorded as the feature similarity of the corresponding image pair. Since both the feature vector of the identity to be verified and the feature vector of the registered identity are L2 normalized feature vectors, no further normalization is performed on the resulting accumulated value.

[0156] The author verification threshold is determined based on the model verification image set and is updated according to the evaluation results of each training epoch during model training. Based on the author identifier, two standardized handwriting images with the same author identifier are selected from the model verification image set to form a same-author verification image pair, and two standardized handwriting images with different author identifiers are selected to form a different-author verification image pair. When constructing verification image pairs, same-author verification image pairs prioritize image combinations with significant differences in page numbers or line numbers, while different-author verification image pairs prioritize image combinations sharing the same content identifier. After the verification image pairs are generated, they are written to a fixed evaluation file. The evaluation file carries data partition signatures and random seed information and is regenerated when the data partition or random seed changes to prevent the reuse of expired verification image pairs. During model training, the same fixed evaluation file is loaded for evaluation epoch by epoch. Standardized handwriting images in the model verification image set do not participate in the backpropagation of the joint contrastive loss or model parameter modification.

[0157] Under a preset threshold search step size, thresholds within the range of 0 to 1 are set as candidate thresholds. For each candidate threshold, the corresponding false acceptance rate, false rejection rate, precision, recall, F1 score, and accuracy are calculated. Thresholds corresponding to equal error rates are determined based on the principle that the false acceptance rate and false rejection rate are closest. Within the range where candidate thresholds are not lower than the thresholds corresponding to equal error rates, the candidate threshold with the highest F1 score is selected as the author verification threshold. When the F1 score, precision, and accuracy are all consistent, the candidate threshold with the larger value is selected. The determined author verification threshold is associated with and saved with the handwriting author feature extraction model parameters saved in the corresponding training round. When author verification is performed, the handwriting author feature extraction model parameters and author verification threshold corresponding to the same training round are loaded, and the feature similarity of the current image pair is compared according to the author verification threshold. When the feature similarity of the current image pair is greater than or equal to the author verification threshold, the image pair is written into the author consistency flag; when the feature similarity of the current image pair is less than the author verification threshold, the image pair is written into the author inconsistency flag. The author consistency verification results include the handwriting image to be verified, the registered handwriting image, the corresponding feature similarity, and the author consistency marker or the author inconsistency marker. Following the original pairwise correspondence between the handwriting image to be verified and the registered handwriting image, the above information is written into the same verification result record.

[0158] Example 1: To verify the feasibility of this invention in practice, it was applied to a handwritten handwriting author verification scenario to check whether two single-line handwritten handwriting images were from the same writer. The input images were organized by author identifier, the filenames included page and line numbers, and the handwriting content was single-line text.

[0159] This embodiment collected approximately 52,000 single-line handwriting images from 1,326 writers. The data was divided according to author identification: 41,937 images corresponding to 1,061 writers were included in the handwriting training image set; 5,116 images corresponding to 132 writers were included in the model validation image set; and 5,343 images corresponding to 133 writers were reserved for author validation testing. The author identifications within the three data ranges did not overlap. Each author had at least four images. The original images underwent grayscale conversion, OTSU binarization, 2×2 structuring element morphological denoising, and proportional scaling, and were placed in a 224×224 pixel canvas. Single-channel images were copied to three-channel images and then pixel normalization was performed. During training, the normalized images underwent preset enhancements: random rotation within 3 degrees, random translation within 5%, and random scaling from 0.95 to 1.05 times were performed with a 50% probability; and random erasure was performed with a 10% probability.

[0160] The handwriting training image set is grouped by author identifier, and page numbers and line numbers are parsed from the filenames to form content identifiers. For each handwriting training batch, 48 writers are randomly selected, and 4 images are chosen without replacement from the image set corresponding to each writer, resulting in 192 images per batch. When selecting images, priority is given to page and line number content identifiers shared by at least two writers, and this is supplemented within each author's group based on page and line number diversity. Images with the same author identifier are associated with the same author, while images with different author identifiers are associated with different authors. Each image corresponds to 3 positive samples and 188 negative samples. Starting from the 10th round, every 5 rounds, an author-level hard sample map is reconstructed based on the author's prototype features, retaining the 20 different authors with the highest feature similarity as hard neighbors for each author. In each training round, 25% of the handwriting training batches use the author-level hard sample map to select authors.

[0161] The handwriting author feature extraction model adopts a ConvNeXt structure with pre-trained weights, removes the classification output layer, and retains the initial convolutional layer, four feature extraction stages, and a global average pooling layer. The global average pooling layer outputs 1024-dimensional visual features. The global feature mapping process sequentially performs flattening, mapping to 512-dimensional intermediate features in the first fully connected layer, one-dimensional batch normalization, GELU activation, random deactivation with a deactivation rate of 0.2, mapping to 256-dimensional global mapping features in the second fully connected layer, and one-dimensional batch normalization. The local feature mapping process performs attention-weighted spatial aggregation on the feature map output from the fourth feature extraction stage, mapping to 256-dimensional intermediate features in the third fully connected layer, one-dimensional batch normalization, GELU activation, random deactivation with a deactivation rate of 0.2, mapping to 128-dimensional local mapping features in the fourth fully connected layer, and one-dimensional batch normalization. The fusion process concatenates the 256-dimensional global mapping features with the 128-dimensional local mapping features, maps to 256-dimensional identity mapping features in the fifth fully connected layer, performs one-dimensional batch normalization, and then performs L2 normalization to output a 256-dimensional handwriting identity feature vector. At the start of training, the initial convolutional layer and the first three feature extraction stages are frozen, while the fourth feature extraction stage and the feature mapping process are retained for training. At the start of the 13th round, the second and third feature extraction stages are unfrozen, leaving only the initial convolutional layer and the first feature extraction stage fixed, and the optimizer is rebuilt.

[0162] The model training employs supervised contrastive loss, semi-hard sample contrastive loss, and local consistency loss. The temperature coefficient for supervised contrastive loss is set to 0.07. For semi-hard sample contrastive loss, cosine similarity before temperature scaling is used. The sample with the lowest similarity among the same author is selected as the hard positive sample. Among the samples from different authors with similarities between the difference between the similarity of the hard positive sample and a preset interval threshold and the similarity of the hard positive sample, the five semi-hard negative samples with the highest similarity are selected. If no semi-hard negative samples exist, the model degenerates into all negative samples. The similarity of the hard positive samples is subtracted by the sum of the preset interval threshold of 0.55 and the average similarity of the semi-hard negative samples, and a linear correction is applied. The hard loss weight is set to 0.5 and takes effect from the 4th round. The local consistency loss constrains the local feature distribution of the same author samples based on the local feature mapping results, with a weight set to 0.03 and takes effect from the 21st round. The total loss is the sum of supervised contrastive loss, 0.5 times the semi-hard sample contrastive loss, and 0.03 times the local consistency loss.

[0163] The training configuration is as follows: a maximum of 80 training epochs, with a fixed 327 training batches per epoch. The AdamW optimizer is used, with a backbone learning rate of 5e-6, a feature map layer learning rate of 5e-5, and weight decay of 0.01. The first 10 epochs use a linear learning rate warm-up, followed by cosine annealing for decay. The gradient clipping norm is set to 1.0, and automatic mixed precision is enabled. The early stopping strategy is as follows: using the best composite score from the last 5 epochs as the benchmark, the number of waiting periods is accumulated until the new best composite score improves by no more than 0.0005 relative to the benchmark. Training stops when the number of waiting periods reaches 20.

[0164] After each training epoch, a fixed evaluation file (containing 10,000 same-author validation image pairs and 10,000 different-author validation image pairs) generated and locked based on the model validation image set is loaded. AUC, F1 score, precision, recall, equal error rate, and feature separation are calculated. In the composite score, AUC has a weight of 0.45, (1 - equal error rate) has a weight of 0.35, F1 score has a weight of 0.10, precision has a weight of 0.05, and recall has a weight of 0.05. The model parameters corresponding to the training epoch with the highest composite score, the optimal model parameters for equal error rate, and the optimal model parameters for AUC are saved, and the author validation thresholds for each training epoch are also saved accordingly.

[0165] The author validation threshold is determined based on the model validation results of the highest training epoch of the composite score. Candidate thresholds are generated in steps of 0.01 within the range of 0 to 1. First, the threshold corresponding to the equal error rate is determined based on the minimum absolute difference between the false acceptance rate and the false rejection rate. Then, within the range where the candidate threshold is not lower than the threshold corresponding to the equal error rate, the candidate threshold with the highest F1 score is selected as the author validation threshold. When the F1 score, precision, and accuracy are all consistent, the candidate threshold with the larger value is selected. In this embodiment, the author validation threshold is determined to be 0.36, and it is associated with and saved with the corresponding model parameters. The model validation metrics corresponding to the highest training epoch of the composite score are: AUC 0.9832, F1 score 0.9354, equal error rate 6.47%, and feature separation 0.6843.

[0166] During the author verification testing phase, 10,000 image pairs with the same author and 10,000 image pairs with different authors were constructed. Each image pair contained one handwriting image to be verified and one registered handwriting image. Both images underwent the same preprocessing and were then input into the same set of model parameters. The model was in evaluation mode; the random deactivation layer stopped deactivation processing, the one-dimensional batch normalization layer used the running mean and variance saved during training, and gradient calculation was disabled. Cosine similarity was calculated between the feature vector of the identity to be verified and the feature vector of the registered identity. If the feature similarity was not less than 0.36, an author consistency marker was written; otherwise, an author inconsistency marker was written.

[0167] The comparative tests employed a single global head feature extraction model, a dual-head feature extraction model without author-level hard sample mining, and the system of this invention. All methods used the same author-validated test image pairs, and the same preprocessing, sampling, and training configurations. The overall accuracy, AUC, error rate, F1 score, precision, recall, number of false acceptances, and number of false rejections in Table 1 are all statistically analyzed for the corresponding test image pairs. The comparison results are shown in the table below:

[0168] Table 1: Performance Comparison Results of Handwriting Author Verification

[0169] Overall accuracy (%) 91.74 92.18 92.76 AUC 0.9756 0.9781 0.9785 EER (%) 8.26 7.84 7.25 F1 value 0.9172 0.9216 0.9277 Accuracy 0.9199 0.9242 0.9254 Recall rate 0.9144 0.9190 0.9301 Number of misreceived image pairs 796 754 750 Number of false rejection image pairs 856 810 699 Author verification threshold 0.35 0.35 0.36

[0170] The data in the table shows that the overall accuracy of the system of this invention is 92.76%, which is higher than the 91.74% of the single global head feature extraction model and the 92.18% of the dual-head feature extraction model without author-level mining, representing improvements of 1.02 percentage points and 0.58 percentage points, respectively. The AUC reaches 0.9785, which is 0.0029 and 0.0004 higher than the 0.9756 and 0.9781 of the two comparison models, respectively, indicating that the system of this invention has a stronger overall ability to distinguish between handwriting by the same author and handwriting by different authors.

[0171] The EER of this invention is 7.25%, lower than the 8.26% of the single global head feature extraction model and the 7.84% of the dual-head feature extraction model without author-level mining, representing reductions of 1.01 percentage points and 0.59 percentage points, respectively. The F1 score reaches 0.9277, an improvement of 0.0105 and 0.0061 compared to the 0.9172 and 0.9216 of the two comparative models, respectively. Regarding the false positive distribution, the number of falsely rejected image pairs in this invention is 699, a reduction of 157 and 111 compared to the 856 and 810 of the two comparative models, respectively; the number of falsely accepted image pairs is 750, a reduction of 46 compared to the single global head feature extraction model, and roughly the same as the dual-head model without author-level mining. This indicates that dual-head feature fusion and author-level hard sample mining can significantly improve the recall of samples from the same author while maintaining a low false acceptance level. The main benefits of author-level hard sample mining are concentrated in reducing the number of false rejections and improving the recall rate.

[0172] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based handwriting author verification system, characterized in that, include: The handwriting acquisition and processing module is used to acquire handwritten handwriting images, perform preprocessing, and generate handwriting training image sets and model verification image sets; The sample construction module is used to establish same-author and different-author sample relationships based on the author identifier and content identifier corresponding to the handwriting training image, and generate a handwriting comparison sample set. The feature extraction module is used to input the handwriting comparison sample set into the handwriting author feature extraction model with the ConvNeXt structure, remove the classification output layer in the ConvNeXt structure, retain the feature extraction layer, and perform global feature mapping, local feature mapping and fusion processing on the visual features to generate handwriting identity feature vectors. The feature optimization module is used to establish an author verification feature space based on the handwriting identity feature vector, perform feature distance constraints on handwriting sample pairs of the same author, perform feature distance separation constraints on handwriting sample pairs of different authors, optimize model parameters using supervised contrast loss, semi-hard sample contrast loss and local consistency loss, and generate an optimized handwriting author feature extraction model. The verification feature generation module is used to input the paired handwriting images to be verified and the registered handwriting images into the optimized handwriting author feature extraction model to generate the feature vector of the identity to be verified and the feature vector of the registered identity. The similarity calculation module is used to calculate the feature similarity between the feature vector of the identity to be verified and the feature vector of the registered identity based on the handwriting identity feature vector set; The author verification output module is used to compare feature similarity with the author verification threshold and generate author consistency verification results based on the comparison results.

2. The handwriting author verification system based on deep learning according to claim 1, characterized in that, The operations for generating the handwriting training image set and the model validation image set include: Acquire the original handwritten handwriting image, perform grayscale conversion on the original handwritten handwriting image, and generate a grayscale handwriting image; Binarization is performed on the grayscale handwriting image to distinguish the handwriting area from the background area, generating a binary handwriting image; Perform morphological denoising on the binary handwriting image to generate a denoised handwriting image; Perform resizing on the denoised handwriting image to generate a standardized handwriting image; The single-channel image of the standardized handwriting image is copied into a three-channel image, and pixel normalization is performed. While the model is in training mode, pre-defined training enhancement processing is performed on the standardized handwriting images; According to the preset data partitioning rules, the standardized handwriting images are divided into the handwriting training image set and the model validation image set, with the author identifier as the partitioning unit, while retaining the author identifier corresponding to the standardized handwriting images.

3. The handwriting author verification system based on deep learning according to claim 2, characterized in that, The operation of generating the handwriting comparison sample set includes: The handwriting training images in the handwriting training image set are grouped according to the author identifier to form a handwriting training image group corresponding to each author. The page number and line number are parsed from the path information of the handwriting training images to form the content identifier corresponding to each handwriting training image. Select a preset number of authors from the handwriting training image groups corresponding to different authors, select a preset number of handwriting training images from the handwriting training image groups corresponding to each selected author, and combine the selected handwriting training images into a handwriting training batch. When selecting handwriting training images, priority is given to handwriting training images corresponding to content identifiers shared by multiple authors, and handwriting training images with significant differences in page numbers or line numbers are given priority when the author identifiers are the same. Establish same-author sample relationship between any handwriting training image in the handwriting training batch and other handwriting training images with the same author identifier, and establish different-author sample relationship between any handwriting training image with a different author identifier; The handwriting training images that establish the relationship between samples from the same author and samples from different authors are collected into a handwriting comparison sample set.

4. The handwriting author verification system based on deep learning according to claim 3, characterized in that, The operation of generating handwriting identity feature vectors using the handwriting author feature extraction model with the ConvNeXt structure includes: The ConvNeXt structure with pre-trained weight configuration is adopted. The classification output layer after the global average pooling layer is removed, while the initial convolutional layer, four feature extraction stages and the global average pooling layer are retained. The output of the global average pooling layer is connected to the global feature mapping process, and the output of the fourth feature extraction stage is connected to the local feature mapping process. The global feature mapping result and the local feature mapping result are concatenated and then fused to generate the identity feature vector. At the start of training, the gradient update state of the initial convolutional layer and the parameters in the first three feature extraction stages is turned off, and the fourth feature extraction stage and the feature mapping process are set to trainable state. At the start of the preset unfreezing round, the fixed state of the second and third feature extraction stages is released, and only the model parameters of the initial convolutional layer and the first feature extraction stage are fixed, and the optimizer is rebuilt.

5. The deep learning-based handwriting author verification system according to claim 4, characterized in that, The operations of global feature mapping, local feature mapping, and fusion processing include: The global feature mapping process connects the flattening layer, the first fully connected layer, the first one-dimensional batch normalization layer, the GELU activation layer, the random deactivation layer, the second fully connected layer, and the second one-dimensional batch normalization layer sequentially at the output of the global average pooling layer. The local feature mapping process performs attention-weighted spatial aggregation on the feature map output from the fourth feature extraction stage, and then sequentially connects the third fully connected layer, the third one-dimensional batch normalization layer, the GELU activation layer, the random deactivation layer, the fourth fully connected layer, and the fourth one-dimensional batch normalization layer after the aggregation result. The fusion process concatenates the global feature mapping results with the local feature mapping results, maps them to identity mapping features through the fifth fully connected layer and the fifth one-dimensional batch normalization layer, and normalizes each feature component according to the L2 norm of the identity mapping features to obtain the handwriting identity feature vector.

6. The deep learning-based handwriting author verification system according to claim 5, characterized in that, The operation of establishing the author verification feature space includes: Arrange the handwriting identity feature vectors according to the handwriting training batch, and maintain the correspondence between the handwriting identity feature vectors and the author identifier; Establish same-author identity feature pairs based on the relationship between same-author samples, and establish different-author identity feature pairs based on the relationship between different-author samples; Calculate the cosine similarity between two handwriting feature vectors in the same author identity feature pair and the different author identity feature pair respectively, and use the difference between 1 and the cosine similarity as the corresponding feature distance; The author verification feature space is composed of the feature vectors of each handwriting identity feature, the feature distances corresponding to the same author identity feature pairs, and the feature distances corresponding to different author identity feature pairs.

7. The deep learning-based handwriting author verification system according to claim 6, characterized in that, The operation of performing feature constraints using supervised contrastive loss includes: The handwriting identity feature vector in the author verification feature space is determined as the anchor vector, the handwriting identity feature vector that establishes the same author identity feature pair with the anchor vector is determined as the positive sample vector, and the handwriting identity feature vector that establishes the opposite author identity feature pair with the anchor vector is determined as the negative sample vector. Calculate the feature similarity between the anchor vector and the positive sample vector and the negative sample vector respectively, and scale each feature similarity according to a preset temperature coefficient; The scaled feature similarity of all positive sample vectors corresponding to the same anchor vector is summarized, and the summarized result is subjected to exponential normalization and negative logarithmic processing with the scaled feature similarity of all non-self sample vectors. Anchor vectors that correspond to at least one positive sample vector are identified as valid anchor vectors. The negative logarithm processing results corresponding to each valid anchor vector in the handwriting training batch are summarized and the average value is calculated to obtain the supervised contrast loss.

8. The handwriting author verification system based on deep learning according to claim 7, characterized in that, The operation of optimizing model parameters using semi-hard sample contrast loss and local consistency loss includes: For each valid anchor vector, the handwriting identity feature vector with the lowest feature similarity among the corresponding positive sample vectors is determined as the difficult positive sample vector; Negative sample vectors whose feature similarity is between the difference between the feature similarity corresponding to the difficult positive sample vector and the preset interval threshold and the feature similarity corresponding to the difficult positive sample vector are determined as a semi-difficult negative sample candidate set. A preset number of negative sample vectors with the highest feature similarity are selected from the semi-difficult negative sample candidate set. When the semi-difficult negative sample candidate set is empty, all negative sample vectors are selected to obtain the difficult negative sample vector set. The feature similarity of the corresponding features of the difficult positive sample vector is subtracted from the sum of the average feature similarity of the corresponding features of the difficult negative sample vector set and a linear correction constraint is applied to form the difficult sample contrast loss. Based on the author-level difficult sample map, author-level difficult sample mining is performed between handwriting training batches, and local consistency constraints are applied to handwriting training samples of the same author based on the local feature mapping results, forming a local consistency loss. Based on the preset hardness loss weight and preset local consistency loss weight, the hard sample comparison loss and local consistency loss are combined with the supervised comparison loss to form a joint comparison loss. The model parameters in the handwriting author feature extraction model that are in a trainable state are updated according to the joint comparison loss.

9. A deep learning-based handwriting author verification system according to claim 8, characterized in that, The operations for generating the feature vector of the identity to be verified and the feature vector of the registered identity include: Receive a pair of handwriting images to be verified and registered handwriting images, generate standardized handwriting images to be verified and standardized handwriting images to be registered according to the same preprocessing process as the model training images, and do not perform training enhancement during the preprocessing process; Load the optimized handwriting author feature extraction model and switch to the evaluation state. Turn off the gradient calculation of the model parameters. Input the standardized handwriting image to be verified and the standardized handwriting image to be registered into the handwriting author feature extraction model using the same set of model parameters to generate the feature vector of the identity to be verified and the feature vector of the registered identity. The feature vectors of the identity to be verified and the feature vectors of the registered identity are associated according to the pairwise correspondence.

10. A deep learning-based handwriting author verification system according to claim 9, characterized in that, The operation of calculating feature similarity and generating author consistency verification results includes: Based on the pairwise correspondence, obtain the identity feature vector to be verified and the registered identity feature vector from the handwriting identity feature vector set; Perform cosine similarity calculation on the feature vector of the identity to be verified and the feature vector of the registered identity to obtain the corresponding feature similarity; Retrieve a pre-determined author verification threshold and compare the feature similarity with the author verification threshold; When the feature similarity is not lower than the author verification threshold, the corresponding image pair is marked as having the same author; when the feature similarity is lower than the author verification threshold, the corresponding image pair is marked as having different authors. Image pairs, feature similarity, and corresponding labels are associated to generate author consistency verification results.

Images