Method for automatic extraction and offline comparison of handwritten signature in file

Abstract

The invention belongs to the technical field of safety quality of nuclear power plants, and particularly relates to a method for automatic extraction and offline comparison of handwritten signatures in files. Comprising the following steps: step 1, preprocessing a scanning copy in a supplier quality file verified and confirmed by a nuclear power plant; 2, importing the preprocessed quality file into a database; step 3, providing a data set for signature region identification by marking signature region positions of a certain number of quality files; and step 4, training a signature region recognition algorithm based on Yolo. Identifying a signature area in the quality file; step 5, performing paired labeling on signatures labeled in the signature area in the quality file; step 6, training an off-line signature test model by using the labeled signature image; and step 7, identifying a signature image by using the trained Yolo model, and then carrying out offline signature inspection. The method has the beneficial effects that the checking efficiency of signature comparison and the accuracy of the counterfeit condition can be greatly improved.

Description

Technical Field

[0001] The invention belongs to the technical field of safety and quality of nuclear power plants, and in particular relates to a method for automatically extracting and off-line comparing handwritten signatures in files. Background Art

[0002] For items purchased by nuclear power plants, suppliers are required to provide relevant quality documents, including certificates, quality plans, test reports, etc.

[0003] At present, nuclear power plants mainly use manual inspection to confirm the accuracy of handwritten signatures on quality documents provided by suppliers. However, manual inspection is inefficient on the one hand, and it is difficult to identify signature forgeries on the other hand, such as counterfeiting historical signature images to sign in quality plans, and finding different people to sign on behalf of others.

[0004] In order to efficiently and accurately detect the problem of signature forgery in supplier quality documents, it is necessary to effectively extract Chinese handwritten signature images from scanned quality documents, and then use relevant computer technology to automatically find signature images that may be forged. However, there are a large number of suppliers providing items and services to nuclear power plants, and their document formats and styles are diverse, which increases the difficulty of regional recognition and screening and extraction of signature images. Summary of the invention

[0005] The purpose of the present invention is to provide a method for automatic extraction and offline comparison of handwritten signatures in documents. By establishing signature area recognition and offline signature data sets, image processing, target detection, offline signature recognition and other technologies are used to realize automatic extraction and comparison of signature images in supplier quality documents, providing a basis for the intelligent recognition of nuclear power plant supplier quality documents.

[0006] The technical solution of the present invention is as follows: A method for automatically extracting and offline comparing handwritten signatures in a document comprises the following steps:

[0007] Step 1: Pre-process the scanned copies of the supplier's quality documents that have been verified and confirmed by the nuclear power plant;

[0008] Step 2: Import the pre-processed quality files into the database;

[0009] Step 3: Provide a data set for signature area recognition by marking the signature area positions of a certain number of quality documents;

[0010] Step 4: Train the Yolo-based signature region recognition algorithm to identify the signature region in the quality file;

[0011] Step 5: Pair the signatures marked in the signature area in the quality file;

[0012] Step 6: Use the annotated signature image to train the offline signature verification model;

[0013] Step 7: Use the signature image recognized by the trained Yolo model to perform offline signature verification.

[0014] The step 1 comprises:

[0015] (1) Convert the PDF version of the test report in the supplier's quality document into image format;

[0016] (2) Preprocess the converted images.

[0017] The step 2 includes: importing the picture obtained in step 1 into the database.

[0018] The step 3 comprises:

[0019] (1) Use the annotation tool to mark the signature area of ​​the image imported in step 2;

[0020] (2) Store the absolute coordinate field of the marked coordinate area into the database and add the image metadata field.

[0021] The step 4 comprises:

[0022] (1) Put a small amount of labeled quality files into the Yolo model for training;

[0023] (2) Obtain and save the weight parameters of the signature image area recognition model.

[0024] The step 5 comprises:

[0025] (1) Crop the signature image obtained in step 4 and select a small amount to put into the signature annotation tool;

[0026] (2) Classify each signature image in the signature annotation tool, and divide each person's signature into one category;

[0027] (3) randomly extract signatures from the signature annotation tool and pair them up to construct signature data pairs for training;

[0028] (4) The true signature is stored in the database as the standard signature for signature verification.

[0029] The step 6 comprises:

[0030] (1) Put the constructed signature data pair into the offline signature verification model for training;

[0031] (2) Construct an offline signature verification model architecture of convolutional neural network-graph residual convolution-forward propagation network;

[0032] (3) Training the offline signature verification model;

[0033] (4) Obtain and save the weight of the offline signature recognition model.

[0034] The step 7 comprises:

[0035] (1) Import the signature region recognition model trained in step 4 and the offline signature recognition model trained in step 6;

[0036] (2) Import all quality documents to be identified into the database through steps 1 and 2;

[0037] (3) Put the quality file image imported in step 7 into the signature area recognition model obtained in step 4 to perform signature area recognition;

[0038] (4) Associating the identified signature area with the quality file and storing the absolute coordinates of the signature area in a database;

[0039] (5) The identified signature area is cropped and paired with the standard signature put into the database in step 5 to construct a data pair for signature verification;

[0040] (6) Put the constructed signature verification data pair into the offline signature verification model trained in step 6 for verification;

[0041] (7) The authenticity label of the signature is saved in the database and mapped to the signature image area of ​​the quality file.

[0042] The beneficial effect of the present invention is that: at present, nuclear power plants mainly rely on manual work to compare signatures in quality documents submitted by suppliers, which requires a lot of human resources, and some signatures are not easy to identify and discover manually. The present invention uses a labeling method, image recognition, target detection, and offline signature verification technologies to automatically extract signature images from quality documents submitted by suppliers of nuclear power plants, providing a basis for intelligently identifying possible frauds, greatly improving the efficiency of signature comparison and the accuracy of frauds, and preventing fraudulent items from entering nuclear power plants in advance, thereby maximizing the safety, availability, and reliability of nuclear power unit systems and equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Extract business architecture diagram for test results

[0044] Figure 2 Flow chart of the method for automatic extraction and comparison of signature images in the implementation of the present invention;

[0045] Figure 3 A diagram of the architecture of the offline signature comparison model in the implementation of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] The present invention utilizes image processing, target detection, offline signature recognition and other technologies, and adopts signature area recognition based on Yolo framework and signature image offline comparison algorithm based on Inception Resnet.

[0048] The method for automatically extracting and offline comparing handwritten signatures in documents provided by the present invention comprises the following steps:

[0049] Step 1: Pre-process the scanned copies of the supplier's quality documents that have been verified and confirmed by the nuclear power plant.

[0050] The step 1 specifically includes:

[0051] (1) Convert the PDF version of the test report in the supplier's quality document into image format;

[0052] (2) Preprocess the converted images.

[0053] Step 2: Import the preprocessed quality files into the database.

[0054] The step 2 specifically includes:

[0055] (1) Import the image obtained in step 1 into the database.

[0056] Step 3: Provide a dataset for signature area recognition by marking the signature area locations of a certain number of quality documents.

[0057] The step 3 specifically includes:

[0058] (1) Use the annotation tool to mark the signature area of ​​the image imported in step 2;

[0059] (2) Store the absolute coordinate field of the marked coordinate area into the database and add the image metadata field.

[0060] Step 4: Train the Yolo-based signature region recognition algorithm to identify the signature region in the quality file.

[0061] The step 4 specifically includes:

[0062] (1) Put a small amount of labeled quality files into the Yolo model for training;

[0063] (2) Obtain and save the weight parameters of the signature image area recognition model.

[0064] Step 5: Pair the signatures marked in the signature area in the quality file.

[0065] The step 5 specifically includes:

[0066] (1) Crop the signature image obtained in step 4 and select a small amount to put into the signature annotation tool;

[0067] (2) Classify each signature image in the signature annotation tool, and divide each person's signature into one category;

[0068] (3) randomly extract signatures from the signature annotation tool and pair them up to construct signature data pairs for training;

[0069] (4) The true signature is stored in the database as the standard signature for signature verification.

[0070] Step 6: Use the annotated signature images to train the offline signature verification model.

[0071] The step 6 specifically includes:

[0072] (1) Put the constructed signature data pair into the offline signature verification model for training;

[0073] (2) Construct an offline signature verification model architecture of convolutional neural network-graph residual convolution-forward propagation network;

[0074] (3) Training the offline signature verification model;

[0075] (4) Obtain and save the weight of the offline signature recognition model.

[0076] Step 7: Use the signature image recognized by the trained Yolo model to perform offline signature verification.

[0077] The step 7 specifically includes:

[0078] (1) Import the signature region recognition model trained in step 4 and the offline signature recognition model trained in step 6;

[0079] (2) Import all quality documents to be identified into the database through steps 1 and 2;

[0080] (3) Put the quality file image imported in step 7 into the signature area recognition model obtained in step 4 to perform signature area recognition;

[0081] (4) Associating the identified signature area with the quality file and storing the absolute coordinates of the signature area in a database;

[0082] (5) The identified signature area is cropped and paired with the standard signature put into the database in step 5 to construct a data pair for signature verification;

[0083] (6) Put the constructed signature verification data pair into the offline signature verification model trained in step 6 for verification;

[0084] (7) The authenticity label of the signature is saved in the database and mapped to the signature image area of ​​the quality file.

[0085] When read in conjunction with the accompanying drawings, a full understanding of the present invention can be obtained based on the process logic in the drawings. Figure 1 This is the architecture diagram of the signature image extraction and automatic comparison service provided by the present invention. Figure 2 This is a flow chart of the method for automatically extracting and comparing signature images in supplier quality documents provided by the present invention. The accompanying drawings are as follows:

[0086] Example

[0087] A method for automatically extracting and comparing signature images in supplier quality documents, the business architecture diagram is as follows Figure 1 As shown. The test result information extraction can be divided into data manager, offline model and annotation platform. The offline model service requires data support from the data manager. The data of the data manager includes training data and document data to be identified. The training data is annotated by the annotation platform based on the original document data. The operation process of the entire signature automatic extraction and comparison system can be divided into three processes: model data annotation, model training and document recognition comparison process, see Figure 2 As shown, the following steps are included:

[0088] Step 1: Preprocess the scanned version of the supplier's quality documents that have been verified and confirmed by the nuclear power plant. Specifically, it includes: converting the PDF version of the test report in the supplier's quality documents that have been verified and confirmed by the nuclear power plant into an image format; performing noise reduction, grayscale, rotation angle determination, and affine transformation on the converted images.

[0089] Step 2: Import the preprocessed quality report image into the database.

[0090] Specifically, the images obtained in step 1 are manually screened to eliminate redundancy and then imported into the database, and reference points are set for positioning of subsequent annotation coordinates;

[0091] Step 3: Provide a dataset for signature area recognition by marking the signature area locations of a certain number of quality documents.

[0092] Specifically, it includes: reading the image imported into the database in step 2 through a labeling tool, manually calibrating the signature image area, labeling the signature area based on the labeling reference point and the border position of the image to be identified, such as the signature positions of "reviewer" and "person in charge" in the quality report, and recording the relative position; after the user triggers the labeling, the labeling data is converted into absolute coordinates and stored in the database.

[0093] Step 4: Train the Yolo-based signature region recognition algorithm to identify the signature region in the quality file.

[0094] Specifically, when the user triggers the completion of the annotation in step 3 and completes uploading the data, the data in step 3 is imported into the Yolo model for training.

[0095] Step 5: Pair the signatures marked in the signature area in the quality file.

[0096] Specifically, the process includes: cropping the image identified in step 4, selecting part of the data and putting it into the signature annotation tool; classifying each signature and classifying signatures of the same person into the same category; constructing data pairs from the classified signatures; and putting the true signature image into the database as a reference image for signature comparison.

[0097] Step 6: Use the annotated signature images to train the offline signature verification model.

[0098] Specifically, the method includes: putting the constructed signature data pair into the offline signature verification model for training; considering that the blank information in the signature image occupies most of the image features, the offline signature verification model will first convert this group of signature images into corresponding graph data structures through multi-layer convolutional neural networks, which are used to replace the spatial segmentation grid in the traditional signature image comparison, reduce the information redundancy brought by the blank background to the model, and improve the learning efficiency of the model for key information in the signature image; in the constructed graph data structure, add trainable position embedding information to strengthen the spatial information relationship in the image; put the graph data structure with added position embedding into the graph residual convolution to extract the detailed features of the signature image, and output the feature vector of the corresponding image through the feedforward network. During the training process, each group of samples contains two signature images and corresponding label information. According to the image features and label information, the distance between the feature vectors of the two images is calculated using the Euclidean distance; after the training is completed, the multi-layer convolutional neural network-graph residual network-feedforward neural network is saved as the weight parameter of the offline signature verification model.

[0099] Step 7: Use the signature image recognized by the trained Yolo model to perform offline signature verification.

[0100] Import the trained signature area recognition model in step 4 and the offline signature recognition model trained in step 6; import all the quality files to be recognized into the database through steps 1 and 2; put the quality file image imported into the database into the signature area recognition model in step 4 to perform signature area recognition; crop the recognized signature area in the manner in step 5, and construct a data pair with the reference signature in the signature database; put the signature data constructed in the previous step into the offline signature verification model in step 6 to obtain the feature vector of the signature area image, and calculate the distance with the feature vector of the reference signature in the signature database, and use the distance between the two as the basis for judging the authenticity of the signature; obtain the authenticity label of the signature data, associate it with the quality file, and judge the authenticity of the quality file.

Claims

1. A method for automatically extracting and offline comparing handwritten signatures in documents, characterized in that: The steps include: Step 1: Pre-process the scanned copies of the supplier's quality documents that have been verified and confirmed by the nuclear power plant; Step 2: Import the pre-processed quality files into the database; Step 3: Provide a data set for signature area recognition by marking the signature area positions of a certain number of quality documents; Step 4: Train the Yolo-based signature region recognition algorithm to identify the signature region in the quality file; Step 5: Pair the signatures marked in the signature area in the quality file; Step 6: Use the annotated signature image to train the offline signature verification model; Step 7: Use the signature image recognized by the trained Yolo model to perform offline signature verification.

2. The method for automatic extraction and offline comparison of handwritten signatures in documents as claimed in claim 1, characterized in that: The step 1 comprises: (1) Convert the PDF version of the test report in the supplier's quality document into image format; (2) Preprocess the converted images.

3. The method for automatic extraction and offline comparison of handwritten signatures in documents as claimed in claim 1, characterized in that: The step 2 includes: importing the picture obtained in step 1 into the database.

4. The method for automatic extraction and offline comparison of handwritten signatures in documents as claimed in claim 1, characterized in that: The step 3 comprises: (1) Use the annotation tool to mark the signature area of ​​the image imported in step 2; (2) Store the absolute coordinate field of the marked coordinate area into the database and add the image metadata field.

5. The method for automatic extraction and offline comparison of handwritten signatures in documents as claimed in claim 1, characterized in that: The step 4 comprises: (1) Put a small amount of labeled quality files into the Yolo model for training; (2) Obtain and save the weight parameters of the signature image area recognition model.

6. The method for automatic extraction and offline comparison of handwritten signatures in documents as claimed in claim 1, characterized in that: The step 5 comprises: (1) Crop the signature image obtained in step 4 and select a small amount to put into the signature annotation tool; (2) Classify each signature image in the signature annotation tool, and divide each person's signature into one category; (3) randomly extract signatures from the signature annotation tool and pair them up to construct signature data pairs for training; (4) The true signature is stored in the database as the standard signature for signature verification.

7. The method for automatic extraction and offline comparison of handwritten signatures in documents as claimed in claim 1, characterized in that: The step 6 comprises: (1) Put the constructed signature data pair into the offline signature verification model for training; (2) Construct an offline signature verification model architecture of convolutional neural network-graph residual convolution-forward propagation network; (3) Training the offline signature verification model; (4) Obtain and save the weight of the offline signature recognition model.

8. The method for automatic extraction and offline comparison of handwritten signatures in documents as claimed in claim 1, characterized in that: The step 7 comprises: (1) Import the signature region recognition model trained in step 4 and the offline signature recognition model trained in step 6; (2) Import all quality documents to be identified into the database through steps 1 and 2; (3) Put the quality file image imported in step 7 into the signature area recognition model obtained in step 4 to perform signature area recognition; (4) Associating the identified signature area with the quality file and storing the absolute coordinates of the signature area in a database; (5) The identified signature area is cropped and paired with the standard signature put into the database in step 5 to construct a data pair for signature verification; (6) Put the constructed signature verification data pair into the offline signature verification model trained in step 6 for verification; (7) The authenticity label of the signature is saved in the database and mapped to the signature image area of ​​the quality file.

Images