A method for identifying a doctoral and master's degree thesis authorization page

Abstract

The application discloses a kind of doctoral and master degree thesis authorization page identification method, comprising: extracting electronic version degree thesis by page to form image set to be examined;According to the image set to be examined, paper version degree thesis is scanned to generate image set to be examined;Image set to be examined is sent into authorization page identification classifier, it is judged whether there is authorization page;If not, output the conclusion without authorization page, if there is, then execute next step;Handwritten signature of author or tutor is detected in authorization page;Output authorization page automatic detection result.The application converts effective authorization page into image classification and target detection problem by automatic detection, and realizes the automatic detection of authorization page and the automatic discrimination of whether authorization page is effective by using image classification and target detection technology based on deep convolution network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic document information recognition, and in particular to a method for identifying authorization pages of doctoral and master's theses. BACKGROUND

[0002] Digital publishing of doctoral and master's theses requires effective authorization of the authors and their schools. In the process of digital processing of doctoral and master's theses, it is an important task to automatically detect whether the theses contain effective authorization pages. The authorization pages are mainly from the electronic or paper versions of the theses submitted by the authors, and an effective authorization page must contain a handwritten signature of the author or the tutor. How to automatically detect the authorization pages from the theses and how to determine whether the authorization pages are effective are the focuses of the present application. Poor-quality authorization pages have problems such as white balance, skew, and shadow, which make it difficult to automatically recognize effective authorization pages.

[0003] In recent years, with the rise of deep learning technology, deep convolution neural networks have been widely used in image classification. The advantage of deep convolution neural networks is that they can automatically discover the feature rules of samples and automatically obtain target features, overcoming the low efficiency and performance of manual feature extraction. Object detection is a major research direction of computer vision and has wide applications in face recognition, pedestrian detection, vehicle detection, traffic sign detection, video target detection, and medical image recognition. With the development of deep learning technology, object detection has made significant breakthroughs.

[0004] The present application aims to automatically detect effective authorization pages, and converts the task of automatically detecting effective authorization pages into image classification and object detection problems, and successfully realizes the automatic detection of authorization pages and the automatic determination of whether the authorization pages are effective by using image classification and object detection technology based on deep convolution networks. SUMMARY

[0005] To solve the above technical problems, the purpose of the present application is to provide a method for identifying authorization pages of doctoral and master's theses.

[0006] The purpose of the present application is achieved by the following technical solutions:

[0007] A method for identifying authorization pages of doctoral and master's theses, comprising:

[0008] A. Extracting the electronic version of the thesis by page to form a set of images to be detected;

[0009] B. Scanning the paper version of the thesis by page to generate a set of images to be detected;

[0010] C. inputting the image set to be detected into the authorized page recognition classifier to determine whether the authorized page exists; if not, outputting the conclusion of no authorized page, and if yes, executing step D;

[0011] D. detecting the handwritten signature of the author or the tutor in the authorized page; if the handwritten signature of the author or the tutor is detected in the authorized page, the authorized page is regarded as a valid authorized page, and if the handwritten signature of the author or the tutor is not detected in the authorized page, the authorized page is regarded as an invalid authorized page;

[0012] E. outputting the automatic detection result of the authorized page.

[0013] In the technical solution, the image set to be detected refers to the 2nd to 10th pages of a doctoral dissertation and the last 5 pages of the dissertation.

[0014] In the technical solution, the authorized page recognition classifier is an image classifier composed of a deep convolution model; the image classifier divides the image to be detected into three categories: authorized page, table of contents page and other pages; the other pages refer to other types of image pages in the image to be detected except the authorized page and the table of contents page, such as Chinese abstract page, English abstract page, dissertation text page, reference page, thanks page, etc.

[0015] In the technical solution, the detection of the handwritten signature of the author or the tutor in the authorized page refers to the detection of the handwritten signature area of the author or the tutor in the authorized page by using a FasterRCNN-based target detector; the target block types of the FasterRCNN-based target detector include two types: handwritten signature block and printed signature block.

[0016] Compared with the prior art, one or more embodiments of the present application can have the following advantages:

[0017] 1) The present application converts the judgment of whether the authorized page exists in the doctoral dissertation into an image classification problem, designs an image classifier by using the deep learning idea, and effectively improves the accuracy of authorized page recognition;

[0018] 2) The judgment of whether the authorized page is valid is regarded as an image target detection problem, and the deep learning-based target detection technology is adopted to successfully realize the automatic discrimination of whether the authorized page is valid;

[0019] 3) The method for recognizing the authorized page of the doctoral dissertation provided by the present application has been actually put into production. In the process of digital processing of the doctoral dissertation, the method not only effectively reduces the labor cost of the authorized page detection link, but also improves the processing speed of the authorized page detection link. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1is a flow chart of the authorization page recognition of the master's degree thesis;

[0021] Figure 2 is an example of the authorization page;

[0022] Figure 3 is an example of the handwritten signature annotation. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with examples and drawings.

[0024] As shown in Figure 1 , the flow chart of the authorization page recognition of the doctoral and master's degree thesis includes:

[0025] A, the electronic version of the degree thesis is extracted by page to form a set of images to be detected;

[0026] B, the paper version of the degree thesis is scanned by page to generate a set of images to be detected;

[0027] C, the set of images to be detected is sent to the authorization page recognition classifier to determine whether there is an authorization page; if not, output the conclusion that there is no authorization page, if there is, then execute step D;

[0028] D, the handwritten signature of the author or the tutor in the authorization page is detected; if the handwritten signature of the author or the tutor in the authorization page is detected, the authorization page is considered as a valid authorization page, if the handwritten signature of the author or the tutor in the authorization page is not detected, the authorization page is considered as an invalid authorization page;

[0029] E, the automatic detection result of the authorization page is output.

[0030] The set of images to be detected in A and B refers to the 2nd to 10th pages of the doctoral and master's degree thesis and the last 5 pages of the thesis.

[0031] According to the statistical analysis of the authorization pages of various doctoral and master's degree theses, the distribution of the authorization pages in the first 10 pages or the last 5 pages accounts for 99.91%. It can be seen that the authorization pages are mostly concentrated in the first 10 pages or the last 5 pages of the thesis. The first page of the degree thesis generally contains information such as the title of the thesis, the author, the tutor, the school, etc., therefore, the 2nd to 10th pages of the degree thesis and the last 5 pages of the thesis are taken as the images to be detected.

[0032] The authorization page recognition classifier is an image classifier composed of a deep convolutional model; the image classifier divides the images to be detected into three categories: authorization pages, table of contents pages and other pages; the other pages refer to other types of image pages in the images to be detected except the authorization pages and the table of contents pages, such as Chinese abstract pages, English abstract pages, thesis text pages, reference pages, thank you pages, etc.

[0033] This example uses the Pytorch deep learning framework to train, validate and test the authorization page recognition classifier. From the 58237 master's degree thesis by page extraction (extracting pages 2 to 10 and the last 5 pages), 814984 images were obtained, of which 55966 authorization pages accounted for 6.87%, 114352 catalog pages accounted for 14.03%, and 644666 other pages accounted for 79.01%. 70%, 10% and 20% of the image set were used as the training set, validation set and test set respectively. The experimental environment for training and testing is as follows: processor: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz, memory (RAM): 16.0GB, operating system: Windows 10 Professional, 64-bit operating system, graphics card: NVIDIA GeForce GTX 1080Ti, graphics memory: 11G; Python: 3.7.1, Pytorch: 1.7.1, cudatoolkit: 10.2.89, cudnn: 7.6.5.

[0034] During the training of the authorization page recognition classifier, the author tried ResNet50, ResNet34, ResNet18, MobileNetv3_large, MobileNetv3_small and other models. The model selection of the authorization page recognition classifier considers the recognition accuracy, recall rate and running speed of each model, as well as the actual running software and hardware environment, and finally selects MobileNetv3_large.

[0035] In the test set, the recall rate of the authorization page recognition classifier for authorization pages reached 99.32%, and the authorization page recognition accuracy reached 98.40%. In the image set identified as authorization pages, 90.49% of the authorization pages had a confidence interval in the [0.99, 1.00] interval, and the authorization page accuracy in this interval reached 99.69%. It can be seen that the accuracy in the confidence interval is higher than that without using the confidence interval. When the accuracy in the confidence interval meets the actual requirements of digital processing, the detection results in the confidence interval do not need to be checked manually, reducing the labor cost of digital processing.

[0036] Detecting the handwritten signature of the author or supervisor in the authorization page refers to detecting the handwritten signature area of the author or supervisor in the authorization page using a FasterRCNN-based target detector. The target block type of the FasterRCNN-based target detector includes two types: handwritten signature block and printed signature block. The feature extraction link (Backbone) of the FasterRCNN-based target detector uses the MobileNetv3_large network.

[0037] This embodiment uses 29,769 license pages as a dataset to train and test a handwritten signature target detector based on Faster R-CNN, specifically including:

[0038] 1) Image preprocessing: The authorized page extracted from the master's and doctoral dissertations has dense pixels, so the original image needs to be scaled proportionally to width = 778 and height = 1100;

[0039] 2) Mark the target area: Mark the handwritten signature block and printed signature block in each authorization page. The marking information includes the width and height of the authorization page image, the type and position of each target block; save the marking information in XML file format and establish a one-to-one correspondence with the corresponding image file;

[0040] 3) Training the target detector: 70% of the dataset is used as the training set and 10% of the dataset is used as the validation set to train the target detector; during training, the optimizer is Mini Batch Gradient Descent (MBGD), Momentum = 0.9, training epoch = 15, the learning rate is 0.01 for the first 10 epochs, and the learning rate is reduced to 0.001 for the last 5 epochs;

[0041] 4) Test the target detector: Use 20% of the dataset as the test set to evaluate the target detector. The evaluation metrics are frames per second (FPS), precision, and recall.

[0042] The formula for calculating accuracy is as follows:

[0043]

[0044] Where n is the number of handwritten signature blocks identified in the test set, iou1 i Let i be the maximum value of the area crossover ratio (OCR) between the i-th identified handwritten signature block and all real handwritten signature blocks, when iou1 i When δ(iou1) > 0.5 i >0.5) = 1, otherwise δ(iou1) i >0.5) = 0;

[0045] The formula for calculating recall is:

[0046]

[0047] Where m is the number of real handwritten signature blocks in the test set, and iou2 iLet i be the maximum value of the area intersection-union ratio (IU2) between the i-th real handwritten signature block and all recognized handwritten signature blocks. i When δ(iou2) > ​​0.5 i >0.5) = 1, otherwise δ(iou2) i >0.5) = 0.

[0048] In the early stages of developing the handwritten signature target detector, only handwritten signature blocks were labeled when annotating the target region, omitting printed signature blocks. During testing, it was discovered that a large number of printed signatures were misidentified as handwritten signatures. To suppress such errors, the authors modified the target region annotation rules to label both handwritten and printed signature blocks. Currently, the handwritten signature target detector achieves a recall rate of 99.21% and an accuracy rate of 98.77% for handwritten signature blocks and is already in production use.

[0049] While the embodiments disclosed in this invention are as described above, the content is merely for the purpose of facilitating understanding of the invention and is not intended to limit the invention. Any person skilled in the art to which this invention pertains may make any modifications and variations in form and detail of the implementation without departing from the spirit and scope disclosed herein; however, the scope of patent protection for this invention shall still be determined by the scope defined in the appended claims.

Claims

1. A method for identifying the authorization pages of doctoral and master's theses, characterized in that, include: A. Extract electronic versions of dissertations page by page to form an image set to be checked; B. Scan the paper version of the thesis page by page to generate a set of images to be checked; C. Send the set of images to be inspected to the authorization page recognition classifier to determine whether an authorization page exists; If it does not exist, output the conclusion that there is no authorized page; if it exists, proceed to step D. D. Detect the author's or supervisor's handwritten signature on the authorization page; If a handwritten signature of the author or supervisor is detected on the authorization page, the authorization page is considered a valid authorization page; if no handwritten signature of the author or supervisor is detected on the authorization page, the authorization page is considered an invalid authorization page. E. Output the automatic detection results of the authorization page; In D, detecting the author's or supervisor's handwritten signature within the authorization page refers to using a Faster R-CNN-based object detector to detect the handwritten signature region of the author or supervisor within the authorization page; the target block types of the Faster R-CNN-based object detector include two types: handwritten signature blocks and printed signature blocks; the feature extraction stage of the Faster R-CNN-based object detector uses the MobileNetv3_large network. Using the valid and invalid authorization pages as a dataset, a Faster R-CNN-based handwritten signature target detector was trained and tested; specifically including: 1) Image preprocessing: The authorized page extracted from the master's and doctoral dissertations has dense pixels, so the original image needs to be scaled proportionally to width = 778 and height = 1100; 2) Mark the target area: Mark the handwritten signature block and printed signature block in each authorization page. The marking information includes the width and height of the authorization page image, the type and position of each target block; save the marking information in XML file format and establish a one-to-one correspondence with the corresponding image file; 3) Training the target detector: Use 70% of the dataset as the training set and 10% of the dataset as the validation set to train the target detector; during training, the optimizer selects mini-batch gradient descent, with Momentum = 0.9 and training epoch = 15. The learning rate is 0.01 in the first 10 epochs and decreases to 0.001 in the last 5 epochs. 4) Test the target detector: Use 20% of the dataset as the test set to evaluate the target detector. The evaluation metrics are frames per second (FPS), precision, and recall. The formula for calculating the accuracy (Precision) is as follows: Where n is the number of handwritten signature blocks identified in the test set, iou1 i Let i be the maximum value of the area crossover ratio (OCR) between the i-th identified handwritten signature block and all real handwritten signature blocks, when iou1 i When δ(iou1) > 0.5 i >0.5) = 1, otherwise δ(iou1) i >0.5) = 0; The formula for calculating recall is: Where m is the number of real handwritten signature blocks in the test set, and iou2 i Let i be the maximum value of the area intersection-union ratio (IU2) between the i-th real handwritten signature block and all recognized handwritten signature blocks. i When δ(iou2) > ​​0.5 i >0.5) = 1, otherwise δ(iou2) i >0.5) = 0; The model selection for the authorization page recognition classifier comprehensively considered the recognition accuracy, recall, and running speed of each model, as well as the actual hardware and software environment, and ultimately selected MobileNetv3_large; The license page recognition classifier directly outputs the result when 90.49% of the license pages in the image set identified as license pages are within the range of [0.99, 1.00], without requiring manual inspection; The image sets to be inspected in A and B refer to pages 2 to 10 of the doctoral and master's theses and the last 5 pages of the theses; The authorized page identification classifier in C is an image classifier composed of a deep convolutional model; the image classifier divides the image to be inspected into three categories: authorized page, directory page and other pages; other pages refer to image pages of other types in the image to be inspected, excluding authorized pages and directory pages. To suppress the error of misidentifying a large number of printed signatures as handwritten signatures during testing, the target area annotation rules were modified to annotate both handwritten and printed signature blocks.

Images