A deep learning-based ancient character open set recognition system and method

Abstract

The application discloses a kind of ancient character open set identification system and method based on deep learning, using deep learning technology to carry out preliminary ancient character identification, not only can the known category of ancient character sample be classified, simultaneously can the category sample unknown in original training set be predicted alone, and be marked as "unknown" category, open set universality is stronger.And for unknown category, provide the function of similar character recommendation, with the most similar multiple character of ancient character of unknown category as recommendation, can help unknown category ancient character to be interpreted according to known category similar character, facilitate the development of subsequent interpretation work, realize ancient character open set identification and recommend similar character to the ancient character judged as unknown category, improve ancient character interpretation efficiency, while can reduce the cost of artificial classification, realize automatic preliminary screening, reduce labor cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a deep learning-based ancient character open set recognition system and method. BACKGROUND

[0002] Ancient Chinese characters have a long history and are of great value. With the continuous development of archaeological work, more and more ancient characters are being discovered. However, due to the difficulty of excavating ancient character images, the overall amount of existing ancient character data is relatively small. There are a large number of samples that exceed the known categories of ancient characters in newly excavated ancient characters. This makes the recognition of newly excavated ancient characters an open set recognition task. The open set recognition task has two key parts, mainly including: identifying and judging the samples in the test set that exceed the sample categories in the training set as unknown categories, and classifying the remaining samples judged as known categories, to ultimately achieve the purpose and effect of open set recognition. Therefore, the open set recognition task greatly increases the difficulty of recognizing ancient character samples.

[0003] In recent years, artificial intelligence technology has been more and more combined with character recognition tasks. Applying artificial intelligence technology to current character recognition tasks helps to improve the efficiency of manual character recognition and has better accuracy, greatly reducing the workload of manual classification in character recognition tasks. However, due to the different characteristics of ancient characters and existing characters, there are a large number of newly excavated ancient character samples that do not have existing sample categories. In addition, due to the use characteristics of characters and the characteristics of ancient characters that cannot be created by modern people, existing character recognition methods do not perform well for this type of task.

[0004] Current deep learning-based character recognition methods are mainly based on more standardized character data sets, and the character data set is a closed set. The good character recognition classification data set is established by modern handwriting, which cannot be replicated for ancient characters with this writing style. For open data sets with unbalanced data, it is not completely applicable. For example, CN108664996A discloses a deep learning-based ancient character recognition method and system, which includes: obtaining an ancient character image data set and making classification labels; preprocessing the images in the image data set and expanding the data set through image augmentation; extracting features from the preprocessed and image-augmented image data set through a deep neural network, and classifying the extracted features through a classifier to recognize the evolution of different time seal scripts; using manually designed features to perform auxiliary training using a template matching method to further classify the classification results of the classifier.

[0005] The existing technology has the following defects:

[0006] (1) The dataset is a closed set: Current deep learning-based text recognition methods are mainly based on closed sets. The text datasets used are closed sets, and text data samples of unknown categories cannot be identified as unknown categories.

[0007] (2) The dataset is relatively balanced: The current text dataset has a consistent number of text samples in each category and can be generated manually, resulting in a high degree of dataset standardization.

[0008] (3) Similar ancient characters recommendation function: Existing technology does not propose identification and processing methods for ancient characters of unknown categories.

[0009] (4) Disadvantages of existing open set identification methods: Open set identification methods mainly include outlier exposure and probability threshold estimation. Outlier exposure treats small sample categories as outliers and open set content, while the remaining samples are considered closed set content. During prediction, the categories of these small samples are directly predicted as open sets, resulting in a hard decision and low accuracy. Probability threshold estimation, because the threshold is manually adjusted, is not stable enough. Furthermore, due to the similarity of ancient characters, this method does not perform well in predicting categories near the threshold. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention proposes a deep learning-based open set recognition system and method for ancient Chinese characters. This system can identify ancient Chinese character samples of known and unknown categories and provide recommendations of several ancient Chinese characters of known categories with similar glyphs for ancient Chinese character samples identified as belonging to unknown categories, thereby assisting in the recognition of ancient Chinese characters of unknown categories.

[0011] To achieve the above objectives, the present invention provides the following technical solution:

[0012] On the one hand, this invention provides a method for recognizing open sets of ancient characters based on deep learning, comprising the following steps:

[0013] S1. A deep learning network is used to divide the ancient character samples into open and closed sets through a two-stage open and closed set screening method, resulting in ancient character samples of closed sets and ancient character samples of open sets.

[0014] S2. Use a deep learning network to classify the ancient script samples that were determined to be closed sets in step S1 according to the known category labels.

[0015] S3. Select the ancient characters that are closest to the ancient characters in the ancient character samples that are identified as open sets, as the ancient characters to be recommended for visualization, and make recommendations for similar-looking characters for visualization.

[0016] Furthermore, in step S1, the specific method for dividing the ancient character samples into open and closed sets is as follows:

[0017] Firstly, the samples in the ancient character category in the training set are set as open set ancient character samples, and the remaining samples in the training set are closed set ancient character samples; a network for preliminarily distinguishing open set and closed set ancient character samples is trained, and then the first open set and closed set ancient character prediction screening is performed on the test set to select the ancient character samples predicted as the open set category in this prediction; the ancient character samples predicted as the open set category are screened and classified again using the original closed set classification network; after the two screenings of open set and closed set ancient character samples, the ancient character samples finally determined as the open set category in both screenings are extracted and taken as the ancient character samples finally determined as the open set.

[0018] Further, the network structure for training is that a threshold is set before the probabilities of each category in the output layer, the ancient character samples with a probability value greater than the set threshold are determined as closed set ancient character samples, and the ancient character samples with a probability value less than the set threshold are determined as open set ancient character samples.

[0019] Further, the training set uses an image enhancement method and a writing style changing method to expand the ancient character data set samples.

[0020] Further, the image enhancement method includes image dilation, image erosion and image rotation.

[0021] Further, the writing style changing method is to expand the ancient character data set by multiple people imitating writing.

[0022] Further, the deep learning network in steps S1 and S2 is a ResNet network or a convolutional neural network.

[0023] Further, the classification method in step S2 is that, for the ancient character samples determined as known categories in the open set and closed set distinguishing module, the category with the highest probability in the prediction result is taken as the output category of the prediction result, and the classification is performed according to the category label in the training set.

[0024] Further, the specific method of step S3 is that, for the ancient characters determined as unknown categories in the open set and closed set distinguishing module, the five categories with the highest probabilities in the prediction result in the closed set recognition network are taken as preselected categories, and the corresponding category labels are recorded, the samples of the corresponding categories are found in the training set, and are put together with the ancient characters of unknown categories as visual similar character recommendations.

[0025] On the other hand, the application also provides an ancient character open set recognition system based on deep learning, which comprises the following modules to realize the method of any one of the above.

[0026] The open set and closed set distinguishing module based on the deep learning network divides the open set and closed set ancient character samples by using the deep learning network.

[0027] The closed-set ancient character classification module based on the deep learning network: using the deep learning network to classify the ancient character samples determined as the closed set according to the known category label;

[0028] The visual similar character recommendation module: the several ancient Chinese characters closest to the ancient character determined as the open set sample learned in the open and closed set distinguishing module based on the deep learning network are taken as the visual recommended ancient Chinese characters, and the visual similar character recommendation is made.

[0029] Compared with the prior art, the beneficial effects of the present application are:

[0030] (1) The ancient character open set recognition system and method based on deep learning of the present application apply the designed network to the ancient character data set of the open set for training, which can not only classify the known category of ancient character samples, but also predict the unknown category samples in the original training set and mark them as the "unknown" category, and the open set is more universal.

[0031] (2) The ancient character open set recognition system and method based on deep learning of the present application provide the function of similar character recommendation for the unknown category, take the multiple characters closest to the unknown category as the recommendation, which can help the unknown category of ancient characters to be interpreted according to the known category of similar characters, facilitate the subsequent interpretation work, realize the open set recognition of ancient characters and recommend similar characters for the ancient characters determined as the unknown category, and improve the interpretation efficiency of ancient characters.

[0032] (3) The ancient character open set recognition system and method based on deep learning of the present application use the deep learning technology for preliminary ancient character recognition, which can not only improve the efficiency, but also reduce the cost of artificial classification, realize automatic preliminary screening, and reduce the labor cost. DETAILED DESCRIPTION

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0034] Figure 1 The ancient character open set recognition system module based on deep learning provided by the embodiment of the present application.

[0035] Figure 2 The open set recognition flowchart provided by the embodiment of the present application.

[0036] Figure 3 The network structure schematic diagram taken ResNet as an example provided by the embodiment of the present application.

[0037] Figure 4 Three window view angles in a display provided by an embodiment of the present application.

[0038] Figure 5 An image rotation schematic diagram provided by an embodiment of the present application.

[0039] Figure 6 Handwriting extension dataset effect provided by an embodiment of the present application.

[0040] Figure 7 Visual similar character recommendation result instance provided by an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to better understand the technical solutions, the method of the present application will be described in detail below with reference to the accompanying drawings.

[0042] The ancient character open set recognition system based on deep learning of the present application, as shown in FIG. 1, comprises the following modules: Figure 1 The open set and closed set distinguishing module based on deep learning network: the deep learning network is used to distinguish the open set and closed set ancient character samples;

[0043] The closed set ancient character classification module based on deep learning network: the deep learning network is used to classify the ancient character samples determined as the closed set according to the known category label;

[0044] The visual similar character recommendation module: the several ancient Chinese characters closest to the ancient character determined as the open set sample are learned in the open set and closed set distinguishing module based on deep learning network as the visual recommended ancient Chinese characters, and the visual similar character recommendation is made.

[0045] The specific implementation method of the above system is as follows:

[0046] S1: the deep learning network is used to distinguish the open set and closed set ancient character samples through twice open set and closed set screening, and the closed set ancient character samples and the open set ancient character samples are obtained; the deep learning network can be a ResNet network (residual neural network) or a convolutional neural network;

[0047] S2: the deep learning network is used to classify the ancient character samples determined as the closed set in step S1 according to the known category label; the deep learning network can be a ResNet network or a convolutional neural network; the classification method is specifically as follows: for the ancient character samples determined as the known category in the open set and closed set distinguishing module, the category with the highest probability in the prediction result is taken as the output category of the prediction result, and the classification is made according to the category label in the training set;

[0048]

[0049] ​S3, the ancient character closest to the ancient character sample judged as an open set learned in step S1 is taken as a visual recommended ancient character, and a visual similar character recommendation is made. The specific method is: for the ancient character that has been judged as an unknown category in the open and closed set distinguishing module, the five categories with the highest probability in the prediction results in the closed set recognition network are taken as pre-selected categories, and the corresponding category labels are recorded, and the corresponding category samples in the training set are found and put together with the unknown category ancient character as a visual similar character recommendation.

[0050] In step S1 of the present application, as shown in Figure 2 The specific method for dividing the open set and closed set ancient character samples is:

[0051] First, the samples in the ancient character category with small sample size (in the present application, in practice, the sample category with sample size less than 2 is identified as a small sample, and the specific method is different due to different data set data distribution) in the training set are set as open set ancient character samples (unknown ancient character), and the remaining samples in the training set are closed set ancient character samples (known ancient character); a deep learning network for preliminary distinguishing open and closed set ancient character samples is trained, when the categories in the training set that are identified as closed set samples can be identified as closed set categories in the test set, and part of the unknown categories can be identified as open set categories, it is considered that the preliminary network is well trained; then, according to the labeled open set and closed set labels, the first open and closed set ancient character prediction screening is performed on the test set, and the ancient character samples predicted as open set categories in this prediction are selected; then, the ancient character samples predicted as open set categories are classified again using the original closed set classification network (such as resnet network, which is trained based on the data that each ancient character category in the training set is given a label corresponding to the meaning of the ancient character, for example, the ancient character used by ancient people to represent "one" is also given the category "one", and each different ancient character has a one-to-one corresponding category label); this classification includes two parts, the first part is to determine the category samples considered as open set according to the threshold, and the second part is to classify the remaining samples according to the one-to-one corresponding category label of the ancient character itself; after the two times of open and closed set ancient character sample screening, the ancient character samples that are identified as open set in the final two times are extracted as the final ancient character samples identified as open set.

[0052] Among them, as shown in Figure 3As shown, the trained network structure is: before outputting the probability value of each category (the category with the meaning of the original ancient character) in the output layer, a threshold value is set, the ancient character sample with the probability value greater than the set threshold value is determined as the closed set ancient character sample (known ancient character), and the ancient character sample with the probability value less than the set threshold value is determined as the open set ancient character sample (unknown ancient character). Wherein, the threshold setting method is a method set by artificial, the final threshold is determined in a smaller range through artificial adjustment of parameters, and a better network output result is obtained.

[0053] The recognition method screened through two times of open and closed set solves the problem that the existing method causes the small sample closed set category ancient character to be recognized as an open set category, and can also solve the problem of fuzzy boundary near the threshold in the probability threshold method.

[0054] In the application, in order to solve the small sample problem in ancient character recognition, three image enhancement methods and one writing style changing method are mainly used to expand the ancient character data set sample, so that the category with too few samples has more sample quantity.

[0055] As shown in Figure 4 The image enhancement method includes image dilation, image erosion and image rotation.

[0056] Method one: image dilation, which is equivalent to writing the same Chinese character using a relatively thick pen in life without changing the writing style.

[0057] The operator symbol of image dilation is "⊕", and the operation formula is as follows:

[0058]

[0059] The formula indicates that B is used to expand the image A, wherein B is a convolution kernel, which can be square or circular, and is calculated by convolution of the convolution kernel B and the image A. Each pixel point in the scanned image A is added with the element of the convolution kernel B and the element of the binary image A. The maximum value of the pixel points in the region covered by B is calculated, and the pixel value of the reference point is replaced, so as to realize image dilation.

[0060] Method two: image erosion, which is equivalent to writing the same Chinese character using a relatively thin pen in life without changing the writing style.

[0061] The operator symbol of image erosion is "-", and the operation formula is as follows:

[0062]

[0063] The formula shows that the image A is eroded by the convolution kernel B. By convolving the convolution kernel B with the image A, the minimum value of the pixel points in the area covered by B is calculated, and the minimum value is used to replace the pixel value of the reference point. Finally, the operation effect A-B is realized.

[0064] Method three: image rotation, which is equivalent to changing the paper angle when writing in life, but the rotation angle is not more than ±7°, and the writing style is not changed.

[0065] The rotation of the image is similar to the rotation of the XOY point in the coordinate plane, and the rotation is shown in Figure 5 , point P is rotated counterclockwise by an angle β with the origin O as the rotation center to obtain point Q.

[0066] As shown in Figure 6 , the method for changing the writing style is to expand the ancient character data set by multiple people imitating writing.

[0067] The system and method of the present application are tested on the Tsinghua bamboo ancient character data set. In actual testing, the first, second, third, fourth and fifth volumes of the Tsinghua bamboo are used as the training set of the data set of the present application, and there are a total of 1947 categories and 13161 samples in the training set. Then the seventh volume of the Tsinghua bamboo is used as the test set, and there are a total of 731 categories and 3299 samples. Among the 731 categories, 316 categories are unknown in the training set. The final test result of the open and closed set recognition can achieve an accuracy of more than 50% (only about 20% of the test set is open set samples), and on this basis, the unknown category samples are identified from the test set and distinguished from the closed set category samples. Then for the unknown category content identified, five most similar known category ancient Chinese characters are recommended as similar characters, as shown in Figure 7 , which facilitates the development of ancient character interpretation work.

[0068] In summary, the present application improves the effect of existing ancient character recognition, realizes the identification of known and unknown category ancient character samples, and provides several similar known category ancient character recommendations for ancient character samples judged as unknown category, helping to identify unknown category ancient characters.

[0069] The above only describes the preferred embodiments of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application is included in the protection scope of the present application.

Claims

1. A method for recognizing open sets of ancient Chinese characters based on deep learning, characterized in that, Includes the following steps: S1. A deep learning network is used to divide the ancient characters in the open and closed sets through a two-stage open and closed set screening method to obtain ancient characters in the closed set and ancient characters in the open set. The specific method for dividing the ancient characters in the open and closed sets is as follows: First, the samples in the ancient character category in the training set are set as ancient characters in the open set, and the remaining samples in the training set are ancient characters in the closed set. Train a network to initially distinguish between open and closed set ancient character samples, and then perform the first prediction and screening of open and closed set ancient characters on the test set, selecting the ancient character samples that are predicted to be open set in this prediction. The ancient script samples predicted to be in the open set category are then subjected to a second screening and classification using the original closed set classification network. After these two screenings of open and closed set ancient script samples, the ancient script samples that are ultimately identified as being in the open set category in both screenings are extracted and used as the final ancient script samples that are determined to be in the open set category. The trained network structure is as follows: a threshold is set before the probability of each category in the output layer. Ancient characters with a probability value greater than the set threshold are defined as closed set ancient characters, and ancient characters with a probability value less than the set threshold are defined as open set ancient characters. S2. Use a deep learning network to classify the ancient script samples that were determined to be closed sets in step S1 according to the known category labels. S3. Select the ancient characters that are closest to the ancient characters in the ancient character samples that are identified as open sets, as the ancient characters to be recommended for visualization, and make recommendations for similar-looking characters for visualization.

2. The method for recognizing open sets of ancient characters based on deep learning according to claim 1, characterized in that, The training set was augmented with image enhancement and handwriting style alteration methods to expand the ancient script dataset samples.

3. The method for recognizing open sets of ancient characters based on deep learning according to claim 2, characterized in that, Image enhancement methods include image dilation, image erosion, and image rotation.

4. The method for recognizing open sets of ancient characters based on deep learning according to claim 2, characterized in that, The method for changing the writing style is to expand the ancient script dataset by having multiple people imitate the writing style.

5. The method for recognizing open sets of ancient characters based on deep learning according to claim 1, characterized in that, The deep learning network used in steps S1 and S2 is a ResNet network.

6. The ancient script open set recognition method based on deep learning according to claim 1, characterized in that, The classification method in step S2 is as follows: for ancient character samples that have been identified as known categories in the open / closed set differentiation module, the category with the highest probability in the prediction results is taken as the prediction result output category, and classified according to the category labels in the training set.

7. The method for recognizing open sets of ancient characters based on deep learning according to claim 1, characterized in that, The specific method of step S3 is as follows: For ancient characters that have been judged as unknown categories in the open and closed set differentiation module, the five categories with the highest probability in the prediction results of the closed set recognition network are used as pre-selected categories, and the corresponding category labels are recorded. Samples of the corresponding categories are found in the training set and put together with ancient characters of unknown categories as recommendations for visual similar characters.

8. A deep learning-based system for recognizing open sets of ancient Chinese characters, characterized in that, The following modules are included to implement the method of any one of claims 1-7: The open and closed set differentiation module based on deep learning network: The deep learning network is used to divide the ancient characters samples in the open and closed sets to obtain ancient characters samples in the closed set and ancient characters samples in the open set. The specific method for dividing the ancient characters samples in the open and closed sets is as follows: First, the samples in the ancient characters category in the training set are set as ancient characters samples in the open set, and the remaining samples in the training set are ancient characters samples in the closed set. Train a network to initially distinguish between open and closed set ancient character samples, and then perform the first prediction and screening of open and closed set ancient characters on the test set, selecting the ancient character samples that are predicted to be open set in this prediction. The ancient script samples predicted to be in the open set category are then subjected to a second screening and classification using the original closed set classification network. After these two screenings of open and closed set ancient script samples, the ancient script samples that are ultimately identified as being in the open set category in both screenings are extracted and used as the final ancient script samples that are determined to be in the open set category. The trained network structure is as follows: a threshold is set before the probability of each category in the output layer. Ancient characters with a probability value greater than the set threshold are defined as closed set ancient characters, and ancient characters with a probability value less than the set threshold are defined as open set ancient characters. Closed-set ancient script classification module based on deep learning network: Use deep learning network to classify ancient script samples that are determined to be closed sets according to known category labels; Visualized Similar Character Recommendation Module: The module will select the ancient Chinese characters that are most similar to the ancient characters identified as open set samples, which are learned in the open and closed set differentiation module based on deep learning network, as the visually recommended ancient Chinese characters, and make visual similar character recommendations.

Images