An Automatic Recognition Method for the Front and Back Sides of Oracle Bone Pieces Based on Images

Abstract

The present invention relates to an automatic recognition method for the front and back sides of oracle bone pieces based on images. In the classifier training stage, oracle bone images that meet the requirements are selected and used as two independent classifiers after preprocessing. In the real-time classification stage, the oracle bone images to be classified that meet the requirements are imported into Classifier One. When the classification conditions are met, classification labels for the oracle bone pieces are assigned according to the output of Classifier One; when the classification conditions of Classifier One are not met, the image to be classified is segmented to obtain several image fragments, and the image fragments that meet the requirements are sequentially input into Classifier Two to obtain their corresponding classification results, and the weight voting algorithm is used to assign classification results to the image to be classified. The present invention liberates the labor force and also increases the objectivity of classification.

Description

Technical Field

[0001] The present invention relates to an automatic classification method for oracle bones, and specifically to a front and back classification method based on oracle bone images. Background Art

[0002] As a precious traditional heritage of the Chinese nation, oracle bone inscriptions carry the historical and cultural memories of the Shang Dynasty. Studying oracle bone inscriptions is of great significance for revealing ancient civilizations and inheriting the cultural heritage of ancestors. Generally speaking, in a complete oracle bone piece, its front side contains oracle bone inscriptions but no drilling; while its back side contains drilling but no oracle bone inscriptions, and its examples are as Figure 2a 、 Figure 2b shown.

[0003] In recent years, oracle bone experts have tried to introduce information technology into traditional oracle bone studies, that is, using images of oracle bone pieces for research such as oracle bone joining and interpretation. In the research process, the classification of the front and back sides of oracle bone pieces is an essential step. The traditional method for classifying the front and back sides of oracle bone images requires experts and scholars to observe the characteristics of the images according to experience, manually classify and mark them. This kind of work is often time-consuming and laborious, and the classification results are somewhat subjective. Summary of the Invention

[0004] Aiming at the above technical deficiencies, the purpose of the present invention is to provide an automatic front and back classification method for oracle bone images, which liberates the labor force and also increases the objectivity of classification.

[0005] The technical solution adopted by the present invention to solve its technical problems is: an automatic recognition method for the front and back sides of oracle bone pieces based on images, including the following steps:

[0006] Classifier training stage:

[0007] Step 1: Select oracle bone images containing single oracle bone pieces and corresponding annotation information. After cropping, establish an oracle bone pre-training dataset according to the labels indicating the front and back in the annotation information of each oracle bone image;

[0008] Step 2: Randomly extract N1 oracle bone images from the oracle bone pre-training dataset. After annotating the type characteristics of each oracle bone image, form a training set and a test set for the first classifier, and train the first classifier;

[0009] Step 3: Randomly extract N2 oracle bone images from the oracle bone pre-training dataset. Divide each oracle bone image into multiple image fragments; use these image fragments as the training set and test set for the second classifier, and train the second classifier.

[0010] Real-time classification stage:

[0011] Select the image to be tested containing oracle bone pieces, input the image into the first classifier, and determine whether the output meets the classification conditions according to the first classifier;

[0012] If it meets the conditions, assign the classification result of the first classifier to the image, and the classification process ends;

[0013] If it does not meet the conditions, use the segmentation and screening method to segment the image into multiple image fragments, then input them into the second classifier to obtain the classification results of each image fragment, use the weighted voting algorithm to obtain the classification result of the image, and assign it to the image, and the classification process ends.

[0014] The oracle bone image selects color depth images with known sources, resolutions, and quantities.

[0015] The type features include drilling and oracle bone inscriptions.

[0016] The image containing oracle bone pieces is a 16-bit color depth image with a resolution greater than or equal to d*d; d is an integer multiple of 224.

[0017] The conditions for meeting the classification in the real-time classification stage are as follows:

[0018] The sum of the detected number of the first type features and the number of the second type features is not zero. At this time, if the number of the first type features is greater than or equal to the number of the second type features, the output result of the first classifier is a label indicating negative, otherwise the output result is a label indicating positive;

[0019] The conditions for not meeting the classification in the real-time classification stage are as follows: The sum of the detected number of the first type features and the number of the second type features is zero.

[0020] The first classifier is the YOLOv5 network.

[0021] The second classifier is the RESNET101 convolutional neural network.

[0022] The cropping in step one of the training stage of the second classifier includes the following steps:

[0023] After graying and binarizing the image, find the minimum bounding rectangle of the non-connected region of the black pixel points, and then crop the rectangle region with the largest area to form the cropped image.

[0024] The segmentation in step three of the classifier training stage and the real-time classification stage is to segment the image into several sub-images with a resolution of 224*224, and retain the sub-images in which the main surface of the oracle bone and the area ratio of the sub-images are greater than the threshold, including the following steps:

[0025] a. Establish an image coordinate system, set the pixel in the upper left corner of the image as the coordinate origin, the horizontal direction from left to right is the positive direction of the x-axis, the vertical direction from top to bottom is the positive direction of the y-axis, and set the resolution of the original oracle bone image to m*n;

[0026] b. In the established image coordinate system, use a rectangular frame with a length and width of 224 pixels to cover the image. Suppose the coordinates of the upper left vertex of the rectangular frame in each covering position are (g, h), then

[0027]

[0028] The image covered by each rectangular frame is intercepted to determine the area ratio of the oracle bone body to the covered image. The determination method is as follows:

[0029] First, the image is grayed out. The three primary colors of the oracle bone image or the image to be tested extracted from the oracle bone pre-training data set are R, G, and B respectively. The gray value after graying is represented by M. The graying is performed using the following formula:

[0030] M=(R*0.3+G*0.59+B*0.11)

[0031] Then the grayscale image is binarized. If the grayscale value of the binarized image is represented by B, the binarization is performed using the following formula:

[0032]

[0033] Among them, AVG(M) represents the average grayscale value of all elements in the image;

[0034] Finally, the ratio of pixels with non-zero grayscale values ​​to zero pixels is counted. If the ratio is greater than or equal to the threshold, it is considered that the proportion of oracle bone images exceeds the threshold.

[0035] The weighted voting algorithm in the real-time classification stage specifically refers to:

[0036] The coordinates of the upper left vertex of the rectangular box (g, h) represent the number of the image fragment it covers. If the image resolution is m1*n1, then the weight W of the image fragment is g,h It is expressed as follows:

[0037]

[0038] After obtaining the weight of each image fragment, each image fragment is imported into the trained second classifier, and the total weight of the image fragments classified as representing positive or negative labels is counted respectively, and the image classification label is assigned to the category with the largest weight.

[0039] The present invention includes the following beneficial effects and advantages:

[0040] 1. The present invention introduces artificial intelligence technology into the problem of classifying the front and back sides of oracle bone pieces, thus greatly reducing the human labor consumption in the classification process;

[0041] 2. The present invention divides the image into several sub-images with a resolution of 224*224, and retains the sub-images in which the proportion of the main body of the oracle bone is greater than the threshold. Finally, the classification result of the original image is deduced from the classification results of multiple sub-images, reducing errors and increasing robustness;

[0042] 3. The serial classification structure adopted by the present invention is closer to the thinking mode in the classification process of oracle bone experts, thus greatly enhancing the generalization ability of the classifier. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is the flowchart of the method of the present invention;

[0044] Figure 2a is a schematic diagram of the "front" side of the oracle bone image involved in the present invention;

[0045] Figure 2b is a schematic diagram of the "back" side of the oracle bone image involved in the present invention;

[0046] Figure 3a is an example diagram of manually marked oracle bone characters of the present invention;

[0047] Figure 3b is an example diagram of manually marked drilling of the present invention;

[0048] Figure 4 is the overall flowchart of the automatic recognition method for the front and back sides of oracle bone pieces of the present invention;

[0049] Figure 5a is the visual output result of Yolo v5 on which classifier one of the present invention relies for the "characters" on the front side of the oracle bone piece;

[0050] Figure 5b is the visual output result of Yolo v5 on which classifier one of the present invention relies for the "drilling" on the back side of the oracle bone piece;

[0051] Figure 6a is the original image in the present invention;

[0052] Figure 6b is the grayscale image in the present invention;

[0053] Figure 6c is the binary image in the present invention;

[0054] Figure 6d is the cropped image in the present invention;

[0055] Figure 7aThe source oracle bone image in the present invention;

[0056] Figure 7b The fragmented oracle bone image in the present invention;

[0057] Figure 7c The image fragment after removing the background in the present invention. Detailed implementation manners

[0058] The present invention will be further described in detail below with reference to examples.

[0059] An automatic recognition method for the front and back sides of oracle bone pieces based on images, comprising the following steps:

[0060] As Figure 1 shown, the classification and recognition method consists of two parts: an "offline training stage" and a "real-time classification stage".

[0061] In the classifier training stage, select oracle bone images that meet the requirements. After preprocessing, they are used as the training set sources for two independent classifiers: classifier one (a classifier based on object detection) and classifier two (a classifier based on a traditional convolutional neural network). Select a certain number of preprocessed images. After manually annotating the "oracle bone inscriptions" and "drilling" features of the oracle bone images, form the training set of classifier one, and use this training set to train classifier one. Select a certain number of preprocessed images. After splitting each image into several corresponding image fragments, use the image fragments corresponding to all images that meet certain conditions to train classifier two.

[0062] In the real-time classification stage, import the oracle bone images to be classified that meet the requirements into classifier one. When the classification conditions are met, assign the classification label of the oracle bone piece according to the output of classifier one; when the classification conditions of classifier one are not met, split the image to be classified to obtain several image fragments, and input the qualified image fragments into classifier two in sequence to obtain their corresponding classification results. Use the "weighted voting" algorithm to assign the classification result to the image to be classified.

[0063] Classifier training stage

[0064] Step 1: Select 16-bit color depth images with specific sources, specific resolutions, and specific quantities that contain single oracle bone pieces and their corresponding annotation information. After cropping by the cropping method, establish an oracle bone pre-training data set according to the "front" and "back" labels of the annotation information in each image.

[0065] Step 2: Randomly select 1000 oracle bone images from the oracle bone pre-training data set. After manually annotating their "oracle bone inscriptions" and "drilling" features using software, form the training set and test set of the classifier (classifier one) (700 images in the training set and 300 images in the test set), and then train the classifier under the training parameters. An example of manual annotation is asFigure 3a , Figure 3b as shown

[0066] Step 3: Randomly select 3,000 oracle bone images from the oracle bone pre-training dataset, and fragment each oracle bone image using the segmentation and screening method; use these image fragments as the training set and test set of the classifier (Classifier 2) (the training set contains the image fragments corresponding to all 2,000 complete images, and the test set contains the image fragments corresponding to all 1,000 complete images), and then train the classifier under the training method and parameters.

[0067] Real-time classification stage

[0068] Select a 16-bit color depth color image with a resolution greater than 1792*1792 and containing oracle bone pieces, input the image into Classifier 1, and determine whether the classification conditions are met according to the classifier output. If the classification conditions are met, assign the classification result of Classifier 1 to the oracle bone image, and the classification process ends at this time. If the classification conditions are not met, use the segmentation and screening method to fragment the image and then input it into Classifier 2 to obtain the classification results of each fragment image. Use the "weighted voting" algorithm to obtain the classification result of the entire image, and assign it to the oracle bone image, and the classification process ends. The entire classification process is as Figure 4 shown

[0069] In the first step of the classifier training stage, the oracle bone images with "specific source", "specific resolution", and "specific quantity" specifically refer to 6,000 color images with a resolution greater than 1792*1792 from published oracle bone catalogs (such as "The Complete Collection of Oracle Bone Inscriptions").

[0070] The "classifier (Classifier 1)" refers to a classifier based on the open-source project YOLOv5 (open-source address: https: / / github.com / ultralytics / YOLOv5), which is composed of YOLO v5 and a detection box quantity statistic in series. In the real-time classification stage, "meeting the classification conditions" means that the sum of the detected "drilling" quantity and the "oracle bone inscription" quantity is not 0. At this time, if the "drilling" quantity is greater than or equal to the "oracle bone inscription" quantity, the classifier output result is "negative", otherwise the output result is "positive". "Not meeting the classification conditions" in the real-time classification stage means that the sum of the detected "drilling" quantity and the "oracle bone inscription" quantity is 0. Figure 5a , Figure 5b show the target links of the software, which are the detection results of oracle bone inscriptions and drillings respectively.

[0071] In step 2 of the classifier training phase, the "software" refers to the object detection calibration software "LabelImg" attached to the open-source project YOLOv5. In this step, the "training parameters" specifically refer to a batch size of 32 and an Epoch value of 300 for training.

[0072] In step 3 of the classifier training phase, the "classifier (classifier 2)" refers to the RESNET101 convolutional neural network pre-trained on the ImageNet dataset (open-source address: https: / / download.pytorch.org / models / resnet101-63fe2227.pth). In this step, the "training method and parameters" specifically mean freezing the parameters except for the output layer of this convolutional neural network, adjusting the output layer structure to the structure for binary classification problems, using an improved BackPropagation (BP) strategy to update the weights, that is, using the Adam (Adaptive Moment Estimation) method for parameter update, with an initial learning rate of 0.1, an input resolution of the image of 224*224, a batch size of 32, the number of training epochs set to 20, and the cross-entropy (Cross Entropy Loss) used as the loss function.

[0073] In step 1 of the classifier training phase, the "cropping method" specifically refers to after grayscale and binarization of the image, finding the minimum bounding rectangle of the non-connected regions of the black pixel points, and then cropping the rectangle region with the largest area to form a new image. The intermediate images formed during the cropping process are as Figure 6a , Figure 6b , Figure 6c , Figure 6d shown.

[0074] In step 3 of the classifier training phase and the real-time classification phase, the "segmentation and screening method" refers to splitting the image into sub-images with a resolution of 224*224 and retaining the sub-images in which the main body of the oracle bone image accounts for more than 10%. The specific implementation strategy is as follows:

[0075] a. Establish an image coordinate system, set the pixel at the upper left corner of the image as the coordinate origin, the horizontal direction from left to right as the positive x-axis direction, and the vertical direction from top to bottom as the positive y-axis direction. Let the resolution of the original oracle bone image be m*n.

[0076] b. Under the established image coordinate system, use a rectangular frame with a length and width of 224 pixels to cover the image. Let the coordinates of the upper left vertex of the rectangular frame in each covering position be (g, h), then

[0077]

[0078] Crop the image covered by each rectangular box, and use an algorithm to judge the proportion of the main body of the oracle bone in this part of the image. The judgment method is as follows:

[0079] First, perform grayscale processing on the image. Assume that the three primary colors of the original image pixels are R, G, and B, and the grayscale value after grayscale processing is represented by M. Then, perform grayscale processing using the following formula:

[0080] M = (R * 0.3 + G * 0.59 + B * 0.11)

[0081] After that, perform binarization processing on the grayscale image. Assume that the grayscale value of the image after binarization is represented by B. Then, perform binarization processing using the following formula:

[0082]

[0083] where AVG(M) represents the average grayscale of all elements in the image.

[0084] Finally, count the ratio of the number of pixels with non - zero grayscale values to the number of 0 - valued pixels. If the ratio is greater than or equal to 10%, it is considered that the proportion of the oracle bone image exceeds 10%. The process is as Figure 7a 、 Figure 7b 、 Figure 7c shown.

[0085] In the real - time classification stage, the "weighted voting" algorithm specifically refers to:

[0086] Use the upper - left vertex coordinates (g, h) of the rectangular box to represent the label of the image fragment it covers. Assume that the image resolution is m1 * n1. Then, the weight W g,h of the image fragment can be represented by the following formula:

[0087]

[0088] After obtaining the weights of each image fragment, import each image fragment into the trained classifier two, respectively count the total weights of the image fragments classified as "positive" and "negative", and assign the image classification label to the category with the largest weight.

Claims

1. An automatic recognition method for the front and back sides of oracle bone pieces based on images, characterized in that, It includes the following steps: Classifier training stage: Step 1: Select oracle bone images containing single oracle bone pieces and corresponding annotation information. After cropping, establish an oracle bone pre-training dataset according to the labels indicating front and back in the annotation information of each oracle bone image. Step 2: Randomly extract N1 oracle bone images from the oracle bone pre-training dataset. After annotating the type features of each oracle bone image, form the training set and test set of the first classifier, and train the first classifier. Step 3: Randomly extract N2 oracle bone images from the oracle bone pre-training dataset. Divide each oracle bone image into multiple image fragments; use these image fragments as the training set and test set of the second classifier, and train the second classifier. Real-time classification stage: Select a to-be-tested image containing oracle bone pieces, input the image into the first classifier, and determine whether the output meets the classification conditions according to the judgment of the first classifier. If it meets the conditions, assign the classification result of the first classifier to the image, and the classification process ends. If it does not meet the conditions, use the segmentation and screening method to divide the image into multiple image fragments, then input them into the second classifier, obtain the classification results of each image fragment, use the weighted voting algorithm to obtain the classification result of the image, and assign it to the image, and the classification process ends. The image containing oracle bone pieces is an image with a resolution greater than or equal to d*d and a color depth of 16 bits; d is an integer multiple of 224. The segmentation in Step 3 of the classifier training stage and the real-time classification stage is to divide the image into several sub-images with a resolution of 224*224, and retain the sub-images in which the proportion of the main body of the oracle bone to the area of the sub-image is greater than the threshold, including the following steps: a. Establish an image coordinate system, set the pixel at the upper left corner of the image as the coordinate origin, the horizontal direction from left to right as the positive direction of the x-axis, and the vertical direction from top to bottom as the positive direction of the y-axis. Let the resolution of the original oracle bone image be m*n. b. Under the established image coordinate system, use a rectangular frame with a length and width of 224 pixels to cover the image. Let the coordinates of the upper left vertex of the rectangular frame in each covering position be (g,h), then Intercept the image covered by each rectangular frame, and judge the proportion of the main body of the oracle bone in the covered image to the area of the covered image. The judgment method is as follows: First, perform grayscale processing on the image. The three primary colors of the pixels of the oracle bone image extracted from the oracle bone pre-training dataset or the to-be-tested image are R, G, and B respectively. The grayscale value after grayscale processing is represented by M, then use the following formula for grayscale processing: M = (R * 0.3 + G * 0.59 + B * 0.11) After that, perform binary processing on the grayscale image. Let the grayscale value of the image after binary processing be represented by B, then use the following formula for binary processing: Among them, AVG(M) represents the average grayscale of all elements in the image. Finally, count the ratio of the number of pixels with non-zero grayscale value to the number of 0 pixels. If the ratio is greater than or equal to the threshold, it is considered that the proportion of the oracle bone image exceeds the threshold.

2. The automatic recognition method for the front and back sides of oracle bone pieces based on images according to claim 1, characterized in that: The oracle bone images are selected from color depth images with known sources, resolutions, and quantities.

3. The automatic recognition method for the front and back sides of oracle bone pieces based on images according to claim 1, characterized in that: The type features include drilling and oracle bone inscriptions.

4. The automatic recognition method for the front and back sides of oracle bone pieces based on images according to claim 1, characterized in that, The conditions for meeting the classification in the real-time classification stage are as follows: When the sum of the number of detected features of the first type and the number of features of the second type is not zero, if the number of features of the first type is greater than or equal to the number of features of the second type, the output result of the first classifier is a label indicating negative, otherwise the output result is a label indicating positive; The non - satisfaction of the classification condition in the real - time classification stage is as follows: the sum of the number of detected features of the first type and the number of features of the second type is zero.

5. The automatic recognition method for the front and back sides of oracle bone pieces based on images according to claim 1, characterized in that, The first classifier is the YOLOv5 network.

6. The automatic recognition method for the front and back sides of oracle bone pieces based on images according to claim 1, characterized in that, The second classifier is the RESNET101 convolutional neural network.

7. The automatic recognition method for the front and back sides of oracle bone pieces based on images according to claim 1, characterized in that, The cropping in step one of the training stage of the second classifier includes the following steps: After graying and binarizing the image, find the minimum bounding rectangle of the non - connected regions of black pixel points, and then crop the rectangle region with the largest area to form the cropped image.

8. The automatic recognition method for the front and back sides of oracle bone pieces based on images according to claim 1, characterized in that The weight voting algorithm in the real - time classification stage specifically refers to: The label of the image fragment covered by a rectangular box is represented by the coordinates (g, h) of its upper left vertex. Given the image resolution as m1*n1, the weight W of the image fragment g,h is expressed by the following formula: After obtaining the weights of each image fragment, import each image fragment into the trained second classifier, respectively count the total weights of the image fragments classified as positive or negative labels, and assign the image classification label to the category with the largest weight.

Images