Handwritten oracle detection and identification method, system and equipment based on YOLO v8 and medium

Abstract

The invention discloses a handwritten oracle detection and identification method, system and device based on YOLO v8 and a medium, and the method comprises the steps: constructing a handwritten oracle data set, and carrying out the multi-time labeling of the data set; on the basis of a constructed handwritten oracle data set, a target YOLO V8 detection model is built, training data of difficult-to-retrieve characters are added in a targeted manner on the basis of increasing the overall data scale, attention of the YOLO V8 detection model to the difficult-to-retrieve characters is strengthened, and then the accuracy of a deep learning network is improved; sliding window cutting is carried out on a detection image, so that the problem that a target is too small is avoided, and the performance of a YOLO V8 detection model is improved; meanwhile, aiming at the problem of false detection of oracle bone characters with similar characters, filtering wrong prediction results by using a character recognition model; the method has the advantages of being high in accuracy, good in robustness, high in processing speed and high in adaptive capacity, and oracle characters in the oracle blood donation image can be extracted more accurately and stably.

Description

Technical Field

[0001] The present invention relates to the technical field of oracle bone inscription detection, and in particular to a handwritten oracle bone inscription detection and recognition method, system, device and medium based on YOLO v8. Background Art

[0002] Studying oracle bone inscriptions is of great significance for gaining a deeper understanding of ancient Chinese history, culture, and language. Handwritten oracle bone inscription detection is an important part of oracle bone inscription research. Through effective computer recognition and retrieval technology, we can dig out deeper correlations in oracle bone inscription historical information, and provide technical support for breaking through research bottlenecks such as oracle bone inscription splicing and text interpretation. With the popularization of information technology, in order to increase the efficiency of oracle bone inscription research and reduce the difficulty of researchers' retrieval, we urgently need the help of information technology to carry out semantic computing through big data and push oracle bone inscription research towards informatization and intelligence.

[0003] Handwritten oracle bone inscription detection is a challenging task that requires identifying the presence and location of ancient oracle bone inscriptions. This task is significantly different from ordinary text detection because the characteristics of oracle bone inscriptions and the complexity of the background make it a unique problem with the following difficulties:

[0004] First, one of the characteristics of handwritten oracle bone inscriptions is their small size. Each oracle bone inscription image usually contains very limited information and is regarded as a small target, so it exhibits feature scarcity, high coordinate accuracy requirements, and relatively large IOU errors.

[0005] Second, in oracle bone images, the number of positive samples (oracle bone characters) is usually far less than the number of negative samples (other characters or background). This imbalance between positive and negative samples may cause the model to be more inclined to learn negative samples and insufficiently learn the features of positive samples. This imbalance may affect the performance of the model, especially for small object detection tasks.

[0006] 3. Although the original oracle bone images have a high resolution, preliminary experiments have found that in order to ensure image clarity, directly putting oracle bone images with a pixel count of about 4,000 to 6,000 into the deep learning network for training will lead to many problems, including NaN training loss, failure to detect target information, or memory overflow. On the other hand, a single oracle bone character usually contains only a few dozen pixels, and reducing the image pixels will cause image blur, reducing the accuracy of oracle bone detection.

[0007] 4. Radicals or compound characters are difficult to detect. Oracle bone script is a pictographic character, similar to modern Chinese characters, with components similar to radicals. In order to express more complex meanings, some characters are composed of multiple components, which are called compound characters. However, Oracle bone script is also different from modern Chinese characters. The shape of the components is not deformed by becoming "radicals". Its shape is still the same as that of a single character. Therefore, compound characters with multiple components are easily misdetected as multiple Oracle bone script single characters.

[0008] 5. Due to the long history of Chinese characters, many characters similar to oracle bone characters but not oracle bone characters appeared in the process of font evolution, such as bronze inscriptions, etc. At the same time, due to the large difference in font styles between handwritten oracle bone characters and the similarity of some handwritten sloppy characters to oracle bone characters, the accuracy of oracle bone character detection is affected.

[0009] The patent application document with publication number CN114092700A discloses an ancient character recognition method based on target detection and knowledge graph, which constructs an ancient character component recognition model to identify the components and the position coordinates of the components contained in the ancient character image; constructs an ancient character component position relationship recognition model based on the components and the position coordinates of the components contained in the identified ancient character image, and obtains the position relationship of the components to determine the structure of the text; constructs an ancient character knowledge graph; and uses the ancient character knowledge graph to infer the text results based on the components and the position relationship of the components; however, since there are errors in component recognition or the judgment of component position relationship, these errors will propagate to the construction and reasoning process of the knowledge graph, affecting the accuracy of the final text recognition result; in addition, ancient characters have diverse forms and variants, and even component-based recognition and reasoning are difficult to cover all variants, resulting in some specific styles or variants of ancient characters being difficult to accurately recognize.

[0010] The patent application document with publication number CN115497102A discloses an oracle component recognition system based on Capsule network and transfer learning. The activation vector is regarded as a logical unit through the Capsule network to determine the existence of instantiated entities in the image. Then, through the data enhancement method, the original image is rotated, translated, reversed, scaled and other random interferences are performed to generate new samples. The expanded data set samples are normalized, scaled to the (0,1) interval, and split into training set and test set at a ratio of 4:1. At the same time, the sample labels are one-hot encoded. However, since the Capsule network is more complex in concept than the traditional convolutional neural network, it requires more computing resources and larger models to train and run, thereby increasing the cost of deployment and operation; in addition, one-hot encoding converts the classification label into a vector form, increasing the feature dimension. Too many types of labels will result in a high-dimensional sparse matrix, which requires more computing resources to process. Summary of the invention

[0011] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a handwritten oracle bone inscription detection and recognition method based on YOLO v8. Firstly, an oracle bone inscription data set is established and annotated. Then, the credibility of the data set is improved by manually expanding the data set and annotating multiple times. On the basis of the YOLO v8 detection model, the YOLO v8 detection model is optimized by using sliding window clipping and character recognition model to improve the accuracy of oracle bone inscription detection. The present invention has the characteristics of high accuracy, good robustness, fast processing speed and strong adaptability, and can more accurately and stably extract oracle bone inscriptions in oracle bone document images.

[0012] In order to achieve the above object, the technical solution adopted by the present invention is:

[0013] A handwritten oracle bone inscription detection and recognition method based on YOLO v8, comprising the following steps:

[0014] Step 1: Scan the handwritten oracle bone inscription document into a digital image, find the oracle bone inscription characters in the digital image, annotate them multiple times, and assign corresponding digital codes to the annotated oracle bone inscription characters;

[0015] Step 2: Perform data augmentation on the annotated oracle bone inscriptions dataset in step 1;

[0016] Step 3: Take the text instances of the oracle bone script dataset after data augmentation in step 2 as positive samples, and further fine-tune the oracle bone script dataset after data augmentation to generate negative samples; divide the positive samples and negative samples into training sets and test sets in proportion;

[0017] Step 4: Input the public HWOBC oracle bone recognition dataset into the Effcientnet recognition network for training until convergence to obtain a character recognition model;

[0018] Step 5: After sliding window cropping, the training set in step 3 is input into the character recognition model in step 4 for tuning training. The training process is continued until the model converges.

[0019] Step 6: After sliding window cropping, the test set in step 3 is input into the YOLO V8 detection model for prediction to obtain a preliminary detection box and the confidence score S1 of the YOLO V8 detection model; the character recognition model optimized in step 5 is used to identify and classify the preliminary prediction box, and each preliminary prediction box will obtain the confidence score S2 of the character recognition model; the confidence score S1 and the confidence score S2 are used to calculate the comprehensive score S of the preliminary prediction box, and the preliminary prediction box with the highest comprehensive score S is taken as the final detection result.

[0020] The step 1 is specifically as follows:

[0021] Step 1.1: Scan the handwritten oracle bone inscriptions to obtain a digital image of the oracle bone inscriptions collection;

[0022] Step 1.2: Use LabelImg to select the oracle bone characters in the digital image in step 1.1 and set corresponding labels;

[0023] Step 1.3: After screening the oracle bone characters annotated in step 1.2 for the wrong annotations and identifying the missing annotations, repeat the operation of step 1.2 to re-annotate the wrong annotations and the missing annotations;

[0024] Step 1.4: Assign corresponding digital codes to the oracle bone characters marked in steps 1.2 and 1.3.

[0025] The step 2 is specifically as follows:

[0026] Randomly adjust the stroke thickness of the characters in the annotated oracle-bone script dataset in step 1.4, and perform affine transformation, scaling, rotation, tilting, and flipping on the annotated character instances in the oracle-bone script dataset, while adding random noise.

[0027] The step 3 is specifically as follows:

[0028] Step 3.1: Take the text instances of the oracle bone script dataset after data augmentation in step 2 as positive samples;

[0029] Step 3.2: Randomly cut an image region from each detection image in the oracle bone script dataset after data augmentation in step 2, and compare the image region with each character instance on the detection image; when the overlapping area between the image region and each character instance is less than 0.1 times the area of ​​the character instance, and the sum of the overlapping areas of all character instances is less than 0.1 times the image region, then the image region is a negative sample in the training task;

[0030] Step 3.3: Take 80% of the positive samples in step 3.1 and the negative samples in step 3.2 as the training set, and the remaining 20% ​​as the test set.

[0031] The step 5 is specifically as follows:

[0032] Step 5.1: Slide and crop the original resolution dataset images in the training set in step 3.3, and set the pixels of the overlapping part of the sliding window to less than 256;

[0033] Step 5.2: Input the image cropped by sliding window in step 5.1 into the character recognition model in step 4 for training, and set the initial learning rate to 0.5 to 0.8 times of the recognition task until the model converges.

[0034] The step 6 is specifically as follows:

[0035] Step 6.1: Slide and crop the original resolution dataset images in the test set of step 3.3, and set the pixels of the overlapping part of the sliding window to less than 256;

[0036] Step 6.2: Input the image cropped by sliding window in step 6.1 into the YOLO V8 detection model, use NMS processing to filter out redundant and repeated detection frames, and obtain the preliminary detection frame and the confidence score S1 of the YOLO V8 detection model;

[0037] Step 6.3: Use the character recognition model optimized in step 5.2 to identify and classify the preliminary prediction boxes in step 6.2. Each preliminary prediction box will obtain the confidence score S2 of the character recognition model;

[0038] Step 6.4: Use the confidence score S1 in step 6.2 and the confidence score S2 in step 6.3 to calculate the comprehensive score S of each preliminary prediction box, and take the preliminary prediction box with the highest comprehensive score as the final detection result.

[0039] The calculation formula of the comprehensive score S of the preliminary prediction frame in step 6.4 is:

[0040]

[0041] Among them: i is 1 or 2; m1 and m2 are set to 0.9 and 0.1 respectively; S1 is the confidence score of the preliminary detection model, and S2 is the confidence score of the recognition model.

[0042] A handwritten oracle bone inscription detection and recognition system based on YOLO v8, characterized by:

[0043] Dataset construction and division module: perform data augmentation on the annotated oracle bone script dataset; use the text instances of the augmented oracle bone script dataset as positive samples, and further fine-tune the augmented oracle bone script dataset to generate negative samples; divide the positive samples and negative samples into training sets and test sets in proportion;

[0044] Model training module: input the public HWOBC oracle bone recognition data set into the Effci entnet recognition network for training until convergence to obtain a character recognition model;

[0045] Model tuning and training module: The training set is trimmed by sliding window and then input into the character recognition model for tuning and training. The training process lasts until the model converges.

[0046] Detection and recognition module: The test set is cropped by sliding window and input into the YOLO V8 detection model for prediction to obtain the preliminary detection box and the confidence score of the YOLO V8 detection model; the preliminary prediction box is identified and classified using the optimized character recognition model, and each preliminary prediction box will obtain the confidence score of the character recognition model; the confidence score and confidence score are used to calculate the comprehensive score of the preliminary prediction box, and the preliminary prediction box with the highest comprehensive score is taken as the final detection result.

[0047] A handwritten oracle bone detection and recognition device based on YOLO v8, comprising:

[0048] Memory: used to store a computer program to implement the handwritten oracle bone inscription detection and recognition method based on YOLO v8 as described above;

[0049] Processor: used to implement the handwritten oracle bone detection and recognition method based on YOLO v8 as described above when executing the computer program.

[0050] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the handwritten oracle bone inscription detection and recognition method based on YOLO v8 are implemented.

[0051] Compared with the prior art, the beneficial effects of the present invention are:

[0052] 1. The present invention marks and encodes oracle bone characters, performs data augmentation after improving the credibility of the data set, and specifically increases the training data of difficult-to-check characters, thereby improving the accuracy of the model in detecting oracle bone characters.

[0053] 2. The present invention uses an optimized character recognition model to assist in correcting the results detected by the YOLO V8 detection model, eliminating target boxes with low confidence, thereby effectively ensuring the accuracy of detection.

[0054] 3. The present invention performs data augmentation on the annotated handwritten oracle bone script data to specifically increase the training data of difficult-to-detect characters, thereby improving the accuracy of the model in detecting oracle bone script.

[0055] 4. The present invention avoids the problems of training loss (Loss is NaN), failure to detect target information or memory overflow by inputting the image cropped by sliding window into the deep learning network, thereby improving the performance of the detection model and greatly reducing the cost of deployment and operation.

[0056] In summary, the present invention constructs a handwritten oracle bone character dataset and annotates the dataset multiple times. High-quality annotated data can improve the accuracy of model prediction. Based on the constructed handwritten oracle bone character dataset, a target YOLO V8 detection model is built. Aiming at the problem that the oracle bone character target is small and difficult to detect and the scale of well-annotated training data is too small, the training data of difficult-to-detect characters is increased on the basis of increasing the overall data scale, and the attention of the YOLO V8 detection model to difficult-to-detect characters is strengthened, thereby improving the accuracy of the deep learning network. By sliding window cropping the detection image, the problem of too small targets is avoided, and the performance of the detection model is improved. At the same time, for the problem of false detection of oracle bone characters with similar characters, the oracle bone character recognition model is used to filter the wrong prediction results. The experimental results show that the YOLO V8 detection model can predict difficult-to-detect characters more robustly, and the character recognition model can recall the correct predictions with low confidence and filter the wrong predictions. The present invention has the characteristics of high accuracy, good robustness, fast processing speed, and strong adaptive ability, and can more accurately and stably extract oracle bone characters in oracle bone document images. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The figure is a flow chart of the handwritten oracle bone inscription detection and recognition method based on YOLO v8 of the present invention.

[0058] Figure 2 The oracle bone script single word marking diagram of the present invention Figure 1 .

[0059] Figure 3 The oracle bone script single word marking diagram of the present invention Figure 2 .

[0060] Figure 4 It is a schematic diagram of sliding cutting of the present invention.

[0061] Figure 5 is the true value graph of oracle bone characters; Figure 5 (a) is an example 1 of the oracle bone character truth value of the data set constructed by the present invention; Figure 5 (b) is the second example of the oracle bone character truth value constructed by the present invention; Figure 5 (c) is the third example of the oracle bone character truth value constructed from the data set of the present invention; Figure 5 (d) is the fourth example of the oracle bone character truth value constructed by the present invention; Figure 5 (e) in the figure is the fifth true value example of oracle bone characters from other datasets.

[0062] Figure 6 is the model detection result of the present invention's one-time labeled data set training; wherein, Figure 6 (a) in the figure is the detection result of the true value instance 1 after one labeling; Figure 6(b) in the figure is the detection result of the second true value instance after one annotation; Figure 6 (c) in the figure is the detection result of the third true value instance after one labeling; Figure 6 (d) in the figure is the detection result of the fourth true value instance after one labeling; Figure 6 (e) in the figure is the detection result of the fifth true value instance after one labeling.

[0063] Figure 7 It is the detection result diagram after secondary annotation of the present invention; wherein, Figure 7 (a) in the figure is the detection result of the true value instance 1 after secondary annotation; Figure 7 (b) in the figure is the detection result of the second true value instance after secondary annotation; Figure 7 (c) in the figure is the detection result of the third true value instance after secondary annotation; Figure 7 (d) in the figure is the detection result of the fourth true value instance after secondary annotation; Figure 7 (e) in the figure is the detection result of the true value instance 5 after secondary annotation.

[0064] Figure 8 This is a comparison diagram before and after the optimization of the oracle bone inscription detection model of the present invention; Figure 8 (a) is the oracle bone character truth value graph of the data set constructed by the present invention; Figure 8 (b) in the figure is the detection result without using the character recognition model; Figure 8 (c) in the figure uses the character recognition model to assist in the detection result. DETAILED DESCRIPTION

[0065] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0066] See also Figure 1 , a handwritten oracle bone detection and recognition method based on YOLO v8, comprising the following steps:

[0067] Step 1: Scan the handwritten oracle bone document into a digital image, find the oracle bone characters in the digital image, mark them twice, and assign corresponding digital codes to the oracle bone characters after the two markings; the handwritten oracle bone document comes from the public "Oracle Bone Document Collection".

[0068] The step 1 is specifically as follows:

[0069] Step 1.1: Scan the handwritten oracle bone inscriptions to obtain a digital image of the oracle bone inscriptions collection;

[0070] Step 1.2: Use LabelImg to select the oracle bone characters in the digital image in step 1.1 and set corresponding labels;

[0071] The dataset is mainly produced by relying on the open source labeling tool LabelImg, and the labeling information is converted into the PASCAL VOC dataset labeling format, which is widely used in the field of target detection, for storage, thus reducing the migration cost between detection tasks directed by different applications. Figure 2 and Figure 3 As shown in the figure, the image annotation adopts single-word annotation, and LabelImg is used to select the corresponding oracle bone characters on the image. The annotation information of each character is the coordinates of its selection position (x1, y1, x2, y2), where (x1, y1) is the coordinate of the top left corner of the rectangular box, and (x2, y2) is the coordinate of the bottom right corner of the rectangular box. In this annotation, the ancient characters appearing in the oracle bone documents are marked as positive samples, and the label is set to 1.

[0072] In the first stage of data set construction, the present invention completed the character detection and annotation of 20 volumes of oracle bone documents. A total of 32,345 oracle bone characters were annotated in the data set. Each page of oracle bone document image includes an average of about 7 character examples.

[0073] After marking the above-mentioned oracle bone characters, it is also necessary to screen out erroneous markings and supplement the missing markings to ensure the completeness and accuracy of the entire marking.

[0074] Step 1.3: After screening the oracle bone characters annotated in step 1.2 for wrong annotations and identifying the missing annotations, repeat the operation of step 1.2 to annotate the wrong annotations and the missing annotations for the second time;

[0075] Step 1.4: Introduce coding rules to assign corresponding digital codes to the oracle characters annotated twice in steps 1.2 and 1.3; the coding rules are shown in Table 1:

[0076] Table 1 Oracle-bone script coding rules

[0077]

[0078]

[0079] The above encoding rules allow the annotation results to be used seamlessly in different fonts and platforms, improving the flexibility and reusability of the dataset.

[0080] Step 2: Data augmentation is performed on the annotated oracle bone script dataset in step 1. Since the dataset contains a large number of low-frequency categories and a relatively high-frequency category, the sample size gap between the dataset categories is large, so the present invention uses data augmentation to increase the training samples.

[0081] The step 2 is specifically as follows:

[0082] Randomly adjust the stroke thickness of the text in the annotated oracle-bone script dataset in step 1.4, and perform affine transformation, scaling, rotation, tilting, and flipping on the text instances in the annotated oracle-bone script dataset, while adding random noise to increase the training dataset, thereby improving the accuracy of the deep learning network.

[0083] At the same time, there are characters in the document collection that are similar to oracle bone characters but are not oracle bone characters, such as bronze inscriptions. At the same time, because most oracle bone inscription documents in the past were handwritten documents, there are large differences in font styles between documents, and some handwritten characters are similar to oracle bone characters when they are sloppy, which causes these characters to be misdetected as oracle bone characters. Therefore, on the basis of increasing the overall data scale, the present invention strengthens the detection model's attention to difficult-to-detect characters and increases the number of text images that are easily missed or misdetected.

[0084] Step 3: Take the text instances of the oracle bone script dataset after data augmentation in step 2 as positive samples, and further fine-tune the oracle bone script dataset after data augmentation to generate negative samples; divide the positive samples and negative samples into training sets and test sets in proportion;

[0085] The step 3 is specifically as follows:

[0086] Step 3.1: Take the text instances of the oracle bone script dataset after data augmentation in step 2 as positive samples;

[0087] Step 3.2: Randomly cut an image region from each detection image in the oracle bone script dataset after data augmentation in step 2, and compare the image region with each character instance on the detection image; when the overlapping area between the image region and each character instance is less than 0.1 times the area of ​​the character instance, and the sum of the overlapping areas of all character instances is less than 0.1 times the image region, then the image region is a negative sample in the training task;

[0088] Step 3.3: Take 80% of the positive samples in step 3.1 and the negative samples in step 3.2 as the training set, and the remaining 20% ​​as the test set.

[0089] The oracle bone script detection dataset annotates and encodes the oracle bone script categories, but the number of annotations and encodings only accounts for a small part of all annotations. There are only 676 annotated oracle bone script categories, and most of the data annotation information only contains the location of the text box. Therefore, the character detection model trained in the current oracle bone script detection task lacks more detailed classification capabilities in terms of glyphs, so the character detection model needs to be further trained.

[0090] Step 4: Input the public HWOBC oracle bone recognition dataset with 3881 types of characters into the Effcientnet recognition network for training until convergence to obtain a character recognition model; by using the character recognition model with prior knowledge of multiple types of characters to assist in correcting the prediction results of the current detection task, the false detection in the detection task is reduced.

[0091] Step 5: After sliding window cropping, the training set in step 3 is input into the character recognition model in step 4 for tuning training. The training process is continued until the model converges.

[0092] Since there are many characters in the dataset images, in order to ensure the image clarity, directly inputting the image into the deep learning network will cause memory overflow, and reducing the image pixels and inputting it into the network will cause the image to be blurred, thereby reducing the accuracy of oracle bone inscription detection. Therefore, the present invention performs sliding window cropping on the image and then inputs it into the deep learning network. Figure 4 As shown in the figure, a large pixel image is cropped with a sliding window of 512×512 pixels. If an object is just at the edge of the window and is cut into two pieces, the object itself will occupy fewer pixels and be truncated, which will make it more difficult to detect. Therefore, the overlapping area required during sliding window cropping is selected as 128 pixels.

[0093] The step 5 is specifically as follows:

[0094] Step 5.1: Slide and crop the original resolution dataset images in the training set in step 3.3, and set the pixels of the overlapping part of the sliding window to less than 256;

[0095] like Figure 3 As shown, a sliding window of 512×512 pixels is used for cropping, and the overlapping pixels are 128 during sliding window cropping.

[0096] Step 5.2: Input the image cropped by sliding window in step 5.1 into the character recognition model in step 4 for training, and set the initial learning rate to 0.5 to 0.8 times of the recognition task until the model converges.

[0097] Step 6: After sliding window cropping, the test set in step 3 is input into the YOLO V8 detection model for prediction to obtain a preliminary detection box and the confidence score S1 of the YOLO V8 detection model; the character recognition model optimized in step 5 is used to identify and classify the preliminary prediction box, and each preliminary prediction box will obtain the confidence score S2 of the character recognition model; the confidence score S1 and the confidence score S2 are used to calculate the comprehensive score S of the preliminary prediction box, and the preliminary prediction box with the highest comprehensive score S is taken as the final detection result.

[0098] By combining the confidence scores of the YOLO v8 detection model and the character recognition model, the reliability of each prediction box can be evaluated more accurately. The comprehensive confidence score can reflect the positioning accuracy and recognition accuracy of the detection box, thereby reducing false detections and missed detections, thereby improving the overall recognition effect of the model.

[0099] The step 6 is specifically as follows:

[0100] Step 6.1: Slide and crop the original resolution dataset images in the test set of step 3.3, and set the pixels of the overlapping part of the sliding window to less than 256;

[0101] Step 6.2: Input the image cropped by sliding window in step 6.1 into the YOLO V8 detection model, use NMS processing to filter out redundant and repeated detection frames, and obtain the preliminary detection frame and the confidence score S1 of the YOLO V8 detection model;

[0102] Step 6.3: Use the character recognition model optimized in step 5.2 to identify and classify the preliminary prediction boxes in step 6.2. Each preliminary prediction box will obtain the confidence score S2 of the character recognition model;

[0103] Step 6.4: Use the confidence score S1 in step 6.2 and the confidence score S2 in step 6.3 to calculate the comprehensive score S of each preliminary prediction box, and take the preliminary prediction box with the highest comprehensive score as the final detection result.

[0104] The calculation formula of the comprehensive score S of the preliminary prediction frame in step 6.4 is:

[0105]

[0106] Among them: i is 1 or 2; m1 and m2 are set to 0.9 and 0.1 respectively; S1 is the confidence score of the preliminary detection model, and S2 is the confidence score of the recognition model.

[0107] A handwritten oracle bone inscription detection and recognition system based on YOLO v8, characterized by:

[0108] Dataset construction and division module: perform data augmentation on the annotated oracle bone script dataset; use the text instances of the augmented oracle bone script dataset as positive samples, and further fine-tune the augmented oracle bone script dataset to generate negative samples; divide the positive samples and negative samples into training sets and test sets in proportion;

[0109] Model training module: input the public HWOBC oracle bone recognition data set into the Effci entnet recognition network for training until convergence to obtain a character recognition model;

[0110] Model tuning and training module: The training set is trimmed by sliding window and then input into the character recognition model for tuning and training. The training process lasts until the model converges.

[0111] Detection and recognition module: The test set is cropped by sliding window and input into the YOLO V8 detection model for prediction to obtain the preliminary detection box and the confidence score of the YOLO V8 detection model; the preliminary prediction box is identified and classified using the optimized character recognition model, and each preliminary prediction box will obtain the confidence score of the character recognition model; the confidence score and confidence score are used to calculate the comprehensive score of the preliminary prediction box, and the preliminary prediction box with the highest comprehensive score is taken as the final detection result.

[0112] A handwritten oracle bone detection and recognition device based on YOLO v8, comprising:

[0113] Memory: used to store a computer program to implement the handwritten oracle bone inscription detection and recognition method based on YOLO v8 as described above;

[0114] Processor: used to implement the handwritten oracle bone detection and recognition method based on YOLO v8 as described above when executing the computer program.

[0115] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the handwritten oracle bone inscription detection and recognition method based on YOLO v8 are implemented.

[0116] The application effect of the present invention is described in detail below in conjunction with simulation experiments.

[0117] In order to compare the impact of oracle bone characters on model training, all oracle bone characters after two annotations were classified into one category, and the datasets after two annotations were divided into training set and test set in a ratio of 4:1. The training set and test set were iteratively trained 50 times by YOLO v8 model. The results are shown in Table 2 and Table 3. Figure 3 As shown, after the dataset is labeled and encoded, the credibility of the dataset is improved, thereby improving the accuracy of the model in detecting oracle bones.

[0118] Table 2 Quantitative evaluation table of one-time and two-time annotation model training

[0119]

[0120]

[0121] As can be seen from Table 2, after the test set was annotated twice, the quality of the data set was improved, which slightly improved the overall performance of the model. Compared with the cases of single annotation and single annotation (the test set was annotated twice), the F1 value of the secondary annotation was 91.8%, which was significantly improved. This shows that more detailed annotation can significantly improve the overall performance of the detection model. The precision of the first annotation (the test set was annotated twice) was 86.9%, which was slightly higher than the first annotation, indicating that the annotation quality of the test set had a positive impact on the precision of the model. The precision of the secondary annotation was 90.0%, indicating that through more precise annotation, the model can more accurately identify oracle bone inscriptions and reduce misjudgments. The recall rate of the secondary annotation was 93.6%, indicating that more detailed annotation helps the model better identify all oracle bone inscription samples and improves the recall rate of the model.

[0122] This paper conducts a qualitative evaluation of model training, see Figure 5 (a)-(e) Figure 6 (a)-(e) Figure 7 (a)-(e), by comparing and analyzing the true value map of oracle bone characters, the true value map after one annotation, and the true value map after two annotations, it can be found that the model trained with the second annotation dataset has fewer false detections and missed detections than the model trained with the first annotation dataset.

[0123] Table 3 Quantitative evaluation table of reduced resolution and original resolution sliding window cropping model training

[0124]

[0125]

[0126] The present invention conducts experimental simulations for different resolutions and whether or not to use the sliding window cropping method. It can be seen from Table 3 that when the original resolution picture is directly input into the model without sliding window cropping, insufficient video memory occurs; the original resolution picture is used for model training. Since the input image is clearer, its F1 score, accuracy rate Precision and recall rate Recall are better than the model trained by uniformly reducing it to 2048*1530 pixels and then cropping the data.

[0127] like Figure 8 As shown in (a)-(c) in the figure, by analyzing the true value graph and the images before and after using the character recognition model, it can be concluded that the character recognition model can better filter out erroneous prediction results and recognize oracle bone inscriptions more accurately.

[0128] The present invention constructs a handwritten oracle bone character data set, and annotates the data set multiple times. High-quality annotated data can improve the accuracy of model prediction. Based on the constructed handwritten oracle bone character data set, a target YOLO V8 detection model is built. Aiming at the problem that the oracle bone character target is small and difficult to detect and the scale of well-annotated training data is too small, the training data of difficult-to-detect characters is increased on the basis of increasing the overall data scale, and the attention of the YOLO V8 detection model to difficult-to-detect characters is strengthened, thereby improving the accuracy of the deep learning network. By performing sliding window cropping on the detection image, the problem of too small targets is avoided, and the performance of the detection model is improved. At the same time, for the problem of false detection of oracle bone characters with similar characters, an oracle bone character recognition model is used to filter the wrong prediction results. Experimental results show that the YOLO V8 detection model can predict difficult-to-detect characters more robustly, and the character recognition model can recall the correct predictions with low confidence and filter the wrong predictions. The present invention has the characteristics of high accuracy, good robustness, fast processing speed, and strong adaptability, and can more accurately and stably extract oracle bone characters in oracle bone document images.

[0129] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A handwritten oracle bone inscription detection and recognition method based on YOLO v8, characterized by: The following steps are involved: Step 1: Scan the handwritten oracle bone inscription document into a digital image, find the oracle bone inscription characters in the digital image, annotate them multiple times, and assign corresponding digital codes to the annotated oracle bone inscription characters; Step 2: Perform data augmentation on the annotated oracle bone inscriptions dataset in step 1; Step 3: Take the text instances of the oracle bone script dataset after data augmentation in step 2 as positive samples, and further fine-tune the oracle bone script dataset after data augmentation to generate negative samples; divide the positive samples and negative samples into training sets and test sets in proportion; Step 4: Input the public HWOBC oracle bone recognition dataset into the Effcientnet recognition network for training until convergence to obtain a character recognition model; Step 5: After sliding window cropping, the training set in step 3 is input into the character recognition model in step 4 for tuning training. The training process is continued until the model converges. Step 6: After sliding window cropping, the test set in step 3 is input into the YOLO V8 detection model for prediction to obtain a preliminary detection box and the confidence score S1 of the YOLO V8 detection model; the character recognition model optimized in step 5 is used to identify and classify the preliminary prediction box, and each preliminary prediction box will obtain the confidence score S2 of the character recognition model; the confidence score S1 and the confidence score S2 are used to calculate the comprehensive score S of the preliminary prediction box, and the preliminary prediction box with the highest comprehensive score S is taken as the final detection result.

2. The handwritten oracle bone inscription detection and recognition method based on YOLO v8 according to claim 1, characterized in that: The step 1 is specifically as follows: Step 1.1: Scan the handwritten oracle bone inscriptions to obtain a digital image of the oracle bone inscriptions collection; Step 1.2: Use LabelImg to select the oracle bone characters in the digital image in step 1.1 and set corresponding labels; Step 1.3: After screening the oracle bone characters annotated in step 1.2 for the wrong annotations and identifying the missing annotations, repeat the operation of step 1.2 to re-annotate the wrong annotations and the missing annotations; Step 1.4: Assign corresponding digital codes to the oracle bone characters marked in steps 1.2 and 1.

3.

3. A handwritten oracle bone inscription detection and recognition method based on YOLO v8 according to claim 1 or 2, characterized in that: The step 2 is specifically as follows: Randomly adjust the stroke thickness of the characters in the annotated oracle-bone script dataset in step 1.4, and perform affine transformation, scaling, rotation, tilting, and flipping on the annotated character instances in the oracle-bone script dataset, while adding random noise.

4. The handwritten oracle bone inscription detection and recognition method based on YOLO v8 according to claim 3 is characterized in that: The step 3 is specifically as follows: Step 3.1: Take the text instances of the oracle bone script dataset after data augmentation in step 2 as positive samples; Step 3.2: Randomly cut an image region from each detection image in the oracle bone script dataset after data augmentation in step 2, and compare the image region with each character instance on the detection image; when the overlapping area between the image region and each character instance is less than 0.1 times the area of ​​the character instance, and the sum of the overlapping areas of all character instances is less than 0.1 times the image region, then the image region is a negative sample in the training task; Step 3.3: Take 80% of the positive samples in step 3.1 and the negative samples in step 3.2 as the training set, and the remaining 20% ​​as the test set.

5. The handwritten oracle bone inscription detection and recognition method based on YOLO v8 according to claim 4 is characterized in that: The step 5 is specifically as follows: Step 5.1: Slide and crop the original resolution dataset images in the training set in step 3.3, and set the pixels of the overlapping part of the sliding window to less than 256; Step 5.2: Input the image cropped by sliding window in step 5.1 into the character recognition model in step 4 for training, and set the initial learning rate to 0.5 to 0.8 times of the recognition task until the model converges.

6. The handwritten oracle bone inscription detection and recognition method based on YOLO v8 according to claim 5, characterized in that: The step 6 is specifically as follows: Step 6.1: Slide and crop the original resolution dataset images in the test set of step 3.3, and set the pixels of the overlapping part of the sliding window to less than 256; Step 6.2: Input the image cropped by sliding window in step 6.1 into the YOLO V8 detection model, use NMS processing to filter out redundant and repeated detection frames, and obtain the preliminary detection frame and the confidence score S1 of the YOLO V8 detection model; Step 6.3: Use the character recognition model optimized in step 5.2 to identify and classify the preliminary prediction boxes in step 6.

2. Each preliminary prediction box will obtain the confidence score S2 of the character recognition model; Step 6.4: Use the confidence score S1 in step 6.2 and the confidence score S2 in step 6.3 to calculate the comprehensive score S of each preliminary prediction box, and take the preliminary prediction box with the highest comprehensive score as the final detection result.

7. The handwritten oracle bone inscription detection and recognition method based on YOLO v8 according to claim 6, characterized in that: The calculation formula of the comprehensive score S of the preliminary prediction frame in step 6.4 is: Among them: i is 1 or 2; m1 and m2 are set to 0.9 and 0.1 respectively; S1 is the confidence score of the preliminary detection model, and S2 is the confidence score of the recognition model.

8. A handwritten oracle bone inscription detection and recognition system based on YOLO v8, characterized by: include: Dataset construction and division module: data augmentation for the annotated oracle bone inscriptions dataset; The text instances of the data-augmented oracle bone script dataset are used as positive samples, and the data-augmented oracle bone script dataset is further fine-tuned to generate negative samples; the positive samples and negative samples are divided into training sets and test sets in proportion; Model training module: input the public HWOBC oracle bone recognition data set into the Effci entnet recognition network for training until convergence to obtain a character recognition model; Model tuning and training module: The training set is trimmed by sliding window and then input into the character recognition model for tuning and training. The training process lasts until the model converges. Detection and recognition module: The test set is cropped by sliding window and input into the YOLO V8 detection model for prediction to obtain the preliminary detection box and the confidence score of the YOLO V8 detection model; the preliminary prediction box is identified and classified using the optimized character recognition model, and each preliminary prediction box will obtain the confidence score of the character recognition model; the confidence score and confidence score are used to calculate the comprehensive score of the preliminary prediction box, and the preliminary prediction box with the highest comprehensive score is taken as the final detection result.

9. A handwritten oracle bone inscription detection and recognition device based on YOLO v8, characterized in that: include: Memory: used to store a computer program to implement a handwritten oracle bone inscription detection and recognition method based on YOLO v8 as described in any one of claims 1 to 7; Processor: used to implement a handwritten oracle bone detection and recognition method based on YOLOv8 as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the handwritten oracle bone inscription detection and recognition method based on YOLO v8 are implemented.

Images