Dongba pictographic character recognition method and device

Abstract

The present application provides a Dongba pictograph recognition method and device, the method comprising: obtaining a target image containing at least one Dongba pictograph; recognizing the Dongba pictograph in the target image through a target model, and obtaining a recognition result; wherein the target model is obtained by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM. The Dongba pictograph recognition method and device provided by the present application, after training the improved YOLOv4 convolutional neural network model, obtain a model capable of recognizing Dongba pictographs, thereby realizing the recognition of Dongba pictographs.

Description

Technical Field

[0001] The present application relates to the field of pictographic character recognition, and in particular to a Dongba pictographic character recognition method and device. Background Art

[0002] Dongba pictographic writing has a history of more than 1,000 years. It is the only pictographic writing still in use in the world and has been listed as a "Memory of the World Heritage" by UNESCO. The identification and preservation of Dongba writing is of great significance to the protection and inheritance of my country's traditional ethnic culture.

[0003] In the actual Dongba pictographic recognition, the Dongba ancient books themselves were written by different old Dongba writers on specific Dongba paper. Dongba paper has different texture characteristics, which causes a certain degree of noise interference for the recognition of Dongba pictographic characters. And because different old Dongba writers have different writing habits, the Dongba characters on the Dongba ancient books have different shapes and sizes, and there are variant characters, which adds difficulty to the recognition of Dongba pictographic characters. Summary of the invention

[0004] The purpose of this application is to provide a Dongba pictograph recognition method and device for Dongba pictograph recognition.

[0005] The present application provides a Dongba pictographic character recognition method, comprising:

[0006] Acquire a target image containing at least one Dongba pictograph; recognize the Dongba pictograph in the target image through a target model and obtain a recognition result; wherein the target model is obtained by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM.

[0007] Optionally, before identifying the Dongba hieroglyphs in the target image through the target model and obtaining the recognition result, the method also includes: acquiring a target data set; the target data set contains annotated Dongba hieroglyphs, and the number of each annotated Dongba hieroglyphs exceeds a preset value; using the target data set to train the improved YOLOv4 convolutional neural network model and obtain the target model; wherein the path aggregation network PANet structure of the improved YOLOv4 convolutional neural network model includes three levels, each level outputs images of different sizes, and CBAMs are set between the convolution blocks of each level.

[0008] Optionally, the PANet structure includes a first layer, a second layer and a third layer; the first layer outputs a first feature map, the second layer outputs a second feature map, and the third layer outputs a third feature map; the use of the target data set to train the improved YOLOv4 convolutional neural network model and obtain the target model includes: downsampling the feature image output by the convolution layer of the first layer through a first feature fusion module and performing feature fusion with the feature image output by the convolution layer of the second layer to obtain the second feature map; the feature image output by the convolution layer of the first layer is obtained based on the first upsampled image; the The first up-sampled image is obtained by upsampling the feature image output by the convolution layer of the second level; the feature image output by the convolution layer of the second level is down-sampled by the second feature fusion module and then feature fused with the feature image output by the convolution layer of the third level to obtain the third feature map; the feature image output by the convolution layer of the second level is obtained based on the second up-sampled image; the second up-sampled image is obtained by upsampling the feature image output by the convolution layer of the third level; wherein the first feature fusion module and the second feature fusion module are both composed of a 1×1 convolution layer and a 2×2 pooling layer.

[0009] Optionally, before using the target data set to train the improved YOLOv4 convolutional neural network model and obtaining the target model, the method also includes: using a k-means algorithm to optimize and classify the annotation boxes of the images containing Dongba hieroglyphs annotated in the target data set to obtain k detection boxes of different sizes; using the k detection boxes of different sizes as the detection boxes of the improved YOLOv4 convolutional neural network model; wherein the k detection boxes of different sizes are respectively applied to the images output by the three levels of the PANet structure; k is determined based on the number of levels of the PANet structure.

[0010] Optionally, the use of the k-means algorithm to optimize and classify the detection frames of the images annotated with Dongba hieroglyphs in the target data set to obtain k detection frames of different sizes includes: selecting k initial clustering centers within the size range of the samples contained in the target data set; calculating the Euclidean distance between the center of the hieroglyphic annotation frame of each sample in the target data set and the k initial clustering centers, dividing the center of the hieroglyphic annotation frame into clusters corresponding to the initial clustering centers with the shortest Euclidean distance, and recalculating the cluster center of each cluster; after satisfying preset conditions, obtaining the k detection frames of different sizes.

[0011] Optionally, before using the target data set to train the improved YOLOv4 convolutional neural network model and obtaining the target model, the method also includes: performing a Gaussian filtering operation on the image samples in the target data set to remove excess noise in the image samples; performing a binarization operation on the image samples and performing a dilation operation on the processed image samples to remove excess burrs in the image samples.

[0012] Optionally, after identifying the Dongba hieroglyphs in the target image through the target model and obtaining the recognition result, the method also includes: matching the identified Dongba hieroglyphs with the Dongba hieroglyphs in the Dongba hieroglyph template library, and replacing the identified Dongba hieroglyphs with corresponding standard Dongba hieroglyphs.

[0013] The present application also provides a Dongba pictographic character recognition device, comprising:

[0014] An acquisition module is used to acquire a target image containing at least one Dongba pictograph; a recognition module is used to recognize the Dongba pictographs in the target image through a target model and obtain a recognition result; wherein the target model is obtained by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM.

[0015] Optionally, the device also includes: a training module; the acquisition module is also used to acquire a target data set; the target data set contains annotated Dongba pictographs, and the number of each annotated Dongba pictograph exceeds a preset value; the training module is used to use the target data set to train the improved YOLOv4 convolutional neural network model and obtain the target model; wherein the path aggregation network PANet structure of the improved YOLOv4 convolutional neural network model includes three levels, each level outputs images of different sizes, and CBAM is set between the convolution blocks of each level.

[0016] Optionally, the PANet structure includes a first layer, a second layer and a third layer; the first layer outputs a first feature map, the second layer outputs a second feature map, and the third layer outputs a third feature map; the training module is specifically used to downsample the feature image output by the convolution layer of the first layer through a first feature fusion module, and then perform feature fusion with the feature image output by the convolution layer of the second layer to obtain the second feature map; the feature image output by the convolution layer of the first layer is obtained based on the first up-sampled image; the first up-sampled image is a feature image output by the convolution layer of the second layer. The feature image is obtained by upsampling; the training module is specifically used to downsample the feature image output by the second-level convolution layer through the second feature fusion module, and then perform feature fusion with the feature image output by the third-level convolution layer to obtain the third feature map; the feature image output by the second-level convolution layer is obtained based on the second upsampled image; the second upsampled image is obtained by upsampling the feature image output by the third-level convolution layer; wherein the first feature fusion module and the second feature fusion module are both composed of a 1×1 convolution layer and a 2×2 pooling layer.

[0017] Optionally, the device also includes: a detection frame optimization module; the detection frame optimization module is used to use the k-means algorithm to optimize and classify the annotation frames of the images containing Dongba hieroglyphs in the target data set to obtain k detection frames of different sizes; the detection frame optimization module is also used to use the k detection frames of different sizes as the detection frames of the improved YOLOv4 convolutional neural network model; wherein the k detection frames of different sizes are respectively applied to the images output by the three levels of the PANet structure; k is determined based on the number of levels of the PANet structure.

[0018] Optionally, the detection frame optimization module is specifically used to select k initial clustering centers within the size range of samples included in the target data set; the detection frame optimization module is specifically used to calculate the Euclidean distance between the center of the pictographic annotation frame of each sample in the target data set and the k initial clustering centers, divide the center of the pictographic annotation frame into clusters corresponding to the initial clustering centers with the shortest Euclidean distance, and recalculate the cluster center of each cluster; the detection frame optimization module is specifically used to obtain the k detection frames of different sizes after satisfying preset conditions.

[0019] Optionally, the device includes: an image processing module; the image processing module is used to perform a Gaussian filtering operation on the image samples in the target data set to remove excess noise in the image samples; the image processing module is also used to perform a binarization operation on the image samples, and perform a dilation operation on the processed image samples to remove excess burrs in the image samples.

[0020] Optionally, the recognition module is further configured to match the recognized Dongba pictographs with Dongba pictographs in a Dongba pictograph template library, and replace the recognized Dongba pictographs with corresponding standard Dongba pictographs.

[0021] The present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of any of the above-mentioned Dongba pictograph recognition methods.

[0022] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the above-mentioned Dongba pictograph recognition methods are implemented.

[0023] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned Dongba pictograph recognition methods are implemented.

[0024] The Dongba pictograph recognition method and device provided in the present application obtain a target model by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM. After a target image containing at least one Dongba pictograph is input into the target model, the Dongba pictograph in the target image is recognized by the target model, and a recognition result is obtained, thereby realizing the recognition of Dongba pictographs and having the advantages of high recognition accuracy and fast recognition speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 This is one of the flow charts of the Dongba pictograph recognition method provided by this application;

[0027] Figure 2 This is the second flow chart of the Dongba pictograph recognition method provided by this application;

[0028] Figure 3 This is a schematic diagram of the structure of the improved YOLOv4 convolutional neural network model provided by this application;

[0029] Figure 4 It is a schematic diagram of the structure of the Dongba pictograph recognition device provided by the present application;

[0030] Figure 5 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0033] In the actual Dongba pictographic recognition, the Dongba ancient books themselves were written by different old Dongba writers on specific Dongba paper. Dongba paper has different texture characteristics, which causes a certain degree of noise interference for the recognition of Dongba pictographic characters. And because different old Dongba writers have different writing habits, the Dongba characters on the Dongba ancient books have different shapes and sizes, and there are variant characters, which adds difficulty to the recognition of Dongba pictographic characters.

[0034] The existing Dongba pictograph recognition method has many recognition steps, the recognition process is relatively complicated, and there is no specific system to comprehensively display the Dongba pictograph recognition results. Therefore, the embodiment of the present application provides an improved YOLOv4 convolutional neural network model for the above-mentioned series of problems. On the basis of the YOLOv4 algorithm, the image target feature information is effectively screened by inserting the convolutional block attention module (CBAM), the feature fusion module is used to deepen the extraction depth of the network model, and the k-means algorithm is combined to optimize the detection frame size in the network structure by calculating the data set. The Dongba pictograph training set is input into the improved YOLOv4 convolutional neural network model for training, and the trained Dongba pictograph recognition model is used to identify, detect and locate the image information, and mark the confidence. The Dongba pictograph recognition result is interfaced and visualized, so that it can truly realize the efficiency and convenience of Dongba pictograph recognition.

[0035] Specifically, Figure 1 As shown, the Dongba pictograph recognition model provided by the embodiment of the present application is obtained by training based on the improved YOLOv4 convolutional neural network model of CBAM, and the detection frame size of the Dongba pictograph recognition model is obtained after optimization based on the k-means algorithm. After inputting an image containing Dongba pictographs into the Dongba pictograph recognition model, the Dongba pictographs in the image can be recognized. Afterwards, the Chinese corresponding to each Dongba pictograph can be obtained by matching with the Dongba pictograph template in the database.

[0036] The following is a detailed description of the Dongba pictograph recognition method provided by the embodiment of the present application through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0037] like Figure 2 As shown, a Dongba pictograph recognition method provided in an embodiment of the present application may include the following steps 201 and 202:

[0038] Step 201: Acquire a target image containing at least one Dongba pictograph.

[0039] Exemplarily, the content in the above target image includes: Dongba hieroglyphs written on Dongba paper. Since Dongba paper has different texture features, these texture features may cause certain interference to the recognition of Dongba hieroglyphs.

[0040] Exemplarily, the target image contains other contents due to environmental or human factors, such as horizontal and vertical lines, black blocks, etc. These contents do not belong to Dongba pictographs, so it is necessary to identify and extract the Dongba pictographs in the target image.

[0041] Step 202: Recognize the Dongba pictographs in the target image through the target model and obtain a recognition result.

[0042] Among them, the target model is obtained by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM.

[0043] Exemplarily, the target model is as follows Figure 1 The improved YOLOv4 convolutional neural network model shown in FIG. 1 is trained to obtain the above target model, that is, Figure 1 The Dongba hieroglyphic recognition model shown.

[0044] Exemplarily, after the target image is input into the target model, the target model outputs a recognition result of the target image, which includes the Dongba pictographs recognized from the target image.

[0045] Exemplarily, the recognition result may also include the Chinese characters corresponding to each Dongba pictograph recognized in the target image.

[0046] In this way, after training the improved YOLOv4 convolutional neural network model based on the convolutional block attention module CBAM, a target model is obtained. After a target image containing at least one Dongba pictograph is input into the target model, the Dongba pictograph in the target image is recognized by the target model, and a recognition result is obtained, thereby realizing the recognition of Dongba pictographs with the advantages of high recognition accuracy and fast recognition speed.

[0047] In a possible implementation, the target model can be obtained by training the improved YOLOv4 convolutional neural network model according to the following steps.

[0048] Exemplarily, before the above step 202, the Dongba pictograph recognition method provided in the embodiment of the present application may further include the following steps 203 and 204:

[0049] Step 203: Obtain the target data set.

[0050] The target data set includes annotated Dongba pictographs, and the number of each annotated Dongba pictograph exceeds a preset value.

[0051] Step 204: Use the target data set to train the improved YOLOv4 convolutional neural network model and obtain the target model.

[0052] Among them, the path aggregation network (PANet) structure of the improved YOLOv4 convolutional neural network model includes three levels, each level outputs images of different sizes, and CBAM is set between the convolution blocks of each level.

[0053] Exemplarily, the target data set is used to train the improved YOLOv4 convolutional neural network model. In order to obtain the target data set, images containing Dongba pictographs can be collected from ancient books. Specifically, an image device with a charge coupled device (CCD) image sensor is used to collect Dongba pictograph images, and the images contain a large number of Dongba pictographs for character detection.

[0054] Exemplarily, after collecting an image containing Dongba pictographs, the labelimg annotation tool can be used to manually annotate the collected image with Dongba pictographs. The specific label content is operated in combination with the Naxi pictograph spectrum. Since the quality and quantity of annotations have an important impact on the accuracy of the recognition model trained by the neural network, in the embodiment of the present application, the number of annotations for each type of Dongba pictographs must reach 100 (i.e., the above-mentioned preset value) or more to ensure the recognition function of the Dongba pictograph recognition model. Afterwards, the annotated xml file and the corresponding jpg image are organized into a Dongba pictograph data set, i.e., the above-mentioned target data set.

[0055] For example, after obtaining the above target data set, the annotated Dongba pictograph data set is randomly divided into a training set and a validation set at a ratio of 9:1, and a suitable number of pictures are randomly selected as a test set. The divided training set is input into the above improved YOLOv4 convolutional neural network model for model training. By comparing the recognition effect of the model with the validation set, the model parameters are adjusted to obtain the image recognition model with the best effect.

[0056] Specifically, the PANet structure includes a first layer, a second layer and a third layer; the first layer outputs a first feature map, the second layer outputs a second feature map, and the third layer outputs a third feature map.

[0057] Exemplarily, the above step 204 may include the following steps 204a1 and 204a2:

[0058] Step 204a1: downsample the feature image output by the convolution layer of the first level through a first feature fusion module, and perform feature fusion with the feature image output by the convolution layer of the second level to obtain the second feature map.

[0059] Among them, the feature image output by the convolution layer of the first level is obtained based on the first up-sampled image; the first up-sampled image is obtained by upsampling the feature image output by the convolution layer of the second level.

[0060] Step 204a2: downsample the feature image output by the second-level convolution layer through a second feature fusion module, and perform feature fusion with the feature image output by the third-level convolution layer to obtain the third feature map.

[0061] The feature image output by the convolution layer of the second level is obtained based on the second upsampled image; the second upsampled image is obtained by upsampling the feature image output by the convolution layer of the third level. The first feature fusion module and the second feature fusion module are both composed of a 1×1 convolution layer and a 2×2 pooling layer.

[0062] It should be noted that CBAM stands for the attention mechanism module of the convolution module. It is an attention mechanism module that combines the spatial submodule and the channel submodule. The two submodules extract features from the feature map in the network from the spatial and channel dimensions respectively, thereby enhancing the useful features in the image and removing useless features. The lightweight and universal nature of this module saves the computational cost for the network model.

[0063] For example, Figure 3 As shown in the figure, the PANet structure of the improved YOLOv4 convolutional neural network model is improved on the basis of the YOLOv4 neural network. The CBAM modules are added to the convolution blocks in the three levels of the PANet structure, which are respectively Figure 3 a3, b3, b7, c3 in the figure. And add the feature fusion module to perform multi-layer feature fusion. The introduction of CBAM greatly improves the representation ability of the network and effectively selects the feature information in the image. The feature fusion module realizes multi-level feature fusion and improves the feature extraction depth of the network.

[0064] It should be noted that the three images output by the above PANet structure include annotation boxes, which are created based on the detection boxes in the recognition process. At this time, it is necessary to calculate the size conversion relationship between the input image and the images output by the above three levels, and map the annotation boxes on the output image to the input image according to the conversion relationship to obtain the recognition image containing the annotation boxes.

[0065] Optionally, in an embodiment of the present application, before using the above-mentioned target data set to train the above-mentioned improved YOLOv4 convolutional neural network model, it is also necessary to optimize the detection box of the improved YOLOv4 convolutional neural network model through the k-means algorithm.

[0066] Exemplarily, before the above step 204, the Dongba pictograph recognition method provided in the embodiment of the present application may further include the following steps 205 and 206:

[0067] Step 205: Use the k-means algorithm to optimize and classify the annotated boxes of the images containing Dongba pictographs in the target data set to obtain k detection boxes of different sizes.

[0068] Step 206: Use the k detection boxes of different sizes as detection boxes of the improved YOLOv4 convolutional neural network model.

[0069] The k detection frames of different sizes are respectively applied to the images output by the three levels of the PANet structure; k is determined based on the number of levels of the PANet structure.

[0070] Specifically, in the above step 205, using the k-means algorithm to optimize and classify the annotation boxes of the images containing Dongba pictographs in the target data set may include the following steps 205a1 to 205a3:

[0071] Step 205a1: Select k initial cluster centers within the size range of samples included in the target data set.

[0072] Step 205a2, calculate the Euclidean distance between the center of the pictographic annotation box of each sample in the target data set and the k initial clustering centers, divide the center of the pictographic annotation box into the cluster corresponding to the initial clustering center with the shortest Euclidean distance, and recalculate the clustering center of each cluster.

[0073] Step 205a3: After the preset conditions are met, the k detection frames of different sizes are obtained.

[0074] Exemplarily, the preset condition may be a preset number of iterations, or a change in the center point of the cluster is less than a preset change.

[0075] Exemplarily, the k initial cluster centers are randomly selected. Since the PANet structure contains three levels, and the size of the image output by each level is different, the size of the Dongba pictographs contained is also different, and the size of the detection frame to be used is also different. Therefore, three detection frames of different sizes can be assigned to each level, and a total of 9 detection frames of different sizes are required for the three levels.

[0076] Exemplarily, the above improved YOLOv4 convolutional neural network model can normalize the input image into an image of 416×416 size and 3 channels, and then divide the image into three different scale feature maps of 52×52, 26×26, and 13×13, and configure three different scale detection frames for the three different scale feature maps to perform bounding box detection. The sizes of the 9 different detection frames obtained by the above k-means algorithm are: (11,56), (13,42), (15,69), (18,29), (18,51), (22,38), (25,50), (28,30), (50,30). The three different scale feature maps correspond to three detection frames respectively.

[0077] Exemplarily, after obtaining the above-mentioned target data set and optimizing the detection frame of the above-mentioned improved YOLOv4 convolutional neural network model by the k-means algorithm, the training set and the verification set are input into the improved YOLOv4 convolutional neural network model for training, the learning rate is 0.0001, the batch size of the number of images input into the network each time is 2, and the total number of training iterations is set to 100.

[0078] It is understandable that the above-mentioned k-means algorithm is a commonly used clustering algorithm based on Euclidean distance, which believes that the closer the distance between two targets is, the greater the similarity is. In target detection, the intersection over union (IoU) between the annotated boxes is calculated as a distance metric. The k-means algorithm mainly calculates the ratio of the intersection area of ​​the annotated box and the initial box. If it is close to 1, the position is consistent, or completely overlapped; if it is less than 1, it does not overlap, or does not overlap at all. By calculating IoU again, the degree of offset between the prior box and the initial box can be obtained, and the ones with the smallest offset degree are classified into one category, and then the centers of the prior boxes of each category are averaged to obtain the new clustering center, and so on, and the calculation is iterated continuously until the above preset conditions are met.

[0079] In a possible implementation, in order to reduce the influence of noise in the image samples, the image samples may be processed before the model training.

[0080] Exemplarily, before the above step 204, the Dongba pictograph recognition method provided in the embodiment of the present application may further include the following steps 207 and 208:

[0081] Step 207: Perform a Gaussian filtering operation on the image samples in the target data set to remove excess noise in the image samples.

[0082] Step 208: perform a binarization operation on the image samples, and perform a dilation operation on the processed image samples to remove unnecessary burrs in the image samples.

[0083] For example, when processing image samples, the image samples may be first grayscale processed, and then Gaussian filtering may be performed to remove excess noise in the image samples. After that, a binarization operation may be performed to reduce the amount of data to be processed and highlight the outline of the target. Finally, a dilation operation may be performed to remove excess burrs in the image samples.

[0084] Optionally, in the embodiment of the present application, the target model can only identify the Dongba pictographs contained in the target image, and by matching the identified Dongba pictographs with the templates stored in the database, obtain the standard Dongba pictograph corresponding to each Dongba pictograph in the target image. Afterwards, by identifying the standard Dongba pictograph, the corresponding Chinese is determined.

[0085] Exemplarily, the above step 202 may include the following steps 202a:

[0086] Step 202a1, matching the recognized Dongba pictographs with the Dongba pictographs in the Dongba pictograph template library, and replacing the recognized Dongba pictographs with the corresponding standard Dongba pictographs.

[0087] It is understandable that since Dongba pictographs are written by different old Dongba writers according to their writing habits, in order to improve the accuracy of Dongba pictograph recognition, it is necessary to replace the Dongba pictographs in the target image with standard Dongba imaginary texts, and then translate them into Chinese.

[0088] For example, a standard Dongba pictographic template library is created using a MySQL database, and the standard Dongba characters in the template library and the corresponding Chinese characters are stored in the database. The recognition result is replaced with the corresponding Chinese information in the template library, and the recognition box size is adaptively replaced with the Chinese corresponding to the Dongba characters, and the processing result is displayed on the interface.

[0089] Specifically, an application interface can be designed through Qt desiner to display the image after replacing the standard Dongba pictograms and the image translated into Chinese respectively.

[0090] Exemplarily, after obtaining the above-mentioned recognition image, in order to facilitate user viewing, the background of the above-mentioned recognition image can be removed.

[0091] Specifically, a completely white background image can be created, and the Dongba hieroglyph recognition results output by the target model can be matched to the corresponding Chinese interpretations in the MySQL database. The standard Chinese interpretations can be written using the truetype font toolkit, and the corresponding scaling factor can be obtained in combination with the size of the recognition box to ensure that the Chinese replacement result is consistent with the original Dongba character size. According to the recognition results, the Chinese characters are replaced at the corresponding positions, and the image processing results are displayed in the interface.

[0092] The Dongba pictograph recognition method provided in the embodiment of the present application obtains a target model by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM. After a target image containing at least one Dongba pictograph is input into the target model, the Dongba pictograph in the target image is recognized by the target model, and a recognition result is obtained, thereby realizing the recognition of Dongba pictographs and having the advantages of high recognition accuracy and fast recognition speed.

[0093] It should be noted that the Dongba pictograph recognition method provided in the embodiment of the present application uses an improved YOLOv4 convolutional neural network model to perform iterative operations on the input image information, realizes the feature extraction of Dongba pictographs in the image, and modifies the weight parameters in the model to obtain a Dongba pictograph recognition model, and then uses the Dongba pictograph recognition model to detect and recognize Dongba pictographs in the image. By adding CBAM and feature fusion modules to the PANet structure of the existing YOLOv4 convolutional neural network model, and optimizing the detection frame of the YOLOv4 convolutional neural network model by the k-means algorithm, the effective selection of Dongba pictograph feature information is realized respectively, thereby retaining important feature data, and the feature information of different levels is integrated to improve the feature extraction ability of the network model and the size optimization of the model detection frame by the k-means algorithm. The Dongba pictograph recognition method provided in the embodiment of the present application is effective and fast enough to realize the recognition of Dongba pictographs, significantly improving the robustness and accuracy of the model. At the same time, the translation function of the recognition results, the visual output of standard character replacement and background removal effects optimize the Dongba pictograph recognition results, thereby achieving the effect of simplicity and easy operation of the recognition work.

[0094] It should be noted that, in the embodiments of the present application, the Dongba pictograph recognition method shown in the above-mentioned method drawings is illustrated by taking one of the drawings in the embodiments of the present application as an example. In specific implementation, the Dongba pictograph recognition method shown in the above-mentioned method drawings can also be implemented in combination with any other drawings that can be combined as shown in the above-mentioned embodiments, which will not be repeated here.

[0095] The Dongba pictograph recognition device provided by the present application is described below, and the Dongba pictograph recognition method described below and above can be referenced to each other.

[0096] Figure 4 The structure diagram of the Dongba pictographic character recognition device provided in one embodiment of the present application is as follows: Figure 4 As shown, specifically including:

[0097] The acquisition module 401 is used to acquire a target image containing at least one Dongba pictograph; the recognition module 402 is used to recognize the Dongba pictograph in the target image through a target model and obtain a recognition result; wherein the target model is obtained by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM.

[0098] Optionally, the device also includes: a training module; the acquisition module 401 is also used to acquire a target data set; the target data set contains annotated Dongba pictographs, and the number of each annotated Dongba pictograph exceeds a preset value; the training module is used to use the target data set to train the improved YOLOv4 convolutional neural network model and obtain the target model; wherein the path aggregation network PANet structure of the improved YOLOv4 convolutional neural network model includes three levels, each level outputs images of different sizes, and CBAM is set between the convolution blocks of each level.

[0099] Optionally, the PANet structure includes a first layer, a second layer and a third layer; the first layer outputs a first feature map, the second layer outputs a second feature map, and the third layer outputs a third feature map; the training module is specifically used to downsample the feature image output by the convolution layer of the first layer through a first feature fusion module, and then perform feature fusion with the feature image output by the convolution layer of the second layer to obtain the second feature map; the feature image output by the convolution layer of the first layer is obtained based on the first up-sampled image; the first up-sampled image is a feature image output by the convolution layer of the second layer. The feature image is obtained by upsampling; the training module is specifically used to downsample the feature image output by the second-level convolution layer through the second feature fusion module, and then perform feature fusion with the feature image output by the third-level convolution layer to obtain the third feature map; the feature image output by the second-level convolution layer is obtained based on the second upsampled image; the second upsampled image is obtained by upsampling the feature image output by the third-level convolution layer; wherein the first feature fusion module and the second feature fusion module are both composed of a 1×1 convolution layer and a 2×2 pooling layer.

[0100] Optionally, the device also includes: a detection frame optimization module; the detection frame optimization module is used to use the k-means algorithm to optimize and classify the annotation frames of the images containing Dongba hieroglyphs in the target data set to obtain k detection frames of different sizes; the detection frame optimization module is also used to use the k detection frames of different sizes as the detection frames of the improved YOLOv4 convolutional neural network model; wherein the k detection frames of different sizes are respectively applied to the images output by the three levels of the PANet structure; k is determined based on the number of levels of the PANet structure.

[0101] Optionally, the detection frame optimization module is specifically used to select k initial clustering centers within the size range of samples included in the target data set; the detection frame optimization module is specifically used to calculate the Euclidean distance between the center of the pictographic annotation frame of each sample in the target data set and the k initial clustering centers, divide the center of the pictographic annotation frame into clusters corresponding to the initial clustering centers with the shortest Euclidean distance, and recalculate the cluster center of each cluster; the detection frame optimization module is specifically used to obtain the k detection frames of different sizes after satisfying preset conditions.

[0102] Optionally, the device includes: an image processing module; the image processing module is used to perform a Gaussian filtering operation on the image samples in the target data set to remove excess noise in the image samples; the image processing module is also used to perform a binarization operation on the image samples, and perform a dilation operation on the processed image samples to remove excess burrs in the image samples.

[0103] Optionally, the recognition module 402 is further configured to match the recognized Dongba pictographs with the Dongba pictographs in the Dongba pictograph template library, and replace the recognized Dongba pictographs with corresponding standard Dongba pictographs.

[0104] The Dongba pictograph recognition device provided by the present application uses the improved YOLOv4 convolutional neural network model to perform iterative operations on the input image information, realizes the feature extraction of Dongba pictographs in the image, and modifies the weight parameters in the model to obtain the Dongba pictograph recognition model, and then uses the Dongba pictograph recognition model to detect and recognize the Dongba pictographs in the image. By adding CBAM and feature fusion modules to the PANet structure of the existing YOLOv4 convolutional neural network model, and optimizing the detection frame of the YOLOv4 convolutional neural network model by the k-means algorithm, the effective selection of Dongba pictograph feature information is realized respectively, thereby retaining important feature data, and the fusion of feature information at different levels improves the feature extraction ability of the network model and the size optimization of the model detection frame by the k-means algorithm. The Dongba pictograph recognition method provided by the embodiment of the present application can effectively and quickly realize the recognition of Dongba pictographs, significantly improves the robustness and accuracy of the model, and at the same time optimizes the translation function of the recognition result, the visual output of the standard character replacement and background removal effect, so that the Dongba pictograph recognition result is optimized, and the recognition work is concise and easy to operate.

[0105] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the Dongba pictograph recognition method, which includes: obtaining a target image containing at least one Dongba pictograph; recognizing the Dongba pictograph in the target image through a target model, and obtaining a recognition result; wherein the target model is obtained by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM.

[0106] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0107] On the other hand, the present application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the Dongba hieroglyph recognition method provided by the above methods, and the method includes: obtaining a target image containing at least one Dongba hieroglyph; recognizing the Dongba hieroglyphs in the target image through a target model, and obtaining a recognition result; wherein the target model is obtained by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM.

[0108] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned Dongba hieroglyphic recognition method, the method comprising: obtaining a target image containing at least one Dongba hieroglyph; recognizing the Dongba hieroglyphs in the target image through a target model, and obtaining a recognition result; wherein the target model is obtained by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM.

[0109] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0110] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A Dongba pictographic character recognition method, characterized in that: include: Acquire a target image containing at least one Dongba pictograph; Recognize the Dongba pictographs in the target image through the target model and obtain a recognition result; Wherein, the target model is obtained by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM; the path aggregation network PANet structure of the improved YOLOv4 convolutional neural network model includes three levels, each level outputs images of different sizes, and CBAM is set between the convolution blocks of each level; the PANet structure includes a first level, a second level and a third level; the first level outputs a first feature map, the second level outputs a second feature map, and the third level outputs a third feature map; The PANet structure is used to: perform feature fusion on the feature image output by the convolution layer of the first level after downsampling through the first feature fusion module and then perform feature fusion on the feature image output by the convolution layer of the second level to obtain the second feature map; the feature image output by the convolution layer of the first level is obtained based on the first upsampled image; the first upsampled image is obtained by upsampling the feature image output by the convolution layer of the second level; The PANet structure is also used to: downsample the feature image output by the second-level convolution layer through a second feature fusion module and perform feature fusion with the feature image output by the third-level convolution layer to obtain the third feature map; the feature image output by the second-level convolution layer is obtained based on the second upsampled image; the second upsampled image is obtained by upsampling the feature image output by the third-level convolution layer; The first feature fusion module and the second feature fusion module are both composed of a convolution layer and a pooling layer.

2. The method according to claim 1, characterized in that Before identifying the Dongba pictographs in the target image by using the target model and obtaining the recognition result, the method further includes: Acquire a target data set; the target data set includes annotated Dongba pictographs, and the number of each annotated Dongba pictograph exceeds a preset value; The improved YOLOv4 convolutional neural network model is trained using the target data set to obtain the target model.

3. The method according to claim 2, characterized in that The first feature fusion module and the second feature fusion module are both composed of a 1×1 convolution layer and a 2×2 pooling layer.

4. The method according to claim 2, characterized in that: Before using the target data set to train the improved YOLOv4 convolutional neural network model and obtain the target model, the method further includes: Using the k-means algorithm to optimize and classify the labeled boxes of the images containing Dongba pictographs in the target data set, and obtain k detection boxes of different sizes; Using the k detection frames of different sizes as the detection frames of the improved YOLOv4 convolutional neural network model; The k detection frames of different sizes are respectively applied to the images output by the three levels of the PANet structure; k is determined based on the number of levels of the PANet structure.

5. The method according to claim 4, characterized in that The k-means algorithm is used to optimize and classify the detection frames of the images containing Dongba pictographs annotated in the target data set to obtain k detection frames of different sizes, including: Select k initial cluster centers within the size range of samples included in the target data set; Calculating the Euclidean distance between the center of the pictographic annotation box of each sample in the target data set and the k initial clustering centers, dividing the center of the pictographic annotation box into the cluster corresponding to the initial clustering center with the shortest Euclidean distance, and recalculating the clustering center of each cluster; After the preset conditions are met, the k detection frames of different sizes are obtained.

6. The method according to claim 2, characterized in that Before using the target data set to train the improved YOLOv4 convolutional neural network model and obtain the target model, the method further includes: Performing a Gaussian filtering operation on the image samples in the target data set to remove redundant noise in the image samples; A binarization operation is performed on the image samples, and a dilation operation is performed on the processed image samples to remove redundant burrs in the image samples.

7. The method according to claim 1, characterized in that After the Dongba pictographs in the target image are recognized by the target model and the recognition result is obtained, the method further includes: The recognized Dongba hieroglyphs are matched with the Dongba hieroglyphs in the Dongba hieroglyph template library, and the recognized Dongba hieroglyphs are replaced with the corresponding standard Dongba hieroglyphs.

8. A Dongba pictographic character recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring a target image containing at least one Dongba pictograph; A recognition module, used for recognizing the Dongba pictographs in the target image through a target model and obtaining a recognition result; Wherein, the target model is obtained by training an improved YOLOv4 convolutional neural network model based on a convolutional block attention module CBAM; the path aggregation network PANet structure of the improved YOLOv4 convolutional neural network model includes three levels, each level outputs images of different sizes, and CBAM is set between the convolution blocks of each level; the PANet structure includes a first level, a second level and a third level; the first level outputs a first feature map, the second level outputs a second feature map, and the third level outputs a third feature map; The PANet structure is used to: perform feature fusion on the feature image output by the convolution layer of the first level after downsampling through the first feature fusion module and then perform feature fusion on the feature image output by the convolution layer of the second level to obtain the second feature map; the feature image output by the convolution layer of the first level is obtained based on the first upsampled image; the first upsampled image is obtained by upsampling the feature image output by the convolution layer of the second level; The PANet structure is also used to: downsample the feature image output by the second-level convolution layer through a second feature fusion module and perform feature fusion with the feature image output by the third-level convolution layer to obtain the third feature map; the feature image output by the second-level convolution layer is obtained based on the second upsampled image; the second upsampled image is obtained by upsampling the feature image output by the third-level convolution layer; The first feature fusion module and the second feature fusion module are both composed of a convolution layer and a pooling layer.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the Dongba pictograph recognition method as claimed in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the Dongba pictograph recognition method described in any one of claims 1 to 7 are implemented.

Images