Oracle bone script denoising method, device and medium based on channel attention and glyph aggregation

Abstract

The present invention relates to a method, device, and medium for oracle bone script denoising based on channel attention and glyph aggregation. The method comprises the following steps: obtaining a noisy oracle bone script image; inputting the oracle bone script image into a denoising model, and outputting a denoised oracle bone script image and a character skeleton. The denoising model comprises an input mapper, an output mapper, an ECharFormer module, and a glyph corrector. The input image is converted into shallow features by the input mapper, and the shallow features are converted into reconstructed features and glyph features by the ECharFormer module. The reconstructed features are converted into a denoised oracle bone script image by the output mapper and output, and the glyph features are converted into a character skeleton by the glyph corrector and output. Compared with the prior art, the present invention can achieve fast and good denoising effects while preserving the oracle bone script glyphs.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an oracle bone script denoising method, device and medium based on channel attention and glyph aggregation. Background Art

[0002] Oracle bone script is one of China's earliest written languages ​​and is of great significance to the development of anthropology and archaeology. However, many unrecognized characters remain among the unearthed oracle bones, and manual identification and annotation consumes significant manpower and resources. With the development of artificial intelligence, technologies such as convolutional neural networks and Transformers have been applied in various fields, making automatic oracle bone recognition and annotation possible. However, unearthed oracle bone rubbings contain a significant amount of noise, which poses significant challenges to automatic oracle bone recognition. Existing technologies for denoising oracle bone images destroy the glyph structure, and most existing models are very complex, consuming significant time and computing resources during denoising. Summary of the Invention

[0003] The purpose of the present invention is to provide an oracle bone script denoising method, device and medium based on channel attention and glyph aggregation.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] An oracle bone inscription denoising method based on channel attention and glyph aggregation includes the following steps:

[0006] Obtaining an oracle bone image containing noise;

[0007] An oracle bone script image is input into a denoising model, and a denoised oracle bone script image and a character skeleton are output. The denoising model includes an input mapper, an output mapper, an ECharFormer module and a font corrector. The input image is converted into shallow features by the input mapper, and the shallow features are converted into reconstructed features and font features by the ECharFormer module. The reconstructed features are converted into a denoised oracle bone script image by the output mapper and output, and the font features are converted into a character skeleton by the font corrector and output.

[0008] The ECharFormer module is composed of N+1 encoding ECharFormer blocks and N decoding EcharFormer blocks cascaded, numbered 1, 2,,,,N,N+1,,,2N+1, and the encoding ECharFormer block numbered i is residually linked to the decoding EcharFormer block numbered 2N+2-i, and the encoding ECharFormer block numbered N+1 is not residually linked to any decoding EcharFormer block.

[0009] The ECharFormer block consists of a channel self-attention block, a font structure block, and a selective kernel feature fusion module.

[0010] The input of the ECharFormer block is converted into corresponding reconstructed intermediate features and glyph intermediate features through the channel-word attention block and the glyph structure block respectively;

[0011] The reconstructed intermediate features and the font intermediate features are converted into feature fusion attention maps through the selective kernel feature fusion module;

[0012] The reconstructed intermediate features and the glyph intermediate features are respectively added to the feature fusion attention map to output the reconstructed features and glyph features of the current level as the input of the next level ECharFormer block; among them, the input of the first level encoding ECharFormer block is the shallow feature, and the output of the last level decoding EcharFormer block is the reconstructed features and glyph features finally output by the ECharFormer module.

[0013] The channel self-attention block integrates the channel attention mechanism and the multi-head self-attention mechanism, processes the input features and outputs the reconstructed intermediate features;

[0014] The font structure block sequentially passes the input features through two convolution blocks and then adds the input features to obtain the font intermediate features, wherein the convolution block is composed of a convolution layer and a batch normalization layer with ReLU as the activation function;

[0015] The selective kernel feature fusion module sequentially performs cascade, reconstruction, element-level addition, pooling and convolution operations on the input reconstructed intermediate features and font intermediate features to obtain a first feature. The first feature is sequentially cascaded, reconstructed and softmax layered after reconstructed convolution and font convolution to obtain a second feature. The second feature is respectively matrix multiplied with the input reconstructed intermediate features and font intermediate features and then added to obtain a feature fusion attention map.

[0016] The input mapper consists of a 3*3 convolutional layer and a LeakyReLU activation function.

[0017] The output mapper consists of a 3*3 convolutional layer.

[0018] The font corrector consists of a 3*3 convolutional layer.

[0019] The loss function of the denoising model includes image loss and structural loss, wherein the image loss measures the difference between the denoised oracle bone image output by the model and the true image, and the structural loss measures the difference between the character skeleton output by the model and the clean character skeleton, which is calculated by an existing method based on mathematical morphology.

[0020] An oracle bone script denoising device based on channel attention and glyph aggregation comprises a memory, a processor, and a program stored in the memory, wherein the processor implements the above-mentioned method when executing the program.

[0021] A storage medium stores a program thereon, and when the program is executed, the method described above is implemented.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) The present invention utilizes the selective kernel feature fusion technology to integrate the font information into the reconstructed features, so that the font structure of the oracle bone script can be well preserved during denoising.

[0024] (2) This paper uses channel attention to improve the calculation method of self-attention in Transformer, thereby greatly reducing the complexity of the model and having the advantages of lightweight and fast calculation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Schematic diagram of the oracle bone inscription denoising process of the present invention;

[0026] Figure 2 The figures are comparison images before and after denoising in an embodiment, wherein (2a) is the oracle bone script image before denoising, (2b) is the oracle bone script image after denoising using the present invention, (2c) is the true value image, and (2d) is the oracle bone script character skeleton extracted using the present invention. DETAILED DESCRIPTION

[0027] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0028] This embodiment provides an oracle bone script denoising method based on channel attention and glyph aggregation, comprising the following steps:

[0029] S1, obtain the oracle bone image containing noise;

[0030] S2, inputs the oracle bone script image into the denoising model and outputs the denoised oracle bone script image and character skeleton.

[0031] like Figure 1 As shown, the denoising model includes an input mapper, an output mapper, an ECharFormer module and a font corrector. The input image is converted into shallow features through the input mapper, and the shallow features are converted into reconstructed features and font features through the ECharFormer module. Among them, the reconstructed features are converted into denoised oracle bone script images through the output mapper and output, and the font features are converted into character skeletons through the font corrector and output.

[0032] (1) Input Projector

[0033] The input mapper consists of a 3*3 convolutional layer and a LeakyReLU activation function.

[0034] (2) ECharFormer module

[0035] The ECharFormer module is composed of N+1 encoding ECharFormer blocks and N decoding EcharFormer blocks cascaded, numbered 1, 2,,,,N,N+1,,,2N+1, and the encoding ECharFormer block numbered i is residually linked to the decoding EcharFormer block numbered 2N+2-i, and the encoding ECharFormer block numbered N+1 is not residually linked to any decoding EcharFormer block.

[0036] The ECharFormer block consists of a channel self-attention block (CSAB), a glyph structure block (GSNB), and a selective kernel feature fusion module (SKFF).

[0037] The input of the ECharFormer block is converted into corresponding reconstructed intermediate features and glyph intermediate features through the channel-word attention block and the glyph structure block respectively;

[0038] The reconstructed intermediate features and the font intermediate features are converted into feature fusion attention maps through the selective kernel feature fusion module;

[0039] The reconstructed intermediate features and the glyph intermediate features are respectively added to the feature fusion attention map to output the reconstructed features and glyph features of the current level as the input of the next level ECharFormer block; among them, the input of the first level encoding ECharFormer block is the shallow feature, and the output of the last level decoding EcharFormer block is the reconstructed features and glyph features finally output by the ECharFormer module.

[0040] like Figure 1 As shown in Figure 2, the channel self-attention block integrates the channel attention mechanism and the multi-head self-attention mechanism, processes the input features, and then outputs the reconstructed intermediate features;

[0041] The font structure block passes the input features through two convolution blocks in sequence and then adds them to the input features to obtain the font intermediate features. The convolution block is composed of a convolution layer and a batch normalization layer with ReLU as the activation function.

[0042] The selective kernel feature fusion module reconstructs the intermediate feature F of the input R and font intermediate feature F G The first feature F is obtained by sequentially performing cascade, reconstruction, element-level addition, pooling and convolution operations. C , the first feature F C After reconstruction convolution Conv R and font convolution Conv G Then, cascade, reconstruction and softmax layers are performed in sequence to obtain the second feature. The second feature is added after matrix multiplication with the input reconstructed intermediate feature and the font intermediate feature to obtain the feature fusion attention map.

[0043] (3) Output Projector

[0044] The output mapper consists of a 3*3 convolutional layer.

[0045] (4) Glyph Corrector

[0046] The font corrector consists of a 3*3 convolutional layer.

[0047] The loss function of the denoising model includes image loss and structural loss. The image loss measures the difference between the denoised oracle bone script image output by the model and the true image, and the structural loss measures the difference between the character skeleton output by the model and the clean character skeleton. The clean character skeleton is calculated by existing mathematical morphology-based methods.

[0048] Figure 2a shows an oracle bone script image before denoising. After processing it using the method of the present invention, the denoised image (Figure 2b) and the character skeleton (Figure 2d) are obtained. Comparing them with Figure 2c, it can be seen that the present invention can well preserve the oracle bone script structure and effectively denoise it. Equipped with an NVIDIA RTX 4090 GPU, the present invention only takes 10ms to denoise a 256*256 pixel image, demonstrating its fast computational speed.

[0049] This embodiment also provides an oracle bone script denoising device based on channel attention and glyph aggregation, comprising a memory, a processor, and a program stored in the memory, wherein the processor implements the above-mentioned method when executing the program.

[0050] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0051] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A method for oracle bone inscription denoising based on channel attention and glyph aggregation, characterized in that: The following steps are involved: Obtaining an oracle bone image containing noise; Inputting the oracle bone script image into a denoising model and outputting the denoised oracle bone script image and character skeleton, wherein the denoising model includes an input mapper, an output mapper, an ECharFormer module and a font corrector, wherein the input image is converted into shallow features by the input mapper, and the shallow features are converted into reconstructed features and font features by the ECharFormer module, wherein the reconstructed features are converted into the denoised oracle bone script image by the output mapper and output, and the font features are converted into the character skeleton by the font corrector and output; The ECharFormer module is composed of N+1 encoding ECharFormer blocks and N decoding EcharFormer blocks cascaded, numbered 1, 2,,,, N, N+1,,, 2N+1, and the encoding ECharFormer block numbered i is residually linked to the decoding EcharFormer block numbered 2N+2-i, and the encoding ECharFormer block numbered N+1 is not residually linked to any decoding EcharFormer block; The ECharFormer block consists of a channel self-attention block, a font structure block, and a selective kernel feature fusion module. The input of the ECharFormer block is converted into corresponding reconstructed intermediate features and font intermediate features through the channel self-attention block and the font structure block respectively; The reconstructed intermediate features and the font intermediate features are converted into feature fusion attention maps through the selective kernel feature fusion module; The reconstructed intermediate features and the glyph intermediate features are respectively added to the feature fusion attention map to output the reconstructed features and glyph features of the current level as the input of the next level ECharFormer block; among them, the input of the first level encoding ECharFormer block is the shallow feature, and the output of the last level decoding EcharFormer block is the reconstructed features and glyph features finally output by the ECharFormer module.

2. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 1, characterized in that: The channel self-attention block integrates the channel attention mechanism and the multi-head self-attention mechanism, processes the input features and outputs the reconstructed intermediate features; The font structure block sequentially passes the input features through two convolution blocks and then adds the input features to obtain the font intermediate features, wherein the convolution block is composed of a convolution layer and a batch normalization layer with ReLU as the activation function; The selective kernel feature fusion module sequentially performs cascade, reconstruction, element-level addition, pooling and convolution operations on the input reconstructed intermediate features and font intermediate features to obtain a first feature. The first feature is sequentially cascaded, reconstructed and softmax layered after reconstructed convolution and font convolution to obtain a second feature. The second feature is respectively matrix multiplied with the input reconstructed intermediate features and font intermediate features and then added to obtain a feature fusion attention map.

3. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 1, characterized in that: The input mapper consists of a 3*3 convolutional layer and a LeakyReLU activation function.

4. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 1, characterized in that: The output mapper consists of a 3*3 convolutional layer.

5. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 1, characterized in that: The font corrector consists of a 3*3 convolutional layer.

6. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 1, characterized in that: The loss function of the denoising model includes image loss and structural loss, wherein the image loss measures the difference between the denoised oracle bone image output by the model and the true image, and the structural loss measures the difference between the character skeleton output by the model and the clean character skeleton, which is calculated by an existing method based on mathematical morphology.

7. An oracle bone script denoising device based on channel attention and glyph aggregation, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

8. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 6 is implemented.

Images