Method and device for denoising oracle based on channel attention and font aggregation, and medium

Abstract

The invention relates to an oracle denoising method and device based on channel attention and font aggregation, and a medium. The method comprises the following steps: obtaining an oracle image containing noise; the oracle bone image is input into a denoising model, the denoised oracle bone image and a character skeleton are output, the denoising model comprises an input mapper, an output mapper, an ECharFormer module and a font corrector, the input image is converted into shallow layer features through the input mapper, the shallow layer features are converted into reconstruction features and font features through the ECharFormer module, and the reconstruction features and the font features are reconstructed through the output mapper. The reconstruction features are converted into denoised oracle images through an output mapper and output, and the font features are converted into character skeletons through a font corrector and output. Compared with the prior art, the method can achieve a rapid and good denoising effect while keeping the oracle alphabet fonts.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an oracle bone inscription denoising method, device and medium based on channel attention and glyph aggregation. Background Art

[0002] Oracle bone script is one of the earliest Chinese characters and is of great significance to the development of anthropology and archaeology. However, there are still many unrecognized characters in the unearthed oracle bone scripts, and manual recognition and annotation will consume a lot of manpower and material resources. With the development of artificial intelligence, technologies such as convolutional neural networks and Transformer have been applied in various fields, making automatic recognition and annotation of oracle bone scripts possible. However, the unearthed oracle bone script rubbings contain a lot of noise, which has caused a lot of trouble for the automatic recognition of oracle bone scripts. The existing technology will destroy the font structure when denoising oracle bone script images, and most of the models of the existing technology are very complex, which will consume a lot of time and computing resources when denoising. Summary of the invention

[0003] The purpose of the present invention is to provide an oracle bone inscription denoising method, device and medium based on channel attention and glyph aggregation.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] An oracle bone inscription denoising method based on channel attention and glyph aggregation comprises the following steps:

[0006] Obtaining an oracle bone image containing noise;

[0007] An oracle bone script image is input into a denoising model, and a denoised oracle bone script image and a character skeleton are outputted, wherein the denoising model comprises an input mapper, an output mapper, an ECharFormer module and a font corrector, the input image is converted into shallow features by the input mapper, and the shallow features are converted into reconstruction features and font features by the ECharFormer module, wherein the reconstruction features are converted into a denoised oracle bone script image by the output mapper and outputted, and the font features are converted into a character skeleton by the font corrector and outputted.

[0008] The ECharFormer module is composed of N+1 encoding ECharFormer blocks and N decoding EcharFormer blocks cascaded, which are numbered 1, 2,,,,N,N+1,,,2N+1 respectively, and the encoding ECharFormer block numbered i is residually linked with the decoding EcharFormer block numbered 2N+2-i, and the encoding ECharFormer block numbered N+1 is not residually linked with any decoding EcharFormer block.

[0009] The ECharFormer block consists of a channel self-attention block, a font structure block, and a selective kernel feature fusion module.

[0010] The input of the ECharFormer block is converted into corresponding reconstructed intermediate features and font intermediate features through the channel character attention block and the font structure block respectively;

[0011] The reconstructed intermediate features and the font intermediate features are converted into feature fusion attention maps through the selective kernel feature fusion module;

[0012] The reconstructed intermediate features and the font intermediate features are respectively added to the feature fusion attention map to output the reconstructed features and font features of the current level as the input of the next level ECharFormer block; among them, the input of the first level encoding ECharFormer block is the shallow feature, and the output of the last level decoding EcharFormer block is the reconstructed features and font features finally output by the ECharFormer module.

[0013] The channel self-attention block integrates the channel attention mechanism and the multi-head self-attention mechanism, processes the input features and outputs the reconstructed intermediate features;

[0014] The font structure block sequentially passes the input features through two convolution blocks and then adds the input features to obtain the font intermediate features, wherein the convolution block is composed of a convolution layer and a batch normalization layer with ReLU as the activation function;

[0015] The selective kernel feature fusion module sequentially performs cascading, reconstruction, element-level addition, pooling and convolution operations on the input reconstructed intermediate features and font intermediate features to obtain a first feature. The first feature is sequentially cascaded, reconstructed and softmax layered after reconstructed convolution and font convolution to obtain a second feature. The second feature is respectively matrix multiplied with the input reconstructed intermediate features and font intermediate features and then added to obtain a feature fusion attention map.

[0016] The input mapper consists of a 3*3 convolutional layer and a LeakyReLU activation function.

[0017] The output mapper consists of 3*3 convolutional layers.

[0018] The font corrector consists of a 3*3 convolutional layer.

[0019] The loss function of the denoising model includes image loss and structural loss, wherein the image loss measures the difference between the denoised oracle bone image output by the model and the true image, and the structural loss measures the difference between the character skeleton output by the model and the clean character skeleton, which is calculated by an existing method based on mathematical morphology.

[0020] An oracle denoising device based on channel attention and glyph aggregation comprises a memory, a processor, and a program stored in the memory, wherein the processor implements the method described above when executing the program.

[0021] A storage medium stores a program, which implements the method described above when executed.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) The present invention utilizes the selective kernel feature fusion technology to integrate the font information into the reconstructed features, so that the font structure of the oracle bone script can be well preserved during denoising.

[0024] (2) The present invention uses channel attention to improve the calculation method of self-attention in Transformer, thereby greatly reducing the complexity of the model and having the advantages of lightweight and fast calculation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the oracle bone inscription denoising process of the present invention;

[0026] Figure 2 The figures are comparison diagrams before and after denoising in an embodiment, wherein (2a) is the oracle bone script image before denoising, (2b) is the oracle bone script image after denoising using the present invention, (2c) is the true value image, and (2d) is the oracle bone script character skeleton extracted using the present invention. DETAILED DESCRIPTION

[0027] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0028] This embodiment provides an oracle bone inscription denoising method based on channel attention and glyph aggregation, comprising the following steps:

[0029] S1, obtain the oracle bone image containing noise;

[0030] S2, inputs the oracle bone image into the denoising model and outputs the denoised oracle bone image and character skeleton.

[0031] like Figure 1 As shown, the denoising model includes an input mapper, an output mapper, an ECharFormer module and a font corrector. The input image is converted into shallow features through the input mapper, and the shallow features are converted into reconstruction features and font features through the ECharFormer module. The reconstruction features are converted into denoised oracle bone images through the output mapper and output, and the font features are converted into character skeletons through the font corrector and output.

[0032] (1) Input Projector

[0033] The input mapper consists of a 3*3 convolutional layer and a LeakyReLU activation function.

[0034] (2) ECharFormer module

[0035] The ECharFormer module is composed of N+1 encoding ECharFormer blocks and N decoding EcharFormer blocks cascaded, numbered 1, 2,,,,N,N+1,,,2N+1 respectively, and the encoding ECharFormer block numbered i is residually linked to the decoding EcharFormer block numbered 2N+2-i, and the encoding ECharFormer block numbered N+1 is not residually linked to any decoding EcharFormer block.

[0036] The ECharFormer block consists of a channel self-attention block (CSAB), a glyph structure block (GSNB), and a selective kernel feature fusion module (SKFF).

[0037] The input of the ECharFormer block is converted into corresponding reconstructed intermediate features and font intermediate features through the channel character attention block and the font structure block respectively;

[0038] The reconstructed intermediate features and the font intermediate features are converted into feature fusion attention maps through the selective kernel feature fusion module;

[0039] The reconstructed intermediate features and the font intermediate features are respectively added to the feature fusion attention map to output the reconstructed features and font features of the current level as the input of the next level ECharFormer block; among them, the input of the first level encoding ECharFormer block is the shallow feature, and the output of the last level decoding EcharFormer block is the reconstructed features and font features finally output by the ECharFormer module.

[0040] like Figure 1 As shown in the figure, the channel self-attention block integrates the channel attention mechanism and the multi-head self-attention mechanism, processes the input features and outputs the reconstructed intermediate features;

[0041] The font structure block passes the input features through two convolution blocks in sequence and then adds them to the input features to obtain the font intermediate features. The convolution block is composed of a convolution layer and a batch normalization layer with ReLU as the activation function.

[0042] The selective kernel feature fusion module reconstructs the intermediate feature F of the input R and font intermediate feature F G The first feature F is obtained by sequentially performing cascading, reconstruction, element-level addition, pooling, and convolution operations. C , the first feature F C After reconstruction convolution Conv R And font convolution Conv G Then, cascade, reconstruction and softmax layers are performed in sequence to obtain the second feature. The second feature is added after matrix multiplication with the input reconstructed intermediate feature and the font intermediate feature to obtain a feature fusion attention map.

[0043] (3) Output Projector

[0044] The output mapper consists of a 3*3 convolutional layer.

[0045] (4) Glyph Corrector

[0046] The font corrector consists of a 3*3 convolutional layer.

[0047] The loss function of the denoising model includes image loss and structural loss. The image loss measures the difference between the denoised oracle bone image output by the model and the true image, and the structural loss measures the difference between the character skeleton output by the model and the clean character skeleton. The clean character skeleton is calculated by the existing mathematical morphology-based method.

[0048] As shown in Figure (2a), an oracle bone image before denoising is processed by the method of the present invention to obtain a denoised image as shown in Figure (2b) and a character skeleton as shown in Figure (2d). By comparing with Figure (2c), it can be found that the present invention can well preserve the oracle bone character structure and effectively denoise. Equipped with NVIDIA RTX 4090 GPU, the present invention only needs 10ms to denoise a picture with a pixel size of 256*256, indicating that the present invention has a fast calculation speed.

[0049] This embodiment also provides an oracle denoising device based on channel attention and glyph aggregation, comprising a memory, a processor, and a program stored in the memory, wherein the processor implements the method described above when executing the program.

[0050] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0051] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. An oracle bone inscription denoising method based on channel attention and glyph aggregation, characterized in that: The following steps are involved: Obtaining an oracle bone image containing noise; An oracle bone script image is input into a denoising model, and a denoised oracle bone script image and a character skeleton are outputted, wherein the denoising model comprises an input mapper, an output mapper, an ECharFormer module and a font corrector, the input image is converted into shallow features by the input mapper, and the shallow features are converted into reconstruction features and font features by the ECharFormer module, wherein the reconstruction features are converted into a denoised oracle bone script image by the output mapper and outputted, and the font features are converted into a character skeleton by the font corrector and outputted.

2. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 1, characterized in that: The ECharFormer module is composed of N+1 encoding ECharFormer blocks and N decoding EcharFormer blocks cascaded, which are numbered 1, 2,,,,N,N+1,,,2N+1 respectively, and the encoding ECharFormer block numbered i is residually linked with the decoding EcharFormer block numbered 2N+2-i, and the encoding ECharFormer block numbered N+1 is not residually linked with any decoding EcharFormer block.

3. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 2, characterized in that: The ECharFormer block consists of a channel self-attention block, a font structure block, and a selective kernel feature fusion module. The input of the ECharFormer block is converted into corresponding reconstructed intermediate features and font intermediate features through the channel character attention block and the font structure block respectively; The reconstructed intermediate features and the font intermediate features are converted into feature fusion attention maps through the selective kernel feature fusion module; The reconstructed intermediate features and the font intermediate features are respectively added to the feature fusion attention map to output the reconstructed features and font features of the current level as the input of the next level ECharFormer block; among them, the input of the first level encoding ECharFormer block is the shallow feature, and the output of the last level decoding EcharFormer block is the reconstructed features and font features finally output by the ECharFormer module.

4. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 3, characterized in that: The channel self-attention block integrates the channel attention mechanism and the multi-head self-attention mechanism, processes the input features and outputs the reconstructed intermediate features; The font structure block sequentially passes the input features through two convolution blocks and then adds the input features to obtain the font intermediate features, wherein the convolution block is composed of a convolution layer and a batch normalization layer with ReLU as the activation function; The selective kernel feature fusion module sequentially performs cascading, reconstruction, element-level addition, pooling and convolution operations on the input reconstructed intermediate features and font intermediate features to obtain a first feature. The first feature is sequentially cascaded, reconstructed and softmax layered after reconstructed convolution and font convolution to obtain a second feature. The second feature is respectively matrix multiplied with the input reconstructed intermediate features and font intermediate features and then added to obtain a feature fusion attention map.

5. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 1, characterized in that: The input mapper consists of a 3*3 convolutional layer and a LeakyReLU activation function.

6. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 1, characterized in that: The output mapper consists of 3*3 convolutional layers.

7. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 1, characterized in that: The font corrector consists of a 3*3 convolutional layer.

8. The oracle bone inscription denoising method based on channel attention and glyph aggregation according to claim 1, characterized in that: The loss function of the denoising model includes image loss and structural loss, wherein the image loss measures the difference between the denoised oracle bone image output by the model and the true image, and the structural loss measures the difference between the character skeleton output by the model and the clean character skeleton, which is calculated by an existing method based on mathematical morphology.

9. An oracle bone inscription denoising device based on channel attention and glyph aggregation, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 8 is implemented.

Images