A method and system for oracle bone script recognition based on a hybrid CNN-Transformer network, an electronic device, and a storage medium

Abstract

This invention discloses an oracle bone script recognition method, system, electronic device, and storage medium based on a hybrid CNN-Transformer network. The method first preprocesses the oracle bone script image and constructs a multi-source feature extraction network including local perception and global cognition branches. Then, a frequency-enhanced Transformer module is introduced into the global cognition branch to adaptively suppress high-frequency background noise through a frequency domain filtering mechanism, and a compact cross-channel interactive attention mechanism is combined to extract fine-grained stroke features of the oracle bone script to compensate for detail loss. Next, a selective branch gating fusion mechanism is introduced to dynamically evaluate the semantic importance of features and adaptively aggregate local perception and global cognition features. Finally, the oracle bone script recognition result is output. This invention can effectively overcome noise interference from complex rubbings and intra-class differences in character shapes, significantly improving the accuracy and robustness of oracle bone script recognition.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, deep learning, pattern recognition, and image processing, and particularly to a method, system, electronic device, and computer-readable storage medium for recognizing oracle bone script based on a hybrid CNN-Transformer network. This method is applicable to the recognition of oracle bone script rubbings or scanned images under conditions of complex background noise interference and significant intra-class differences in character shapes. It can be widely applied in scenarios such as digital cultural heritage protection, automatic interpretation of ancient characters, smart archaeology, and digitization of ancient books and documents. Background Technology

[0002] Oracle bone script is the earliest extant pictographic writing system discovered in China. It is not only key to exploring the origins of Chinese characters but also a cultural cornerstone for decoding ancient civilization. Therefore, automatic oracle bone script recognition technology has immeasurable value for the protection and inheritance of digital cultural heritage. However, oracle bone script recognition faces severe challenges. Due to long-term burial and weathering, oracle bone script has suffered severe degradation, resulting in dense dotted noise, bone cracks, and character fragmentation in rubbings. Furthermore, due to cultural differences across different ancient regions and eras, oracle bone script characters exhibit significant intraclass morphological variations.

[0003] The introduction of deep convolutional neural networks (CNNs) has significantly advanced the field of oracle bone script recognition. However, the inherent local inductive bias of CNNs limits their ability to capture long-range spatial dependencies. Meanwhile, the Visual Transformer (ViT) addresses the shortcomings of CNNs by capturing long-range dependencies and modeling global structural information through a self-attention mechanism, but ViT is extremely sensitive to high-frequency noise.

[0004] Recent studies have proposed network architectures integrating CNN and ViT, but these networks still suffer from insufficient recognition performance when faced with severe high-frequency noise and significant intra-class differences in oracle bone inscription images. While some hybrid models have improved the recognition performance of long-tailed oracle bone inscriptions through multi-expert strategies, they lack explicit high-frequency rubbing noise suppression mechanisms, limiting their robustness in complex scenarios.

[0005] In summary, while existing oracle bone script recognition methods have improved performance to some extent, they still have the following shortcomings: Traditional CNNs struggle to fully extract the global structural features of oracle bone inscriptions, while conventional ViT models are highly susceptible to high-frequency noise interference from oracle bone rubbings. In the process of trying to filter noise, it is often unavoidable to smooth out the fine-grained stroke details necessary to distinguish similar variant characters; Existing multi-source feature fusion strategies are mostly static fusions, which cannot adapt to the differences in semantic dependence of different samples and are difficult to adaptively aggregate local details and global context.

[0006] Therefore, there is an urgent need for a robust oracle bone script recognition method that can accurately capture fine-grained local features while suppressing high-frequency noise and achieve efficient cross-branch fusion. Summary of the Invention

[0007] Purpose of the invention The purpose of this invention is to overcome the problems in existing technologies, such as the difficulty of traditional convolutional neural networks in fully extracting the global structural features of oracle bone inscriptions, the susceptibility of conventional visual Transformers to high-frequency noise (such as dot noise and bone cracks) in oracle bone rubbings, the easy loss of fine-grained stroke features due to simple frequency domain denoising, and the inability of existing static feature fusion methods to adapt to the semantic dependency differences of different samples. This invention provides an oracle bone inscription recognition method, system, electronic device, and storage medium based on a hybrid CNN-Transformer network, to improve the accuracy of oracle bone inscription recognition, its robustness against high-frequency interference, and its ability to capture and preserve fine-grained stroke details under complex background noise, severe intra-class variations in character shapes, and long-tailed data distribution. Technical solution

[0008] To achieve the above objectives, the present invention adopts the following technical solution: 1. A method for recognizing oracle bone inscriptions based on a hybrid CNN-Transformer network, comprising the following steps: Step S1: Acquire the image of the oracle bone script to be identified and perform preprocessing. Obtaining images of oracle bone inscriptions The image is then processed by at least one of normalization, resizing, and data augmentation to obtain the network input image. Preferably, the input image is uniformly resized to... The pixels were analyzed, and data augmentation techniques such as random affine mapping and random erasure were employed to enhance sample diversity.

[0009] Step S2: Construct a multi-source feature extraction network and extract features. The preprocessed image is input into a multi-source feature extraction network constructed based on a hybrid CNN-Transformer architecture. This network is mainly divided into a local perception branch and a global cognition branch. The input image features are first downsampled by the Stem module, and then sequentially undergo four stages of feature extraction, with each stage's basic building block being a multi-source fusion block.

[0010] Step S3: Extract local features in the local perception branch and enhance global structure and fine-grained features in the global cognition branch. Within the multi-source fusion block, the local perception branch primarily relies on convolutional operations to extract local texture features. Output features of the global cognitive branch It is composed of the outputs of the frequency-enhanced Transformer module and the compact cross-channel interactive attention mechanism.

[0011] Preferably, the specific processing procedure of the Frequency Enhancement Transformer module (FATB) is as follows: To overcome the sensitivity of traditional Transformers to high-frequency rubbing noise while acquiring global context, the input features are sequentially passed through a multi-depth convolutional head transposed attention layer and a frequency domain feedforward layer.

[0012] In the MDTA layer, using Convolution and Deep convolution generates query features Key features Sum value characteristics : Cross-channel attention map computation reduces the computational complexity of high-resolution feature maps, resulting in aggregated features. : Subsequently, the features are fed into the frequency domain feedforward layer and subjected to two-dimensional fast Fourier transform. Transform it to the spectral domain and introduce a learnable frequency weight tensor. Modulating frequency components to suppress high-frequency noise textures, and finally using inverse fast Fourier transform. Transform back to the spatial domain: Preferably, the specific processing procedure of the Compact Cross-Channel Interactive Attention (CCIA) mechanism is as follows: To compensate for the fine-grained stroke details smoothed out by frequency domain denoising, a spatial dimensionality reduction strategy is adopted. The input features are compressed into a compact representation Z by average pooling to reduce computational redundancy.

[0013] The compact feature Z is divided into two subspaces along the channel dimension. and Depthwise separable convolutions generate projections for each group. , and By leveraging the attention maps of one set to modulate the value features of another set, efficient cross-channel semantic interaction can be achieved. Features after interaction and splicing position Furthermore, the original spatial resolution is restored through upsampling using transposed convolution (Deconv) with the stride, thereby refining the edges of oracle bone script strokes and obtaining local detail enhancement features. : Step S4: Adaptively aggregate cross-branch features using the Selective Branch Gated Fusion (SBGF) mechanism. To adapt to the differences in semantic dependencies among different samples, the local perceptual features are dynamically aggregated. With global cognitive features .

[0014] First, the information from each branch is integrated by adding elements one by one, and then global average pooling is applied. and dimensionality reduction convolution Encode the dependencies between channels to generate a compact context feature descriptor Z: Based on the feature descriptor Z, a dedicated projection head is used to generate soft attention vectors that quantify the importance of each branch, and then normalized using the Softmax function: The original features are recalibrated and weighted using adaptive weights to output the final multi-source fusion features. : Step S5: Perform classification prediction and output recognition results After four levels of multi-source fusion feature extraction, the deep features are fed into the global pooling layer and the fully connected layer to calculate the category probability distribution, and finally output the classification and recognition results of the oracle bone image.

[0015] 2. Preferred Solution: Supervised Training Method During the model training phase, the prediction results output by the multi-source feature extraction network are subject to end-to-end supervision and optimization. Preferably, the AdamW optimizer is used to optimize the network, and a cosine annealing learning rate decay strategy combined with a linear warm-up phase is employed for learning rate updates. Simultaneously, a cross-entropy loss function with label smoothing mechanism is used to quantify the difference between the predicted results and the true class labels, preventing model overfitting and improving recognition accuracy. Furthermore, automatic mixed precision calculation is enabled during training to accelerate the training process and further enhance the model's generalization ability.

[0016] 3. An Oracle Bone Script Recognition System Based on a Hybrid CNN-Transformer Network This invention also provides an oracle bone script recognition system, comprising: an image acquisition module for acquiring and preprocessing oracle bone script images; a feature extraction module for constructing a multi-source feature extraction network, extracting local textures in the local perception branch, suppressing high-frequency noise using FATB in the global cognition branch, and extracting fine-grained stroke details using CCIA; a feature fusion module for adaptively calculating branch weights using SBGF and fusing local and global features; and a recognition prediction module for outputting oracle bone script recognition results based on the fused features.

[0017] The present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory. When the computer program is executed by the processor, it causes the electronic device to perform the above-described oracle bone script recognition method based on a hybrid CNN-Transformer network.

[0018] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described oracle bone script recognition method based on a hybrid CNN-Transformer network. Beneficial effects

[0019] Compared with the prior art, the present invention has at least the following beneficial effects: 1. By introducing a frequency enhancement Transformer module into the global cognition branch, the features are transformed to the frequency domain using two-dimensional fast Fourier transform, and the spectral distribution is modulated by a learnable frequency weight tensor. This can adaptively suppress high-frequency background noise while extracting the global context, thus alleviating the problem that traditional Transformer is extremely sensitive to dense noise points and bone cracks in rubbings. 2. By using a compact cross-channel interactive attention mechanism, cross-group attention maps are generated by reducing spatial dimensionality and dividing channel subspaces. Features are cross-modulated and interacted, which effectively compensates for the loss of fine-grained stroke details during frequency domain denoising and improves the ability to distinguish significant intra-class variations and easily confused similar characters. 3. This invention adopts a selective branch gating fusion strategy to replace the traditional static feature splicing or addition method, which can better dynamically allocate weights according to the semantic dependencies of different input samples, coordinate local perceptual details and global cognitive architecture, and improve the effectiveness of multi-source feature cross-branch fusion; 4. The method described above is highly adaptable to complex scenarios such as high background noise, severe character incompleteness, significant intraclass variation, and long-tail data distribution, and can significantly improve the accuracy, robustness, and generalization ability of automatic oracle bone script recognition. 5. This invention can be deployed on cloud computing platforms, edge computing devices, portable scanning terminals, and smart archaeology systems, and has good practical application prospects in scenarios such as automatic interpretation of ancient characters, protection of digital cultural heritage, and digitization of ancient books and documents. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall network structure of the present invention; Figure 2 This is a schematic diagram of the multi-source feature fusion module structure of the present invention; Figure 3 This is a schematic diagram of the frequency domain enhancement Transformer module structure of the present invention; Figure 4 This is a schematic diagram of the gate-compact cross-channel interactive attention mechanism of the present invention; Figure 5 This is a schematic diagram of the selective branch gating fusion module of the present invention. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but the present invention is not limited to the following embodiments.

[0022] Obtain an Oracle image dataset (such as OBC306 or Oracle-MNIST dataset) and preprocess the original images. Preferably, the input images are uniformly adjusted to 224×224 pixels, and data augmentation is performed using random affine transformation and random erasing to improve sample diversity.

[0023] The preprocessed image is input into a multi-source feature extraction network based on a hybrid CNN-Transformer architecture. Features flow in parallel within the network: local texture features are extracted in the local perception branch; in the global cognition branch, high-frequency rubbing noise is suppressed through the FATB module, and fine-grained stroke enhancement is performed through the CCIA module. Subsequently, the features from the two branches are adaptively weighted and aggregated using a selective branch gating fusion (SBGF) mechanism, and the predicted category is output at the end of the network.

[0024] For model optimization and training supervision, a cross-entropy loss function with label smoothing (smoothing parameter of 0.1) was used to calculate the difference between the predicted results and the true labels. The AdamW optimizer was employed, with a batch size of 128, an initial learning rate of 3 × 10⁻⁴, and weight decay of 5 × 10⁻³. A cosine annealing strategy with 5 epochs of linear warmup was used for learning rate updates. Automatic mixed precision (AMP) was preferably enabled during training to accelerate computation and improve model generalization ability.

[0025] The single-frame image of oracle bone script to be detected is input into the trained network model. It undergoes feature downsampling, multi-scale extraction via multi-source fusion blocks, local texture perception, frequency domain global structure acquisition and denoising, and fine-grained stroke enhancement. After adaptive fusion of features at each level through a gating mechanism, the probability distribution of the target category is finally output by the global pooling layer and fully connected layers. The category with the highest probability is selected as the final oracle bone script recognition result.

[0026] This embodiment is applicable to the interpretation of complex oracle bone script rubbings, including severe dense dot noise, bone crack interference, character incompleteness, and significant variations in character shape within the same period and region.

[0027] The method described in this invention can be deployed in devices such as cloud servers, edge computing terminals, portable ancient book scanning terminals, and smart archaeology digital platforms. The device includes a processor, a memory, an image input interface, and a recognition result output interface. When the processor calls the program in the memory, it executes the aforementioned oracle bone script recognition method.

[0028] The operations in this invention, such as global pooling, fast Fourier transform, depthwise separable convolution, softmax activation, transposed convolution, and feature concatenation, can all be implemented using conventional methods in the field.

[0029] This invention is not limited to the specific number of network layers, channel dimensions, compression ratio, convolution kernel size, loss function form, and training hyperparameters given in the paper. As long as it is based on the overall technical concept of "local perception and global cognition dual-branch extraction + frequency domain filtering and denoising and cross-channel feature enhancement + selective branch gating fusion", it should fall within the protection scope of this invention.

[0030] Without departing from the essential concept of this invention, any equivalent substitutions or modifications made by those skilled in the art to the module structure, connection method, and parameter configuration should fall within the scope of protection of this invention.

Claims

1. A robust method for recognizing oracle bone script, characterized in that, Includes the following steps: S1. Obtain the image of the oracle bone script to be identified; S2. The image features are input into the local perception branch and the global cognition branch respectively for feature extraction; wherein, the local perception branch mainly relies on convolution for feature extraction; S3. In the global cognitive branch, the background noise is suppressed and the global structure is obtained by using the Frequency Enhancement Transformer module (FATB), and the fine-grained stroke detail features of the oracle bone script are extracted by combining the Compact Cross-Channel Interactive Attention (CCIA). S4. Features from the local perception branch and the global cognition branch are adaptively aggregated through the Selective Branch Gated Fusion (SBGF) mechanism; S5. Output the oracle bone script recognition result based on the fused features.

2. The oracle bone script recognition method based on a hybrid CNN-Transformer network according to claim 1, characterized in that: Step S3 includes: In the global cognition branch, the input features are first aggregated into a global context through a multi-depth convolutional head transposed attention layer, then enter a frequency domain feedforward layer, and the features are transformed to the frequency domain using fast Fourier transform. The frequency components are modulated by a learnable weight tensor to suppress high-frequency noise, and then restored to spatial features through inverse fast Fourier transform. At the same time, a compact cross-channel interactive attention mechanism is used to pool and reduce the dimensionality of the input features and divide the subspace into channel dimensions to generate query, key, and value features respectively. Channel interaction is achieved through cross-group attention maps. Finally, the interactive features are concatenated and restored to spatial resolution through transposed convolution to obtain fine-grained stroke enhancement features. Step S4 includes: The features of the local perception branch and the global cognition branch are added element-wise. The dependencies between channels are encoded through global average pooling and dimensionality reduction convolution to generate a compact context descriptor. Based on the descriptor, soft attention vectors of each branch are generated. After normalization by the Softmax function, the features of the local perception branch and the global cognition branch are weighted and aggregated to obtain adaptive cross-branch fusion features.

Images