Oracle bone word decoding method and system based on comparative learning and visual ideology description sequence

Abstract

The invention belongs to the field of pattern recognition and artificial intelligence, and discloses an oracle-bone word decoding method and system based on comparative learning and a visual ideology description sequence, and the method comprises the steps: carrying out the feature extraction of an input to-be-decoded oracle-bone word image part through a trained oracle-bone word part recognizer, visual embedding features of the oracle bone character image are obtained; a visual encoder is used for extracting overall visual features of an input to-be-decoded oracle-bone character image and a modern Chinese character set, and oracle-bone character overall visual features and Chinese character overall visual features are obtained; performing representation enhancement on the oracle-bone character image visual embedding features, the oracle-bone character overall visual features, the Chinese character image visual embedding features and the Chinese character overall visual features by using a feature enhancer based on IDS mutual inquiry to obtain oracle-bone character enhancement features and Chinese character enhancement features; and performing comparative calculation on the oracle-bone character enhancement features and the Chinese character enhancement features by using comparative learning to obtain a comparative vector, and obtaining an oracle-bone text decoding result based on the comparative vector.

Description

Technical Field

[0001] The present invention relates to the technical field of pattern recognition and artificial intelligence, and in particular to an oracle bone inscription deciphering method and system based on contrast learning and visual ideographic description sequence. Background Art

[0002] As one of the oldest pictographic scripts in the world, the recognition, composition and deciphering of oracle bone characters have received increasing attention from researchers in recent years. These ancient characters are key to exploring ancient China (especially the Shang Dynasty culture, about 1600-1046 BC), but most of the discovered categories have not yet been deciphered. Deciphering these characters will not only help their preservation, but also contribute to the inheritance of cultural heritage. Existing research on the interpretation of oracle bone characters mainly focuses on the reasoning and reconstruction of the evolutionary stages. Previously, researchers used generative models such as CycleGAN to simulate the evolutionary process of oracle bone characters. At each intermediate stage of the evolution (such as bronze inscriptions, seal scripts, etc.), reconstructed glyphs were generated as input clues for subsequent stages to advance their simulation process.

[0003] However, this type of method has two major limitations: (1) The method flow involves expensive and complex implementation costs. It requires the compilation of real-value data from all intermediate evolutionary stages from oracle bone inscriptions to modern Chinese characters for supervision. Due to the extensive and complex nature of the evolutionary process, many character categories may have key clues that are blurred or lost in the intermediate stages, so this method often fails in this scenario. (2) The information mining on the label side is not deep enough. In this type of decryption method, the use of the label side (i.e., modern Chinese characters) is only introduced in the form of glyph images in the final evolutionary stage. However, for the complex and challenging oracle bone decryption task, other important clues of modern Chinese characters on the label side should also be deeply mined to assist in decryption, such as ideographic description sequences (IDS). IDS includes components and structural symbols, which can provide valuable knowledge and clues for understanding the structural information and composition details of Chinese characters. Therefore, how to overcome the problem of insufficient evolutionary clues in existing methods and how to introduce IDS knowledge to assist model reasoning and complete end-to-end oracle bone decryption has become an urgent problem to be solved today. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an oracle bone inscription deciphering method based on contrastive learning and visual ideographic description sequence, the method comprising:

[0005] Step S1, using the trained oracle bone script component identifier to extract features of the input oracle bone script image components to be deciphered, to obtain visual embedding features of the oracle bone script components;

[0006] Step S2, using a visual encoder to extract overall visual features from the input oracle bone inscription image to be deciphered and the modern Chinese character set, to obtain overall visual features of the oracle bone inscription and overall visual features of the Chinese character;

[0007] Step S3, using a feature enhancer based on IDS mutual inquiry to characterize and enhance the oracle bone character component visual embedding features and the oracle bone character overall visual features, the Chinese character component visual embedding features and the Chinese character overall visual features, to obtain oracle bone character enhanced features and Chinese character enhanced features;

[0008] Step S4: using contrast learning to perform contrast calculation on the oracle bone script enhanced features and the Chinese character enhanced features to obtain a contrast vector, and obtaining an oracle bone script deciphering result based on the contrast vector.

[0009] Optionally, in step S1, the training process of the oracle bone script component identifier specifically includes:

[0010] The visual encoder is used to extract the overall visual features of the oracle bone inscription image to be deciphered;

[0011] Using a visual embedding layer to extract visual embedding features from the component set of the oracle bone inscription image to be deciphered and the visual IDS of the component set;

[0012] Using a contrastive learner based on cosine distance, calculating the similarity of a feature sequence of the overall visual features of the oracle bone inscription image to be deciphered and a feature sequence of the visual embedding features, and obtaining a contrast vector based on the similarity;

[0013] A counting loss function is used to calculate the loss value of the comparison vector, and the oracle bone component identifier is optimized and trained based on the loss value to obtain a trained oracle bone component identifier.

[0014] Optionally, in step S1, the process of extracting features of the input oracle bone script image components to be deciphered using a trained oracle bone script component identifier to obtain visual embedding features of the oracle bone script components specifically includes:

[0015] The visualization embedding layer of the oracle bone inscription component identifier includes a CNN feature extractor, a one-hot encoder, and a fused convolutional layer;

[0016] Inputting the IDS of the oracle bone character image component to be deciphered into a one-hot encoder to obtain the oracle bone character component IDS feature;

[0017] Inputting the IDS of the oracle bone character image component to be deciphered into a CNN feature extractor to obtain the visual IDS features of the oracle bone character component;

[0018] The oracle bone character component IDS feature and the oracle bone character component visual IDS feature are input into the fusion convolution layer for feature fusion to obtain the oracle bone character component visual embedding feature.

[0019] Optionally, in step S3, the process of using a feature enhancer based on IDS mutual inquiry to characterize and enhance the oracle bone character image visual embedding features and the oracle bone character overall visual features, the Chinese character image visual embedding features and the Chinese character overall visual features to obtain the oracle bone character enhanced features and the Chinese character enhanced features specifically includes:

[0020] Performing feature fusion on the oracle bone character image visual embedding feature and the oracle bone character overall visual feature to obtain an oracle bone character fusion feature;

[0021] Fusing the Chinese character image visual embedding feature with the Chinese character overall visual feature to obtain a Chinese character fusion feature;

[0022] Use the tag dictionary to perform one-hot encoding on the TopK candidates of the comparison vector and the IDS of the oracle bone script fusion feature and the IDS of the Chinese character fusion feature to obtain the TopK candidate embedding features and the label item embedding features;

[0023] Preprocessing the tag item embedding features and the Chinese character visual overall features after splicing to obtain tag item fusion features;

[0024] Preprocessing the embedded features of the TopK candidate items and the overall visual features of the oracle bone inscriptions to obtain fused features of the TopK candidate items;

[0025] The tag item fusion feature and the TopK candidate item fusion feature are dually input into the feature enhancer based on IDS mutual inquiry, and Chinese character enhancement features and oracle bone inscription enhancement features are output.

[0026] The present invention also discloses an oracle bone character deciphering system based on contrastive learning and visual ideographic description sequence, the system comprising:

[0027] A component feature extraction module is used to extract features of the input oracle bone character image components to be deciphered using a trained oracle bone character component identifier to obtain visual embedding features of the oracle bone character components;

[0028] The overall feature extraction module is used to extract overall visual features from the input oracle bone characters image to be deciphered and the modern Chinese character set using a visual encoder to obtain overall visual features of the oracle bone characters and overall visual features of the Chinese characters;

[0029] A feature enhancement module, used for using a feature enhancer based on IDS mutual inquiry to characterize and enhance the oracle bone character component visual embedding features and the oracle bone character overall visual features, the Chinese character component visual embedding features and the Chinese character overall visual features, to obtain oracle bone character enhanced features and Chinese character enhanced features;

[0030] The oracle bone deciphering module is used to use contrast learning to perform contrast calculation on the oracle bone inscription enhancement features and the Chinese character enhancement features to obtain a contrast vector, and obtain an oracle bone inscription deciphering result based on the contrast vector.

[0031] Optionally, in the component feature extraction module, the training process of the Oracle component identifier specifically includes:

[0032] The visual encoder is used to extract the overall visual features of the oracle bone inscription image to be deciphered;

[0033] Using a visual embedding layer to extract visual embedding features from the component set of the oracle bone inscription image to be deciphered and the visual IDS of the component set;

[0034] Using a contrastive learner based on cosine distance, calculating the similarity of a feature sequence of the overall visual features of the oracle bone inscription image to be deciphered and a feature sequence of the visual embedding features, and obtaining a contrast vector based on the similarity;

[0035] A counting loss function is used to calculate the loss value of the comparison vector, and the oracle bone component identifier is optimized and trained based on the loss value to obtain a trained oracle bone component identifier.

[0036] Optionally, the workflow of the component feature extraction module specifically includes:

[0037] The visualization embedding layer of the oracle bone inscription component identifier includes a CNN feature extractor, a one-hot encoder, and a fused convolutional layer;

[0038] Inputting the IDS of the oracle bone character image component to be deciphered into a one-hot encoder to obtain the oracle bone character component IDS feature;

[0039] Inputting the IDS of the oracle bone character image component to be deciphered into a CNN feature extractor to obtain the visual IDS features of the oracle bone character component;

[0040] The oracle bone character component IDS feature and the oracle bone character component visual IDS feature are input into the fusion convolution layer for feature fusion to obtain the oracle bone character component visual embedding feature.

[0041] Optionally, the workflow of the feature enhancement module specifically includes:

[0042] Performing feature fusion on the oracle bone character image visual embedding feature and the oracle bone character overall visual feature to obtain an oracle bone character fusion feature;

[0043] Fusing the Chinese character image visual embedding feature with the Chinese character overall visual feature to obtain a Chinese character fusion feature;

[0044] Use the tag dictionary to perform one-hot encoding on the TopK candidates of the comparison vector and the IDS of the oracle bone script fusion feature and the IDS of the Chinese character fusion feature to obtain the TopK candidate embedding features and the label item embedding features;

[0045] Preprocessing the tag item embedding features and the Chinese character visual overall features after splicing to obtain tag item fusion features;

[0046] Preprocessing the embedded features of the TopK candidate items and the overall visual features of the oracle bone inscriptions to obtain fused features of the TopK candidate items;

[0047] The tag item fusion feature and the TopK candidate item fusion feature are dually input into the feature enhancer based on IDS mutual inquiry, and Chinese character enhancement features and oracle bone inscription enhancement features are output.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The present invention predicts the deciphering relationship based on the feature comparison algorithm between the oracle bone inscriptions and the modern Chinese characters, which can effectively avoid the problem of missing clues in the intermediate stage of evolution, so that the entire model can be trained end-to-end and output the final deciphering result; the oracle bone inscription component identifier based on visual IDS and contrastive learning can effectively identify components of unseen categories, which has positive significance in the oracle bone inscription scenario with scarce training data; the feature enhancement based on IDS mutual inquiry can fully expand the evaluation scope of TopK component candidates, effectively improving the IDS characterization capability of oracle bone inscriptions. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 A method step diagram of an oracle bone inscription deciphering method based on contrastive learning and visual ideographic description sequence according to an embodiment of the present invention;

[0052] Figure 2 This is a pre-training flow chart of the oracle bone script component identifier according to an embodiment of the present invention;

[0053] Figure 3 This is a flow chart of a feature enhancer based on IDS mutual inquiry according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0056] First, some technical terms used in the present invention are explained:

[0057] IDS (Ideographic Description Sequence) is a method for describing the structure of Chinese characters, especially when Chinese characters need to be digitized, decomposed, reconstructed or analyzed. IDS is based on part of the Unicode encoding standard and is mainly used to represent the structural information of complex Chinese characters that cannot be directly encoded. It describes the composition and arrangement of Chinese characters through a set of special symbols and basic components. Main features 1. Structured representation: IDS is based on components and uses descriptors (such as left-right structure, top-bottom structure, etc.) to break down Chinese characters into basic components, providing a modular and structured method for representing Chinese characters.

[0058] Descriptors support a variety of structural relationships:

[0059] IDS uses specific descriptors, such as:

[0060] Represents left-right structure

[0061] Indicates the upper and lower structure

[0062] Represents a cross structure

[0063] These descriptors can be flexibly combined to express the diverse structures of Chinese characters. 3. Applicable to non-encoded Chinese characters: IDS is particularly suitable for rare Chinese characters or newly created characters that are not included in the Unicode encoding. Through the combination of descriptors and components, users can create digital representations of these Chinese characters. 4. Support for Chinese character component decomposition and recognition: IDS helps to decompose and analyze Chinese characters at the component level, making it easier for computers to perform tasks such as pattern recognition, Chinese character learning, and glyph retrieval. At the same time, it provides technical support for philological research and historical document compilation.

[0064] Contrastive Learning is an unsupervised or self-supervised learning method whose main purpose is to extract useful feature representations by learning the similarities and differences between samples. This method has been widely used in many fields such as computer vision, natural language processing, and recommendation systems. Basic principles 1. Positive and negative sample pairs: Positive sample pairs: two samples are similar in some sense, such as from the same category or different perspectives of the same instance. Negative sample pairs: two samples are different in some sense, such as from different categories or different instances. 2. Loss function: Contrastive learning optimizes model parameters by designing a specific loss function so that the feature representations of positive sample pairs are closer and the feature representations of negative sample pairs are farther apart. Common loss functions include contrastive loss, triplet loss, etc. Common contrastive learning methods 1. SimCLR: SimCLR is a method based on self-supervised learning. It generates positive sample pairs by applying different data augmentation techniques on the same image, and then optimizes the model through contrastive loss functions. The specific steps include: data augmentation, feature extraction, projection head, and contrastive loss calculation. 2. MoCo (Momentum Contrast): MoCo solves the problem of limited number of negative samples by maintaining a momentum-updated queue to store negative samples. The momentum update mechanism allows the negative samples in the queue to be gradually updated, maintaining the diversity and stability of negative samples. 3. BYOL (Bootstrap Your Own Latent): BYOL does not use negative sample pairs, but learns feature representations through two independent encoders (online encoder and target encoder). The online encoder updates parameters by optimizing the output of the predicted target encoder, and the target encoder is gradually updated through the momentum update mechanism. 4. "HUST-OBS" dataset, the HUST-OBS dataset is used to evaluate and compare the performance of different OCR algorithms, especially in the field of handwritten Chinese character recognition. It is a commonly used dataset for oracle bone deciphering tasks. 5. CLIP (Contrastive Language-Image Pretraining) is a multimodal model proposed by OpenAI that jointly trains text and image embeddings through contrastive learning. It uses large-scale image-text pair data to achieve cross-modal matching and can efficiently recognize image content in zero-shot tasks. CLIP performs well in unsupervised learning and multi-task adaptation, and has the ability to recognize unseen categories.

[0065] Application scenarios 1. Image recognition: In computer vision, contrastive learning can help the model learn more discriminative feature representations and improve the performance of tasks such as image classification and object detection. 2. Natural language processing: In natural language processing, contrastive learning can be used for tasks such as word embedding and sentence representation to help the model better understand the semantics of the text. 3. Recommendation system: In the recommendation system, contrastive learning can be used to learn the feature representations of users and items and improve the accuracy and personalization of recommendations.

[0066] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] Embodiment 1

[0068] A method for deciphering oracle bone characters based on contrastive learning and visual ideographic description sequences, such as Figure 1 As shown, the method includes:

[0069] Step S1: Use the trained oracle bone script component identifier to extract features of the input oracle bone script image components to be deciphered, and obtain visual embedding features of the oracle bone script components.

[0070] like Figure 2 As shown in FIG. 1 , the training process of the oracle bone script component identifier specifically includes: using a visual encoder based on DenseNet-44 to extract visual features from the input oracle bone script image, and the feature is recorded as F rad ; Use the visualization embedding layer VE(·) to embed the component set π of the existing oracle bone inscription image and the visual IDS(Q π ) Extract visual embedding features G vt ;

[0071] G vt =VE(π,Q π )=Conv(Enc or (Q π )||Embed(π))

[0072] Among them, Conv(·), Enc or (·), Embed(·) and (·||·) refer to a 1*1 convolutional layer, a ResNet-18 network, a one-hot encoder and a vector concatenation operation respectively; the implementation of visual IDS here directly uses the skeleton image corresponding to the component;

[0073] The visual feature F of the existing oracle bone inscription image is calculated using a contrastive learner based on cosine distance. rad And the visual embedding feature G vt The cosine distance of , and the comparison vector is obtained based on the cosine distance;

[0074]

[0075] Where M and T represent the preset component set size and F respectively. rad The length after vectorization.

[0076] The loss value of the comparison vector is calculated using a counting loss function, and the oracle bone component identifier is optimized and trained based on the loss value to obtain a trained oracle bone component identifier.

[0077] The process of using a trained oracle bone script component identifier to extract features of an input oracle bone script image component to be deciphered to obtain visual embedding features of the oracle bone script component specifically includes: the visual embedding layer of the oracle bone script component identifier includes a CNN feature extractor, a one-hot encoder and a fused convolutional layer; the IDS of the oracle bone script image component to be deciphered is input into the one-hot encoder to obtain the oracle bone script component IDS features; the IDS of the oracle bone script image component to be deciphered is input into the CNN feature extractor to obtain the oracle bone script component visual IDS features; the oracle bone script component IDS features and the oracle bone script component visual IDS features are input into the fused convolutional layer for feature fusion to obtain the oracle bone script component visual embedding features.

[0078] Step S2, using a visual encoder to extract overall visual features from the input oracle bone inscription image to be deciphered and the modern Chinese character set, to obtain overall visual features of oracle bone inscriptions and overall visual features of Chinese characters. Input the I DS of the modern Chinese character set into a one-hot encoder to obtain the I DS features of modern Chinese characters; input the I DS of the visual features into a CNN feature extractor to obtain visual I DS features; input the I DS features of modern Chinese characters and the visual I DS features into the fusion convolution layer for feature fusion, to obtain overall visual features of oracle bone inscriptions and overall visual features of Chinese characters.

[0079] The DenseNet-44 framework is used to construct two visual encoders with independent parameters to extract visual features of the input oracle bone script image and the overall glyph graph of the modern Chinese character set respectively.

[0080] Step S3, using a feature enhancer based on IDS mutual inquiry to characterize and enhance the oracle bone character component visual embedding features and the oracle bone character overall visual features, the Chinese character component visual embedding features and the Chinese character overall visual features, to obtain oracle bone character enhanced features and Chinese character enhanced features.

[0081] like Figure 3As shown, the visual embedding features of the oracle bone characters and the overall visual features of the oracle bone characters are fused to obtain the oracle bone characters fusion features; the visual embedding features of the Chinese character components and the overall visual features of the Chinese characters are fused to obtain the Chinese character fusion features; the TopK candidates of the comparison vector and the IDS of the token features are one-hot encoded using the tag dictionary to obtain the TopK candidate embedding features and the label item embedding features; the output TopK component candidates (oracle bone characters) and IDS label items (Chinese characters) are one-hot encoded using the tag dictionary, wherein the role of the tag dictionary is to apply the existing deciphering knowledge so that the tags of the determined oracle bone characters and the corresponding modern Chinese character components are consistent, so that the one-hot encodings generated by the two are consistent;

[0082] The tag item embedding features and token features are concatenated and preprocessed to obtain the tag item fusion features; a linear mapping layer is used to map the output TopK component candidate embedding (oracle bone characters) and tag item embedding (Chinese characters), where the linear mapping layer is composed of a fully connected layer;

[0083] The embedding features of the TopK candidate items are concatenated with the token features and preprocessed to obtain the fused features of the TopK candidate items; the embedding features and token features of the tag items are concatenated (denoted as C), reshaped (denoted as R) and dimensionally expanded (denoted as E) to obtain the fused features of the tag items (Chinese characters), denoted as Where N, L N , K and C represent the number of labels, the preset maximum length of Chinese character IDS, the preset number of candidates and the hidden feature length of Transformer respectively; similarly, the embedding features and token features of TopK candidates are concatenated and reshaped to obtain the fusion features of TopK candidates (oracle bone characters), which are recorded as Where M and L M They represent the size of the Oracle component set and the preset maximum length of the Oracle IDS respectively; according to the experimental results, the setting of K is 3 to achieve better results;

[0084] The dual input of the label item fusion feature and the TopK candidate item fusion feature is input into the feature enhancer based on IDS mutual inquiry, and the oracle bone inscription enhanced feature and the Chinese character enhanced feature are output. The Chinese character enhanced feature and the TopK candidate item fusion feature (oracle bone inscription enhanced feature) are used as Query (denoted as Q) and Key (denoted as K) and Value (denoted as V) respectively, and input into two Transformer-based mutual inquiry enhancers (with the same internal design, but the parameters are not shared) for mutual inquiry modeling. The two Transformers are designed with an encoder layer (Encoder Layer) and a decoder layer (DecoderLayer). When the number of layers is set to 2, the experimental effect is better; the label item enhanced feature (Chinese character) obtained after the reshaping operation and self-attention calculation is denoted as The input item enhanced features (oracle bone characters) obtained after the reshaping operation and the linear pooling layer are denoted as

[0085] Step S4: using contrast learning to perform contrast calculation on the oracle bone script enhanced features and the Chinese character enhanced features to obtain a contrast vector, and obtaining an oracle bone script deciphering result based on the contrast vector.

[0086] The contrast model in the contrast learning module based on cosine distance of the present invention comprises the following steps:

[0087] The cosine distance-based comparison model has two steps. The first step is to concatenate the label item enhanced features (Chinese characters, i.e., Chinese character enhanced features) with the output features of the visual encoder of the corresponding Chinese character glyph graph, and at the same time concatenate the input item enhanced features (oracle bone characters, i.e., oracle bone character enhanced features) and the output features of the visual encoder of the oracle bone characters; the second step is to use the cosine distance to calculate the similarity between the input features after concatenation in the first step and the label-end features to obtain the similarity vector; the cross entropy is used to calculate the loss value of the similarity vector of the previous step and its true value, and the model is optimized based on the loss value; in the inference stage, the argmax function is used to calculate the similarity vector, and the calculation result is the final decryption result.

[0088] Embodiment 2

[0089] An oracle bone inscription deciphering system based on contrastive learning and visual ideographic description sequence, the system comprising:

[0090] A component feature extraction module is used to extract features of the input oracle bone character image components to be deciphered using a trained oracle bone character component identifier to obtain visual embedding features of the oracle bone character components;

[0091] The training process of the oracle bone script component identifier specifically includes: using a visual encoder based on DenseNet-44 to extract visual features from the input oracle bone script image, and this feature is recorded as Frad ; Use the visualization embedding layer VE(·) to embed the component set π of the existing oracle bone inscription image and the visual IDS(Q π ) Extract visual embedding features G vt ;

[0092] G vt =VE(π,Q π )=Conv(Enc or (Q π )||Embed(π))

[0093] Among them, Conv(·), Enc or (·), Embed(·) and (·||·) refer to a 1*1 convolutional layer, a ResNet-18 network, a one-hot encoder and a vector concatenation operation respectively; the implementation of visual IDS here directly uses the skeleton image corresponding to the component;

[0094] The visual feature F of the existing oracle bone inscription image is calculated using a contrastive learner based on cosine distance. rad And the visual embedding feature G vt The cosine distance of , and the comparison vector is obtained based on the cosine distance;

[0095]

[0096] Where M and T represent the preset component set size and F respectively. rad The length after vectorization.

[0097] The loss value of the comparison vector is calculated using a counting loss function, and the oracle bone component identifier is optimized and trained based on the loss value to obtain a trained oracle bone component identifier.

[0098] The overall feature extraction module uses a visual encoder to extract overall visual features from the input oracle bone inscriptions to be deciphered and the modern Chinese character set to obtain overall visual features of the oracle bone inscriptions and the Chinese characters. Specifically, the visual embedding layer includes a CNN feature extractor, a one-hot encoder and a fused convolutional layer; the IDS of the modern Chinese character set is input into the one-hot encoder to obtain the IDS features of modern Chinese characters; the IDS of the visual features is input into the CNN feature extractor to obtain the visual IDS features; the modern Chinese character IDS features and the visual IDS features are input into the fused convolutional layer for feature fusion to obtain token features.

[0099] The DenseNet-44 framework is used to construct two visual encoders with independent parameters to extract visual features of the input oracle bone script image and the glyph graph of the modern Chinese character set respectively.

[0100] The feature enhancement module is used to use a feature enhancer based on IDS mutual inquiry to characterize and enhance the oracle bone character component visual embedding features and the oracle bone character overall visual features, the Chinese character component visual embedding features and the Chinese character overall visual features to obtain oracle bone character enhanced features and Chinese character enhanced features.

[0101] The oracle bone character image visual embedding feature and the oracle bone character overall visual feature are feature-fused to obtain the oracle bone character fusion feature; the Chinese character image visual embedding feature and the Chinese character overall visual feature are fused to obtain the Chinese character fusion feature; the TopK candidate items of the comparison vector and the IDS of the oracle bone character fusion feature and the IDS of the Chinese character fusion feature are one-hot encoded using a tag dictionary to obtain the TopK candidate item embedding feature and the label item embedding feature; the output TopK component candidates (oracle bone characters) and IDS label items (Chinese characters) are one-hot encoded using a tag dictionary, wherein the role of the tag dictionary is to apply the existing deciphering knowledge so that the tags of the determined oracle bone characters and the corresponding modern Chinese character components are consistent, so that the one-hot encodings generated by the two are consistent;

[0102] The tag item embedding features and token features are concatenated and preprocessed to obtain the tag item fusion features; a linear mapping layer is used to map the output TopK component candidate embedding (oracle bone characters) and tag item embedding (Chinese characters), where the linear mapping layer is composed of a fully connected layer;

[0103] The embedding features of the TopK candidate items are concatenated with the token features and preprocessed to obtain the fused features of the TopK candidate items; the embedding features and token features of the tag items are concatenated (denoted as C), reshaped (denoted as R) and dimensionally expanded (denoted as E) to obtain the fused features of the tag items (Chinese characters), denoted as Where N, L N , K and C represent the number of labels, the preset maximum length of Chinese character IDS, the preset number of candidates and the hidden feature length of Transformer respectively; similarly, the embedding features and token features of TopK candidates are concatenated and reshaped to obtain the fusion features of TopK candidates (oracle bone characters), which are recorded as Where M and L M They represent the size of the Oracle component set and the preset maximum length of the Oracle IDS respectively; according to the experimental results, the setting of K is 3 to achieve better results;

[0104] Based on the label item fusion feature and the TopK candidate item fusion feature, the enhanced original output feature and the enhanced token feature are obtained. The label item fusion feature and the TopK candidate item fusion feature are used as Query (denoted as Q), Key (denoted as K) and Value (denoted as V) respectively, and input into two Transformer-based mutual query enhancers (with the same internal design, but the parameters are not shared) for mutual query modeling. The two Transformers are each designed with an encoder layer (Encoder Layer) and a decoder layer (Decoder Layer). When the number of layers is set to 2, the experimental effect is better; the label item enhanced feature (Chinese character) obtained after the reshaping operation and self-attention calculation is denoted as The input item enhanced features (oracle bone characters) obtained after the reshaping operation and the linear pooling layer are denoted as

[0105] The oracle bone deciphering module is used to use contrast learning to perform contrast calculation on the oracle bone inscription enhancement features and the Chinese character enhancement features to obtain a contrast vector, and obtain an oracle bone inscription deciphering result based on the contrast vector.

[0106] The contrast model in the contrast learning module based on cosine distance of the present invention comprises the following steps:

[0107] The cosine distance-based comparison model has two steps. The first step is to concatenate the label item enhanced features (Chinese characters) with the output features of the visual encoder of the corresponding Chinese character glyph graph, and at the same time concatenate the input item enhanced features (oracle bone characters) and the output features of the visual encoder of the oracle bone characters; the second step is to use the cosine distance to calculate the similarity between the input features after concatenation in the first step and the label-end features to obtain the similarity vector; the cross entropy is used to calculate the loss value of the similarity vector of the previous step and its true value, and the model is optimized based on the loss value; in the inference stage, the argmax function is used to calculate the similarity vector, and the calculation result is the final decryption result.

[0108] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for deciphering oracle bone characters based on contrastive learning and visual ideographic description sequences, characterized in that: The method comprises: Step S1, using the trained oracle bone script component identifier to extract features of the input oracle bone script image components to be deciphered, to obtain visual embedding features of the oracle bone script components; Step S2, using a visual encoder to extract overall visual features from the input oracle bone inscription image to be deciphered and the modern Chinese character set, to obtain overall visual features of the oracle bone inscription and overall visual features of the Chinese character; Step S3, using a feature enhancer based on IDS mutual inquiry to characterize and enhance the oracle bone character component visual embedding features and the oracle bone character overall visual features, the Chinese character component visual embedding features and the Chinese character overall visual features, to obtain oracle bone character enhanced features and Chinese character enhanced features; Step S4: using contrast learning to perform contrast calculation on the oracle bone script enhanced features and the Chinese character enhanced features to obtain a contrast vector, and obtaining an oracle bone script deciphering result based on the contrast vector.

2. The oracle bone inscription deciphering method based on contrastive learning and visual ideographic description sequence according to claim 1, characterized in that: In step S1, the training process of the oracle bone inscription component identifier specifically includes: The visual encoder is used to extract the overall visual features of the oracle bone inscription image to be deciphered; Using a visual embedding layer to extract visual embedding features from the component set of the oracle bone inscription image to be deciphered and the visual IDS of the component set; Using a contrastive learner based on cosine distance, calculating the similarity of a feature sequence of the overall visual features of the oracle bone inscription image to be deciphered and a feature sequence of the visual embedding features, and obtaining a contrast vector based on the similarity; A counting loss function is used to calculate the loss value of the comparison vector, and the oracle bone component identifier is optimized and trained based on the loss value to obtain a trained oracle bone component identifier.

3. The oracle bone inscription deciphering method based on contrastive learning and visual ideographic description sequence according to claim 2, characterized in that: In step S1, the process of extracting features of the input oracle bone inscription image components to be deciphered using the trained oracle bone inscription component identifier to obtain visual embedding features of the oracle bone inscription components specifically includes: The visualization embedding layer of the oracle bone inscription component identifier includes a CNN feature extractor, a one-hot encoder, and a fused convolutional layer; Inputting the IDS of the oracle bone character image component to be deciphered into a one-hot encoder to obtain the oracle bone character component IDS feature; Inputting the IDS of the oracle bone character image component to be deciphered into a CNN feature extractor to obtain the visual IDS features of the oracle bone character component; The oracle bone character component IDS feature and the oracle bone character component visual IDS feature are input into the fusion convolution layer for feature fusion to obtain the oracle bone character component visual embedding feature.

4. The oracle bone inscription deciphering method based on contrastive learning and visual ideographic description sequence according to claim 3, characterized in that: In step S3, the process of using the feature enhancer based on IDS mutual inquiry to characterize and enhance the oracle bone character image visual embedding features and the oracle bone character overall visual features, the Chinese character image visual embedding features and the Chinese character overall visual features to obtain the oracle bone character enhanced features and the Chinese character enhanced features specifically includes: Performing feature fusion on the oracle bone character image visual embedding feature and the oracle bone character overall visual feature to obtain an oracle bone character fusion feature; Fusing the Chinese character image visual embedding feature with the Chinese character overall visual feature to obtain a Chinese character fusion feature; Use the tag dictionary to perform one-hot encoding on the TopK candidates of the comparison vector and the IDS of the oracle bone script fusion feature and the IDS of the Chinese character fusion feature to obtain the TopK candidate embedding features and the label item embedding features; Preprocessing the tag item embedding features and the Chinese character visual overall features after splicing to obtain tag item fusion features; Preprocessing the embedded features of the TopK candidate items and the overall visual features of the oracle bone inscriptions to obtain fused features of the TopK candidate items; The tag item fusion feature and the TopK candidate item fusion feature are dually input into the feature enhancer based on IDS mutual inquiry, and Chinese character enhancement features and oracle bone inscription enhancement features are output.

5. A system for deciphering oracle bone characters by contrastive learning and visual ideographic description sequence, the system being used to implement the oracle bone character deciphering method according to any one of claims 1 to 4, characterized in that: The system comprises: A component feature extraction module is used to extract features of the input oracle bone character image components to be deciphered using a trained oracle bone character component identifier to obtain visual embedding features of the oracle bone character components; The overall feature extraction module is used to extract overall visual features from the input oracle bone characters image to be deciphered and the modern Chinese character set using a visual encoder to obtain overall visual features of the oracle bone characters and overall visual features of the Chinese characters; A feature enhancement module, used for using a feature enhancer based on IDS mutual inquiry to characterize and enhance the oracle bone character component visual embedding features and the oracle bone character overall visual features, the Chinese character component visual embedding features and the Chinese character overall visual features, to obtain oracle bone character enhanced features and Chinese character enhanced features; The oracle bone deciphering module is used to use contrast learning to perform contrast calculation on the oracle bone inscription enhancement features and the Chinese character enhancement features to obtain a contrast vector, and obtain an oracle bone inscription deciphering result based on the contrast vector.

6. The oracle bone inscription deciphering system of contrastive learning and visual ideographic description sequence according to claim 5, characterized in that: In the component feature extraction module, the training process of the Oracle component identifier specifically includes: The visual encoder is used to extract the overall visual features of the oracle bone inscription image to be deciphered; Using a visual embedding layer to extract visual embedding features from the component set of the oracle bone inscription image to be deciphered and the visual IDS of the component set; Using a contrastive learner based on cosine distance, calculating the similarity of a feature sequence of the overall visual features of the oracle bone inscription image to be deciphered and a feature sequence of the visual embedding features, and obtaining a contrast vector based on the similarity; A counting loss function is used to calculate the loss value of the comparison vector, and the oracle bone component identifier is optimized and trained based on the loss value to obtain a trained oracle bone component identifier.

7. The oracle bone inscription deciphering system of contrastive learning and visual ideographic description sequence according to claim 6, characterized in that: The workflow of the component feature extraction module specifically includes: The visualization embedding layer of the oracle bone inscription component identifier includes a CNN feature extractor, a one-hot encoder, and a fused convolutional layer; Inputting the IDS of the oracle bone character image component to be deciphered into a one-hot encoder to obtain the oracle bone character component IDS feature; Inputting the IDS of the oracle bone character image component to be deciphered into a CNN feature extractor to obtain the visual IDS features of the oracle bone character component; The oracle bone character component IDS feature and the oracle bone character component visual IDS feature are input into the fusion convolution layer for feature fusion to obtain the oracle bone character component visual embedding feature.

8. The oracle bone inscription deciphering system based on contrastive learning and visual ideographic description sequence according to claim 7, characterized in that: The workflow of the feature enhancement module specifically includes: Performing feature fusion on the oracle bone character image visual embedding feature and the oracle bone character overall visual feature to obtain an oracle bone character fusion feature; Fusing the Chinese character image visual embedding feature with the Chinese character overall visual feature to obtain a Chinese character fusion feature; Use the tag dictionary to perform one-hot encoding on the TopK candidates of the comparison vector and the IDS of the oracle bone script fusion feature and the IDS of the Chinese character fusion feature to obtain the TopK candidate embedding features and the label item embedding features; Preprocessing the tag item embedding features and the Chinese character visual overall features after splicing to obtain tag item fusion features; Preprocessing the embedded features of the TopK candidate items and the overall visual features of the oracle bone inscriptions to obtain fused features of the TopK candidate items; The tag item fusion feature and the TopK candidate item fusion feature are dually input into the feature enhancer based on IDS mutual inquiry, and Chinese character enhancement features and oracle bone inscription enhancement features are output.

Images