Ancient book relation identification method and system based on entity pair information enhancement

Abstract

The invention discloses an ancient book relation recognition method and system based on entity pair information enhancement, and belongs to the technical field of natural language processing. The method comprises the following steps: firstly, encoding an ancient book text added with a positioning mark by utilizing a pre-training model GuwenBERT, and extracting characteristics of the ancient book text; meanwhile, entity pair features are extracted from three dimensions of semantics, part of speech and relative positions. In order to effectively utilize the features, a channel weighted fusion algorithm introducing a gating mechanism is designed, and the multi-dimensional features of the entity pairs are dynamically fused and screened. And finally, fusing ancient book text features and enhanced entity pair features through a multi-head cross attention mechanism to realize accurate classification of character relationships. Experimental results show that the F1 value on the twenty-four history corpus reaches 90.28%, the accuracy and robustness of ancient book character relation recognition are remarkably improved, and the problems that the semantic meaning of ancient book texts is obscure and the features are not fully utilized are effectively solved.

Description

Technical Field

[0001] This invention relates to the fields of Natural Language Processing (NLP), deep learning, and digital humanities, and in particular to a method for recognizing relationships between figures in ancient books based on entity pair information enhancement, which can integrate multi-source features and utilize attention mechanisms for deep semantic interaction. Background Technology

[0002] Chinese ancient books are vast and numerous, containing thousands of years of historical wisdom and cultural essence of the Chinese nation. Within these texts, the intricate relationships between historical figures (such as fathers and sons, rulers and subjects, enemies, and friends) play a crucial role in understanding the context of historical events and the destinies of the individuals involved. With the rise of digital humanities, how to utilize artificial intelligence technologies, especially deep learning, to automatically and accurately identify these relationships from massive amounts of ancient texts has become a core element in constructing knowledge graphs of ancient books and realizing their intelligentization.

[0003] However, existing relationship recognition technologies are mainly designed for modern Chinese or English corpora, and face significant challenges when directly applied to the identification of relationships between figures in ancient texts: The differences between ancient and modern languages ​​are vast: the writing style, vocabulary, and semantics of ancient texts (classical Chinese) differ significantly from modern vernacular Chinese. Classical Chinese syntax is flexible and varied, with frequent occurrences of inverted sentences, elliptical sentences (such as subject omission), and allusions, increasing the difficulty of semantic understanding. Data sparsity and difficulty in feature extraction: compared to the massive annotated corpora of modern texts, annotated corpora in the field of ancient texts are extremely scarce. Traditional rule-based or statistical machine learning methods rely on manual feature engineering and have poor generalization ability; while existing general pre-trained models (such as BERT) are mostly trained on modern corpora, making it difficult to accurately capture the deep semantics of classical texts.2

[0004] Entity pairs do not fully utilize features: Most existing models only focus on the overall semantic features of the text sequence, while ignoring the rich information contained within the entity pairs themselves. For example, the part-of-speech of an entity in a sentence (person's name, official title, place name), the relative positional distance between entities, and other micro-features often hint at specific relationship types. Existing methods have failed to effectively integrate these multi-dimensional entity features with the global text features.

[0005] Therefore, there is an urgent need to develop a high-precision method for recognizing relationships between figures in ancient books that can adapt to the characteristics of ancient language and fully explore and integrate multi-dimensional information about entity pairs. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and propose an entity-pair-based method for recognizing relationships between figures in ancient Chinese texts (EPERRM). This method significantly improves the accuracy and robustness of recognizing relationships between figures in ancient Chinese texts by introducing the GuwenBERT pre-trained model specifically for Classical Chinese and designing a multi-dimensional entity-pair feature extraction and gating fusion mechanism.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for recognizing relationships between figures in ancient books based on entity-pair information enhancement, comprising the following steps: 1. Data preprocessing: Obtain the ancient text data to be identified, determine the entity pairs (head entity and tail entity) to be identified, and perform special marking processing on the ancient text. Explicitly guide the model to focus on the entity positions by inserting positioning markers.

[0008] 2. Ancient text encoding: The preprocessed ancient text is input into the pre-trained language model GuwenBERT, and its Transformer encoder is used to obtain ancient text vectors containing contextual semantic information.

[0009] 3. Multidimensional feature extraction: Ancient text feature extraction: Based on the ancient text vectors, deep text semantic features are extracted through a feedforward network.

[0010] Entity pair feature extraction: Semantic features, part-of-speech (POS) features, and relative position features of entity pairs are extracted respectively. Semantic features are derived from pooling of character vectors; POS features are based on the embedding mapping of POS tagging results; and relative position features are based on the embedding mapping of entity spacing.

[0011] 4. Dynamic Feature Fusion: A channel-weighted fusion algorithm with a gating mechanism is adopted. First, the semantic features and part-of-speech features of entity pairs are weighted and gating to remove noise information. Then, the residual connection and relative position features are combined to generate enhanced entity pair features.

[0012] 5. Deep Interaction and Classification: Utilizing a multi-head cross-attention mechanism, the query vector is the ancient text features, and the key vector is the enhanced entity pair features. The interaction attention between the two is calculated to generate global features. Finally, the relationship probability distribution is calculated through a fully connected layer and a softmax function to output the recognition result.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. Optimization for ancient books: using GuwenBERT as the base model is more adapted to the grammatical and semantic habits of ancient Chinese.

[0014] 2. Enhanced Multidimensional Features: This innovative approach integrates explicit features such as part-of-speech and relative position, overcoming the limitations of single semantic features in handling complex classical Chinese sentence structures. For example, part-of-speech features help distinguish between personal names and official titles, while relative position features aid in understanding long-distance dependencies.

[0015] 3. Gated fusion mechanism: The designed gated channel weighting algorithm can dynamically learn the weights of different features, effectively filter out information useful for relationship judgment, suppress irrelevant noise, and improve the robustness of the model.

[0016] 4. High recognition accuracy: Experiments on the standard dataset of the Twenty-Four Histories show that the F1 score of the method of this invention reaches 90.28%, which is significantly better than the existing baseline model. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the overall process architecture of the method provided in this embodiment of the invention. Figure 2 This is a schematic diagram of the channel weighted fusion algorithm with gating mechanism introduced in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] Example 1: System Architecture and Hardware Environment The method of this invention can run on a computer terminal, server, or cloud computing platform. In specific implementation, the experimental environment is configured as follows: the operating system is Windows 11 or a Linux distribution; the programming language is Python 3.9.0; and the deep learning framework is PyTorch 2.4.1. For hardware, an NVIDIA GeForce RTX 4060 Ti GPU is used to accelerate model training and inference.

[0020] Example 2: Specific process for identifying relationships between figures in ancient books like Figure 1 As shown, the EPERRM model proposed in this invention mainly includes four core modules: ancient text encoding module, ancient text feature extraction module, entity pair feature extraction and fusion module, and recognition result output module.

[0021] Step 1: Data Preprocessing and Location Marking An ancient book text is usually a sequence of characters. To enable the model to clearly identify the relationship between which two entities, this embodiment adopts an explicit marking strategy. Suppose the input ancient book sentence S is: Huiye's younger brother, ZhaoYe, was quite learned and held the position of Remonstrance Official. The entity pair to be recognized is: the head entity (HeadEntity, He) ZhaoYe, and the tail entity (TailEntity, Te) Remonstrance Official. The system will insert special positioning markers * before and after the head entity and the tail entity respectively. The processed input text becomes: Huiye's younger brother *ZhaoYe*, was quite learned and held the position of *Remonstrance Official*. This marking method can guide the attention mechanism of Transformer to focus more on the entity boundaries and their contexts.

[0022] Step 2: Ancient Book Text Encoding (GuwenBERT) Input the marked text sequence into the GuwenBERT model. GuwenBERT is a model based on the RoBERTa architecture and obtained by continuing pre-training (ContinueTraining) on a large-scale ancient Chinese corpus (such as the Daizhige corpus). Compared with the general BERT, it has a deeper understanding of the meanings of ancient Chinese characters and syntax (such as the usage of zhi, hu, zhe, ye). Input: Character sequence . Output: The sequence of ancient book text vectors processed by multiple layers of Transformer encoders , where is the d dimensional vector representation of the i-th character.

[0023] Step 3: Ancient Book Text Feature Extraction To further integrate the global semantics of the sentence, input the vector sequence output by GuwenBERT (or take the [CLS] position vector therein, or the entire vector sequence) into a feed-forward layer (FeedForwardLayer). The calculation formula is: where and are the weight matrix and the bias term respectively, is the hyperbolic tangent activation function. The obtained G is the ancient book text feature, which contains the deep semantic information of the sentence.

[0024] Step 4: Entity Pair Feature Extraction This is one of the key steps of this invention, aiming to characterize the properties of the entity pair from three dimensions.

[0025] 1. Entity Pair Semantic Feature Extraction: From the ancient book text vector T , according to the position index of the entity, extract the vector sequence He corresponding to the head entity and the tail entity Te Corresponding vector sequence . Since the entity lengths are different (such as the two characters of Zhao Ye and the four characters of Imperial Censor), average pooling (AveragePooling) is used to convert them into vectors of a fixed dimension. Formula: , where j { He , Te}. In this way, the semantic features of the head entity and the semantic features of the tail entity are obtained.

[0026] 2. Extraction of entity pair part-of-speech features: Judging the part-of-speech pair relationship in ancient texts is of great value (for example, the official relationship usually connects a person's name and an official position name). Use an ancient Chinese part-of-speech tagging tool (such as Jiayan) to tag the original sentence. For example: Zhao Ye is tagged as ns (person's name), and Imperial Censor is tagged as n (noun / official position). Input these discrete part-of-speech tags into the part-of-speech embedding layer (POSEmbedding) to obtain part-of-speech vectors. In order to align with the semantic feature dimension, pass through a linear projection layer (LinearProjection) again. Formula: . The part-of-speech features of the head entity and the part-of-speech features of the tail entity are obtained.

[0027] 3. Extraction of entity pair relative position features: In ancient texts, entities with a close relationship are often close in distance, but there are also long-distance dependencies. Calculate the difference between the index of the first character of the head entity and the index of the first character of the tail entity : . In order to prevent excessive distance from causing data sparsity, use the clipping function clamp to limit the difference within the range of [-max_dist, max_dist] (for example, 60). Map the clipped value to a positive integer index and input it into the relative position embedding layer (RelativePositionEmbedding). Formula: . The relative position feature R is obtained.

[0028] Step 5: Feature fusion of entity pairs (channel weighting algorithm introducing a gating mechanism) Simple concatenated features cannot distinguish the importance of features. In this embodiment, a complex fusion strategy is designed (as shown in Figure 2 ): 1. Channel weighting and gating: First, concatenate the semantic features L and the part-of-speech features P . Input the concatenated result into a gating network (composed of a fully connected layer and a Sigmoid activation function) to generate a gating value Gate . Should Gate The value ranges from 0 to 1, which is equivalent to a filter.

[0029] 2. Feature selection: The gate value is multiplied element-wise with the initially fused features to suppress irrelevant features and amplify key features. 3. Residual Connection: To avoid gradient vanishing and preserve the original semantic information, the original fused features are added to the gated features. 4. Final concatenation: Combine the enhanced semantic-part-of-speech features obtained in step 4 with the relative position features obtained in step 4. R Then, the parts are assembled. .

[0030] Step 6: Output the recognition results (multi-head cross-attention mechanism) To enable the model to understand the relationship between entity pairs in conjunction with the context of the current sentence, this embodiment introduces a multi-head cross-attention mechanism.

[0031] Query (Query Vector): Ancient text features from step 3 G .

[0032] Key (key vector) & Value (value vector): The final entity pair features from step 5. Through an attention mechanism, the model calculates which parts of the text are most important for explaining the relationship between the entity pairs. The output consists of fused features. F The input is processed by a fully connected layer and normalized using Softmax to obtain the probability of belonging to each relationship category (such as parent and child, official, and those with the same name). The category with the highest probability is selected as the final recognition result.

[0033] Example 3: Experimental Verification and Effect Analysis In order to objectively verify the technical effect of the present invention, the applicant conducted a detailed experiment on the standard entity relation corpus of the Twenty-Four Histories.

[0034] 1. Experimental Data Setup: The corpus contains 4413 annotated instances, covering 12 high-frequency relationships between figures in ancient texts, including those serving as officials, sharing the same name, father and son, brothers, and those going to or leaving. The dataset is divided into training, validation, and test sets in a 7:2:1 ratio.

[0035] 2. Model parameter settings: Maximum sequence length is set to 128; BatchSize is set to 16; Number of training epochs is set to 20; Learning rate (lr) is set to 1e-5; Dropout ratio is set to 0.1. The optimizer used is AdamW.

[0036] 3. Comparative Experimental Results Analysis: The applicant compared the model of this invention (EPERRM) with several mainstream models, including the general-purpose Qwen3-8B / 30B (Zero-shot / Few-shot), DeepSeek-v3.1, MacBERT, and Chinese-RoBERTa, as well as Siku-BERT and GuwenBERT optimized for classical Chinese. The experimental results show: The F1 score of general large models (such as Qwen3-8bZero-shot) is only 55.92%, indicating that large models lacking domain fine-tuning are difficult to understand classical Chinese.

[0037] The baseline model, GuwenBERT, has an F1 score of 85.94%.

[0038] The improved model (GuwenBERT-BiGRU-ATT) incorporating BiGRU and Attention has an F1 score of 86.87%.

[0039] The model (EPERRM) of this invention achieved an F1 score of 90.28%. This result is an improvement of 4.34% and 2.96% compared to GuwenBERT and R-GuwenBERT, respectively. This fully demonstrates the superiority of the feature fusion strategy proposed in this invention in the field of ancient books. 4. Ablation Experiment Analysis: To demonstrate the independent contribution of each innovation point (part-of-speech, position, gating fusion) in this invention, an ablation experiment was conducted: Baseline (GuwenBERT only): F1=87.84%, semantic features: F1 improved to 87.92%.

[0040] Relative position feature: F1 score improved to 89.00%. This indicates that distance information between entities is very effective in determining relationships (such as long-distance dependencies).

[0041] Part-of-speech features: F1 score improved to 89.17%. This indicates that part-of-speech (such as distinguishing between official titles and place names) is crucial for determining relationships such as "serving as an official" and "going to".

[0042] All features (in this invention): F1 reaches 90.28%.

[0043] It is evident that each type of feature extracted and the fusion mechanism in this invention makes a positive contribution to the final result.

[0044] In summary, this invention constructs the EPERRM model, integrating multi-dimensional features (semantics, part-of-speech, and relative position) of ancient texts and entity pairs. It also designs a channel-weighted fusion algorithm with gating and a multi-head cross-attention mechanism. This method effectively solves the problems of difficult semantic understanding and limited feature utilization in ancient texts, significantly improving the performance of character relationship recognition in ancient texts. It possesses high academic value and application prospects.

[0045] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for recognizing relationships between characters in ancient books based on entity-pair information enhancement, characterized in that, Includes the following steps: Step 1: Obtain the ancient text data to be identified, and determine the entity pairs of the figures to be identified, wherein the entity pairs include a head entity and a tail entity; preprocess the ancient text, and add positioning markers before and after the head entity and the tail entity; Step 2: Input the preprocessed ancient text into the pre-trained language model GuwenBERT to obtain the ancient text vector; Step 3: Based on the ancient text vectors, extract the ancient text features and entity pair features respectively; The extraction of entity pair features includes: extracting semantic features of entity pairs, part-of-speech features of entity pairs, and relative position features of entity pairs; Step 4: Using a channel-weighted fusion algorithm with a gating mechanism, the semantic features and part-of-speech features of the entity pairs are fused together, and combined with the relative position features of the entity pairs, the final entity pair features are generated. Step 5: Utilize the multi-head cross-attention mechanism to interactively fuse the ancient text features with the final entity pair features to obtain global features; Step 6: Input the global features into the fully connected layer and the Softmax classifier, calculate the probability distribution of the relationship categories, and output the results of the ancient book character relationship recognition.

2. The method according to claim 1, characterized in that, In step 3, the extraction of part-of-speech features of the entities includes: obtaining part-of-speech tags for the head and tail entities using an ancient Chinese part-of-speech tagging tool; mapping the part-of-speech tags into part-of-speech vectors through a part-of-speech embedding layer; and mapping the part-of-speech vectors to the same dimension as the semantic features through linear projection to obtain the part-of-speech features of the head entity. Suffix entity part-of-speech features .

3. The method according to claim 1, characterized in that, In step 4, the channel-weighted fusion algorithm with gating mechanism specifically includes: weighting the semantic features and part-of-speech features of entity pairs respectively; introducing a gating unit to concatenate the semantic features and part-of-speech features and input them into the gating network to generate a gating value. The gating values ​​are used to selectively regulate the features, and the original features are fused with the regulated features through residual connections to obtain optimized entity pair semantic-part-of-speech features.

4. The method according to claim 1, characterized in that, In step 3, the extraction of the relative position features of the entities includes: calculating the difference in the relative position indices of the head entity and the tail entity in the text; and using a clipping function to limit the difference to a preset range. Internally, the clipped indices are mapped to relative position vectors through a relative position embedding layer. .

5. The method according to claim 1, characterized in that, In step 5, the use of the multi-head cross-attention mechanism specifically involves: applying the ancient text features obtained in step S3... G The final entity pair features obtained in step S4 are used as the query vector; the interaction weights of the two are calculated through a multi-head attention mechanism, and the fused feature representation is output.

6. A method for recognizing relationships between figures in ancient books based on entity-pair information enhancement, used to implement the method described in any one of claims 1-5, characterized in that, include: Ancient text encoding module: used to preprocess and mark ancient texts and encode them using the GuwenBERT model; Ancient text feature extraction module: used to extract deep semantic features of text through a feedforward layer; Entity Pair Feature Extraction and Fusion Module: Used to extract semantic, part-of-speech, and relative position features of entity pairs, and to perform multi-dimensional feature fusion using a gated channel weighting algorithm; Recognition result output module: used to fuse text and entity pair features through a multi-head cross-attention mechanism and output relationship classification results.

Images