A method for constructing cultural knowledge graph based on W2ner model

Abstract

The present invention discloses a method for constructing a cultural knowledge graph based on a W2ner model, which belongs to the field of culture and artificial intelligence technology, comprising S1: data collection; S2: complex entity recognition of the collected data based on the W2ner model; S3: open domain relationship extraction based on a large language model; S4: using a K-means clustering algorithm to effectively merge relational phrases and perform relationship normalization; S5: storing the normalized knowledge triples obtained after identification, extraction and alignment in a database, and performing visual output; the present invention integrates specialized technical modules for complex entity recognition, open domain relationship extraction and relationship normalization, and combines them with graph database storage to form a knowledge graph construction technical solution that is more adaptable to cultural text characteristics and has a higher degree of automation.

Description

Technical Field

[0001] The present invention belongs to the field of culture and artificial intelligence technology, and specifically relates to a method for constructing a cultural knowledge graph based on the W2ner model. Background Art

[0002] Cultural data (such as historical documents, local chronicles, art reviews, and folklore records) contains a wealth of information. However, most cultural data is unstructured. While rich in information, its disorganized format and lack of a clear organizational structure make it extremely difficult to retrieve, understand, analyze, and apply in depth. Knowledge graphs are a data representation technology based on semantic networks that can transform unstructured text into structured knowledge. Their basic structure consists of entities, relations, and attributes. Using triples as the fundamental representation unit, they organize real-world objects, concepts, and their connections into a networked knowledge system. By explicitly expressing the relationships between entities, knowledge graphs can effectively improve the efficiency of cultural data retrieval and the depth of information comprehension, thereby supporting the digital preservation, research, and dissemination of cultural heritage.

[0003] In recent years, pre-trained models represented by Large Language Models (LLMs) have achieved major breakthroughs in the field of artificial intelligence. Their powerful contextual understanding, semantic reasoning, and text generation capabilities have given them great potential for automatically extracting semantic knowledge and identifying entities and relationships from cultural data across a wide range of fields. This provides a promising new technical paradigm for overcoming the current bottleneck in knowledge graph construction and achieving more efficient and large-scale automated construction.

[0004] Currently, knowledge graphs are divided into domain knowledge graphs and general knowledge graphs. General knowledge graphs are domain-wide knowledge graphs, aiming to encompass a wide range of commonsense knowledge. Due to their broad coverage, general knowledge graphs emphasize the breadth of entities. However, general knowledge graphs lack domain depth and fine-grained semantic representation capabilities, and specialized data is sparse and of low quality. Two core steps in the automated process of constructing cultural knowledge graphs are named entity recognition (NER) and relation extraction (RE). Named entity recognition technology is used to automatically identify specific entities such as names of people, places, and institutions from text. This technology has evolved from early rule-based and dictionary-based approaches to statistical machine learning methods (such as HMMs, SVMs, and CRFs), and in recent years, deep learning-based models have become the mainstream. NER tasks are typically treated as sequence labeling problems. The principle is to assign a predefined category label (such as B-PER, I-PER, or O) to each basic unit (e.g., character) in the input text sequence. The label sequence is then parsed to determine the entity boundaries and types. However, this sequence labeling approach has inherent flaws when dealing with complex cultural domain text. Cultural texts often contain nested entities (one entity contains another entity) and non-contiguous entities (text segments of the same entity are not adjacent). Because sequence tagging assigns a unique label to each character, it cannot effectively represent a character belonging to multiple nested entities, or associate non-adjacent character segments as the same entity. This leads to poor recognition performance in these common scenarios.

[0005] Relationship extraction technology is used to identify and classify semantic relationships between pairs of identified entities. Its technology has evolved from pattern matching to machine learning (including supervised learning, which requires large amounts of labeled data, and unsupervised clustering, which does not rely on annotations), and finally to deep learning models. However, applying these technologies to the cultural field also faces challenges. The relationships between cultural entities are often open-ended, semantically complex, and closely tied to deep cultural and historical contexts. This makes traditional methods that rely on predefined relationship categories and large amounts of labeled data difficult to apply. Due to the flexibility of natural language expression, the same semantic relationship may be expressed through different words or phrases (synonymous relationships). If these semantically equivalent but differently expressed relationships are not effectively merged, the resulting knowledge graph will contain a large amount of redundant information, reducing the quality and usability of knowledge.

[0006] Furthermore, existing technologies attempting to automatically construct knowledge graphs from unstructured cultural text have found that mainstream named entity recognition techniques, particularly those based on sequence labeling, struggle to effectively handle the nested and discontinuous entity structures common in cultural texts, resulting in inaccurate and incomplete entity recognition. Traditional methods relying on predefined relationship patterns are unable to adapt to the open nature of cultural relationships, their complex semantics, and their heavy reliance on background knowledge. Left unaddressed, these methods severely impact the quality and consistency of the resulting knowledge graph. Summary of the Invention

[0007] In response to the problems mentioned in the background technology, the present invention proposes a method for constructing a cultural knowledge graph based on the W2ner model. Its core purpose is to overcome the limitations of existing technologies in processing specific regional cultural knowledge, and to achieve systematic integration, refined management and convenient application of the cultural heritage knowledge system by constructing a structured, networked and semantic domain knowledge graph.

[0008] Technical solution: In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0009] A method for constructing a cultural knowledge graph based on the W2ner model includes the following steps:

[0010] S1: Data collection;

[0011] S2: Perform complex entity recognition on the collected data based on the W2ner model;

[0012] S21: Define any two characters x in the text i and x j the categories of relationships between them;

[0013] S22: Use the pre-trained BERT model and bidirectional LSTM network to process the input sentence and generate a character vector representation containing context information;

[0014] S23: Using conditional layer normalization, the character vector representation is converted into a two-dimensional character pair relationship grid. Then, a multi-layer perceptron and multi-granularity dilated convolution are used to capture the interactive features of character pairs at different distances on this grid to form the final character pair feature representation Q.

[0015] First, construct the initial character pair grid and perform conditional layer normalization;

[0016] Then perform MLP mapping, for each grid point Dimensionality reduction and nonlinear transformation are performed through multi-layer perceptron, specifically:

[0017] ;

[0018] ;

[0019] ;

[0020] Among them, F i,j Represents the dimensionality reduction feature of the multilayer perceptron output of the position pair (i, j), d f Represents the output dimension of the multilayer perceptron, W (1) 、b (1) Both represent the parameters of the first layer of the multilayer perceptron; W (2) 、b (2) Both represent the parameters of the second layer of the multilayer perceptron; d m represents the hidden layer dimension; The input vector representing the position pair (i, j); U i,j Represents the first layer output of the multilayer perceptron for position pair (i, j); ReLU represents the activation function; x represents the input of the linear transformation; X represents the output value of the activation function;

[0021] Finally, multi-granularity dilated convolution is performed;

[0022] The specific content of multi-granularity dilated convolution is:

[0023] Assume that the expansion rate of the kth particle size is d k , convolution kernel K （k） The convolution operation is performed on the input dimensionality reduction feature F and the dilated convolution granularity K, specifically:

[0024] ;

[0025] Among them, d k represents the kth expansion rate, k h 、k w Respectively represent the height and width of the convolution kernel, d g Indicates the number of channels of each dilated convolution output, K （k） represents the convolution kernel weight of the kth granularity, u and v represent the two indexes inside the convolution kernel;

[0026] Convolution is performed on multiple granularities d1, d2, and d3 to obtain G1, G2, and G3, specifically:

[0027] ;

[0028] Where K represents the number of dilated convolution granularity, and k represents the kth dilated convolution granularity; represents the result vector of the convolution at position (i, j) under the kth dilation rate; d g Indicates the number of channels of each dilated convolution output;

[0029] Finally, sum to get Q i，j ;

[0030] S24: Based on the character pair feature representation Q, use the multi-layer perceptron and the bi-affine predictor to predict each pair of characters (x i ,x j ) and decode the relationship to output the final entity recognition result;

[0031] First, the multi-layer perceptron is projected to the head and tail representations, and for each position pair (i, j), the existing feature vector Q i，j , using two independent single-layer MLPs to obtain the head and tail representations;

[0032] ;

[0033] ;

[0034] ;

[0035] Among them, W (h) , b (h) Represent the weight matrix and bias vector of the head mapping respectively; W (t) , b (t) Represent the weight matrix and bias vector of tail mapping, u i,j Indicates the head represents the vector; v i,j ReLU represents the activation function; x represents the input of the linear transformation of this layer; X represents the output value of the activation function; d u Indicates that the head and tail represent the dimensions of the vector;

[0036] Then, double affine scoring is performed. For the r-th type of relationship, the affine score is calculated using the parameters W and U. Specifically,

[0037] ;

[0038] Among them, || represents vector splicing, W (r) 、U (r) Denote the dual affine parameters, bias, and b respectively. (r) represents the bias scalar; represents the original score of the position pair (i, j) belonging to the rth class; u i,j Indicates the head represents the vector; v i,j The tail represents a vector; T represents the transpose operator;

[0039] Finally, perform relationship classification and training loss, perform Softmax on all R-class scores at position (i, j), and obtain the probability distribution, specifically:

[0040] ;

[0041] Where exp represents the exponential function e x , represents the probability that the position pair (i, j) is predicted to be relation r, and R represents the number of relation categories; represents the raw score of the position pair (i, j) belonging to the rth class; Indicates that the position pair (i, j) belongs to the The raw score of the class, Indicates traversal of all possible relationship categories;

[0042] Then get the predicted relationship category for each position pair (i, j);

[0043] In the prediction stage, for each (i, j) take:

[0044] ;

[0045] in, Represents the probability that the position pair (i, j) is predicted to be relation r; Represents the predicted relationship category, argmax represents the operator;

[0046] S3: Open domain relation extraction based on large language model;

[0047] S31: Integration and expansion of entity information;

[0048] Including entity information integration, entity type supplementation, hidden information reasoning and error propagation correction;

[0049] S32: Implementation of a three-step prompting strategy based on the integration and expansion of entity information;

[0050] Requirements: Clarify task objectives, output format, and entity representation specifications;

[0051] Input format prompts, unit consistency, output format prompts;

[0052] Domain knowledge tips: Incorporate cultural domain background knowledge and expert experience;

[0053] Case Tip: Use a small number of typical examples to guide the model to learn how to extract high-quality, semantically diverse relationships;

[0054] S33: Provide the entity information identified in S2 as reference input to the large language model for entity recognition correction;

[0055] S4: Use K-means clustering algorithm to effectively merge relational phrases and perform relation normalization;

[0056] The specific content of applying K-means clustering algorithm to the vector representation of all relationship phrases to reduce the relationship types is as follows:

[0057] S41: Randomly select m samples as the initial cluster centers;

[0058] S42: For each sample x i , find the nearest centroid and assign label c;

[0059] ;

[0060] in, represents the centroid of the jth cluster after the tth iteration, ||·|| represents the vector modulus; Represents x i The cluster label assigned in the tth iteration; M represents the number of clusters; represents the centroid of the jth cluster after the t-1th iteration;

[0061] S43: For each cluster, use the mean of all samples in the cluster as the new centroid;

[0062] ;

[0063] ;

[0064] in, Indicates the sample index set contained in cluster j in the tth iteration; x i Represents the i-th sample character; represents the centroid of the jth cluster after the tth iteration;

[0065] S5: The normalized knowledge triples obtained after identification, extraction and alignment are stored in the database and output visually.

[0066] Preferably, in S1, text data is obtained from historical documents, news reports, statistical yearbooks, academic papers, Baidu Encyclopedia, and cultural encyclopedias.

[0067] Preferably, in S21, the relationship categories include adjacent relationship, head-tail relationship and no relationship;

[0068] Adjacent relationship: x i and x j Adjacent and belonging to the same entity instance;

[0069] Head-to-tail relationship: If x i is the last character of entity A, x j is the first character of entity A, then (x i ,x j ) have a head-to-tail relationship.

[0070] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0071] (1) This paper uses the W2ner model to transform the named entity recognition task from the traditional sequence labeling paradigm to the character-character relationship classification paradigm. This method can directly model and effectively identify nested entities and non-continuous entities that are prevalent in cultural texts, overcoming the recognition bottleneck of traditional sequence labeling methods on these complex structures. Experimental results show that this method outperforms the baseline sequence labeling model when processing cultural datasets containing such entities, and can extract entity information more accurately and comprehensively.

[0072] (2) With the help of the powerful semantic understanding ability of the large language model (LLM) and the "three-step" prompting strategy, the present invention can automatically extract diverse and deep semantic relationships from cultural texts without pre-defining relationship patterns. This greatly improves the flexibility and coverage of relationship extraction, and is particularly suitable for cultural fields with complex relationship types that are difficult to pre-examine. At the same time, incorporating the entities identified by W2ner as reference information into the large language model helps guide the LLM and may correct errors in the NER stage, thereby improving the robustness of end-to-end information extraction. The present invention introduces the K-means clustering method to achieve effective merging of relational phrases, significantly reducing the redundancy of the knowledge graph, improving the structural quality of the knowledge graph, and facilitating subsequent efficient query and knowledge management.

[0073] (3) This invention integrates specialized technical modules for complex entity recognition, open domain relationship extraction, and relationship normalization, and combines them with graph database storage to form a knowledge graph construction method that is more adaptable to the characteristics of cultural texts and has a higher degree of automation. This method provides a more effective and precise technical approach for the deep mining and structured utilization of massive, unstructured cultural heritage data resources, effectively supporting the in-depth application and digital protection of cultural data. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a flow chart of the method for constructing a cultural knowledge graph based on the W2ner model of the present invention;

[0075] Figure 2 Schematic diagram of the present invention modeling entity recognition as a relationship between words;

[0076] Figure 3 This is a schematic diagram of the overall architecture of W2ner of the present invention;

[0077] Figure 4 It is a schematic diagram of the three-step prompting strategy of the present invention;

[0078] Figure 5It is a schematic diagram comparing the performance of different models of the present invention;

[0079] Figure 6 It is a schematic diagram showing the difference between the simple entity and the complex entity of W2ner of the present invention. DETAILED DESCRIPTION

[0080] The present invention will be further illustrated below with reference to specific examples. The examples are implemented based on the technical solutions of the present invention. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0081] This embodiment provides a method for constructing a cultural knowledge graph based on the W2ner model. This method addresses the shortcomings of constructing a knowledge graph for cultural text in terms of complex entity recognition, open-domain relationship extraction, and relationship synonymy processing. This method integrates deep learning, a large language model (LLM), and a clustering algorithm. It aims to automatically and efficiently extract entities and relationships from unstructured cultural texts, thereby constructing a high-quality cultural knowledge graph. Specifically, it includes the following steps:

[0082] S1: Data collection;

[0083] like Figure 1 As shown in Figure 2, text data is obtained from historical documents, news reports, statistical yearbooks, academic papers, Baidu Encyclopedia, and cultural encyclopedias.

[0084] S2: Perform complex entity recognition on the collected data based on the W2ner model;

[0085] The cultural ontology-instance classification conceptual structure hierarchy is constructed using the Protégé tool.

[0086] This step aims to solve the problem of identifying nested and non-contiguous entities in cultural texts. The core idea is to abandon the traditional sequence labeling method and remodel the entity recognition task as a relationship classification problem between characters (tokens).

[0087] like Figure 1 As shown, in this embodiment, entities include literature and art, celebrities, geography, scenic spots and historical sites, products, academics and documents, science and education, history, customs, and clans; among them, ancient towns include Dongpu Ancient Town, Doumen Ancient Town, Jindongmen Old Street, Luzhi, Mudu, Zhouzhuang, Wuzhen, Wan'an, Tongli, Shimen Town, Shengze, Shagou Ancient Town, Qinglong Town, Pingwang, Yansi, Chengkan, Dangkou Town, and Daohe Ancient Street.

[0088] S21: Define any two characters x in the text i and x j the categories of relationships between them;

[0089] First, define any two characters x in the text i and x j The categories of relationships that may exist between entities, related to entity boundaries and types.

[0090] like Figure 2 As shown, this embodiment mainly includes the following types of relationships:

[0091] Next-Neighboring-Word (NHW): If x i and x j If two entities are adjacent and belong to the same entity instance, then there is an adjacent relationship between them.

[0092] Head-to-tail relationship ( ): If x i is the last character of entity A (Tail), x j is the first character of entity A (Head), then (x i ,x j ) have a head-to-tail relationship, where Represents the type of entity A (such as PER, LOC, ORG, etc.). This relationship connects the head and tail of entities, even if they are not adjacent (handles non-contiguous entities).

[0093] No relationship (NONE): There is no such correlation between the character pairs.

[0094] Among them, Ye Gui and Zi Tian Shi are both entities. There is an adjacent relationship and a head-tail relationship between the character Ye and the character Gui, an adjacent relationship between the character Ye and the character Tian, ​​an adjacent relationship between the character Tian and the character Shi, and a head-tail relationship between the character Shi and the character Ye.

[0095] S22: The W2ner model first uses the pre-trained model BERT and the bidirectional LSTM network to process the input sentence and generate a character vector representation containing contextual information.

[0096] The BERT encoding process mainly consists of three parts: position encoding (Position embeddings), sentence encoding (Segmentation embeddings) and word embedding encoding (Token embeddings). Position encoding is used to mark the position information of characters in a sentence, sentence encoding is responsible for determining the sentence information of the character, and word embedding encoding is the feature vector of the character. Finally, the three parts of the encoding are summed as the final feature vector, thereby completing the high-dimensional representation of natural semantics.

[0097] A tokenized input sequence of length n will have three different representations, specifically:

[0098] Token embedding, shape (1, n, 768), which is the vector representation of the word.

[0099] Segment embedding, shape (1, n, 768), which is the vector representation of the sentence.

[0100] Position embedding, shape (1, n, 768), which is the vector representation of the position.

[0101] This example uses a BiLSTM (bidirectional long short-term memory) architecture. BiLSTM adds a reverse operation to the unidirectional LSTM. The forward LSTM captures valuable information from previous context and transmits it to the current moment. The backward LSTM captures the impact of subsequent context on the current moment, allowing the model to take context into account. LSTM has three gate structures: the forget gate, the input gate, and the output gate.

[0102] 1) Forget gate;

[0103] For the cell state in the LSTM at the previous moment, some information may be "outdated". In order to prevent excessive memory from affecting the neural network's processing of current input, some components in the previous cell state are selectively forgotten.

[0104] ;

[0105] Among them, f t Represents the output vector of the sigmoid neural layer; represents the activation function sigmoid; W f Represents the weight vector of the current step; h t-1 Indicates the output value of LSTM at the previous moment; x t represents the input of the network at the current moment; b f Represents the bias term of the forget gate.

[0106] 2) Input gate;

[0107] The input gate layer is combined with the sigmoid function to determine which values ​​are used for updating; the tanh layer is used to generate new candidate values ​​and add them to obtain the candidate values; finally, the update of the cell state C is completed; the calculation formula of the input gate is as follows:

[0108]

[0109]

[0110]

[0111] Among them, i t represents the acceptance weight; b t represents the bias term of the admission weight; W f Represents the weight vector of the current step; Indicates candidate status; represents an activation function that outputs a real value of (-1, 1); W c represents the weight vector (Weight); b c represents the bias term; C t Indicates the current cell state (after update); f t represents the output of the forget gate; x t Represents the input of the network at the current moment; represents the activation function sigmoid; h t-1 Indicates the output value of LSTM at the previous moment.

[0112] 3) Output gate;

[0113] The forget gate and input gate are the process of removing unnecessary information, adding new information, and finally updating the cell state to achieve output. The output gate outputs the input of the current step LSTM.

[0114] ;

[0115] ;

[0116] Among them, O t Indicates that the vector after integrating the current input value and the output value of the previous moment is used to extract the information obtained by the sigmoid function; h t Represents the output of the model; Indicates the compression processing of previously learned information, which plays a role in stabilizing the value; b o represents the bias term; W o represents the weight vector; represents the activation function sigmoid; x t Represents the input of the network at the current moment; h t-1 Indicates the output value of LSTM at the previous moment.

[0117] The features of the two moments are concatenated as the output vector.

[0118] The forward LSTML inputs “Jiang”, “Nan”, “Wen” and “Hua” in sequence to obtain four vectors {h L0 , h L1 , h L2 , h L3}. The backward LSTMR sequentially inputs "化", "文", "南", "江" to obtain four vectors {h R0 , h R1 , h R2 , h R3}. Finally, the forward and backward hidden vectors are concatenated to obtain {[h L0 , h R3 , [h L1 , h R2 , [h L2 , h R1 , [h L3 , h R0 s}, that is, {h0, h1, h2, h3}. For the entity recognition task, the sentence representation [h L2 , h R2 adopted in this embodiment contains both forward and backward information.

[0119] S23: Subsequently, the character vector representation is converted into a two-dimensional character pair relationship grid through conditional layer normalization (CLN), which integrates character information, relative position information, and region information. Then, a multi-layer perceptron (MLP) and multi-grained dilated convolution are applied to capture the interaction features of character pairs at different distances on this grid to form the final character pair feature representation Q. Multi-grained dilated convolution is also known as multi-grained dilation convolution. <s

[0120] Figure 3 As shown in

[0121] Step 1: Construct an initial character pair grid and perform conditional layer normalization (CLN);

[0122] Assume the sequence length is n, and each character passes through BiLSTM / BERT to obtain a vector H. The specific calculation formula of H is:

[0123] ;

[0124] For any position (i, j), directly concatenate the following four vectors along the channel dimension, specifically:

[0125] ;

[0126] Among them, H i and H j are character representations, is the relative position encoding, is the region information encoding; || represents vector concatenation, i, j represent the position indices of any pair of characters in the sequence, and n represents the sequence length. d x represents the vector X​i,j The total dimension of d h represents the vector dimension of each character after passing through BERT+BiLSTM; d p Represents the dimension of relative position encoding; d r The dimension of the region encoding.

[0127] Then perform conditional layer normalization on each (i, j), specifically:

[0128] ;

[0129] ;

[0130] Among them, μ represents the 2 X i,j The mean is calculated on the vector dimension, σ represents the standard deviation (including Stable term, is a small constant to prevent division by zero); i, j represent the position index of any pair of characters in the sequence, and n represents the length of the sequence.

[0131] ;

[0132] Among them, CLN represents the conditional layer normalization operation; represents the element-by-element product; μ represents the product of all n 2 X i,j The mean is calculated on the vector dimension, σ represents the standard deviation; the scaling parameter γ and the offset parameter β are dynamically generated according to the actual condition vector C. x For X i,j This operation is performed in parallel on all (i, j) and outputs a normalized grid, specifically:

[0133] ;

[0134] in, represents the character pair feature grid obtained after (conditional) layer normalization; n represents the sequence length; d x Represents the vector dimension at each grid point, that is, the concatenated vector X before normalization i,j The number of channels.

[0135] Step 2: MLP mapping: For each grid point A two-layer multi-layer perceptron (MLP) is used to perform dimensionality reduction and nonlinear transformation, specifically:

[0136] ;

[0137] ;

[0138] ;

[0139] Among them, F i,j represents the dimensionality reduction feature of the MLP output of the position pair (i, j), d f represents the MLP output dimension, W (1) 、b (1) represents the parameters of the first layer of MLP; W (2) 、b (2) Denotes the parameters of the second layer of MLP. m represents the hidden layer dimension, that is, the length of the first layer output vector; U represents the output of the first layer of MLP; The input vector representing the position pair (i, j); U i,j represents the output of the first layer of the MLP for the position pair (i, j); ReLU represents the activation function; x represents the input of the linear transformation of this layer. X represents the output value of the activation function.

[0140] Step 3: Multi-granularity dilated convolution;

[0141] Perform multiple sets of different dilated 2D convolutions on F. Let the dilation rate of the kth granularity be d k , convolution kernel K （k） Perform convolution operation on input F and K, specifically:

[0142] ;

[0143] Among them, K represents the number of dilated convolution granularity, d k represents the kth expansion rate, k h , k w Respectively represent the height and width of the convolution kernel, d g Indicates the number of channels of each dilated convolution output, K （k） Represents the convolution kernel weight of the kth granularity, u and v are two indexes inside the convolution kernel, used to traverse the size of k h ×k w The convolution kernel.

[0144] Convolution is performed on multiple granularities d1, d2, and d3 to obtain G1, G2, and G3, specifically:

[0145] ;

[0146] Where K is the number of dilated convolution granularity, and k is the kth dilated convolution granularity. represents the result vector of the convolution at position (i, j) under the kth dilation rate; d g Indicates the number of channels of each dilated convolution output.

[0147] Finally, sum to get Q i，jThat is the final interactive feature representation of each position pair (i, j) in the model.

[0148] S24: Finally, based on the character pair feature representation Q, MLP and BiaffinePredictor are used to predict each pair of characters (x i ,x j ) between predefined NER relationship categories (such as NHW, , NONE, etc.).

[0149] The predicted relationship matrix output by the model can be converted into the final entity recognition result through decoding (identifying specific relationship paths).

[0150] Step 1: MLP projects to head and tail representations;

[0151] For each position pair (i, j), the existing feature vector Q i，j， Two independent single-layer MLPs (including activations) are used to obtain the "head" and "tail" representations.

[0152] ;

[0153] ;

[0154] ;

[0155] Among them, W (h) , b (h) is the weight matrix and bias vector of the head mapping; W (t) , b (t) is the weight matrix and bias vector of the tail mapping, u i,j Head represents the vector; v i,j tail represents a vector. ReLU represents the activation function; x represents the input of the linear transformation of this layer. X represents the output value of the activation function. u Indicates that head and tail represent the dimensions of the vector.

[0156] Step 2: Biaffine scoring;

[0157] This embodiment predefines three relationship categories. For the rth relationship, the affine score is calculated using parameters W and U. The specific values ​​of W and U are obtained through training.

[0158] ;

[0159] Among them, || represents vector splicing, W (r) 、U (r) is the biaffine parameter, bias, b (r)is the bias scalar. is the raw score of the position pair (i, j) belonging to the rth class. i,j Head represents the vector; v i,j tail represents a vector. T represents the transpose operator.

[0160] Step 3: Relation classification and training loss;

[0161] Perform Softmax on all R-class scores at (i, j) to obtain the probability distribution, specifically:

[0162] ;

[0163] Where exp is the exponential function e x , is the probability that (i, j) is predicted to be relation r, and R is the number of relation categories. is the raw score of the position pair (i, j) belonging to the rth class; Indicates that the position pair (i, j) belongs to the The raw score of the class, Indicates traversal of all possible relationship categories.

[0164] In the prediction stage, for each (i, j) take:

[0165] ;

[0166] in, The probability of (i, j) being predicted as relation r; is the predicted relationship category, argmax is the operator, and The largest r and all (NA indicates irrelevant category labels) character pairs (i, j) are considered to be from character x i to x j There is an entity of type .

[0167] S3: Open domain relation extraction based on large language model;

[0168] This step leverages the powerful understanding capabilities of the Large Language Model (LLM) to address the problem of open and difficult-to-predefine relationship types in the cultural field, and attempts to mitigate the errors caused by the pipeline method.

[0169] S31: Integration and expansion of entity information;

[0170] like Figure 4 As shown, it includes entity information integration, entity type supplementation, hidden information reasoning and error propagation correction.

[0171] S32: Implementation of a three-step prompting strategy based on the integration and expansion of entity information;

[0172] In order to guide LLM to perform open domain relation extraction that meets the requirements, the method of the present invention adopts a "three-step" prompt strategy in prompt design and construction.

[0173] Requirements: Clarify task objectives, output format, and entity representation specifications.

[0174] Input format prompts, unit consistency, output format prompts.

[0175] Domain knowledge tip: Incorporate necessary cultural domain background knowledge and expert experience.

[0176] Case Tip: Through a small number of typical examples (instance extraction results), guide the model to learn how to extract high-quality, semantically diverse relations.

[0177] S33: Provide the entity information identified in S2 as reference input to the large language model for entity recognition correction;

[0178] The entity information identified by the W2ner model is provided as reference input to the LLM. Emphasizing its reference role allows the LLM to correct entity recognition errors based on the context, further improving the accuracy of relation extraction and reducing error propagation.

[0179] S4: Use K-means clustering algorithm to effectively merge relational phrases and perform relation normalization;

[0180] K-means clustering algorithm is applied to the vector representations of all relation phrases to reduce the relation types.

[0181] S41: Randomly select m samples as the initial cluster centers ;

[0182] Where M represents the number of clusters.

[0183] S42: For each sample x i , find the nearest centroid and assign it a label c, specifically:

[0184] ;

[0185] in, represents the centroid of the jth cluster after the tth iteration, ||·|| represents the vector modulus; Represents sample x i The cluster label assigned in the tth iteration; M represents the number of clusters; represents the centroid of the jth cluster after the t-1th iteration.

[0186] S43: For each cluster j, use the mean of all samples in the cluster as the new centroid, specifically:

[0187] ;

[0188] ;

[0189] in, Indicates the set of sample indices contained in cluster j in the tth iteration. i Represents the feature vector of the i-th sample. represents the centroid of the jth cluster after the tth iteration.

[0190] If the centroid or sample label does not change in this iteration (or the change is less than the threshold), stop; otherwise, let Repeat steps 2 and 3.

[0191] In this embodiment, the K-means clustering algorithm is used to achieve effective merging of relational phrases. The result of relational normalization can be seen in Table 1 below.

[0192] Table 1. Some K-means relationship clustering results

[0193]

[0194] Table 1 shows some of the K-means relationship clustering results, which merges the extracted synonymous words; for example, the four words occupation, position, function and position are merged into position.

[0195] S5: Finally, the normalized knowledge triples (entity, relationship, entity) and entity attributes obtained after identification, extraction and alignment are stored in the graph database to complete the construction of the cultural field knowledge graph; the cultural field knowledge graph is obtained and visualized.

[0196] The present invention uses three basic indicators, precision, recall and F1 score, to evaluate the results of entity extraction and relationship extraction.

[0197] Precision measures how many of the results identified as positive examples by the model are actually positive examples.

[0198] Recall measures how many of all true positive examples are successfully identified by the model.

[0199] The F1 score is the harmonic mean of precision and recall, which comprehensively measures the accuracy and completeness of the model.

[0200] The experimental results are as follows Figure 5As shown in the figure, the W2ner-based named entity recognition model proposed in this invention outperforms the sequence tagging model using BERT+CRF or BERT+BiLSTM+CRF encoders in the cultural domain entity recognition task. Compared with the sequence tagging method of multi-label classification of each character of the named entity, W2ner adopts a unique character-character relationship classification method to simultaneously identify continuous, nested, and non-continuous entities in the text. It uses multiple two-dimensional dilated convolutions to capture the semantic relationship between character pairs at different distances. By defining two main relationships, NHW and THW, to identify entity boundaries and entity types, the method can more accurately determine entity boundaries and entity types.

[0201] Experimental results show that the proposed method not only outperforms commonly used sequence labeling models, but can also identify complex entities such as nested and irregular entities. Overall, W2ner solves the problem of identifying various entities in cultural data very well.

[0202] like Figure 6 As shown, the F1 score for both simple entities (label) and complex entities (nested, irregular entities) (Entity) gradually increases with the number of training epochs. This indicates that the model becomes increasingly accurate and complete in recognizing both flat and complex entities. Overall, the proposed method effectively identifies both simple and complex entities in cultural data.

[0203] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for constructing a cultural knowledge graph based on the W2ner model, characterized by: The following steps are involved: S1: Data collection; S2: Perform complex entity recognition on the collected data based on the W2ner model; S21: Define any two characters x in the text i and x j the categories of relationships between them; S22: Use the pre-trained BERT model and bidirectional LSTM network to process the input sentence and generate a character vector representation containing context information; S23: Using conditional layer normalization, the character vector representation is converted into a two-dimensional character pair relationship grid. Then, a multi-layer perceptron and multi-granularity dilated convolution are used to capture the interactive features of character pairs at different distances on this grid to form the final character pair feature representation Q. First, construct the initial character pair grid and perform conditional layer normalization; Then perform MLP mapping, for each grid point Dimensionality reduction and nonlinear transformation are performed through multi-layer perceptron, specifically: ； ； ； Among them, F i,j Represents the dimensionality reduction feature of the multilayer perceptron output of the position pair (i, j), d f Represents the output dimension of the multilayer perceptron, W (1) 、b (1) Both represent the parameters of the first layer of the multilayer perceptron; W (2) 、b (2) Both represent the parameters of the second layer of the multilayer perceptron; d m represents the hidden layer dimension; The input vector representing the position pair (i, j); U i,j Represents the first layer output of the multilayer perceptron for position pair (i, j); ReLU represents the activation function; x represents the input of the linear transformation; X represents the output value of the activation function; Finally, multi-granularity dilated convolution is performed; The specific content of multi-granularity dilated convolution is: Assume that the expansion rate of the kth particle size is d k , convolution kernel K （k） The convolution operation is performed on the input dimensionality reduction feature F and the dilated convolution granularity K, specifically: ； Among them, d k represents the kth expansion rate, k h 、k w Represents the height and width of the convolution kernel, d g Indicates the number of channels of each dilated convolution output, K （k） represents the convolution kernel weight of the kth granularity, u and v represent the two indexes inside the convolution kernel; Convolution is performed on multiple granularities d1, d2, and d3 to obtain G1, G2, and G3, specifically: ； Where K represents the number of dilated convolution granularity, and k represents the kth dilated convolution granularity; represents the result vector of the convolution at position (i, j) under the kth dilation rate; d g Indicates the number of channels of each dilated convolution output; Finally, sum to get Q i，j ; S24: Based on the character pair feature representation Q, use the multi-layer perceptron and the bi-affine predictor to predict each pair of characters (x i ,x j ) and decode the relationship to output the final entity recognition result; First, the multi-layer perceptron is projected to the head and tail representations, and for each position pair (i, j), the existing feature vector Q i，j , using two independent single-layer MLPs to obtain the head and tail representations; ； ； ； Among them, W (h) , b (h) Represent the weight matrix and bias vector of the head mapping respectively; W (t) , b (t) Represent the weight matrix and bias vector of tail mapping, u i,j Indicates the head represents the vector; v i,j ReLU represents the activation function; x represents the input of the linear transformation of this layer; X represents the output value of the activation function; d u Indicates that the head and tail represent the dimensions of the vector; Then, double affine scoring is performed. For the r-th type of relationship, the affine score is calculated using the parameters W and U. Specifically, ； Among them, || represents vector splicing, W (r) 、U (r) Denote the dual affine parameters, bias, and b respectively. (r) represents the bias scalar; represents the original score of the position pair (i, j) belonging to the rth class; u i,j represents the head representation vector; v i,j The tail represents a vector; T represents the transpose operator; Finally, perform relationship classification and training loss, perform Softmax on all R-class scores at position (i, j), and obtain the probability distribution, specifically: ； Where exp represents the exponential function e x , represents the probability that the position pair (i, j) is predicted to be relation r, and R represents the number of relation categories; represents the raw score of the position pair (i, j) belonging to the rth class; Indicates that the position pair (i, j) belongs to the The raw score of the class, Indicates traversal of all possible relationship categories; Then get the predicted relationship category for each position pair (i, j); In the prediction stage, for each (i, j) take: ； in, Represents the probability that the position pair (i, j) is predicted to be relation r; Represents the predicted relationship category, argmax represents the operator; S3: Open domain relation extraction based on large language model; S31: Integration and expansion of entity information; Including entity information integration, entity type supplementation, hidden information reasoning and error propagation correction; S32: Implementation of a three-step prompting strategy based on the integration and expansion of entity information; Requirements: Clarify task objectives, output format, and entity representation specifications; Input format prompts, unit consistency, output format prompts; Domain knowledge tips: Incorporate cultural domain background knowledge and expert experience; Case Tip: Use a small number of typical examples to guide the model to learn how to extract high-quality, semantically diverse relationships; S33: Provide the entity information identified in S2 as reference input to the large language model for entity recognition correction; S4: Use K-means clustering algorithm to effectively merge relational phrases and perform relation normalization; The specific content of applying K-means clustering algorithm to the vector representation of all relationship phrases to reduce the relationship types is as follows: S41: Randomly select m samples as the initial cluster centers; S42: For each sample x i , find the nearest centroid and assign label c; ； in, represents the centroid of the jth cluster after the tth iteration, ||·|| represents the vector modulus; Represents x i The cluster label assigned in the tth iteration; M represents the number of clusters; represents the centroid of the jth cluster after the t-1th iteration; S43: For each cluster, use the mean of all samples in the cluster as the new centroid; ； ； in, Indicates the sample index set contained in cluster j in the tth iteration; x i Represents the i-th sample character; represents the centroid of the jth cluster after the tth iteration; S5: The normalized knowledge triples obtained after identification, extraction and alignment are stored in the database and output visually.

2. The method for constructing a cultural knowledge graph based on the W2ner model according to claim 1 is characterized in that: In S1, text data is obtained from historical documents, news reports, statistical yearbooks, academic papers, Baidu Encyclopedia, and cultural encyclopedias.

3. The method for constructing a cultural knowledge graph based on the W2ner model according to claim 1 is characterized in that: In S21, the relationship categories include adjacent relationships, head-tail relationships, and no relationship; Adjacent relationship: x i and x j Adjacent and belonging to the same entity instance; Head-to-tail relationship: If x i is the last character of entity A, x j is the first character of entity A, then (x i ,x j ) have a head-to-tail relationship.

Images