A Thangka cultural knowledge graph display system and its construction method

Abstract

The present invention discloses a thangka culture knowledge graph display system and a construction method thereof. The system completes the construction of a thangka culture knowledge graph display system based on a web terminal through four parts: the construction of a thangka data set, the recognition of thangka culture named entities, the joint extraction of entities and relationships between entities based on the Bs-Spert model, and the construction of a thangka culture knowledge graph display system. The system realizes the named entity recognition and relationship extraction visualization query function of thangka natural language text, and alleviates the current predicament of portal websites with thangka culture as the theme, such as the lack of resources, scattered data, and shallow knowledge. The system can divide roles according to the characteristics of different audiences and grant different degrees of authority, so that the system can increase and continuously improve functions based on the needs of different groups of people. In addition, the system is simple to operate, with a concise and rich interface, and is suitable for users who are older or not very good at using the Internet.

Description

Technical Field

[0001] The present invention relates to the field of computer image description technology, and in particular to a thangka culture knowledge graph display system and a construction method thereof. Background Art

[0002] Thangka cultural texts refer to the natural language texts left behind through the recording, research, description, and creation of thangka painting content over thousands of years of development. These natural language texts mostly exist in the form of paper books or oral transmission. Their limited form and the limitations of the medium of thangka cultural texts often lead to missing, omitted, and inaccurate information about thangka culture. This phenomenon undoubtedly contradicts my country's philosophy of protecting cultural heritage. Therefore, the protection of thangka cultural texts has become an urgent task.

[0003] With the passage of time, in addition to more traditional methods like the China Federation of Literary and Art Circles' (CFA) publications and archiving programs to preserve thangka culture, digital thangka cultural resource websites and online Baidu Encyclopedia entries are emerging. University research institutions and libraries in ethnic minority areas are primarily responsible for the development of these websites, but these websites suffer from fragmented resources and disparate research efforts, often driven by disparate institutions and regions. Baidu Encyclopedia websites primarily present content in the form of Baidu Encyclopedia entries, which are relatively simple and fail to fully interpret the inherent meanings of thangka texts. Therefore, a solution that can unify these disparate resources and provide a deeper understanding of thangka culture has become a pressing challenge in the preservation of thangka culture.

[0004] The definition of the knowledge graph given is also different according to the difference of application scenarios and in different technical categories. Since this application is based on the knowledge graph construction under Thangka culture, it mainly focuses on the Thangka corpus text in Thangka culture. It is a kind of knowledge graph constructed from the natural language perspective. Therefore, from the perspective of natural language processing (Natural Language Processing, NLP), the construction process of the knowledge graph can be regarded as extracting semantics from Thangka text and the structured information in Thangka text, that is, extracting the entity in Thangka text and the dependency relationship between entities. In short, the knowledge graph can be regarded as a tool for expressing the knowledge of the real world in the form of a graph, and each node in the graph represents an entity, and the edge in the graph represents the relationship between two entities in the knowledge graph. Summary of the Invention

[0005] In order to solve the above technical problems, the purpose of the present invention is to provide a thangka culture knowledge graph display system and a construction method thereof.

[0006] The present invention protects a method for constructing a Thangka culture knowledge graph display system, which specifically includes the following steps:

[0007] S1, Thangka knowledge graph dataset construction: consists of two parts: collecting and organizing the Thangka cultural dataset and annotating the Thangka cultural dataset using the Brat tool;

[0008] S1-1, Dataset Acquisition: First, we conducted a targeted analysis of Buddhist scripture websites with strong relevance to Thangka culture, wrote a corresponding web crawler, and used it to crawl the dataset. Then, we manually acquired and corrected the data. We manually searched and extracted terms that were strongly related to Thangka culture, identified them using OCR technology, and further corrected the crawled dataset.

[0009] S1-2, Data Arrangement and Cleaning: Arrange and clean the above-obtained dataset for data noise, missing data, duplication, and outliers. Fill missing data with the global constant Unknown; directly delete data outliers or assign them to the global variable Unknown; directly delete data outliers or assign them to the global variable Unknown; remove data duplication and redundancy; refer to the term data recognized by OCR technology during data arrangement and cleaning;

[0010] S1-3, Dataset Annotation: The Brat annotation tool was used to annotate the dataset. First, a file with the same name and suffix "ann" was generated for the extracted thangka text data. Then, the initial entities and entity relationships, as well as the location information of the text, were configured in the annotation.conf file. BIOES was selected as the entity annotation method. Based on the obtained entities and entity relationships, the dataset was divided into a training set:test set ratio of 8:2. An Ubuntu system environment was installed using VMware virtual machine software in a Windows environment, and the Brat annotation tool was then deployed.

[0011] S2, named entity recognition in Thangka culture: We use the Bi-Lstm+CRF model for named entity recognition in Thangka culture. We introduce a conditional random field (CRF) based on the Bi-Lstm model, using the Bi-Lstm as a feature extractor. The final output of the model is used as the input of the CRF model, which is then used to obtain state transition rules between annotation sequences. Finally, we use the stochastic gradient descent algorithm for network learning.

[0012] S3, joint extraction of entities and relationships between entities based on the Bs-Spert model:

[0013] S3-1, Bs-Spert model construction: This model mainly consists of a Bert pre-training model module, a beam search module, a span classification module, a span filtering module, and a relationship classification module. The Bert pre-training model is used as the basis for the Bs-Spert model to carry out joint extraction of thangka text entities and inter-entity relationships.

[0014] S3-2, Bs-Spert model training: The Bs-Spert model is trained based on the thangka text dataset constructed in step 2, and the Bert pre-training model uses Bert-Base-Chinese;

[0015] S3-3, Thangka text entity and entity joint extraction task experiments: First, test the performance of the Bs-Spert model under the influence of different beam widths; then, select the pooling function by testing the Precision, Recall, and F1-Score values ​​of different pooling functions on the span classification module; finally, based on the results of the first two steps, compare it with the classic information extraction models Bert-CNN and LSTM-RNN to obtain the experimental performance of the Bs-Spert model on the Thangka culture dataset;

[0016] S4, the construction of the Thangka culture knowledge graph display system: First, the storage of the Thangka culture knowledge graph is carried out through two steps: importing Thangka culture entities and the relationship between Thangka culture entities to save Thangka culture knowledge into the Neo4j graph database; then, the construction of the Thangka culture knowledge graph web display system is completed through three steps: system requirements analysis, system design, and system testing.

[0017] Furthermore, in step S3-3, different beam widths are set to 3, 5, 7, 9, and 11; and different pooling functions are Average Pooling, Sum Pooling, and Max Pooling.

[0018] Furthermore, in step S4, the demand analysis includes interface demand analysis and functional demand analysis; the system design includes interface design and functional design; the interface design includes login interface and functional interface; the system test includes login test and functional test, and the login test includes visitor mode, user mode and management mode; functional test includes thangka named entity recognition, thangka entity query, thangka entity relationship query and adding entities or entity relationships.

[0019] The present invention also protects a thangka cultural knowledge graph display system, including a demand analysis layer, an interface login layer, a function layer and a data storage layer;

[0020] The demand analysis layer includes interface demand analysis and functional demand analysis. The interface demand analysis is used to analyze the interface requirements that meet the needs of different user operations and whether the font color and size are reasonably matched with the main color of the interface; the functional demand analysis is used to analyze the functional requirements of three roles: thangka culture enthusiasts, thangka culture researchers, and platform administrators.

[0021] The interface login layer is used for the login interface of different users. Based on the data analyzed by the demand analysis layer, it distinguishes the goals of different users and pops up the access login interface adapted to them;

[0022] The functional layer is used for operation functions of different users, and the operation functions include thangka named entity recognition, thangka entity query, thangka relationship query, adding thangka entities or thangka entity relationships, modifying and correcting thangka entities or thangka entity relationships, and deleting thangka entities or thangka entity relationships;

[0023] The data storage layer is used to store thangka cultural entities and the relationships between thangka cultural entities.

[0024] Furthermore, the interface login layer includes a login interface, a new user registration interface and a password modification interface. The login interface is suitable for Thangka culture enthusiasts, Thangka culture researchers and platform administrators. Under normal default circumstances, the administrator option is not checked on the login interface, and the system defaults to ordinary user login. When accessing in visitor mode, you can directly click on the visitor mode in the lower right corner to automatically jump to the platform interface with restricted functions. When the administrator checkbox is checked, it will switch to administrator login mode.

[0025] Furthermore, the users who access this system in visitor mode will be automatically classified as thangka enthusiasts and will be granted only restricted functions. They can only use the three functions of named entity recognition, entity query, and entity relationship query of this system. Users who register and log in normally to this system will be granted additional functions for thangka culture and thangka cultural relationships of this system on the basis of being granted the above three functional permissions; system administrators will be granted all control permissions for the thangka culture knowledge graph display platform.

[0026] Compared with the existing technology, the present invention has the following beneficial effects:

[0027] The proposed method constructs a web-based knowledge graph display system for Thangka culture through four steps: constructing a Thangka dataset, recognizing named entities in Thangka culture, jointly extracting entities and their relationships using the Bs-Spert model, and building a Thangka culture knowledge graph display system. This method implements named entity recognition and relationship extraction visualization for Thangka natural language text, alleviating the current predicament of Thangka culture-themed portals, which suffer from limited resources, fragmented data, and shallow knowledge. Specifically, the Bi-Lstm+CRF model achieves superior Thangka entity recognition performance compared to the CRF and Bi-Lstm models on the Thangka culture dataset, while balancing time overhead. The Bs-Spert model exhibits excellent performance in jointly extracting entities and entity relationships on the Thangka culture dataset. The system can also assign roles based on the characteristics of different audiences and grant varying degrees of authority, enabling the system to continuously improve its functionality while meeting the needs of different populations. Furthermore, the system is simple to operate, with a concise and comprehensive interface, making it suitable for older users or those less adept at using the internet. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Flow chart of the method of the present invention;

[0029] Figure 2 Annotate the thangka dataset for the Brat tool;

[0030] Figure 3 This is the Bi-Lstm+CRF model architecture diagram;

[0031] Figure 4 The experimental results of Bi-Lstm+CRF under different Epochs;

[0032] Figure 5 The experimental results of Bi-Lstm+CRF under different Hidden_Size;

[0033] Figure 6 This is the overall architecture diagram of the Bs-Spert model;

[0034] Figure 7 Beam Search

[0035] Figure 8 The experimental results are shown below under different Beam Width values.

[0036] Figure 9 Design an architecture diagram for the web-based Thangka cultural knowledge graph display system;

[0037] Figure 10 Named entity recognition for thangka texts;

[0038] Figure 11 Query for thangka entities;

[0039] Figure 12 For thangka relationship inquiries;

[0040] Figure 13 Added for Thangka cultural entity;

[0041] Figure 14 Modified for Thangka cultural relationship. DETAILED DESCRIPTION

[0042] The following examples clearly and completely describe the technical solutions of the present invention. Obviously, the examples described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the examples of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0043] Example 1

[0044] A method for constructing a thangka cultural knowledge graph display system

[0045] 1. Construction of Thangka Knowledge Graph Dataset

[0046] The work mainly consists of two parts: collecting and organizing the Thangka culture dataset and annotating the Thangka culture dataset using the Brat tool;

[0047] (1) Dataset acquisition

[0048] First, through targeted analysis of Buddhist scripture websites that have a strong correlation with Thangka culture, corresponding web crawlers were written and used to crawl data; for example, when crawling entries in the relevant Thangka culture field websites (Buddhist Encyclopedia), a total of about 95,000 entries were obtained; data (a total of about 95,000 entries were obtained); targeted data crawling work was carried out on special websites such as "China Thangka Network", and 259 entries on Thangka culture were accurately obtained, totaling about 340,000 words; then, data was obtained manually and supplemented and corrected. The entries that were strongly related to Thangka culture were manually checked and extracted, and identified through OCR technology, and then the data set obtained by the above web crawler was supplemented and corrected, totaling more than 150 entries and more than 36,000 words.

[0049] (2) Data organization and cleaning

[0050] The data obtained above is sorted and cleaned for problems such as high noise, missing data, duplicate data, and data anomalies. The specific methods include using the global constant Unknown to fill in missing data; directly deleting data anomalies or assigning them to the global variable Unknown; directly deleting data anomalies or assigning them to the global variable Unknown; and eliminating redundant data duplications. During data sorting and cleaning, more than 150 entries recognized by OCR technology are also referenced.

[0051] (3) Dataset annotation

[0052] The Brat annotation tool is used to implement dataset annotation. First, a file with the same name and suffix ann is generated for the thangka text data to be extracted. Then, the initial entities and entity relationships as well as the location information of the text are configured in the annotation.conf file. The entities collected so far are [fo]_[age]_[name], [pusa], [tianmu], [tinanv], [shangshi], [jingang], [hufashen], [monk], [zunzhe], [book], [direction], [zuozi], [Loc], [jiaopai]_[believers], [shuliang], [shoushi], [faqi], [zhenyan], [yuyi], [faxiang], [qiguan], [nation], [job], [zuoji], etc. 24 categories and 3942 entities, and the entity relationships are [Mentoring], [ToEast], [OtherName], [belong], [as], 1756 items in 26 categories including [previouslife], [previouslifeMother], [previouslifejob], [previouslifeBelong], [home], [relatives], [wife], [son], [verb], [own], [Means], [successful], [othername], [Create], [Birthtime], [Dietime], [BirthLoc], [DieLoc], [father], [mother], and [xieshi], annotated with the Brat annotation dataset (see Appendix for details). Figure 2); and BIOES is selected as the entity annotation method, where the label "B" indicates the beginning of the entity, the "I" label indicates the inside of the entity, the "O" label indicates no entity, the "E" label indicates the end of the entity, and the "S" label indicates a separate entity (see Table 1 for details); based on the above entities and entity relationships, the dataset is divided into a training set:test set ratio of 8:2 (see Table 2 for details).

[0053] Table 1 Named entity recognition data annotation format

[0054]

[0055] Table 2 Classification of Thangka culture dataset

[0056]

[0057] In the Windows environment, VMware virtual machine software is used to install the Ubuntu system environment, and then the Brat annotation tool is deployed (see Table 3 for details).

[0058] Table 3 Experimental environment for annotation of the Thangka dataset

[0059]

[0060] 2. Thangka Culture Named Entity Recognition

[0061] (1) Construction of Bi-Lstm+CRF model

[0062] The final output of Bi-Lstm is to classify the labels through a fully connected neural network, and only the label with the maximum probability of the current word is selected for output, so the output of each label is determined only by the context. From the construction of the conditional random field model, it can be seen that the hidden state between different characters in the thangka text is the state transition rule between the annotation sequences. On the basis of the Bi-Lstm model, the conditional random field is introduced to solve the shortcomings of CRF in facing the large data scale and slow speed, and the Bi-Lstm model cannot make good use of the implicit state transition rules of the sequence, and finally achieve the purpose of stabilizing and improving the accuracy of entity recognition. This application uses Bi-Lstm as a feature extractor, and regards the final output of the model as the input of the conditional random field model. The conditional random field can be used to obtain the state transition rules between the annotation sequences. Bi-Lstm+CRF model architecture diagram (see attached for details) Figure 3 ).

[0063] Assume that the output dimension of Bi-Lstm is L1, which represents the probability value of each word mapped to the label. The output matrix of Bi-Lstm model is P, P i,j Represents word x iThe non-normalized probability mapped to the jth label; combined with the state transfer matrix of the conditional random field above, for the input sequence X={x1,x2,x3,...,x m-1 ,x m} Output variable sequence Y = {y1,y2,y3,...,y m-1 ,y m}Define the scoring function as shown in formula (1):

[0064]

[0065] Then use the Softmax function to define the probability value for each label sequence as shown in formula (2):

[0066]

[0067] in, Represents all label sequences, including those that are impossible to appear. Then, in training, the maximum log-likelihood probability log(P(Y|X)) is used, and the loss function can be defined as log(P(Y|X)). The loss function of the Bi-Lstm+CRF model is shown in formula (3):

[0068]

[0069] Finally, the stochastic gradient descent algorithm is used to learn the network.

[0070] (2) Comparison and analysis of Bi-Lstm+CRF experiments under different treatments

[0071] The experimental results in this paper are the average precision, average recall, and corresponding average F1-Score values ​​for all entities in the dataset. The experiments were conducted on an Ubuntu 20.04.2 LTS system, and the experimental environment configuration is shown in Table 4.

[0072] Table 4 Experimental environment of Thangka named entity recognition task

[0073]

[0074] 1) Comparison and analysis of Bi-Lstm+CRF experiments under different Epochs

[0075] First, in order to determine the value of Epoch, the Bi-Lstm+CRF model can achieve the best experimental results, as shown in the attached Figure 4 The Epoch values ​​shown are the precision, recall and F1-Score values ​​of Bi-Lstm+CRF when Epoch=50, 100, 150, 200, 250 and 300 respectively. Figure 4 It can be seen that the model can achieve better experimental results when Epoch = 100. Therefore, the Epoch variable is set to 100 in the experimental comparisons in the following text.

[0076] 2) Comparison and analysis of Bi-Lstm+CRF experiments under different Hidden_Size

[0077] Under the premise of obtaining the optimal Epoch, different hidden variables Hidden_Size=64, 128, 256, 512 in Bi-Lstm were explored, and the final experimental results are as follows Figure 5 As shown in FIG, when the hidden variable Hidden_Size=64, the model achieves the best experimental results.

[0078] 3) Comparison and analysis of Bi-Lstm+CRF experiments with different Bi-Lstm layers

[0079] In the experiment, this paper found that changing the number of Bi-Lstm layers has a greater impact on the final results of the experiment. Therefore, the number of Bi-Lstm layers was experimented. In this experiment, the number of layers was first selected from one to ten layers, and then ten layers and twenty layers were extracted for comparison. The experimental results are shown in Table 5 below. The model experimental effect first gradually increases and then gradually decreases, and reaches the optimal level at the fourth layer.

[0080] Table 5 Experimental results of Bi-Lstm+CRF under different Bi-Lstm layers

[0081]

[0082] 4) Experimental comparison and analysis under different models

[0083] Based on the above experiments, we selected a Bi-Lstm+CRF iteration count of 100 and a Bi-Lstm model hidden layer feature dimension of 64. We then compared the performance of the CRF, Bi-Lstm, and Bi-Lstm+CRF models on the Thangka dataset using a four-layer Bi-Lstm model. The detailed parameter settings for the Bi-Lstm+CRF model are shown in Table 6.

[0084] Table 6 Parameter settings in the Bi-Lstm+CRF model

[0085]

[0086] The experimental results, shown in Table 7, show that the CRF, Bi-Lstm, and Bi-Lstm+CRF models performed poorly on the Thangka Culture dataset. The Bi-Lstm model performed the worst, with the CRF model improving by 0.78% in Precision, 2.46% in Recall, and 1.64% in F1-Score. The traditional statistical CRF model outperformed the deep learning-based Bi-Lstm model on the Thangka dataset for three main reasons. First, the CRF model is a statistical probabilistic graphical model. By scanning the input text with a feature template, it can more comprehensively consider the linear weighted combination of local features of the input sequence. The CRF, on the other hand, computes the joint probability over the sequence, optimizing the entire sequence rather than simply concatenating the optimal solution at each moment. Second, the Bi-Lstm model's output for each time slice fails to consider the output of the previous time slice, failing to effectively model the annotated rules in the sequence. Using the Bi-Lstm model alone for named entity recognition can potentially lose contextual semantic dependencies. Third, the Thangka culture dataset is relatively small, and CRF excels at processing small datasets. Furthermore, as shown in Table 8, in the time cost comparison experiment among the three models, the CRF model requires a greater time cost than the Bi-Lstm model. The CRF model effectively trades time resources for improved performance.

[0087] Because the Bi-Lstm model is based on a recurrent neural network, NVIDIA offers a variety of data training acceleration technologies for neural networks, such as GPU acceleration and multi-stage asynchronous training. Compared to the CRF model, it can complete feature extraction from observation sequences more quickly. The CRF model can better leverage state transition rules within the sequence's hidden states and better learn contextual semantic dependencies. The CRF and Bi-Lstm models complement each other, ultimately achieving performance improvements of 0.7% in Precision, 1.01% in Recall, and 0.86% in F1-Score over the CRF model, as shown in Table 4.4. Furthermore, when comparing the time costs of different models, the Bi-Lstm+CRF model significantly reduces the time cost of the CRF model.

[0088] Table 7 Experimental results of different models

[0089]

[0090] Table 8 Time cost of different models

[0091]

[0092] 3. Joint extraction of entities and relationships between entities based on the Bs-Spert model

[0093] (1) Bs-Spert Model Construction

[0094] The main components of this model are composed of the Bert pre-trained model module, beam search module, span classification module, span filtering module, and relation classification module; using the Bert pre-trained model as the basis of the Bs-Spert model, joint extraction of entities and relationships between entities in Thangka text is carried out. Input a Thangka text P {"Shi", "Jia", "Mou", "Ni", "Zhi", "Dao", "A", "Nan", "Xiu", "Xing"} into the Bs-Spert model, and then after data preprocessing, a set of byte-encoded pairs P = {p1, p2, p3,..., p m-1 , p m} with a length of m is obtained. Then, after post-processing by the Bert pre-trained model, a sequence of embedding vectors Q = {q1, q2, q3,..., q m-1 u , q m , c} with a length of m + 1 can be obtained, where c represents a special classifier in the processing of Thangka text, and its main function is to capture the historical context information in Thangka text; for example, the entity spans in Thangka text P can be expressed as {("Shi", "Jia"), ("Shi", "Jia", "Mou", "Ni"), ("Shi", "Jia", "Mou", "Ni", "Zhi", "Dao"), ("Shi", "Jia", "Mou", "Ni", "Zhi", "Dao", "A", "Nan")} etc. as part of the span sequences of text P; The overall architecture diagram of the Bs-Spert model (see attached Figure 6 );

[0095] 1) Span Classification Module

[0096] Any possible entity candidate spans can be input into the span classifier (the spanclass lfler module in the attached figure Figure 6 ). Here, first assume that the input span sequence is, and the span length is k + 1. Then refer to the predefined entity set of Thangka text entity categories in the third chapter as, where represents all existing recognizable Thangka text entity categories, and O represents non-existent or unrecognizable entity spans.

[0097] The input of the span classifier module consists of three parts:

[0098] A. Span Embedding: The span embeddings obtained through the Bert pre-trained model are combined using the fusion function f as, f(q i , q i+1 ,..., q i+k-1 , q i+k), the influence of different fusion functions on the final experimental results was deeply studied, and it was finally found that the experimental accuracy was the highest under the max-pooling function.

[0099] B. Span length embedding: Embed the span length and splice it with f(q i , q i+1 ,..., q i+k-1 , q i+k ) to obtain a span representation with a width of k + 1.

[0100] q(s) = f(q i , q i+1 ,..., q i+k-1 , q i+k ) · w k+1 (4)

[0101] C. Sentence vector c: Obtained from the pre-trained model Bert, this vector represents the historical information of the context of the sentence text and functions similar to the keyword in a sentence. For example: in the text {"now", "Buddha", "is", "Sakyamuni", "Buddha"}, the keyword "Buddha" in the entity category is an important label for "fo". At the same time, the sentence vector c can effectively eliminate the ambiguity problem of entities by using the historical information of the context. Finally, after splicing the sentence vector c, the input of the span classification module is as shown in formula (5).

[0102] x s = q(s) · c (5)

[0103] Finally, input x s into the span classification module, and here the softmax function is mainly used as the classifier function. As shown in formula (6), the posterior values for each entity category will be generated, and of course, the posterior of O (non-existent or unrecognized entity span) is also included here.

[0104] y s = soft max(W s · x s + b s ) (6)

[0105] Here, W s represents the weight matrix of the input x s , and b s represents the bias term.

[0106] 2) Span classification module

[0107] Formula (5) scores each entity category separately, and then selects the entity category with the highest score as the entity recognition result. Finally, the O category, i.e., non-entity spans, is filtered out, and only the span embeddings belonging to the ψ entity category are concatenated with the span length embedding and passed to the relation classification module.

[0108] 3) Beam search module

[0109] There is only one parameter in Beam Search, Beam Width, which is set to k here. The k best results with the largest current conditional probability are selected as the first word in the candidate output sequence. The k best results with the largest conditional probability among all combinations are selected based on the current word. The entire selection process keeps the k best candidates, and finally the best result is selected from all candidates. Figure 7 The figure shows the search process of the beam search when the search tree width k=2.

[0110] 4) Relationship Classification Module

[0111] The input to the relation classification module consists primarily of the span embedding and span length q(s) processed by the BERT model, as well as the contextual history between two valid entities. From the span classification module, it's easy to derive the fused representations of the candidate entity pair (s1, s2) as q(s1) and q(s2).

[0112] Another input component of the relation classification module is the contextual history between the entity pair (s1, s2). Since we are acquiring contextual history information, we naturally need to refer to the sentence embedding c obtained by the Bert model. This is then combined with the Bert pre-trained model and pooled to obtain the range embedding d(s1, s2) between the two valid entity pairs. This d(s1, s2) is input into the relation classification module as contextual information. If entities overlap, the range between the valid entity pairs will be empty, resulting in an anomaly. Therefore, the range embedding between the valid entity pairs is defined as d(s1, s2) = 0.

[0113] Next, a concatenation operation similar to the span classification module is performed to concatenate the span embeddings of the two valid entity pairs with the range embeddings between the entity pairs. At the same time, the asymmetric relationship between the entity pairs is taken into account. The input representation of the relation classification module is shown in formula (7):

[0114]

[0115] Will and Input into the single-layer classifier in the relation classification module, as shown in the following formula (8):

[0116]

[0117] here and Respectively and The weight matrix, and Then they represent the corresponding bias terms. σ represents the Sigmoid function with a dimension of R. Any high response phenomenon on the Sigmoid layer indicates that there is a corresponding relationship between the valid entity pair (s1, s2). This paper sets a response threshold α, which means that when the response score of any relationship is greater than or equal to α, it can be considered that there is a known relationship r∈R between the two valid entity pairs. On the contrary, if the response score is less than the response threshold α, it can be considered that there is no known relationship between the two valid entity pairs.

[0118] (2) Bs-Spert model training

[0119] The Bs-Spert model is trained based on the thangka text dataset constructed in step 2. The Bert pre-training model uses Bert-Base-Chinese. The parameters of the pre-training model are as follows: the number of Bert model layers of the encoder is 12, and the number of self-attention heads of the Bert model is 12; the word vector dimension Bert_Dimension is 768; the initial weight of the relation classifier in the model uses a random number from the normal distribution (0, 0.02); the model uses the Adam optimizer with a warm-up learning rate and a linear decay learning rate; at the same time, in order to prevent overfitting, the Dropout Rate is set to 0.5 for the entity extraction and relation extraction modules respectively; the relation response threshold in the relation classification module is set to = 0.4; the detailed parameter settings of the Bs-Spert model are shown in Table 9.

[0120] Table 9 Parameter settings of Bs-Spert model for joint extraction of entities and entity relationships

[0121]

[0122] (3) Thangka text entity and entity joint extraction task experiment:

[0123] First, the performance of the Bs-Spert model under the influence of different beam widths (Width = {3, 5, 7, 9, 11}) was tested, and the optimal beam width value was selected to determine the further experiments; other experimental parameters remained unchanged. Figure 8As can be seen from the figure, a Beam Width of 11 achieved significant results. Next, we selected a pooling function by testing the Precision, Recall, and F1-Score values ​​of different pooling functions (AveragePooling, Sum Pooling, and Max Pooling) on ​​the span classification module. As shown in Table 10, Max Pooling showed a relatively significant improvement. Finally, based on the results of the first two steps, we compared the BS-Spert model with the classic information extraction models Bert-CNN and LSTM-RNN to determine its experimental performance on the Tangka Culture dataset (see Table 11 for details).

[0124] Table 10 Experimental comparison under different pooling functions

[0125]

[0126] Table 11 Experimental comparison of different models on the Thangka dataset

[0127]

[0128] 4. Construction of Thangka Culture Knowledge Graph Display System

[0129] The storage of Thangka culture knowledge graph consists of two steps: importing Thangka culture entities and the relationship between Thangka culture entities. The Thangka culture knowledge is stored in the Neo4j graph database. The construction of the Thangka culture knowledge graph web display system is completed in three steps: system requirements analysis, system design, and system testing (see the attached Figure 9 ); the demand analysis includes interface demand analysis and functional demand analysis; the system design includes interface design and functional design; the interface design includes login interface and functional interface; the system test includes login test and functional test, and the login test includes visitor mode, user mode and management mode; functional test includes thangka named entity recognition, thangka entity query, thangka entity relationship query and adding entities or entity relationships.

[0130] (1) Storage of Thangka Culture Knowledge Graph

[0131] 1) Storage of Thangka Cultural Entities

[0132] Place the Thangka_Entity.csv file of the Thangka culture entity in the import folder under the "file: / / / neo4j-community-3.5.14 / import" path, and then use the command Load csv withheaders from "file: / / / Thangka_Entity1.csv" as linecreate(:Entity1{Entiy:line.Entity}) to import the Thangka culture entity data into the Neo4j graph database.

[0133] 2) Storage of relationships between thangka cultural entities

[0134] The entity relationship Thangka_Relation.csv file of Thangka culture is placed in the import folder under the "file: / / / neo4j-community-3.5.14 / import" path, and then the command Load CSV withheaders from "file: / / / Thangka_Relation.csv" As lineMatch(entity1:fo{name:line.fo}),(entity2:faqi{name:line.faqi}) Create(entity1)-[r1:line.relation]->(entity2) is used to connect the entities that are related to each other.

[0135] (2) System requirements analysis

[0136] 1) Interface requirements analysis: analyze the interface requirements to meet different user operations and whether the font color and size are reasonably matched with the main color of the interface.

[0137] 2) Functional requirement analysis: analyzing the functional requirements of three roles: Thangka culture enthusiasts, Thangka culture researchers, and platform administrators.

[0138] (3) System design

[0139] 1) Interface design: Design of user login and registration interface and functional interface. Based on the data analyzed by the demand analysis layer, distinguish the goals of different users and design an access platform interface that is suitable for them.

[0140] 2) Functional design: different login functions and operation functions for the Thangka culture knowledge graph are set for different users. Among them, the login function has the function of registering and changing the password; the operation function includes query operation, addition operation, modification operation and deletion operation on the Thangka culture knowledge graph. These four basic operations are implemented for two objects: entities and relationships between entities. Among them, the query operation is used to identify and query all data in the Thangka text entered by the user, and it can also query the relationship between two entities; the addition operation can enter Thangka entities into this platform to add Thangka entities and the relationship between Thangka entities; the modification operation can modify and correct Thangka entities or the relationship between Thangka entities that have been entered into this platform; the deletion operation can delete Thangka entities and the relationship between Thangka entities that have been entered into this platform. Users who access this system in visitor mode will be automatically classified as thangka enthusiasts and granted only restricted functions. They can only use the three functions of named entity recognition, entity query, and entity relationship query of this system. Users who register and log in normally to this system will be granted additional functions for thangka culture and thangka cultural relationships on the basis of the above three functional permissions. System administrators will be granted all control permissions for the thangka culture knowledge graph display platform.

[0141] (4) System testing

[0142] 1) Login interface test

[0143] After users with different roles log in to the system, test the effects displayed based on the different permissions granted by their role characteristics, the registration interface for first-time users, and the user password modification operation. The login interface is suitable for Thangka culture enthusiasts, Thangka culture researchers and platform administrators; by default, the administrator option is not checked on the login interface, and the system defaults to ordinary user login. When a visitor visits, they can directly click on the visitor mode in the lower right corner to automatically jump to the platform interface with restricted functions. When the administrator checkbox is checked, it will switch to administrator login mode. When a user visits the system for the first time, they need to register as a user. Clicking "Register Now" will jump to the user registration interface. When a user needs to change the password, click Change Password on the login interface to jump to the user password modification interface.

[0144] 2) Functional testing

[0145] Five functional modules including thangka named entity recognition, thangka cultural entity query, thangka relationship query, thangka entity addition, and thangka relationship modification were selected for functional testing and demonstration.

[0146] A. Thangka named entity recognition function test: as shown in the attached Figure 10As shown in the figure, the administrator role enters the overall functional overview of the system. The default homepage is the thangka named entity recognition interface. This interface primarily allows users to enter the thangka text for entity recognition in the input text box, such as the text in the figure below {"White Umbrella Goddess, with her two main arms placed in front of her chest, holding a vajra in her left hand and a white umbrella in her right hand, and stepping on the six realms of sentient beings, symbolizing their protection."}. After clicking the confirm button, named entity recognition is performed. The recognition result is {"White Umbrella Goddess / tianmu|main arms / qiguan|chest / qiguan|left hand / qiguan|vajra / dharma instrument|right hand / qiguan|white umbrella / dharma instrument|feet / qiguan|six realms of sentient beings / dharma instrument"}. Outputs such as / tianmu indicate that the entity type of "White Umbrella Goddess" is "tianmu," and "|" is used as a separator.

[0147] B. Thangka cultural entity inquiry: as attached Figure 11 As shown, the user enters the Manjusri Bodhisattva field in the input bar and clicks on the query to display the relationship below. Figure 1 The column displays entity relationships related to Manjushri Bodhisattva. The center point in the diagram represents the Manjushri Bodhisattva entity, and the surrounding circles represent other types of entities. Lines connecting entities represent relationships between them. This feature not only allows you to query whether a Tangka field is a Tangka entity, but also displays its related entities and the relationships between them if it is.

[0148] C. Thangka relationship query: as attached Figure 12 As shown, the user enters two specific thangka entities to query whether an entity relationship exists between them. The user enters "Shakyamuni Buddha" and "Manjushri Bodhisattva" to query the relationship between the two specific thangka entities. Selecting the "Unknown" field in the drop-down bar indicates that the relationship between the two input fields is unknown. The primary goal is to query the relationship between the two fields. As shown in the figure, the "xieshi" relationship indicates that "Manjushri Bodhisattva" is the attendant of "Shakyamuni Buddha." A second approach involves entering entity 1 in the first input field and selecting a relationship in the drop-down bar. This query will query entities that match the first input field and the selected relationship. Since the results of this test are similar to the first approach, it will not be demonstrated for now.

[0149] D. Thangka entity addition: as attached Figure 13As shown, this functional module is mainly open to thangka culture researchers who have logged into this system through normal registration. In the current module, users can enter the thangka entity they want to add through the input bar, and click Add to send a request to add the thangka entity to the backend administrator. After the administrator of this platform confirms the correctness of the added entity, he agrees to the user's request to add the entity, and thus the entity can be added to the thangka culture database. Similarly, the operation process for adding relationships between thangka entities is similar.

[0150] E. Modification of Thangka Relationships: During the operation of this system, it is inevitable that some errors will occur in the definition of Thangka entities or the relationships between Thangka entities. Such errors can be discovered and corrected by the platform administrator. For example, if the relationship between the two pairs of entities "Manjushri Bodhisattva" and "Shakyamuni Buddha" in the above text is changed from "xieshi" to "Manjushri Bodhisattva"-[othername]->"Shakyamuni Buddha", you need to use the following method to modify the relationship between the two pairs of entities. Figure 14 As shown, enter the corresponding thangka entity name in the two input fields and select the entity relationship to be modified. Click the Modify button to modify the entity relationship from "Manjushri Bodhisattva" - [xieshi] -> "Shakyamuni Buddha" to "Manjushri Bodhisattva" - [othername] -> "Shakyamuni Buddha".

[0151] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a Thangka cultural knowledge graph display system, characterized in that: The specific steps include: S1, Thangka knowledge graph dataset construction: consists of two parts: collecting and organizing the Thangka cultural dataset and annotating the Thangka cultural dataset using the Brat tool; S1-1, Dataset Acquisition: First, through targeted analysis of Buddhist scripture websites with strong relevance to Thangka culture, a corresponding web crawler was written and used to crawl the dataset. Then, data was acquired manually and supplemented and corrected. Strongly related terms of Thangka culture were manually searched and extracted, and identified using OCR technology to further supplement and correct the crawled dataset. S1-2, Data Arrangement and Cleaning: Arrange and clean the above-obtained dataset for data noise, missing data, duplication, and outliers. Fill missing data with the global constant Unknown; directly delete data outliers or assign them to the global variable Unknown; directly delete data outliers or assign them to the global variable Unknown; remove data duplication and redundancy; refer to the term data recognized by OCR technology during data arrangement and cleaning; S1-3, Dataset Annotation: The Brat annotation tool was used to annotate the dataset. First, a file with the same name and suffix "ann" was generated for the extracted thangka text data. Then, the initial entities and entity relationships, as well as the location information of the text, were configured in the annotation.conf file. BIOES was selected as the entity annotation method. Based on the obtained entities and entity relationships, the dataset was divided into a training set:test set ratio of 8:

2. An Ubuntu system environment was installed using VMware virtual machine software in a Windows environment, and the Brat annotation tool was then deployed. S2, named entity recognition of Thangka culture: The Bi-Lstm+CRF model is used for Thangka named entity recognition. Based on the Bi-Lstm model, a conditional random field is introduced. The Bi-Lstm is used as a feature extractor. The final output of the model is used as the input of the conditional random field model. The conditional random field is used to obtain the state transition rules between the annotation sequences. Finally, the stochastic gradient descent algorithm is used for network learning. S3, joint extraction of entities and relationships between entities based on the Bs-Spert model: S3-1, Bs-Spert model construction: This model mainly consists of a Bert pre-training model module, a beam search module, a span classification module, a span filtering module, and a relationship classification module. The Bert pre-training model is used as the basis of the Bs-Spert model to carry out joint extraction of thangka text entities and inter-entity relationships. S3-2, Bs-Spert model training: The Bs-Spert model is trained based on the thangka text dataset constructed in step 2, and the Bert pre-training model uses Bert-Base-Chinese; S3-3, Thangka text entity and entity joint extraction task experiments: First, test the performance of the Bs-Spert model under the influence of different beam widths; then, select the pooling function by testing the Precision, Recall, and F1-Score values ​​of different pooling functions on the span classification module; finally, based on the results of the first two steps, compare it with the classic information extraction models Bert-CNN and LSTM-RNN to obtain the experimental performance of the Bs-Spert model on the Thangka culture dataset; S4, the construction of the Thangka culture knowledge graph display system: First, the storage of the Thangka culture knowledge graph is carried out through two steps: importing Thangka culture entities and the relationship between Thangka culture entities to save Thangka culture knowledge into the Neo4j graph database; then, the construction of the Thangka culture knowledge graph web display system is completed through three steps: system requirements analysis, system design, and system testing.

2. The method for constructing a Thangka culture knowledge graph display system according to claim 1, characterized in that: In step S3-3, different beam widths are set to 3, 5, 7, 9, and 11; and different pooling functions are Average Pooling, Sum Pooling, and Max Pooling.

3. The method for constructing a Thangka culture knowledge graph display system according to claim 1, characterized in that: In step S4, the demand analysis includes interface demand analysis and functional demand analysis; the system design includes interface design and functional design; the interface design includes login interface and functional interface; the system test includes login test and functional test, and the login test includes visitor mode, user mode and management mode; functional test includes thangka named entity recognition, thangka entity query, thangka entity relationship query and adding entities or entity relationships.

4. A Thangka cultural knowledge graph display system, characterized by: A system constructed by the method for constructing a Thangka culture knowledge graph display system according to any one of claims 1 to 3, comprising a demand analysis layer, an interface login layer, a function layer, and a data storage layer; The demand analysis layer includes interface demand analysis and functional demand analysis. The interface demand analysis is used to analyze the interface requirements that meet the needs of different user operations and whether the font color and size are reasonably matched with the main color of the interface; the functional demand analysis is used to analyze the functional requirements of three roles: thangka culture enthusiasts, thangka culture researchers, and platform administrators. The interface login layer is used for the login interface of different users. Based on the data analyzed by the demand analysis layer, it distinguishes the goals of different users and pops up the access login interface adapted to them; The functional layer is used for operation functions of different users, and the operation functions include thangka named entity recognition, thangka entity query, thangka relationship query, adding thangka entities or thangka entity relationships, modifying and correcting thangka entities or thangka entity relationships, and deleting thangka entities or thangka entity relationships; The data storage layer is used to store thangka cultural entities and the relationships between thangka cultural entities.

5. A Thangka culture knowledge graph display system according to claim 4, characterized in that: The interface login layer includes a login interface, a new user registration interface and a password modification interface. The login interface is suitable for Thangka culture enthusiasts, Thangka culture researchers and platform administrators. Under normal default circumstances, the administrator option is not checked on the login interface, and the system defaults to ordinary user login. When accessing in visitor mode, you can directly click on the visitor mode in the lower right corner to automatically jump to the platform interface with restricted functions. When the administrator checkbox is checked, it will switch to administrator login mode.

6. A Thangka culture knowledge graph display system according to claim 5, characterized in that: Users who access this system in visitor mode will be automatically classified as thangka enthusiasts and granted only restricted functions. They can only use the three functions of named entity recognition, entity query, and entity relationship query of this system. Users who register and log in normally to this system will be granted additional functions for thangka culture and thangka cultural relationships on the basis of the above three functional permissions. System administrators will be granted all control permissions for the thangka culture knowledge graph display platform.

Images