Movie knowledge graph construction system based on improved BERT and professional dictionary

Abstract

The application relates to a movie and television work knowledge graph construction system based on improved BERT and a professional dictionary, which comprises a data acquisition module, a regional field dictionary construction module, a definition module, a labeling module, a training module, an entity relationship extraction module, a knowledge graph construction module and a query display module. The training module further comprises an incremental training submodule. The application realizes automatic construction of a regional characteristic knowledge graph, and the constructed movie and television work regional characteristic knowledge graph is beneficial to cultural propaganda personnel and researchers to establish an intuitive impression of regional characteristics of movie and television works and analyze regional differences of the movie and television works.

Description

Technical Field

[0001] This invention belongs to the field of intelligent text recognition, specifically involving a knowledge graph construction system for film and television works based on an improved BERT and a professional dictionary. Background Technology

[0002] Knowledge graphs were first proposed by Google in 2012, initially primarily used in search engines to accurately describe the relationships between people, things, and events. A knowledge graph comprises three main elements: entities, relationships, and attributes. The construction of a knowledge graph begins by determining the knowledge representation model, then selecting different knowledge acquisition methods based on different knowledge sources and importing relevant knowledge. Next, the corresponding knowledge graph is constructed based on technologies such as knowledge reasoning and knowledge fusion. Finally, the representation of the knowledge graph is designed according to different application scenarios. As a graph-based knowledge storage and management model, knowledge graphs are characterized by clear relationships and flexible structures. Besides applications in knowledge visualization and question answering, they also possess semantic reasoning capabilities, assisting decision-makers in retrieving and utilizing related knowledge.

[0003] Internet platforms possess a vast amount of textual information related to film and television works, which is of significant value to film and television industry research. The regional characteristics showcased in film and television works represent regional images and are a crucial channel for promoting regional culture. However, due to the scattered distribution and inconsistent quality of this information, users need to manually filter information related to regional characteristics in film and television works, which is time-consuming, inefficient, and lacks accuracy. Therefore, it is necessary to mine and analyze regionally related knowledge, and by constructing corresponding knowledge graphs, improve the utilization efficiency of regional characteristic data associated with film and television works, thereby effectively extracting regional characteristic information from film and television works and conveniently obtaining information about related film and television works from a specific region.

[0004] Compared to general knowledge graphs, the construction of domain-specific knowledge graphs needs to be more precise. Currently, there is a wealth of research in China on the construction of knowledge graphs in broad fields such as healthcare, power, finance, and education, but there is a relative lack of research on the construction of knowledge graphs for more refined fields such as film and television works.

[0005] The construction of regional characteristic knowledge graphs for film and television works, through structured analysis of regional data associated with a large number of film and television works, can provide more accurate decision-making information consulting services for local film and television research and film and television industry practices in my country. Its results can better serve the enhancement of national and local cultural soft power. Summary of the Invention

[0006] The purpose of this invention is to address the aforementioned problems by providing a film and television knowledge graph construction system based on an improved BERT model and a specialized dictionary. The improved BERT model serves as the system's deep learning model. After training the deep learning model using entities and entity relationships from labeled regional association data, unlabeled regional association data of film and television works is input into the deep learning model. The deep learning model extracts entity relationships from the film and television works and stores these relationships in a graph database, resulting in a regional characteristic knowledge graph of the film and television works. The film and television regional knowledge graph construction system uses the trained deep learning model to extract entity relationships from the unlabeled regional association data of film and television works, stores these relationships in a graph database, obtains a regional characteristic knowledge graph of the film and television works, and displays it to the system user.

[0007] The technical solution of this invention is a knowledge graph construction system for film and television works based on an improved BERT and a professional dictionary. It adopts an improved BERT model, replacing the random word-level masking mechanism in the BERT model with a phrase-level masking mechanism related to the film and television works domain, so that the improved BERT model can learn the syntactic and grammatical information of the film and television works text data.

[0008] The film and television knowledge graph construction system includes:

[0009] The data acquisition module is used to acquire regional correlation data of film and television works;

[0010] The regional domain dictionary construction module is used to construct a regional domain dictionary for film and television works, and to perform word segmentation on the acquired regional association data of film and television works based on the constructed regional domain dictionary;

[0011] The definition module defines entities in the geographic association data of film and television works and establishes the relationships between entities;

[0012] The annotation module manually annotates entities and entity relationships in the geographic association data of film and television works based on entities and relationships between entities, and obtains annotated entities and entity relationships;

[0013] The training module uses geographic association data from film and television works and labeled entities and entity relationships to train the deep learning model;

[0014] The entity relationship extraction module is used to input the geographic association data of unlabeled film and television works into the trained deep learning model to extract entity relationships and obtain multiple extracted entity relationships.

[0015] The knowledge graph construction module stores the entity relationships extracted by the deep learning model into a graph database to obtain a regional characteristic knowledge graph of film and television works.

[0016] The query and display module utilizes the regional characteristics of film and television works to provide system users with information on related film and television works for the region they are querying.

[0017] Preferably, the training module further includes an incremental training submodule, which is used to periodically collect updated regional association data of film and television works from the Internet, and use the incremental data to retrain the deep learning model to realize the updating and optimization of the deep learning model.

[0018] Furthermore, in the improved BERT model, the hidden layer input is {e1,e2,...,e...} T}, e i Let i = 1, 2, ..., T represent the i-th word vector of the regional association data of film and television works output by the BERT pre-trained model, and let the word vector e be... i The expression is:

[0019] e i =e wi +e li +e si +e 专业词汇

[0020] In the formula e wi e represents the word content information of the i-th word vector. li e represents the word position information of the i-th word vector. si e represents the word context information of the i-th word vector. 专业词汇 This indicates vocabulary information belonging to the regional domain of film and television works in the dictionary.

[0021] Preferably, the film and television knowledge graph construction system adds an attention layer connected to the forward hidden layer in the improved BERT model to enhance the semantic weight of regional domain words in film and television works.

[0022] The Attention layer uses global attention, and its weight coefficients are calculated as follows:

[0023] 1) Calculate the score of the source hidden state.

[0024]

[0025] In the formula The score of the source hidden state is represented by the dot product between the target hidden state and the source hidden state, which is used as the score of the source hidden state at each time step; h t Indicates the target is in a hidden state; This indicates the source is in a hidden state.

[0026] 2) Normalization processing,

[0027]

[0028] In the formula, α represents the attention weight distribution. s' represents the hidden state after encoding changes; s' represents the non-source hidden state.

[0029] 3) Calculate the attention vector c t And the attention vector c t and hidden state h t The hidden states h are concatenated and input into a fully connected layer to obtain the hidden state h. * Hidden state h * The calculation formula is

[0030] h * =tanh(W[c t h t ])

[0031] In the formula, W represents all words input to the improved BERT model.

[0032] 4) Calculate the output result based on the hidden state.

[0033] p(y t |y ＜t ,x)=softmax(W s h * )

[0034] In the formula p(y t |y ＜t (x) represents the probability of the target word at this moment; W s Indicates the target vocabulary, y t The target word is represented by x, the source word is represented by y. ＜t Indicates the words preceding the current target word.

[0035] In the improved BERT model, the hidden vector H = [h1, h2, ..., h T ], where h j j = 1, 2, ..., T represents the positively directed hidden states. and reverse hidden state The summation of the summation; the calculation formula for the attention distribution M of the hidden vector H is M = tanh(H); α represents the attention weight distribution of the hidden layer output data.

[0036] Preferably, the weight matrix r of the attention layer is calculated as follows:

[0037] r = Hα T .

[0038] The method for constructing a knowledge graph system for film and television works includes the following steps:

[0039] Step 1: Collect text data of film and television works, construct a regional domain dictionary of film and television works, and preprocess and segment the text data to obtain regional association data of film and television works;

[0040] Step 2: Define the entities in the regional association data and establish the relationships between the entities;

[0041] Step 3: Based on the entities and entity relationships, label the entities and entity relationships in the regional association data of film and television works to obtain the labeled entities and entity relationships;

[0042] Step 4: Train the BERT model using the regional association data of film and television works, as well as the labeled entities and entity relationships;

[0043] Step 5: Input the unlabeled geographic association data of film and television works into the BERT model, and use the BERT model to extract the entity relationships of the film and television works;

[0044] Step 6: Store the entity relationships of the obtained film and television works into a graph database to obtain a regional characteristic knowledge graph of the film and television works.

[0045] Preferably, step 1 specifically includes acquiring online text data such as promotional websites, Weibo, WeChat, portal websites, news, encyclopedias, and online Q&A communities for film and television works, constructing a regional domain dictionary for film and television works, and preprocessing and segmenting the data to obtain preprocessed regional association data of film and television works, such as film and television work titles, production regions, and work introductions.

[0046] The geographic association data of the film and television works includes the production region of the film and television works, dialects, place names, food, celebrities, landmarks, characteristic industries / products, characteristic festivals and activities, intangible cultural heritage, cultural relics, historical figures, folk customs and legends. This geographic association data maps to the corresponding geographic attributes.

[0047] In step 2, the entities in the regional association data of film and television works are defined by combining expert knowledge, and the relationships between entities are established. The relationships between entities include the relationship between work title and production region, work title and dialect, work title and place name, as well as the relationship between production region and regional name, dialect and regional name, and place name and regional name.

[0048] Preferably, in step 3, the Brat tool is used to manually annotate the entities and entity relationships in the geographic association data of the film and television works.

[0049] Compared with the prior art, the beneficial effects of the present invention include:

[0050] 1) This invention realizes the automatic construction of knowledge graphs of regional characteristics of film and television works, replacing manual labor, saving time and effort. The constructed knowledge graphs of regional characteristics of film and television works are helpful for cultural promotion personnel and researchers to establish an intuitive impression of the regional characteristics of film and television works and to analyze and study the regional differences of film and television works.

[0051] 2) This invention improves the random word-level masking mechanism in the BERT model to a phrase-level masking mechanism, enabling the improved BERT model to learn the syntactic and glyphic information of film and television text data, thereby improving the efficiency and accuracy of entity relationship recognition in regional association data.

[0052] 3) This invention adds an attention layer connected to the hidden layer in the BERT model, which strengthens the semantic weight of regional domain words and further improves the recognition accuracy of entities and entity relationships in regional association data.

[0053] 4) The system of the present invention uses a trained BERT model to automatically extract entities and entity relationships from unlabeled regional association data of film and television works, stores them in a graph database, obtains a regional characteristic knowledge graph of film and television works, and displays it to system users for querying; the system of the present invention collects and updates regional association data of film and television works from the Internet at regular intervals, and uses incremental data to retrain the BERT model, realizing the updating and continuous optimization of the entity and relationship recognition model of the knowledge graph. Attached Figure Description

[0054] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0055] Figure 1 This is a flowchart illustrating the method for constructing a regional knowledge graph of film and television works according to an embodiment of the present invention.

[0056] Figure 2 This is a schematic diagram of the phrase-level masking mechanism of the improved BERT model in an embodiment of the present invention.

[0057] Figure 3 This is a schematic diagram of the improved BERT model according to an embodiment of the present invention.

[0058] Figure 4 This is a schematic diagram of the knowledge graph of the film and television work "Shan Hai Qing" constructed according to an embodiment of the present invention.

[0059] Figure 5 This is a schematic diagram of a film and television knowledge graph construction system according to an embodiment of the present invention. Detailed Implementation

[0060] like Figure 1-3As shown, a knowledge graph construction system for film and television works based on an improved BERT and a professional dictionary is proposed. The BERT model is improved by changing the random word-level masking mechanism in the BERT model to a phrase-level masking mechanism related to the film and television works domain, so that the improved BERT model can learn the syntactic and grammatical information of the film and television works text data. An attention layer connected to the feedforward hidden layer is added to the improved BERT model to strengthen the semantic weight of the regional domain words of film and television works.

[0061] Figure 3 The example shown is an improved BERT model with hidden layer inputs {e1, e2, ..., e}. T}, e i Let i = 1, 2, ..., T represent the i-th word vector of the regional association data of film and television works output by the BERT pre-trained model, and let the word vector e be... i The expression is as follows

[0062] e i =e wi +e li +e si +e 专业词汇

[0063] In the formula e wi e represents the word content information of the i-th word vector. li e represents the word position information of the i-th word vector. si e represents the word context information of the i-th word vector. 专业词汇 This indicates vocabulary information belonging to the regional domain of film and television works in the dictionary.

[0064] The BERT pre-trained model in this embodiment refers to the pre-trained model published in the paper "Pre-Training with Whole Word Masking for Chinese BERT" by Cui Yiming et al., which was included in the arXiv academic institution in 2019.

[0065] Hidden vector H = [h1, h2, ..., h T ], where h j j = 1, 2, ..., T represents the positively directed hidden states. and reverse hidden state The summation of the summation; the attention distribution M of the hidden vector H is calculated as M = tanh(H); α represents the attention weight distribution of the hidden layer output data, and the weight matrix r of the attention layer is calculated as r = Hα. T .

[0066] The Attention layer uses global attention, and its weight coefficients are calculated as follows:

[0067] 1) Calculate the score of the source hidden state.

[0068]

[0069] In the formula The score of the source hidden state is represented by the dot product between the target hidden state and the source hidden state, which is used as the score of the source hidden state at each time step; h t Indicates the target is in a hidden state; Indicates the source is hidden;

[0070] 2) Normalization processing, In the formula, α represents the attention weight distribution. s' represents the hidden state after encoding changes; s' represents the non-source hidden state.

[0071] 3) Calculate the attention vector c t And the attention vector c t and hidden state h t The hidden states h are concatenated and input into a fully connected layer to obtain the hidden state h. * Hidden state h * The calculation formula is

[0072] h * =tanh(W[c t h t ])

[0073] In the formula, W represents all words input to the improved BERT model;

[0074] 4) Calculate the output result based on the hidden state.

[0075] p(y t |y ＜t ,x)=softmax(W s h * )

[0076] In the formula p(y t |y ＜t (x) represents the probability of the target word at this moment; W s Indicates the target vocabulary, y t The target word is represented by x, the source word is represented by y. ＜t Indicates the words preceding the current target word.

[0077] The method for constructing a regional knowledge graph of film and television works in this embodiment includes the following steps:

[0078] Step 1: Obtain online text data from promotional websites, Weibo, WeChat, portal websites, news, encyclopedias, online Q&A communities, etc. of film and television works, construct a regional domain dictionary of film and television works, and preprocess and segment the data to obtain preprocessed regional association data of film and television works such as film and television work titles, production regions, and work introductions.

[0079] The regional association data of film and television works includes the production region of the film and television works, dialects, place names, food, celebrities, landmarks, characteristic industries / products, characteristic festivals and activities, intangible cultural heritage, cultural relics, historical figures, folk customs and legends. This regional association data maps to the corresponding regional attributes.

[0080] Step 2: Define the entities in the regional association data and establish the relationships between the entities;

[0081] By combining expert knowledge, entities in the regional association data of film and television works are defined, and the relationships between entities are established. The relationships between entities include the relationship between work title and production region, work title and dialect, work title and place name, as well as the relationship between production region and regional name, dialect and regional name, and place name and regional name.

[0082] Step 3: Based on entities and entity relationships, use the Brat tool to annotate the entities and entity relationships in the geographic association data of film and television works. Entities in the geographic association data text are labeled as T-type, and entity relationships as R-type, resulting in annotated entities and entity relationships. Brat is a web-based text annotation tool used for structured text annotation. The annotation results from Brat are used to structure the annotated text.

[0083] Step 4: Train the BERT model using the regional association data of film and television works, as well as the labeled entities and entity relationships.

[0084] Step 5: Input the unlabeled regional association data of film and television works into the BERT model, and use the BERT model to extract the entity relationships of film and television works.

[0085] Step 6: Store the entity relationships of the obtained film and television works into a graph database to obtain a regional characteristic knowledge graph of the film and television works.

[0086] The method for constructing a regional knowledge graph of film and television works in the embodiment involves periodically collecting and updating regional association data of film and television works, and using incremental data to retrain the BERT model.

[0087] In the embodiment, the knowledge graph of the film and television work "Shan Hai Qing" obtained by the method of the present invention is as follows: Figure 4 As shown.

[0088] like Figure 5 As shown, the film and television knowledge graph construction system of this embodiment includes:

[0089] The data acquisition module is used to acquire regional correlation data of film and television works;

[0090] The regional domain dictionary construction module is used to construct a regional domain dictionary for film and television works, and to perform word segmentation on the acquired regional association data of film and television works based on the constructed regional domain dictionary;

[0091] The definition module defines entities in the regional association data of film and television works based on expert knowledge and establishes the association relationships between entities;

[0092] The annotation module manually annotates entities and entity relationships in the geographic association data of film and television works based on entities and relationships between entities, and obtains annotated entities and entity relationships;

[0093] The training module uses geographic association data from film and television works and labeled entities and entity relationships to train the deep learning model;

[0094] The incremental training submodule of the training module is used to collect and update the regional association data of film and television works from the Internet at regular intervals, and use the incremental data to retrain the deep learning model to realize the updating and optimization of the deep learning model.

[0095] The entity relationship extraction module is used to input the geographic association data of unlabeled film and television works into the trained deep learning model to extract entity relationships and obtain multiple extracted entity relationships.

[0096] The knowledge graph construction module stores the entity relationships extracted by the deep learning model into a graph database to obtain a regional characteristic knowledge graph of film and television works.

[0097] The query and display module utilizes the regional characteristics of film and television works to provide system users with information on related film and television works for the region they are querying.

Claims

1. A knowledge graph construction system for film and television works based on improved BERT and specialized dictionaries, characterized in that: The film and television knowledge graph construction system adopts an improved BERT model as a deep learning model, and changes the random word-level masking mechanism in the BERT model to a phrase-level masking mechanism related to the film and television domain, so that the improved BERT model can learn the syntactic and grammatical information of the film and television text data. The film and television knowledge graph construction system includes: The data acquisition module is used to acquire regional correlation data of film and television works; The regional domain dictionary construction module is used to construct a regional domain dictionary for film and television works, and to perform word segmentation on the acquired regional association data of film and television works based on the constructed regional domain dictionary; The definition module defines entities in the geographic association data of film and television works and establishes the relationships between entities; The annotation module manually annotates entities and entity relationships in the geographic association data of film and television works based on the entities and relationships between entities, and obtains the annotated entities and entity relationships; The training module uses geographic association data from film and television works and labeled entities and entity relationships to train the deep learning model; The entity relationship extraction module is used to input the geographic association data of unlabeled film and television works into the trained deep learning model to extract entity relationships and obtain multiple extracted entity relationships. The knowledge graph construction module stores the entity relationships extracted by the deep learning model into a graph database to obtain a regional characteristic knowledge graph of film and television works. The query and display module utilizes the regional characteristics of film and television works to provide system users with information on related film and television works for the region they are querying. In the improved BERT model, the hidden layer input is {e1,e2,...,e...} T }, e i Let i = 1, 2, ..., T represent the i-th word vector of the geographic association data of film and television works output by the BERT pre-trained model, and let the word vector e be... i The expression is: And i =and wi +e li +e si +e 专业词汇 In the formula e wi e represents the word content information of the i-th word vector. li e represents the word position information of the i-th word vector. si e represents the word context information of the i-th word vector. 专业词汇 This represents vocabulary information belonging to the regional domain dictionary of film and television works, and T represents the number of hidden layers.

2. The film and television knowledge graph construction system according to claim 1, characterized in that, The knowledge graph construction system for film and television works adds an attention layer connected to the forward hidden layer in the improved BERT model to enhance the semantic weight of regional domain words in film and television works. The Attention layer uses global attention, and its weight coefficients are calculated as follows: 1) Calculate the score of the source hidden state. In the formula The score of the source hidden state is represented by the dot product between the target hidden state and the source hidden state, which is used as the score of the source hidden state at each time step; h t Indicates the target is in a hidden state; Indicates the source is hidden; 2) Normalization processing, In the formula, α represents the attention weight distribution. s' represents the hidden state after encoding changes; s' represents the non-source hidden state. 3) Calculate the attention vector c t And the attention vector c t and hidden state h t The hidden states h are concatenated and input into a fully connected layer to obtain the hidden state h. * Hidden state h * The calculation formula is h * = tanh(W[c t h t ]) In the formula, W represents all words input to the improved BERT model; 4) Calculate the output result based on the hidden state. p(y t |y ＜t ,x)=softmax(W s h * ) In the formula p(y t |y ＜t (x) represents the probability of the target word at this moment; W s Indicates the target vocabulary, y t The target word is represented by x, the source word is represented by y. ＜t Indicates the words preceding the current target word.

3. The film and television knowledge graph construction system according to claim 2, characterized in that, In the improved BERT model, the hidden vector H = [h1, h2, ..., h T ], where h j j = 1, 2, ..., T represents the positively directed hidden states. and reverse hidden state The summation of the summation; the calculation formula for the attention distribution M of the hidden vector H is M = tanh(H); α represents the attention weight distribution of the hidden layer output data.

4. The film and television knowledge graph construction system according to claim 3, characterized in that, The formula for calculating the weight matrix r of the attention layer is: r＝Hα T 。 5. The film and television knowledge graph construction system according to claim 4, characterized in that, The training module also includes an incremental training submodule, which is used to collect and update the regional association data of film and television works from the Internet at regular intervals, and use the incremental data to retrain the deep learning model to realize the updating and optimization of the deep learning model.

6. The method for constructing a knowledge graph of film and television works as described in any one of claims 1-5, characterized in that, Includes the following steps: Step 1: Collect text data of film and television works, construct a regional domain dictionary of film and television works, and preprocess and segment the text data to obtain regional association data of film and television works; Step 2: Define the entities in the regional association data and establish the relationships between the entities; Step 3: Based on the entities and entity relationships, label the entities and entity relationships in the regional association data of film and television works to obtain the labeled entities and entity relationships; Step 4: Train the BERT model using the regional association data of film and television works, as well as the labeled entities and entity relationships; Step 5: Input the unlabeled geographic association data of film and television works into the BERT model, and use the BERT model to extract the entity relationships of the film and television works; Step 6: Store the entity relationships of the obtained film and television works into a graph database to obtain a regional characteristic knowledge graph of the film and television works.

7. The method according to claim 6, characterized in that, In step 2, the entities in the regional association data of film and television works are defined by combining expert knowledge, and the relationships between entities are established. The relationships between entities include the relationship between work title and production region, work title and dialect, work title and place name, as well as the relationship between production region and regional name, dialect and regional name, and place name and regional name.

8. The method according to claim 7, characterized in that, Step 3 involves using the Brat tool to manually annotate the entities and entity relationships in the geographic association data of the film and television works.

9. The method according to claim 7 or 8, characterized in that, The method also includes periodically collecting and updating regional association data of film and television works, and using incremental data to retrain the BERT model.

Images