Text topic segmentation method and device based on knowledge graph and electronic equipment

Abstract

The application discloses a text topic segmentation method based on a knowledge graph, comprising: obtaining a text document; identifying text entities in the text document through an entity recognition algorithm, the entity recognition algorithm comprising a first recognition algorithm and a second recognition algorithm, the first recognition algorithm and the second recognition algorithm being different recognition algorithms; linking the text entities to knowledge graph entities; and based on the linking results of the text entities and the knowledge graph entities, performing topic segmentation on the text document, wherein performing text topic segmentation on the text document comprises taking a sentence as a text topic segmentation unit, calculating semantic similarity between the sentences based on the text entities in the sentences, and performing the text topic segmentation according to the semantic similarity; a text topic segmentation algorithm model with high precision can be trained on a small-scale data set, and the algorithm generalization capability and interpretability are strong.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence, and particularly relates to a text topic segmentation method and device based on a knowledge graph and an electronic device. BACKGROUND

[0002] Text topic segmentation is a technology of segmenting an original document into multiple sub-documents with consistent topics according to whether the topics of the text are transferred.

[0003] The text topic segmentation technology can model the topic structure of a document, and based on the segmented topic structure, downstream tasks such as information retrieval, text summarization, topic detection and tracking can be performed. Common text topic segmentation algorithms are mostly based on supervised learning. The supervised text topic segmentation algorithm needs a large amount of labeled data and has poor generalization ability and interpretability.

[0004] Based on the above problems, the present application provides a text topic segmentation method based on a knowledge graph. SUMMARY

[0005] To solve the problems of the prior art, the present application provides a text topic segmentation method based on a knowledge graph. The text topic segmentation method based on a knowledge graph can train a text topic segmentation algorithm model with high precision on a small-scale data set, and has the technical effects of strong algorithm generalization ability and interpretability.

[0006] The technical effects achieved by the present application are realized by the following solutions:

[0007] In a first aspect, the present application provides a text topic segmentation method based on a knowledge graph, comprising:

[0008] Obtaining a text document;

[0009] Identifying text entities in the text document by an entity recognition algorithm, the entity recognition algorithm comprising a first recognition algorithm and a second recognition algorithm, the first recognition algorithm and the second recognition algorithm being different recognition algorithms; linking the text entities to knowledge graph entities, wherein the knowledge graph entities represent entities in a knowledge graph corresponding to the text entities;

[0010] Segmenting the text document based on the linking results of the text entities and the knowledge graph entities, wherein segmenting the text document includes: taking a sentence as a text topic segmentation unit, calculating semantic similarity between the sentences based on the text entities in the sentences, and performing the text topic segmentation according to the semantic similarity.

[0011] Furthermore, the step of identifying text entities in the text document using an entity recognition algorithm includes:

[0012] A first recognition algorithm is used to identify a first type of text entity in the text document, wherein the first recognition algorithm is a Trie tree recognition algorithm, and the first type of text entity represents an entity that already exists in the knowledge graph;

[0013] A second recognition algorithm is used to identify a second type of text entity in the text document, wherein the second recognition algorithm is the Bert-BiLSTM-CRF recognition algorithm, and the second type of text entity represents an entity that does not exist in the knowledge graph.

[0014] Furthermore, linking the text entity to the knowledge graph entity includes:

[0015] Based on the Bi-encoder candidate entity recall algorithm, N knowledge graph entities corresponding to the text entity are matched from the knowledge graph, where N is a positive integer;

[0016] Based on the Cross-encoder candidate entity disambiguation algorithm, the candidate entity with the highest similarity score to the text entity is determined from the N knowledge graph entities.

[0017] Furthermore, the Bert-BiLSTM-CRF recognition algorithm includes:

[0018] We selected Bert as the embedding layer model, trained the Bert-base-chinese pre-trained model, and obtained the embedding vector output by the Bert embedding layer.

[0019] The embedding vector is input into a bidirectional long short-term memory network to obtain the feature vector output by the bidirectional long short-term memory network;

[0020] The feature vector is input into a conditional random field, and the transition probabilities between sequence labels are learned using the conditional random field. By combining the global information of the sequence labels, the feature vector is decoded into sequence labels.

[0021] Furthermore, the method also includes:

[0022] The embedding vectors of the text entities are trained using a Siamese network, which includes classification and regression tasks.

[0023] The classification task satisfies the following formula:

[0024] o = softmax(W t (u1, v1, |u1-v1|)),

[0025] Where o represents the output vector, u1 and v1 represent the embedding vector, and W t This represents the multiplication of a vector by a parameter matrix. n is the dimension of the embedding vector, and k is the number of classification labels.

[0026] Further, the step of using sentences as text topic segmentation units, calculating semantic similarity between sentences based on the text entities in the sentences, and performing text topic segmentation based on the semantic similarity includes:

[0027] Using sentences as the unit for text topic segmentation, the text topic is divided into a set of text segments S. For each text segment s in the set of text segments S, a sentence text entity relation graph G is constructed.

[0028] Each node in the sentence text entity relationship graph G represents a knowledge graph entity in the knowledge graph. Each node includes frequency information and attribute information, where the frequency information indicates the number of times the knowledge graph entity is mentioned in the text document. The edges in the sentence text entity relationship graph G represent the relationship between two nodes, and the value of the relationship is Kno w. re (u, v) and satisfy the following formula:

[0029]

[0030] Where u and v represent different nodes, E represents the set of all relations in the knowledge graph, e(u, v) indicates that there is a relation e between u and v, KGReMap(u, v) indicates the similarity when there is a relation e between u and v, and PathRe(u, v) indicates the similarity when there is no relation e between u and v.

[0031] Furthermore, the step of using sentences as text topic segmentation units, calculating semantic similarity between sentences based on the text entities in the sentences, and performing text topic segmentation based on the semantic similarity further includes:

[0032] Consistency(i,j) is used to represent the entity relation graph G of the sentence text. i and G j The formula for calculating Consistency(i,j) is as follows:

[0033] Consistency(i,j)=α×MC(i,j)+β×CC(i,j),

[0034] Where MC(i,j) represents the word distribution dissimilarity, CC(i,j) represents the topic cohesion loss, α and β are the importance parameters of MC(i,j) and CC(i,j) respectively, i represents the i-th sentence, and j represents the j-th sentence.

[0035] Furthermore, the step of using sentences as text topic segmentation units, calculating semantic similarity between sentences based on the text entities in the sentences, and performing text topic segmentation based on the semantic similarity further includes:

[0036] Set a topic coherence threshold σ for the sentence text entity relationship graph G, and merge adjacent sentence text entity relationship graphs whose topic coherence is greater than the topic coherence threshold σ;

[0037] The text topic segmentation result is obtained when there are no adjacent text entity relationship graphs that can be merged in the sentence text entity relationship graph array.

[0038] Secondly, embodiments of the present invention provide an apparatus for a text topic segmentation method based on a knowledge graph, comprising:

[0039] The acquisition module is used to acquire text documents;

[0040] The entity linking module is used to identify text entities in the text document using an entity recognition algorithm, wherein the entity recognition algorithm includes a first recognition algorithm and a second recognition algorithm, and the first recognition algorithm and the second recognition algorithm are different recognition algorithms; and to link the text entities to knowledge graph entities, wherein the knowledge graph entities represent entities in the knowledge graph that correspond to the text entities;

[0041] The text topic segmentation module is used to perform topic segmentation on the text document based on the link results between the text entities and the knowledge graph entities. The text topic segmentation on the text document includes: using sentences as text topic segmentation units, calculating the semantic similarity between sentences based on the text entities in the sentences, and performing text topic segmentation based on the semantic similarity.

[0042] Thirdly, embodiments of the present invention provide an electronic device, comprising:

[0043] A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the program, implements the method described in any one of the preceding methods.

[0044] Fourthly, embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method described in any one of the above-mentioned embodiments.

[0045] The knowledge graph-based text topic segmentation method provided in this invention can train a high-precision text topic segmentation algorithm model on a small dataset, achieving the technical effects of strong algorithm generalization ability and interpretability. Attached Figure Description

[0046] To more clearly illustrate the embodiments of this application or the existing technical solutions, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of a knowledge graph-based text topic segmentation method according to an embodiment of this application;

[0048] Figure 2 This is a technical framework diagram of a knowledge graph-based text topic segmentation method in one embodiment of this application;

[0049] Figure 3 This is a schematic diagram illustrating an example of named entity recognition in one embodiment of this application;

[0050] Figure 4 This is a schematic diagram of a Trie tree structure based on a knowledge graph in one embodiment of this application;

[0051] Figure 5 This is a schematic diagram of the network architecture of the Bert-BiLSTM-CRF entity recognition algorithm in one embodiment of this application;

[0052] Figure 6 This is a schematic diagram of the Bert network architecture in one embodiment of this application;

[0053] Figure 7 This is a schematic diagram of an LSTM network architecture in one embodiment of this application;

[0054] Figure 8 This is a schematic diagram of the architecture of a candidate entity recall algorithm based on Bi-encoder in one embodiment of this application;

[0055] Figure 9This is a schematic diagram of the candidate entity disambiguation algorithm architecture based on Cross-encoder in one embodiment of this application;

[0056] Figure 10 This is a schematic diagram of a twin network structure in one embodiment of this application;

[0057] Figure 11 This is a schematic diagram of a Sentence-Bert Siamese network training example in one embodiment of this application;

[0058] Figure 12 This is a schematic diagram of a sentence text entity relationship diagram in one embodiment of this application;

[0059] Figure 13 This is a schematic diagram illustrating an example of merging sentence text entity relationship graphs in one embodiment of this application;

[0060] Figure 14 This is a schematic diagram illustrating a text topic segmentation process in one embodiment of this application;

[0061] Figure 15 This is a schematic diagram of a knowledge graph-based text topic segmentation device according to an embodiment of this application;

[0062] Figure 16 This is a schematic block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar words used in one or more embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0065] The main ideas of the technical solution of this invention include:

[0066] By using text entity linking technology, text entities in text documents (in this invention, "text" and "text document" represent the same meaning) are identified, and these text entities are linked to entities in the knowledge graph.

[0067] Using sentences as the unit for text topic segmentation, semantic similarity between sentences is calculated based on entities within the sentences, and then text topic segmentation is performed based on the similarity. The achievable results are shown in Table 1. For document 1, the topic classification results are topic 1 discussed in sentences 1 to 19, and topic 2 discussed in sentences 20 to 36.

[0068] Table 1. Example of topic segmentation output results

[0069]

[0070] The various non-limiting embodiments of this application are described in detail below with reference to the accompanying drawings.

[0071] First, refer to Figure 1 The knowledge graph-based text topic segmentation method of the present invention will be described in detail below: Figure 1 This is a flowchart of a knowledge graph-based text topic segmentation method according to an embodiment of this application;

[0072] like Figure 1 As shown, text topic segmentation methods based on knowledge graphs include:

[0073] S100, retrieve text document;

[0074] For example, the text document in this invention can be a text document of various formats, such as PDF, Word text, etc. This is only an example and is not a specific limitation.

[0075] S200, using an entity recognition algorithm, identify text entities in the text document. The entity recognition algorithm includes a first recognition algorithm and a second recognition algorithm, which are different recognition algorithms. Link the text entities to knowledge graph entities, wherein the knowledge graph entities represent entities in the knowledge graph that correspond to the text entities.

[0076] S300, based on the linking results between the text entities and the knowledge graph entities, the text document is segmented into topics. The text topic segmentation of the text document includes: using sentences as text topic segmentation units, calculating the semantic similarity between sentences based on the text entities in the sentences, and performing text topic segmentation based on the semantic similarity.

[0077] The aforementioned knowledge graph-based text topic segmentation method identifies text entities and maps them to entities in the knowledge graph through different recognition algorithms, and then performs text topic segmentation. It can train a high-precision text topic segmentation algorithm model on a small-scale dataset, achieving technical effects such as strong algorithm generalization ability and interpretability.

[0078] The following reference Figure 2 This paper provides a detailed technical framework diagram of a knowledge graph-based text topic segmentation method. Figure 2 This is a technical framework diagram of a knowledge graph-based text topic segmentation method in one embodiment of this application;

[0079] like Figure 2 As shown, the overall framework is divided into four parts, including: input data, entity linking module, text topic segmentation module, and output data;

[0080] The two key modules are: the entity linking module and the text topic segmentation module. The entity linking module uses named entity recognition technology to identify entities in the text, and then uses entity disambiguation technology to link these entities to entities in the knowledge graph. Based on the linking results from the entity linking module, the text topic segmentation module designs an entity-based inter-sentence similarity calculation method and a text segmentation algorithm. Using sentences as the segmentation unit, it achieves topic segmentation of the text based on the above methods.

[0081] For example, the input data includes an input knowledge graph and a text document; then, the entity linking module identifies the text entities in the text document using a named entity recognition algorithm based on the knowledge base, and disambiguates the text entities using an entity disambiguation algorithm based on edit distance; next, the text topic segmentation module segments the text document into text topics using a text topic algorithm based on the knowledge graph, obtains the text topics, and finally outputs the segmented text document.

[0082] For example, a first recognition algorithm can be used to identify a first type of text entity in the text document, wherein the first recognition algorithm is a Trie tree recognition algorithm, and the first type of text entity represents an entity that already exists in the knowledge graph;

[0083] For example, a second recognition algorithm can be used to identify a second type of text entity in the text document, wherein the second recognition algorithm is the Bert-BiLSTM-CRF recognition algorithm, and the second type of text entity represents an entity that does not exist in the knowledge graph.

[0084] The Trie tree recognition algorithm and Bert-BiLSTM-CRF recognition algorithm described above are merely examples. Other algorithms known to those skilled in the art that can achieve the above functions are also applicable here, and no limitation is made.

[0085] Different algorithms are used to identify different text entities, which can achieve targeted text recognition. For some entities that do not exist in the knowledge graph, the correct text entities can be identified more accurately, thus improving accuracy.

[0086] In some embodiments, the entity linking module entity includes two key technologies: named entity recognition and entity disambiguation. The relevant technologies of the entity linking module entity are introduced below:

[0087] The following examples illustrate named entity recognition in detail:

[0088] Named entity recognition is a natural language processing task that identifies entities with specific meaning from unstructured text and classifies them into pre-declared entity categories.

[0089] in Figure 3 This is a schematic diagram illustrating a named entity recognition example from one embodiment of this application; as shown below. Figure 3 As shown, a sentence was identified by a named entity recognition algorithm as having three entities: "Zhang Shan," "Beijing Haidian District," and "Police Officer," which were then categorized into "Person Name," "Location," and "Occupation," respectively. Accurate named entity recognition results lay a solid foundation for downstream natural language processing applications such as information retrieval and machine question answering.

[0090] In some embodiments, the present invention may employ an entity recognition algorithm based on a knowledge base and Bert-BiLSTM-CRF. The knowledge base is constructed from a knowledge graph. Entity recognition algorithms based on the knowledge base can identify entities existing in the knowledge graph with high efficiency and accuracy, but cannot identify entities not included in the knowledge base. Therefore, a deep learning model is needed to identify such entities.

[0091] This invention uses a Trie tree to identify and extract entities existing in a knowledge base. A Trie tree is a variant of a hash tree and can be used for string matching. Trie trees offer superior string matching performance by using common prefixes of strings to reduce the number of useless matches and optimize query time. A knowledge base-based Trie tree is attached. Figure 4 As shown in (a), to match "array index", Trie will follow the appendix. Figure 4 (b) shows the path for string matching.

[0092] In some embodiments, the Bert-BiLSTM-CRF recognition algorithm includes:

[0093] We selected Bert as the embedding layer model, trained the Bert-base-chinese pre-trained model, and obtained the embedding vector output by the Bert embedding layer.

[0094] The embedding vector is input into a bidirectional long short-term memory network to obtain the feature vector output by the bidirectional long short-term memory network;

[0095] The feature vector is input into a conditional random field, and the transition probabilities between sequence labels are learned using the conditional random field. By combining the global information of the sequence labels, the feature vector is decoded into sequence labels.

[0096] In some embodiments, the present invention can also utilize a Bert-BiLSTM-CRF-based entity recognition algorithm to identify out-of-vocabulary words (vocabulary words represent entities that do not exist in the knowledge graph) for new word discovery. The network architecture of the Bert-BiLSTM-CRF-based entity recognition algorithm is as follows: Figure 5 As shown in the attached image. BERT is a pre-trained language representation model. Figure 6As shown, the BERT network architecture consists of stacked multi-layer Transformer encoders. BERT training is divided into two types: Masked LM (MLM) and Next Sentence Prediction (NSP). MLM randomly masks 15% of the tokens in the input sequence and uses the output vector at the corresponding position to predict the true value. NSP concatenates two sentences and designs a binary classification problem to determine whether the two sentences are the same sentence. The BERT model has a huge number of parameters, requiring massive training data and computing units for training. Commonly used open-source pre-trained BERT models and their main parameters are shown in the table. This invention selects BERT as the embedding layer model and fine-tunes the BERT-base-chinese pre-trained model. The word sequence obtains word vectors with strong expressive power containing bidirectional information through the BERT embedding layer.

[0097] Table 2 Open Source Pre-trained Bert Models

[0098]

[0099] The embedding vectors output by the BERT embedding layer can represent sufficient semantic information. However, the BERT model does not consider the orientation information of the tokens, which plays a role in named entity recognition. To fully learn the positional and orientation information of the sequence, this invention adds a bidirectional long short-term memory network (Bi-LSTM) after the BERT input layer. The LSTM network structure is attached. Figure 7 As shown, LSTM possesses the ability to learn long-term dependencies through some gating units as shown in Table 3. The forget gate can selectively forget the cell state C. t-1 Some information that does not need to be retained, the forget gate works as shown in formula (1), the input signal x at time t t and t -1 The output signal h at time t t-1 After concatenation, the signal is input into a sigmoid network layer to obtain the forgetting signal f. t f t The elements in the vector are all between 0 and 1, indicating whether the information at that position needs to be retained.

[0100] f t =σ(W f ·[h t-1 x t ]+b f (1)

[0101] The function of memory gates is to selectively retain the state C of memory cells. t-1 The information that needs to be memorized. The working principle of the memory gate is shown in formulas (2) and (3), where the input signal x at time t is... t and t-1 The output signal h at time t t-1 After concatenation, the signal i is input into the sigmoid network layer to obtain the memory signal. t Similar to the forget gate, the value at each position represents whether the information at that position needs to be retained; the input signal x at time t. t and t -1 The output signal h at time t t-1 The concatenated state candidate vector C is input into the tanh network layer. t ′.

[0102] i t =σ(W i ·[h t-1 x t ]+b i (2)

[0103] C t ′=tanh(W c ·[h t-1 x t ]+b C (3)

[0104] Combining the forgetting gate and the memory gate, cell state C t It can be updated according to formula (4).

[0105] C t =f t *C t-1 +i t *C t ′ (4)

[0106] The function of the output gate is to calculate the output signal h. t The working principle of the output gate is shown in formulas (5) and (6). The input signal x at time t t and t -1 The output signal h at time t t-1 After concatenation, the result is input into a sigmoid network layer to obtain the output o. t With C t The output signal h at time t is obtained by multiplying the results obtained through the tanh network layer. t .

[0107] o t =σ(W o [h t-1 x t ]+b o (5)

[0108] h t =o t *tanh(C t (6)

[0109] Table 3 LSTM Gating Units

[0110]

[0111] Directly selecting sequence labels from the feature vectors output by Bi-LSTM fails to consider the transition probability of label appearance. For example, when performing part-of-speech tagging on a sequence, the probability of tagging two consecutive words as verbs is 0.

[0112] This invention adds a Conditional Random Field (CRF) as a decoding layer model after the bidirectional long short-term memory network layer. It uses the CRF to learn the transition probabilities between sequence labels, integrates the global information of the sequence labels, and decodes the feature vector output by Bi-LSTM into sequence labels.

[0113] The following examples illustrate the physical disambiguation technique in detail:

[0114] The purpose of entity disambiguation is to solve the problem of polysemy by mapping named entities in a given context to entities in a knowledge graph.

[0115] In some embodiments, linking the text entity to the knowledge graph entity includes:

[0116] Based on the Bi-encoder candidate entity recall algorithm, N knowledge graph entities corresponding to the text entity are matched from the knowledge graph, where N is a positive integer;

[0117] Based on the Cross-encoder candidate entity disambiguation algorithm, the candidate entity with the highest similarity score to the text entity is determined from the N knowledge graph entities.

[0118] In some embodiments, calculating the matching degree between entity references identified by entity recognition algorithms and entities in the knowledge base one by one can result in high time and computational costs. This invention designs a candidate entity retrieval algorithm based on Bi-encoder. This algorithm quickly matches N entities from the knowledge base that are most similar to the entity reference. Then, by accurately ranking these N candidate entities, the entity reference can be mapped to the entities in the knowledge base, reducing computational complexity. The structure of the Bi-encoder-based candidate entity retrieval algorithm is shown in the attached figure. Figure 8 As shown, the Bi-encoder consists of a context encoder and a knowledge base entity encoder, and recalls candidate entities by comparing the similarity scores between the context embedding vector and the knowledge base entity embedding vector.

[0119] The candidate entity disambiguation algorithm performs entity disambiguation on the candidate entities obtained by the candidate entity recall algorithm, mapping the entity references to the most similar candidate entities. This invention designs a Cross-encoder-based candidate entity disambiguation algorithm, the structure of which is shown in the attached figure. Figure 9 As shown, the algorithm maps entity references to the candidate entities with the highest similarity scores by calculating the similarity scores between entity references and candidate entities.

[0120] This invention selects Sentence-Bert as the context encoder and entity encoder. Sentence-Bert can be used to calculate text embedding vectors, supports over 100 languages, and can be used in text semantic similarity matching, semantic search, and other fields. Sentence-Bert adds a pooling layer after the BERT model to generate fixed-length text embedding vectors. The pooling layer employs three pooling strategies: obtaining the [CLS] character output vector, calculating the mean of the output vectors, and calculating the maximum value of the output vectors. The default pooling strategy is to calculate the mean of the output vectors. The usage of Sentence-Bert is shown in the appendix. Figure 10 The text embedding vectors trained by the Siamese network shown are composed of two networks with identical structures and shared weights. The Siamese network has a simple structure and strong generalization ability. It is often used to determine whether the semantics of two input texts are similar. It is a baseline model with excellent performance in text processing tasks.

[0121] Sentence-Bert usage is as follows (see attached) Figure 11 The training strategy shown divides Siamese network training into two categories: classification tasks and regression tasks. The process of the classification task is shown in formula (7). n is the dimension of the text embedding vector, k is the number of classification labels, and the text embedding vectors u1 and v1 output by the pooling layer are concatenated with the vector difference |u1-v1|, which is then multiplied by the parameter matrix W. t The result of the operation is input into the softmax classifier to obtain the output vector o, and the cross-entropy is used as the loss function to train the model; the regression task compares the cosine similarity of the text embedding vectors u1 and v1. Sentence-Bert also uses the training method of the triplet network, as shown in formula (8). Given the anchor sentence a, the positive sentence p and the negative sentence n, the training objective is to maximize the interval between sentences a and n of different categories and minimize the interval between sentences a and p of the same category; ||·|| represents the interval between vectors, and ∈ is the interval coefficient.

[0122] o = softmax(W t (u1, v1, |u1-v1|)) (7)

[0123] max(||sa -s p ||-||s a -s n ||+∈,0) (8)

[0124] The text topic segmentation module will be described in detail below through some examples:

[0125] Existing text topic segmentation algorithms segment text topics by relying on word distribution variations and cue words, but fail to capture semantic relationships, resulting in poor performance. Deep learning-based text topic segmentation algorithms offer superior performance, utilizing deep learning models to learn the semantics of text segments and segment topics through semantic changes. However, deep learning models operate as black boxes, leading to poor interpretability. Ontology-based text topic segmentation algorithms obtain semantic relationships between entities through a constructed ontology, and then derive semantic relationships between sentences from the semantic relationships between entities, enabling text topic segmentation. Ontology-based algorithms improve the accuracy of text topic segmentation by utilizing semantic relationships between sentences, and the results become interpretable by leveraging the semantic relationships between entities provided by the ontology. This invention utilizes knowledge graphs to design a knowledge graph-based text topic segmentation algorithm. Through an entity linking module, the text contains entities mapped to the knowledge graph.

[0126] The following examples illustrate the sentence-text entity relationship graph in detail:

[0127] In some embodiments, sentences are used as the unit of text topic segmentation. The text topic is divided into a set of text segments S, and a sentence-text entity relation graph G is constructed for each text segment s in the set of text segments S. The sentence-text entity relation graph G is shown in the attached figure. Figure 12 As shown, the nodes of graph G are entities in the knowledge graph. Each node has the number of times it is mentioned and its attributes. The number of mentions is the number of times the entity is mentioned in the text. Attributes include character constants, constants, string constants, integer constants, ASCII tables, etc. The edges of graph G represent the relationship between two nodes u and v in the knowledge graph, and the value of the relationship is Kno w. re (u, v).

[0128]

[0129] E represents the set of all relations in the knowledge graph, and e(u, v) indicates that there is a relation e between u and v. If u and v are adjacent in the knowledge graph and have a relation e, their relevance is KGReMap(u, v), which can be obtained directly by querying the knowledge graph. If u and v are not adjacent in the knowledge graph and do not have a relation e, their relevance is PathRe(u, v), where PathRe represents the Know calculated from all node sequences in the knowledge graph according to the relevance propagation rule. re (u, v) is the maximum value. The relevance transfer rule is shown in formula (10). Entities u and p, and p and v have relationships e1 and e2 in the knowledge graph. The relevance between u and p is Known. re (u, p), the correlation between p and v is Kno w re (p, v), then the correlation between u and v obtained through p is Know re (u, p) × Know re (p, v).

[0130]

[0131] The following examples illustrate topic cohesion in detail:

[0132] The topic cohesion is defined as the degree of topic concentration in the sentence text entity graph G. The higher the cohesion, the more concentrated and unified the topic of the text segment. The cohesion is calculated as shown in formula (11), where n is the number of nodes in the sentence text entity graph G of the text document, and num... i edge is the number of times entity i is mentioned. ij The relevance between entities i and j.

[0133]

[0134] Define CohesionChange, CC(i,j) as the sentence text entity relation graph G. i G j The topic cohesion loss after merging. A larger CC(i,j) indicates a larger sentence text entity relationship graph G. i G j The more inconsistent the content, the more inconsistent the values. CC(i,j) is calculated as shown in formula (12), Cohesion i Cohesion j Representative sentence text entity relation graph G i G j Cohesion ij Representative sentence text entity relation graph G i G jMerged sentence text entity relation graph G ij Thematic cohesion.

[0135] CC(i,j) = 2 × Cohesion ij -Cohesion i -Cohesion j (12)

[0136] An example of merging sentence text entity relationship diagrams is attached. Figure 13 As shown, the sentence text entity relation graph G i G j Merge into G ij G i G j The node merging rules are shown in formula (13). The merged sentence text entity relationship graph G ij The node is G i G j Union of nodes, Node x This represents the set of nodes in the text entity relationship graph x of a text sentence.

[0137] Node ij =Node i ∪Node j (13)

[0138] G i G j The edge merging rules are shown in formula (14). After merging, the sentence text entity relation graph G is obtained. ij Let G be the edge. i G j Union of edges, Edge x Let x be the set of edges in the sentence text entity relation graph.

[0139] Edge ij =Edge i ∪Edge j (14)

[0140] The merging rules for the node mention count attribute are shown in formula (15), num x，u This indicates the number of times entity u is mentioned in the sentence text entity relation graph x. i G j When both appear simultaneously, the entity relationship graph G of the merged sentence text is obtained. ij num ij，u For num i，u With num j，u The sum of, otherwise num ij，u The number of times 'u' is mentioned in the text entity relation graph of a sentence containing 'u'.

[0141]

[0142] Theme changes are often accompanied by changes in vocabulary distribution. This invention defines MentionChange, where MC(i,j) represents the sentence text entity relationship graph G. i G j The difference in word distribution is indicated by the larger the MC(i,j) value, the better the graph G. i and G j The closer the word distributions are, the better. MC(i,j) is calculated as shown in formula (16), where Node x |Node represents the set of nodes in the sentence text entity relation graph x. x | Represents a set of nodes, Node x The number of elements, N = |Node i ∩Node j |,M=|Node i ∪Node j |,mf(node i |node i ∈Node i G represents the entity relation graph of the sentence text. i The number of times node i is mentioned.

[0143]

[0144] This invention integrates topic cohesion loss CC(i,j) and word distribution dissimilarity MC(i,j), defining Consistency(i,j) as the entity relation graph G of the sentence text. i and G j The topic coherence. Consistency(i,j) is calculated as shown in formula (17), where α and β are the importance parameters of MC(i,j) and CC(i,j) respectively. The larger the absolute value of α, the higher the discrimination of MC(i,j) in the topic coherence judgment of the sentence text entity relationship graph. Similarly, β is calculated as follows. The larger the Consistency(i,j), the higher the discrimination of the sentence text entity relationship graph G. i and G j The more coherent the themes between them.

[0145] Consistency(i,j)=α×MC(i,j)+β×CC(i,j) (17)

[0146] The following examples illustrate the sentence-text entity graph merging algorithm in detail:

[0147] Utilizing topic coherence, this invention designs a sentence-text entity graph merging algorithm as shown below. The algorithm's input is an array of sentence-text entity graphs (Array); a function `isMerge` determines whether adjacent sentence-text entity graphs can be merged; a topic coherence threshold `σ` is set for merging sentence-text entity graphs; adjacent sentence-text entity graphs with a topic coherence greater than `σ` can be merged; and a function `merge` merges adjacent sentence-text entity graphs. The algorithm returns an array of sentence-text entity graphs (ArrayMerged) of the merged text segment. The sentence-text entity graph merging algorithm calculates the topic coherence of adjacent sentence-text entity graphs. If the topic coherence is greater than the set topic coherence threshold `σ`, they are merged. This process iterates until no mergingable adjacent sentence-text entity graphs remain in the sentence-text entity graph array. At this point, `ArrayMerged` represents the text topic segmentation result of the text document.

[0148]

[0149] As attached Figure 14 As shown, after the sentence text entity relationship graph merging algorithm is completed, the return value is the merged sentence text entity relationship graph array ArrayMerged. Each sentence entity relationship graph in the array has a different theme in its topic set.

[0150] This invention presents a text topic segmentation method based on knowledge graphs, which can be trained on small-scale datasets to obtain a high-precision text topic segmentation algorithm model, achieving the technical effects of strong algorithm generalization ability and interpretability.

[0151] The following is based on Figure 15 To describe another embodiment of the present invention in detail, Figure 15 This is a schematic diagram of a knowledge graph-based text topic segmentation device according to an embodiment of this application;

[0152] A knowledge graph-based text topic segmentation device includes:

[0153] The acquisition module is used to acquire text documents;

[0154] The entity linking module is used to identify text entities in the text document using an entity recognition algorithm, wherein the entity recognition algorithm includes a first recognition algorithm and a second recognition algorithm, and the first recognition algorithm and the second recognition algorithm are different recognition algorithms; and to link the text entities to knowledge graph entities, wherein the knowledge graph entities represent entities in the knowledge graph that correspond to the text entities;

[0155] The text topic segmentation module is used to perform topic segmentation on the text document based on the link results between the text entities and the knowledge graph entities. The text topic segmentation on the text document includes: using sentences as text topic segmentation units, calculating the semantic similarity between sentences based on the text entities in the sentences, and performing text topic segmentation based on the semantic similarity.

[0156] The knowledge graph-based text topic segmentation device in this embodiment can achieve all the knowledge graph-based text topic segmentation methods in the above embodiments, and can also achieve all the technical effects of the knowledge graph-based text topic segmentation methods, which will not be repeated here.

[0157] It should be noted that the methods of one or more embodiments of the present invention can be executed by a single device, such as a computer or server. The methods of this embodiment can also be applied in a distributed scenario, where multiple devices cooperate to complete the process. In such a distributed scenario, one of these devices may execute only one or more steps of the methods of one or more embodiments of the present invention, and the multiple devices will interact with each other to complete the method described.

[0158] It should be noted that the above description describes specific embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0159] Based on the same inventive concept, and corresponding to any of the above-described embodiments, the present invention also discloses an electronic device;

[0160] Specifically, Figure 15 This diagram illustrates the hardware structure of an electronic device using a knowledge graph-based text topic segmentation method provided in this embodiment. The device may include a processor 410, a memory 420, an input / output interface 430, a communication interface 440, and a bus 450. The processor 410, memory 420, input / output interface 430, and communication interface 440 are interconnected internally via the bus 450.

[0161] The processor 410 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0162] The memory 420 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 420 can store the operating system and other applications. When the technical solutions provided in the embodiments of the present invention are implemented by software or firmware, the relevant program code is stored in the memory 420 and is called and executed by the processor 410.

[0163] Input / output interface 430 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0164] The communication interface 440 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (e.g., USB, Ethernet cable, etc.) or wireless means (e.g., mobile network, Wi-Fi, Bluetooth, etc.).

[0165] Bus 450 includes a pathway for transmitting information between various components of the device (e.g., processor 410, memory 420, input / output interface 430, and communication interface 440).

[0166] It should be noted that although the above-described device only shows the processor 410, memory 420, input / output interface 430, communication interface 440, and bus 450, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of the present invention, and not necessarily all the components shown in the figures.

[0167] The electronic devices described above are used to implement the corresponding knowledge graph-based text topic segmentation methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0168] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, one or more embodiments of the present invention also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the knowledge graph-based text topic segmentation method as described in any of the above embodiments.

[0169] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0170] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the knowledge graph-based text topic segmentation method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0171] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the invention as described above, which are not provided in detail for the sake of brevity.

[0172] Additionally, to simplify the description and discussion, and to avoid obscuring one or more embodiments of the invention, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring one or more embodiments of the invention, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which one or more embodiments of the invention will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuitry) are set forth to describe exemplary embodiments of the invention, it will be apparent to those skilled in the art that one or more embodiments of the invention may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0173] Although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0174] One or more embodiments of the present invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of the present invention should be included within the protection scope of the present invention.

Claims

1. A text topic segmentation method based on knowledge graphs, characterized in that, The method includes: Get the text document; The text entities in the text document are identified using an entity recognition algorithm, which includes a first recognition algorithm and a second recognition algorithm, and the first recognition algorithm and the second recognition algorithm are different recognition algorithms; the text entities are linked to knowledge graph entities, wherein the knowledge graph entities represent entities in the knowledge graph that correspond to the text entities; Based on the linking results between the text entities and the knowledge graph entities, the text document is segmented into topics. This text topic segmentation includes: using sentences as the text topic segmentation unit, calculating the semantic similarity between sentences based on the text entities within those sentences, and performing the text topic segmentation based on the semantic similarity, including: Using sentences as the unit for text topic segmentation, the text topic is divided into a set of text segments S. For each text segment s in the set of text segments S, a sentence text entity relation graph G is constructed. Each node in the sentence text entity relationship graph G represents a knowledge graph entity in the knowledge graph. Each node includes frequency information and attribute information. The frequency information indicates the number of times the knowledge graph entity is mentioned in the text document. The edges in the sentence text entity relationship graph G represent the relationship between two nodes, and the value of the relationship is "Know". re (u, v) and satisfy the following formula: Where u and u represent different nodes, E represents the set of all relations in the knowledge graph, e(u, v) indicates that there is a relation e between u and v, KGReMap(u, v) indicates the similarity when there is a relation e between u and v, and PathRe(u, v) indicates the similarity when there is no relation e between u and v. Also includes: Consistency(i,j) is used to represent the entity relation graph G of the sentence text. i and G j The formula for calculating ConSistency(i,j) is as follows: Consistency(i,j)=α×MC(i,j)+β×CC(i,j), Where MC(i,j) represents the word distribution dissimilarity, CC(i,j) represents the topic cohesion loss, α and β are the importance parameters of MC(i,j) and CC(i,j) respectively, i represents the i-th sentence, and j represents the j-th sentence; Also includes: Set a topic coherence threshold σ for the sentence text entity relationship graph G, and merge adjacent sentence text entity relationship graphs whose topic coherence is greater than the topic coherence threshold σ; The text topic segmentation result is obtained when there are no adjacent text entity relationship graphs that can be merged in the sentence text entity relationship graph array.

2. The text topic segmentation method based on knowledge graphs as described in claim 1, characterized in that, The step of identifying text entities in the text document using an entity recognition algorithm includes: A first recognition algorithm is used to identify a first type of text entity in the text document, wherein the first recognition algorithm is a Trie tree recognition algorithm, and the first type of text entity represents an entity that already exists in the knowledge graph; A second recognition algorithm is used to identify a second type of text entity in the text document, wherein the second recognition algorithm is the Bert-BiLSTM-CRF recognition algorithm, and the second type of text entity represents an entity that does not exist in the knowledge graph.

3. The text topic segmentation method based on knowledge graphs as described in claim 1, characterized in that, The step of linking the text entity to the knowledge graph entity includes: Based on the Bi-encoder candidate entity recall algorithm, N knowledge graph entities corresponding to the text entity are matched from the knowledge graph, where N is a positive integer; Based on the Cross-encoder candidate entity disambiguation algorithm, the candidate entity with the highest similarity score to the text entity is determined from the N knowledge graph entities.

4. The text topic segmentation method based on knowledge graphs as described in claim 2, characterized in that, The Bert-BiLSTM-CRF recognition algorithm includes: We selected Bert as the embedding layer model, trained the Bert-base-chinese pre-trained model, and obtained the embedding vector output by the Bert embedding layer. The embedding vector is input into a bidirectional long short-term memory network to obtain the feature vector output by the bidirectional long short-term memory network; The feature vector is input into a conditional random field, and the transition probabilities between sequence labels are learned using the conditional random field. By combining the global information of the sequence labels, the feature vector is decoded into sequence labels.

5. The text topic segmentation method based on knowledge graphs as described in claim 3, characterized in that, The method further includes: The embedding vectors of the text entities are trained using a Siamese network, which includes classification and regression tasks. The classification task satisfies the following formula: o=softmax(W t (u1, v1, |u1-v1|)) Where o represents the output vector, u1 and v1 represent the embedding vector, and W t This represents the multiplication of a vector by a parameter matrix. n is the dimension of the embedding vector, and k is the number of classification labels.

6. A knowledge graph-based text topic segmentation apparatus for implementing the knowledge graph-based text topic segmentation method according to any one of claims 1 to 5, characterized in that, The device includes: The acquisition module is used to acquire text documents; The entity linking module is used to identify text entities in the text document using an entity recognition algorithm, wherein the entity recognition algorithm includes a first recognition algorithm and a second recognition algorithm, and the first recognition algorithm and the second recognition algorithm are different recognition algorithms; and to link the text entities to knowledge graph entities, wherein the knowledge graph entities represent entities in the knowledge graph that correspond to the text entities; The text topic segmentation module is used to perform topic segmentation on the text document based on the link results between the text entities and the knowledge graph entities. The text topic segmentation on the text document includes: using sentences as text topic segmentation units, calculating the semantic similarity between sentences based on the text entities in the sentences, and performing text topic segmentation based on the semantic similarity.

7. An electronic device, characterized in that, The invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the program, implements the method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method as described in any one of claims 1 to 5.

Images