Target text distribution mining method and device

Abstract

The present disclosure provides a target text distribution mining method and device, electronic equipment and readable storage medium, relating to the technical fields of artificial intelligence such as deep learning, natural language processing, cloud service and big data. The target text distribution mining method comprises: obtaining a document set, extracting at least one target text from the document set; taking the documents in the document set and the at least one target text as nodes, and taking the relationships between the documents and the relationships between the documents and the target text as edges, to construct a heterogeneous graph; obtaining a document attention matrix of a target document according to the heterogeneous graph, wherein each element in the document attention matrix represents the importance of other documents to the target document; and obtaining a final target text distribution of the target document according to the document attention matrix and an initial target text distribution of the target document, wherein the target text distribution contains different target texts and the weights of the target texts.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, particularly to artificial intelligence technologies such as deep learning, natural language processing, cloud services, and big data. It provides a method, apparatus, electronic device, and readable storage medium for mining target text distributions. Background Technology

[0002] With the development of scientific research, new technologies are constantly emerging, and traditional technologies are also constantly iterating. Automatically mining technological trends from massive amounts of scientific literature is an important task in the field of Science of Science, and it has attracted widespread attention from all sectors of society.

[0003] While existing methods can initially achieve the goal of identifying technology trends, many key issues still need to be addressed, such as the sparsity of technical terms, cross-document impact, and time dependence. Summary of the Invention

[0004] According to a first aspect of this disclosure, a method for mining target text distribution is provided, comprising: acquiring a document set; extracting at least one target text from the document set; constructing a heterogeneous graph by using the documents in the document set and the at least one target text as nodes, and using the relationships between documents and the relationships between documents and target text as edges; obtaining a document attention matrix of the target document based on the heterogeneous graph, wherein each element in the document attention matrix represents the importance of other documents to the target document; and obtaining a final target text distribution of the target document based on the document attention matrix and the initial target text distribution of the target document, wherein the target text distribution includes different target texts and the weights of the target texts.

[0005] According to a second aspect of this disclosure, an apparatus for mining target text distribution is provided, comprising: an acquisition unit for acquiring a document set and extracting at least one target text from the document set; a construction unit for constructing a heterogeneous graph by using the documents in the document set and the at least one target text as nodes, and the relationships between documents and between documents and target text as edges; a processing unit for obtaining a document attention matrix of the target document based on the heterogeneous graph, wherein each element in the document attention matrix represents the importance of other documents to the target document; and a mining unit for obtaining a final target text distribution of the target document based on the document attention matrix and an initial target text distribution of the target document, wherein the target text distribution includes different target texts and the weights of the target texts.

[0006] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0007] According to a fifth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described above.

[0008] According to a sixth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described above.

[0009] As can be seen from the above technical solutions, this disclosure improves the accuracy of the obtained document attention matrix by constructing a heterogeneous graph with documents and target text as nodes and the relationships between documents and between documents and target text as edges. Furthermore, by updating the initial target text distribution based on the document attention matrix, the dependencies between documents are fully utilized, thereby improving the accuracy of the final target text distribution.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0013] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0014] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0015] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0016] Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0017] Figure 6 This is a block diagram of an electronic device used to implement the target text distribution mining method of the embodiments of this disclosure. Detailed Implementation

[0018] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and mechanisms are omitted in the following description.

[0019] Figure 1 This is a schematic diagram based on the first embodiment of this disclosure. (See diagram below.) Figure 1 As shown, the target text distribution mining method in this embodiment specifically includes the following steps:

[0020] S101. Obtain a document set and extract at least one target text from the document set;

[0021] S102. Using the documents in the document set and the at least one target text as nodes, and the relationships between documents and the relationships between documents and target text as edges, construct a heterogeneous graph.

[0022] S103. Based on the heterogeneous graph, obtain the document attention matrix of the target document, where each element in the document attention matrix represents the importance of other documents to the target document;

[0023] S104. Based on the document attention matrix and the initial target text distribution of the target document, the final target text distribution of the target document is obtained, wherein the target text distribution includes different target texts and the weights of the target texts.

[0024] The target text distribution mining method in this embodiment first constructs a heterogeneous graph based on the documents and target text in the document set after extracting at least one target text from the document set. Then, it obtains a document attention matrix based on the heterogeneous graph. Finally, it obtains the final target text distribution based on the document attention matrix and the initial target text distribution of the target documents. This embodiment improves the accuracy of the obtained document attention matrix by constructing a heterogeneous graph with documents and target text as nodes and the relationships between documents and between documents and target text as edges. Furthermore, updating the initial target text distribution based on the document attention matrix fully utilizes the dependencies between documents, thereby improving the accuracy of the final target text distribution.

[0025] The document set obtained by executing S101 in this embodiment contains multiple documents. Different documents in the document set may have the same publication year or different publication years.

[0026] Specifically, in this embodiment, when performing S101 to extract at least one target text from the document collection, the optional implementation method may be as follows: obtain the title and / or summary of the documents in the document collection; extract the target entity from the obtained title and / or summary as at least one target text. In this embodiment, the target entity may be a word or phrase in the title and / or summary used to describe the method, task and dataset; wherein, in this embodiment, a BERT-based named entity recognition model may be used to extract the target text.

[0027] In other words, this embodiment extracts target text by using the document's title and / or summary. The target text is technical terminology. Since the words or phrases contained in the title and / or summary are usually highly relevant to the entire document, this embodiment can improve the accuracy of target text extraction. Furthermore, since the title and / or summary are usually short, this embodiment can also improve the efficiency of target text extraction.

[0028] For example, if the title of a document is “Bi-Directional Attention Flow for Machine Comprehension”, where “Bi-Directional Attention Flow” is a phrase describing the method (i.e., bidirectional attention flow) and “Machine Comprehension” is a phrase describing the task (i.e., machine understanding), in this embodiment, when executing S101, “Bi-Directional Attention Flow” and “Machine Comprehension” can be extracted as target text.

[0029] In this embodiment, after executing S101 to extract at least one target text, S102 is executed to construct a heterogeneous graph by using the documents in the document set and at least one target text as nodes, and the relationships between documents and the relationships between documents and target text as edges.

[0030] In this embodiment, the heterogeneous graph constructed by executing S102 contains multiple nodes and edges between multiple nodes; the nodes in the heterogeneous graph represent documents or target text, and the edges in the heterogeneous graph represent the relationships between documents or the relationships between documents and target text.

[0031] The heterogeneous graph constructed in S102 of this embodiment contains four types of edges: appear edges, has edges, citedBy edges, and self edges.

[0032] Among them, occurrence edges and having edges are used to reflect the relationship between a document and a target text. A occurrence edge indicates that the target text appears in the document, and a having edge is the opposite edge of the occurrence edge, indicating that the document contains the target text. Reference edges are used to reflect the relationship between documents, indicating that one document is referenced by another document. Self-edges are used to represent self-links of documents.

[0033] For example, if the document set obtained by executing S101 in this embodiment includes document 1 and document 2, and document 1 references document 2, then target text 1 can be extracted from document 1; when executing S102 to construct the heterogeneous graph in this embodiment, document 1, document 2 and target text 1 can be used as nodes to establish reference edges between document 1 and document 2, self-edges of document 1, self-edges of document 2, edges between document 1 and target text 1, and occurrence edges.

[0034] In this embodiment, when constructing the heterogeneous graph in step S102, the node features of the nodes in the heterogeneous graph can also be initialized.

[0035] Specifically, in this embodiment, when executing S102, the encoding vector corresponding to the document (for example, using the BERT model to encode the text content of the document into a 768-dimensional vector) can be obtained as the original node feature of the document node; the encoding vector corresponding to the target text (for example, using the Word2vec model to encode the text content of the target text into a 300-dimensional vector) can be obtained as the original node feature of the target text node.

[0036] In other words, this embodiment improves the accuracy of heterogeneous graphs by setting two types of nodes and four types of edges between nodes, so that the content contained in the heterogeneous graph can more completely reflect the relationships between documents and between documents and target text.

[0037] In this embodiment, after executing S102 to construct the heterogeneous graph, S103 is executed to obtain the document attention matrix of the target document based on the heterogeneous graph.

[0038] In this embodiment, when executing S103 to obtain the document attention matrix of the target document based on the heterogeneous graph, an optional implementation method can be adopted as follows: determine the neighboring document nodes of the target document node based on the heterogeneous graph, where the target document node is the node in the heterogeneous graph corresponding to the target document, and the neighboring document nodes are the nodes in the heterogeneous graph corresponding to documents that have a reference relationship with the target document; obtain the importance of the neighboring document nodes to the target document node based on the original node features of the target document node, the original node features of the neighboring document nodes, and the transformation matrix corresponding to the reference edges, where the importance can be a score, and the higher the score, the more important the current document is to the target document; obtain the document attention matrix of the target document based on the documents corresponding to the neighboring document nodes and the importance of the neighboring document nodes to the target document node, where the document attention matrix includes the importance of other documents to the target document.

[0039] In other words, this embodiment obtains the importance of neighboring document nodes adjacent to the target document node to the target document node based on the relationship between documents in the heterogeneous graph, and then obtains the document attention matrix that includes the importance of other documents to the target document based on the importance between the nodes. This embodiment can obtain a more accurate document attention matrix through the heterogeneous graph.

[0040] In this embodiment, when executing S103 to obtain the importance of neighboring document nodes to the target document node based on the original node features of the target document node, the original node features of neighboring document nodes, and the transformation matrix corresponding to the reference edge, the following may also be included: using the transformation matrix corresponding to the document node to transform the original node features of the target document node and the original node features of neighboring document nodes; and obtaining the importance of neighboring document nodes to the target document node based on the transformed node features and the transformation matrix corresponding to the reference edge.

[0041] In other words, this embodiment can also transform the original node features of the document node according to the transformation matrix corresponding to the document node before obtaining the document attention matrix, so that the characteristics of the document node can be mapped to a shared latent space, thereby improving the accuracy of the obtained importance.

[0042] In this embodiment, when executing S103, the node feature transformation can be performed using calculation formula (1):

[0043]

[0044] In the calculation formula (1): M represents the node features obtained after the transformation. φ This represents the transformation matrix corresponding to the document node; Represents the original node characteristics of a document node.

[0045] In this embodiment, when executing S103, the importance of adjacent document nodes to the target document node can be obtained using calculation formulas (2) and (3):

[0046]

[0047]

[0048] In formulas (2) and (3): x i Represents the target document node, x j Indicates neighboring document nodes; Represents the transformation matrix corresponding to the referenced edge; || represents the concatenation operation; This indicates the importance of neighboring document nodes to the target document node.

[0049] It is understood that this embodiment can repeat the calculation of the importance of neighboring document nodes to the target document node multiple times, and l in the above formula is the number of calculations.

[0050] In addition, this embodiment can also calculate the importance of the target text node to the document node and the importance of the document node itself to the document node according to the method provided in step S103, that is, w in the above calculation formula (2) and calculation formula (3) τ It can be transformed into a transformation matrix corresponding to the appearing edge, a transformation matrix corresponding to the edge having, and a transformation matrix corresponding to the self-edge.

[0051] In this embodiment, after obtaining the importance of neighboring nodes to the target node by executing S103, the node characteristics of the target node can be updated according to the transformation matrix corresponding to different edges, the node characteristics of neighboring nodes, and the importance of neighboring nodes to the target node.

[0052] In other words, when executing S103 in this embodiment, in addition to obtaining the document attention matrix of the target document based on the constructed heterogeneous graph, it can also output the final node features of the document nodes and target text nodes contained in the heterogeneous graph.

[0053] In this embodiment, the node features of the target node can be updated using calculation formula (4):

[0054]

[0055] In the calculation formula (4); The updated node characteristics of target node i are represented by τ; different types of edges between nodes are represented by w. τ This represents the transformation matrix corresponding to edges of different types; This indicates the importance of neighbor node j to target node i; This represents the node characteristics of neighboring nodes.

[0056] It is understood that, in this embodiment, when executing S103, the heterogeneous graph can also be input into a pre-trained heterogeneous graph neural network model. This heterogeneous graph neural network model contains multiple graph attention layers. Each graph attention layer is used to obtain the importance of the neighboring document node to the target document node based on the original node features of the target document node, the original node features of the neighboring document nodes, and the transformation matrix corresponding to the reference edge.

[0057] Similarly, in this embodiment, the heterogeneous graph neural network model can also update the node features of the nodes contained in the heterogeneous graph according to each graph attention layer, so as to obtain the final node features of the document node and the target text node in the heterogeneous graph after processing through multiple graph attention layers.

[0058] In this embodiment, the heterogeneous graph neural network model used in S103 can optimize the parameters (e.g., the transformation matrix corresponding to different types of edges) in the model according to the pre-set document classification task and publication year regression task.

[0059] In this embodiment, the loss function value corresponding to the document classification task can be calculated using calculation formulas (5) and (6):

[0060]

[0061]

[0062] In calculation formulas (5) and (6): This indicates the prediction result that document i belongs to topic cls; W represents the node features obtained from the last update of document i; cls This represents the transformation matrix corresponding to the topic cls; T represents the loss function value for the document classification task; ik The label indicates that document i belongs to topic cls. A label of 1 indicates that document i belongs to topic cls, and a label of 0 indicates that document i does not belong to topic cls.

[0063] In this embodiment, the loss function value of the regression task for the year of publication can be calculated using formulas (7) and (8):

[0064]

[0065]

[0066] In calculation formulas (7) and (8): W represents the predicted publication year of document i; reg This represents the transformation matrix corresponding to the year reg; This represents the node features obtained from the last update of document i; Y represents the loss function value of the regression task in the year of publication; i This indicates the actual year that document i was published.

[0067] In this embodiment, after executing S103 to obtain the document attention matrix of the target document, S104 is executed to obtain the final target text distribution of the target document based on the document attention matrix and the initial target text distribution of the target document. In this embodiment, the target text distribution obtained by executing S104 includes different target texts and the weights of the target texts. The greater the weight of the target text, the higher the popularity of the target document corresponding to that target text.

[0068] In this embodiment, when executing S104 to obtain the final target text distribution of the target document based on the document attention matrix and the initial target text distribution of the target document, the optional implementation method can be as follows: determine the reference document corresponding to the target document; obtain the initial target text distribution based on the target text extracted from the target document and the target text extracted from the reference document, wherein the weight of the target text in the initial target text distribution can be the number of times the target text appears in the document; and obtain the final target text distribution of the target document based on the document attention matrix and the initial target text distribution of the target document.

[0069] In other words, this embodiment adjusts the weights of each target text in the initial target text distribution based on the importance of other documents in the document attention matrix to the target document, so that the importance in the document attention matrix is ​​propagated to the target text distribution, making the weights of each target text in the final target text distribution more accurate.

[0070] In this embodiment, when executing S104, the final target text distribution can be obtained using calculation formulas (9) and (10):

[0071] B (0) =B

[0072]

[0073] In calculation formulas (9) and (10): B is the initial target text distribution; B l+1 Indicates the final target text distribution; This represents the transpose of the document attention matrix of the target document.

[0074] In this embodiment, after executing S104 to obtain the final target text distribution of the target document, the target text with a weight exceeding a preset weight threshold can be selected based on the weight of each target text in the final target text distribution and fed back to relevant personnel.

[0075] This embodiment can also obtain the final target text distribution of multiple target documents through the above method, and then integrate the same target texts according to the target texts contained in each final target text distribution and their weights, so as to obtain the final target text distribution corresponding to multiple documents.

[0076] Figure 2 This is a schematic diagram according to the second embodiment of this disclosure. (See diagram below.) Figure 2 As shown, after executing S104 "obtaining the final target text distribution of the target document", this embodiment may further include the following:

[0077] S201. Based on the final target text distribution of the target document, the final node features of the document nodes corresponding to the target document, and the time latent variable, obtain the enhanced features of the target document.

[0078] S202. Based on the enhanced features, the first topic distribution is obtained;

[0079] S203. Based on the final node characteristics of the target text node corresponding to the target text and the topic characteristics of the target topic at the target time, the second topic distribution is obtained;

[0080] S204. Based on the first topic distribution and the second topic distribution, obtain the dynamic target text distribution of the target document corresponding to the target topic at the target time. The dynamic target text distribution includes different target texts and the weights of the target texts corresponding to the target topic at the target time.

[0081] In other words, this embodiment can also achieve dynamic modeling of the target text distribution, so that the obtained dynamic target text distribution can reflect the weight of the target text corresponding to the target topic at the target time, thereby improving the correlation between the weight of the target text and the time and topic in the target text distribution. Moreover, the weight contained in the dynamic target text distribution can also reflect the popularity of the target topic corresponding to the target text at the target time. The greater the weight, the higher the popularity.

[0082] In this embodiment, when executing S201, the features of the document nodes output by the graph attention layer of the last layer in the heterogeneous graph neural network model can be used as the final node features of the document nodes corresponding to the target document.

[0083] In this embodiment, when executing S201, the time hidden variable can be obtained using calculation formulas (11) and (12):

[0084]

[0085]

[0086] In calculation formulas (11) and (12): This represents the set of target text distributions for all documents in year yi; This represents the average distribution of target text across all documents in year 1. This represents the time-latent variable obtained by encoding the average distribution of target texts in different publication years.

[0087] In this embodiment, when executing S201, the enhanced features of the target document can be obtained using calculation formula (13):

[0088]

[0089] In the calculation formula (13): d i Indicates the enhanced features of the target document; b i This indicates the final target text distribution of the target document; The final node feature representing the document node corresponding to the target document; This represents a latent time variable.

[0090] In this embodiment, when executing S202 to obtain the first topic distribution based on the enhanced features of the target document, the calculation formulas (14) to (17) can be used to obtain the following:

[0091] μ i =MLP μ (d i )

[0092] σ i =MLP σ (d i )

[0093]

[0094] θ i =softmax(z i )

[0095] In calculation formulas (14) to (17): μ i σ represents the mean of the enhanced features of document i. i The variance of the enhanced features of document i is represented by ε; ε follows a normal distribution N(0,1); θ iLet be the first topic distribution, which represents the topic distribution of document i in year yi.

[0096] In this embodiment, when executing S203, the second topic distribution can be obtained using calculation formula (18):

[0097]

[0098] In the calculation formula (18): ρ represents the distribution of target text for topic k in year yi; ρ is the final node feature of the target text node corresponding to the target text. This represents the thematic characteristics of theme k in year yi.

[0099] In this embodiment, when executing S204 to obtain the dynamic target text distribution of the target document corresponding to the target topic at the target time based on the first topic distribution and the second topic distribution, the calculation formula (19) can be used to obtain it:

[0100]

[0101] In the calculation formula (19): Indicates the dynamic distribution of target text; This represents the transpose of the first topic distribution; This indicates the distribution of the second theme.

[0102] It is understood that, in this embodiment, when S202 is executed, the enhanced features of the target document can be input into the encoder in the topic modeling model, and the encoder can obtain the first topic distribution based on the enhanced features.

[0103] In this embodiment, when executing S204, the first topic distribution and the second topic distribution can be input into the decoder in the topic modeling model. The decoder then obtains the dynamic target text distribution of the target document based on the first topic distribution and the second topic distribution.

[0104] Similarly, the topic modeling model used in this embodiment can obtain the overall loss function based on the pre-set reconstruction loss function, smoothing constraint loss function, and classification cross-entropy loss function, and optimize the model's parameters.

[0105] In this embodiment, when optimizing the parameters in the topic modeling model, the overall loss function used can be obtained using the calculation formula (20):

[0106] loss = loss rec +w α loss α +w β loss β +w η lossη +w cls loss cls

[0107] In the calculation formula (20): loss represents the overall loss function; loss rec Represents the reconstruction loss function; loss α Loss function representing topic features; β The loss function representing the second topic distribution; loss η The loss function representing the time-dependent latent variables; loss cls Represents the classification cross-entropy loss function; w α w β w η and w cls This is a hyperparameter.

[0108] Among them, the reconstruction loss function is loss rec It can be obtained using the calculation formula (21):

[0109]

[0110] In the calculation formula (21): b i This represents the final target text distribution for document i; This represents the dynamic target text distribution of document i.

[0111] loss function for topic features α It can be obtained using the calculation formula (22):

[0112]

[0113] In the formula: This indicates the thematic characteristics of the theme in the first year; This indicates the thematic characteristics of the theme in year yi+1.

[0114] The loss function lossβ of the second topic distribution can be obtained using formula (23):

[0115]

[0116] In the calculation formula (23): This indicates the distribution of the second theme in the first year; This represents the second theme distribution in year yi+1.

[0117] loss function for time-latent variables η It can be obtained using formula (24):

[0118]

[0119] In the calculation formula (24): This represents a latent variable representing time in the first year; This represents the latent variable in year yi+1.

[0120] In this embodiment, the first topic distribution can be considered as the distribution of target topics of the target document at the target time. The first topic distribution includes different target topics and the weights of the target topics. The second topic distribution can be considered as the distribution of target text of the target topics at the target time. The second topic distribution includes different target text and the weights of the target text.

[0121] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure. Figure 3 The diagram illustrates the structure for mining the target text distribution in this embodiment: First, a heterogeneous graph is constructed based on the target text and the text in the document set. The nodes in the heterogeneous graph are then processed using a heterogeneous graph neural network model to obtain the final node features of document nodes, the final node features of target text nodes, and the document attention matrix. Then, based on the document attention matrix and the initial target text distribution of the target documents, the final target text distribution is obtained. The final node features of the target documents, the final target text distribution, and the temporal latent variable obtained from the average target text distribution for each publication year are input into a topic modeling model composed of an encoder and a decoder. This allows the encoder to output the mean and variance of the enhanced features of the target documents, thus obtaining the first topic distribution. Finally, the first topic distribution and the second topic distribution obtained based on the target topic are input into the decoder to obtain the dynamic target text distribution output by the decoder.

[0122] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure. Figure 4 The diagram shows a schematic of the heterogeneous graph constructed in this embodiment: the heterogeneous graph includes a target document, a cited document, and a target text, wherein there is a citedBy edge between the target document and the cited document, a self edge between the target document itself, and an appear edge and a has edge between the target document and the target text.

[0123] Figure 5 This is a schematic diagram according to the fifth embodiment of this disclosure. (See diagram below.) Figure 5 As shown, the target text distribution mining device 500 of this embodiment includes:

[0124] Acquisition unit 501 is used to acquire a document set and extract at least one target text from the document set;

[0125] Construction unit 502 is used to construct a heterogeneous graph by taking the documents in the document set and the at least one target text as nodes, and taking the relationships between documents and the relationships between documents and target text as edges;

[0126] Processing unit 503 is used to obtain a document attention matrix of the target document based on the heterogeneity graph, wherein each element in the document attention matrix represents the importance of other documents to the target document;

[0127] Mining unit 504 is used to obtain the final target text distribution of the target document based on the document attention matrix and the initial target text distribution of the target document, wherein the target text distribution includes different target texts and the weights of the target texts.

[0128] The document collection acquired by the acquisition unit 501 contains multiple documents. Different documents in the document collection may have the same publication year or different publication years.

[0129] Specifically, when the acquisition unit 501 extracts at least one target text from the document collection, the optional implementation method may be: acquiring the title and / or summary of the documents in the document collection; extracting the target entity from the acquired title and / or summary as at least one target text.

[0130] In other words, the acquisition unit 501 extracts the target text through the title and / or summary of the document. Since the words or phrases contained in the title and / or summary are usually highly relevant to the whole document, this embodiment can improve the accuracy of target text extraction. Moreover, since the title and / or summary are usually short, this embodiment can also improve the efficiency of target text extraction.

[0131] In this embodiment, after the acquisition unit 501 extracts at least one target text, the construction unit 502 uses the documents in the document set and at least one target text as nodes, and the relationships between documents and the relationships between documents and target text as edges to construct a heterogeneous graph.

[0132] The heterogeneous graph constructed by building unit 502 contains multiple nodes and edges between multiple nodes; the nodes in the heterogeneous graph represent documents or target text, and the edges in the heterogeneous graph represent the relationships between documents or the relationships between documents and target text.

[0133] The heterogeneous graph constructed by building unit 502 contains four types of edges: appear edges, has edges, citedBy edges, and self edges.

[0134] When building a heterogeneous graph, the building unit 502 can also initialize the node features of the nodes in the heterogeneous graph.

[0135] Specifically, the construction unit 502 can obtain the encoding vector corresponding to the document (e.g., using the BERT model to encode the text content of the document into a 768-dimensional vector) as the original node feature of the document node; it can also obtain the encoding vector corresponding to the target text (e.g., using the Word2vec model to encode the text content of the target text into a 300-dimensional vector) as the original node feature of the target text node.

[0136] In other words, by setting two types of nodes and four types of edges between nodes in the heterogeneous graph, the building unit 502 can improve the accuracy of the heterogeneous graph, so that the content contained in the heterogeneous graph can more completely reflect the relationship between documents and between documents and target text.

[0137] In this embodiment, after the heterogeneous graph is constructed by the construction unit 502, the processing unit 503 obtains the document attention matrix of the target document based on the heterogeneous graph.

[0138] When processing unit 503 obtains the document attention matrix of the target document based on the heterogeneous graph, the optional implementation methods are as follows: determine the neighboring document nodes of the target document node based on the heterogeneous graph; obtain the importance of the neighboring document nodes to the target document node based on the original node features of the target document node, the original node features of the neighboring document nodes, and the transformation matrix corresponding to the reference edge; obtain the document attention matrix of the target document based on the document corresponding to the neighboring document node and the importance of the neighboring document nodes to the target document node, wherein the document attention matrix contains the importance of other documents to the target document.

[0139] In other words, the processing unit 503 obtains the importance of neighboring document nodes adjacent to the target document node to the target document node based on the relationship between documents in the heterogeneous graph, and then obtains the document attention matrix that includes the importance of other documents to the target document based on the importance between the nodes. In this embodiment, a more accurate document attention matrix can be obtained through the heterogeneous graph.

[0140] When processing unit 503 obtains the importance of neighboring document nodes to the target document node based on the original node features of the target document node, the original node features of neighboring document nodes, and the transformation matrix corresponding to the reference edge, it may also include the following: using the transformation matrix corresponding to the document node to transform the original node features of the target document node and the original node features of neighboring document nodes; obtaining the importance of neighboring document nodes to the target document node based on the transformed node features and the transformation matrix corresponding to the reference edge.

[0141] In other words, before obtaining the document attention matrix, the processing unit 503 can also transform the original node features of the document node according to the transformation matrix corresponding to the document node, so that the characteristics of the document node can be mapped into a shared latent space, thereby improving the accuracy of the obtained importance.

[0142] Understandably, the processing unit 503 can repeatedly calculate the importance of neighboring document nodes to the target document node.

[0143] In addition, the processing unit 503 can also calculate the importance of the target text node to the document node and the importance of the document node itself to the document node, which only requires transformation into the transformation matrix corresponding to the edge that appears, the transformation matrix corresponding to the edge that has it, and the transformation matrix corresponding to the self-edge.

[0144] After obtaining the importance of neighboring nodes to the target node, the processing unit 503 can also update the node characteristics of the target node based on the transformation matrix corresponding to different edges, the node characteristics of neighboring nodes, and the importance of neighboring nodes to the target node.

[0145] In other words, in addition to obtaining the document attention matrix of the target document, the processing unit 503 can also output the final node features of the document nodes and target text nodes contained in the heterogeneous graph based on the constructed heterogeneous graph.

[0146] It is understandable that the processing unit 503 can also input the heterogeneous graph into a pre-trained heterogeneous graph neural network model, which contains multiple graph attention layers. Each graph attention layer is used to obtain the importance of the neighboring document node to the target document node based on the original node features of the target document node, the original node features of the neighboring document nodes, and the transformation matrix corresponding to the reference edge.

[0147] Similarly, in this embodiment, the heterogeneous graph neural network model can also update the node features of the nodes contained in the heterogeneous graph according to each graph attention layer, so as to obtain the final node features of the document node and the target text node in the heterogeneous graph after processing through multiple graph attention layers.

[0148] In this embodiment, after the processing unit 503 obtains the document attention matrix of the target document, the mining unit 504 obtains the final target text distribution of the target document based on the document attention matrix and the initial target text distribution of the target document; wherein, the target text distribution obtained by the mining unit 504 includes different target texts and the weights of the target texts.

[0149] When the mining unit 504 obtains the final target text distribution of the target document based on the document attention matrix and the initial target text distribution of the target document, the optional implementation methods are as follows: determine the referenced documents corresponding to the target document; obtain the initial target text distribution based on the target text extracted from the target document and the target text extracted from the referenced documents, wherein the weight of the target text in the initial target text distribution can be the number of times the target text appears in the document; and obtain the final target text distribution of the target document based on the document attention matrix and the initial target text distribution of the target document.

[0150] In other words, the mining unit 504 adjusts the weights of each target text in the initial target text distribution based on the importance of other documents in the document attention matrix to the target document, so that the importance in the document attention matrix is ​​propagated to the target text distribution, making the weights of each target text in the final target text distribution more accurate.

[0151] After the mining unit 504 obtains the final target text distribution of the target document, it can select target texts whose weights exceed a preset weight threshold based on the weights of each target text in the final target text distribution and feed them back to relevant personnel.

[0152] The mining unit 504 can also obtain the final target text distribution of multiple target documents, and then integrate the same target texts according to the target texts contained in each final target text distribution and their weights, so as to obtain the final target text distribution corresponding to multiple documents.

[0153] The target text distribution mining device 500 of this embodiment may further include an update unit 505, which is used to perform the following: obtaining enhanced features of the target document based on the final target text distribution of the target document, the final node features of the document nodes corresponding to the target document, and the time latent variable; obtaining a first topic distribution based on the enhanced features; obtaining a second topic distribution based on the final node features of the target text nodes corresponding to the target text and the topic features of the target topic at the target time; and obtaining a dynamic target text distribution of the target document corresponding to the target topic at the target time based on the first topic distribution and the second topic distribution, wherein the dynamic target text distribution includes different target texts and the weights of the target texts corresponding to the target topics at the target time.

[0154] In other words, the update unit 505 can also realize dynamic modeling of the target text distribution, so that the obtained dynamic target text distribution can reflect the weight of the target text corresponding to the target topic at the target time, thereby improving the correlation between the weight of the target text and the time and topic in the target text distribution.

[0155] The update unit 505 can use the features of the document nodes output by the graph attention layer of the last layer in the heterogeneous graph neural network model as the final node features of the document nodes corresponding to the target document.

[0156] Understandably, the update unit 505 can input the enhanced features of the target document into the encoder in the topic modeling model, and the encoder can obtain the first topic distribution based on the enhanced features.

[0157] The update unit 505 can input the first topic distribution and the second topic distribution into the decoder in the topic modeling model, and the decoder can obtain the dynamic target text distribution of the target document based on the first topic distribution and the second topic distribution.

[0158] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0159] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0160] like Figure 6 The diagram shown is a block diagram of an electronic device for a method of mining target text distribution according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0161] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0162] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0163] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as methods for mining target text distributions. For example, in some embodiments, the methods for mining target text distributions may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 608.

[0164] In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by computing unit 601, one or more steps of the target text distribution mining method described above may be performed. Alternatively, in other embodiments, computing unit 601 may be configured to perform the target text distribution mining method by any other suitable means (e.g., by means of firmware).

[0165] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0166] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other programmable target text distribution mining device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0167] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0168] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for showing information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0169] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0170] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the management difficulties and weak business scalability inherent in traditional physical hosts and VPS (Virtual Private Server) services. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0171] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0172] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for mining a target text distribution, comprising: obtaining a document set, and extracting at least one target text from the document set; constructing a heterogeneous graph by taking documents in the document set and the at least one target text as nodes, and taking relations between documents and relations between documents and target texts as edges; obtaining a document attention matrix of a target document according to the heterogeneous graph, wherein each element in the document attention matrix represents an importance degree of other documents to the target document; obtaining a final target text distribution of the target document according to the document attention matrix and an initial target text distribution of the target document, wherein the target text distribution contains different target texts and weights of the target texts; wherein the obtaining the final target text distribution according to the document attention matrix and the initial target text distribution of the target document comprises: determining a cited document corresponding to the target document; obtaining an initial target text distribution according to target texts extracted from the target document and target texts extracted from the cited document, wherein a weight of a target text in the initial target text distribution is a number of occurrences of the target text in a document; taking a multiplication result between a transposed matrix of the document attention matrix and the initial target text distribution as the final target text distribution.

2. The method of claim 1, wherein, The extracting the at least one target text from the document set comprises: obtaining titles and / or abstracts of documents in the document set; extracting target entities from the titles and / or abstracts as the at least one target text.

3. The method of claim 1, wherein, The constructing the heterogeneous graph comprises: obtaining an encoding vector corresponding to a node in the heterogeneous graph; taking the encoding vector as an original node feature of the node.

4. The method of claim 1, wherein, The obtaining the document attention matrix of the target document according to the heterogeneous graph comprises: determining a neighbor document node of a target document node according to the heterogeneous graph; obtaining an importance degree of the neighbor document node to the target document node according to an original node feature of the target document node, an original node feature of the neighbor document node, and a transformation matrix corresponding to a citation edge; obtaining the document attention matrix of the target document according to a document corresponding to the neighbor document node and the importance degree of the neighbor document node to the target document node.

5. The method of claim 4, wherein, The obtaining the importance degree of the neighbor document node to the target document node according to the original node feature of the target document node, the original node feature of the neighbor document node, and the transformation matrix corresponding to the citation edge comprises: transforming the original node feature of the target document node and the original node feature of the neighbor document node using a transformation matrix corresponding to a document node; obtaining the importance degree of the neighbor document node to the target document node according to the transformed node features and the transformation matrix corresponding to the citation edge.

6. The method of claim 5, further comprising, After obtaining the importance of the neighboring document nodes to the target document node, the node features of the target document node are updated based on the importance, the node features of the neighboring document nodes, and the transformation matrix corresponding to the reference edge.

7. The method of claim 1, wherein, The step of obtaining the document attention matrix of the target document based on the heterogeneous graph includes: The heterogeneous graph is input into a heterogeneous graph neural network model, and the document attention matrix of the target document is obtained based on the multiple graph attention layers contained in the heterogeneous graph neural network.

8. The method according to claim 1, further comprising: After obtaining the final target text distribution of the target document, the enhanced features of the target document are obtained based on the final target text distribution of the target document, the final node features of the document nodes corresponding to the target document, and the temporal latent variable. Based on the enhanced features, the first topic distribution is obtained; The second topic distribution is obtained based on the final node features of the target text node corresponding to the target text and the topic features of the target topic at the target time. Based on the first topic distribution and the second topic distribution, a dynamic target text distribution corresponding to the target topic of the target document at the target time is obtained. The dynamic target text distribution includes different target texts and the weights of the target texts corresponding to the target topic at the target time.

9. The method of claim 8, wherein, The process of obtaining the first topic distribution based on the enhanced features includes: The enhanced features are input into the encoder in the topic modeling model, and the output of the encoder is used as the first topic distribution.

10. The method of claim 8, wherein, The step of obtaining the dynamic target text distribution of the target document corresponding to the target topic at the target time based on the first topic distribution and the second topic distribution includes: The first topic distribution and the second topic distribution are input into the decoder in the topic modeling model, and the output of the decoder is used as the dynamic target text distribution of the target document corresponding to the target topic at the target time.

11. A target text distribution mining device, comprising: An acquisition unit is used to acquire a document set and extract at least one target text from the document set; A construction unit is used to construct a heterogeneous graph by taking the documents in the document set and the at least one target text as nodes, and taking the relationships between documents and the relationships between documents and target text as edges. The processing unit is configured to obtain a document attention matrix of the target document based on the heterogeneity graph, wherein each element in the document attention matrix represents the importance of other documents to the target document; The mining unit is used to obtain the final target text distribution of the target document based on the document attention matrix and the initial target text distribution of the target document, wherein the target text distribution includes different target texts and the weights of the target texts; Specifically, when the mining unit obtains the final target text distribution of the target document based on the document attention matrix and the initial target text distribution of the target document, it performs the following: Identify the referenced documents corresponding to the target document; An initial target text distribution is obtained based on the target text extracted from the target document and the target text extracted from the referenced document, wherein the weight of the target text in the initial target text distribution is the number of times the target text appears in the document; The result of multiplying the transpose of the document attention matrix with the initial target text distribution is taken as the final target text distribution.

12. The apparatus of claim 11, wherein, When the acquisition unit extracts at least one target text from the document set, it specifically performs the following: Obtain the titles and / or summaries of the documents in the document collection; Extract the target entity from the title and / or summary as the at least one target text.

13. The apparatus of claim 11, wherein, When constructing a heterogeneous graph, the building unit specifically performs the following: Obtain the encoding vector corresponding to the node in the heterogeneous graph; The encoded vector is used as the original node feature of the node.

14. The apparatus of claim 11, wherein, When the processing unit obtains the document attention matrix of the target document based on the heterogeneous graph, it specifically performs the following: Based on the heterogeneous graph, determine the neighboring document nodes of the target document node; Based on the original node features of the target document node, the original node features of the neighboring document nodes, and the transformation matrix corresponding to the reference edge, the importance of the neighboring document nodes to the target document node is obtained. Based on the documents corresponding to the neighboring document nodes and the importance of the neighboring document nodes to the target document node, the document attention matrix of the target document is obtained.

15. The apparatus of claim 14, wherein, When the processing unit determines the importance of the neighboring document nodes to the target document node based on the original node features of the target document node, the original node features of the neighboring document nodes, and the transformation matrix corresponding to the reference edge, it specifically performs the following: The original node features of the target document node and the original node features of the neighboring document nodes are transformed using the transformation matrix corresponding to the document node. Based on the node features obtained from the transformation and the transformation matrix corresponding to the reference edge, the importance of the neighboring document node to the target document node is obtained.

16. The apparatus of claim 15, wherein the processing unit is further configured to perform: After obtaining the importance of the neighboring document nodes to the target document node, the node features of the target document node are updated based on the importance, the node features of the neighboring document nodes, and the transformation matrix corresponding to the reference edge.

17. The apparatus of claim 11, wherein, When the processing unit obtains the document attention matrix of the target document based on the heterogeneous graph, it specifically performs the following: The heterogeneous graph is input into a heterogeneous graph neural network model, and the document attention matrix of the target document is obtained based on the multiple graph attention layers contained in the heterogeneous graph neural network.

18. The apparatus of claim 11, further comprising an updating unit for performing: After the mining unit obtains the final target text distribution of the target document, it obtains the enhanced features of the target document based on the final target text distribution of the target document, the final node features of the document nodes corresponding to the target document, and the temporal latent variables. Based on the enhanced features, the first topic distribution is obtained; The second topic distribution is obtained based on the final node features of the target text node corresponding to the target text and the topic features of the target topic at the target time. Based on the first topic distribution and the second topic distribution, a dynamic target text distribution corresponding to the target topic of the target document at the target time is obtained. The dynamic target text distribution includes different target texts and the weights of the target texts corresponding to the target topic at the target time.

19. The apparatus of claim 18, wherein, When the update unit obtains the first topic distribution based on the enhanced features, it specifically performs the following: The enhanced features are input into the encoder in the topic modeling model, and the output of the encoder is used as the first topic distribution.

20. The apparatus of claim 18, wherein, When the update unit obtains the dynamic target text distribution of the target document corresponding to the target topic at the target time based on the first topic distribution and the second topic distribution, it specifically performs the following: The first topic distribution and the second topic distribution are input into the decoder in the topic modeling model, and the output of the decoder is used as the dynamic target text distribution of the target document corresponding to the target topic at the target time.

21. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-10.

22. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-10.

23. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-10.

Images