A multi-document summary generation system and method based on aspect orientation

Abstract

The present invention provides a multi-document summary generation system and method based on aspect orientation, which relates to the field of natural language processing. Multi-document information is obtained through a graph encoder and a heterogeneous graph is constructed to generate central entity information; the first encoder uses a multi-head cross-attention mechanism at the entity-sentence level to screen according to the central entity information to generate central entity related information; the second encoder combines the central entity related information based on the aspect-sentence cross-attention mechanism to determine the target text context vector; the decoder combines the target text context vector by aspect detection to generate aspect distribution analysis results, and generates target summary information according to the aspect distribution analysis results. The aspect orientation of the summary is ensured by aspect detection and summary generation models, and a more comprehensive and accurate summary is generated, thereby improving the practicality of the summary, ensuring the fidelity, comprehensiveness and logic of the information, and meeting the user's needs for information from different aspects.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to an aspect-oriented multi-document summary generation system and method. Background Art

[0002] Text summarization technology refers to the natural language understanding and generation technology that generates a relatively short summary of the main content of the source text after reading the source text. The generated summary can compress the text content but retain the core information in the source text.

[0003] Aspect-based summarization aims to generate summaries from source documents that only contain content related to specific aspects. It is a technique that focuses on analyzing specific aspects in a document given a document and generates summaries that highlight these aspects. This method not only focuses on entities and their relationships in the document, but also explores key aspects or topics in the document in more depth, such as product features, event impacts, or policy outcomes.

[0004] Although multi-document summarization technology based on pre-trained language models has made significant progress and demonstrated strong summary generation capabilities, it still has obvious limitations in practical applications. Existing technologies mainly use a two-stage approach to generate aspect-oriented multi-document summaries, which aims to improve the overall quality of summaries by improving the effect of aspect recognition and integrating aspect information to filter the most relevant content in the source text. However, this approach often ignores the systematic analysis of central entities and does not fully utilize the potential of multi-dimensional analysis, resulting in content dispersion and information redundancy.

[0005] Therefore, how to effectively carry out multi-dimensional analysis and realize central entity recognition, domain adaptation and aspect classification in multiple documents has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] In order to effectively carry out multi-dimensional analysis and realize central entity recognition, domain adaptation and aspect classification in multiple documents, the present application provides an aspect-oriented multi-document summary generation system and method.

[0007] In the first aspect, the present application provides an aspect-oriented multi-document summary generation method using the following technical solution:

[0008] A multi-document summary generation system based on aspect orientation, comprising: a multi-dimensional analysis module and an aspect orientation summary generation module;

[0009] The multi-dimensional analysis module includes a graph encoder and a first encoder;

[0010] The aspect-oriented summary generation module includes a second encoder and a decoder;

[0011] The graph encoder is used to obtain multi-document information and construct a heterogeneous graph according to the multi-document information to generate central entity information;

[0012] The first encoder is used to adopt a multi-head cross attention mechanism at the entity-sentence level to filter according to the central entity information to generate central entity related information;

[0013] The second encoder is used to combine the central entity related information based on an aspect-sentence cross attention mechanism to determine a target text context vector;

[0014] The decoder is used to combine the target text context vector by aspect detection to generate an aspect distribution analysis result, and generate target summary information according to the aspect distribution analysis result.

[0015] Optionally, the graph encoder is further used to determine a sentence set and an entity set using Spacy when receiving multi-document information;

[0016] Encoding the sentence set using a weighted pooling operation to generate a text representation;

[0017] Encoding the entity set using a weighted pooling operation to generate an entity representation;

[0018] A heterogeneous graph is constructed according to the text representation and the entity representation to generate central entity information.

[0019] Optionally, the graph encoder is also used to construct a heterogeneous graph ;

[0020] Among them, V as a node on the heterogeneous graph includes sentence nodes and entity nodes , B represents the undirected edges between nodes, and the undirected edges include: entity-entity and entity-sentence edges;

[0021] For each sentence and each entity , combined with the co-occurrence matrix C, the elements of which Representing Entities In the sentence The number of occurrences in , using the frequency as the element The values ​​of construct the co-occurrence matrix;

[0022] Define the weight of the entity-sentence edge as the number of occurrences of the entity The score is calculated as:

[0023]

[0024] in Is Entity In the sentence The frequency in Is Entity The inverse document frequency of

[0025] Define the weight of the entity-entity edge. For any pair of entities in the entity-entity edge and , calculate the co-occurrence score, the co-occurrence score is:

[0026]

[0027] where m is the number of sentences, represents the indicator function, if the entity and If they appear in the sentence together, then the value is 1, otherwise the value is 0;

[0028] Iterate over all entities and sentences to calculate the co-occurrence score .

[0029] Optionally, the first encoder is further used to update the node representation in the heterogeneous graph using a graph attention network;

[0030] For each node in the heterogeneous graph, calculate the attention coefficient of the node connected to it:

[0031]

[0032] in, is a sentence node, is an entity node, is the attention coefficient, is the learnable attention mechanism parameter, W is the weight parameter matrix in training, is the set of neighbor nodes of node i, LeakyReLU is a nonlinear activation function, and the updated node representation is:

[0033]

[0034] in, represents the updated node representation, W represents the weight parameter matrix during training, ReLU activation function

[0035] Use multi-head attention mechanism to update features:

[0036]

[0037] Concatenate the multi-head attention mechanisms to get multi-head attention:

[0038]

[0039] in, Represents multi-headed attention, Represents the attention calculated by the nth head, each head has an independent and , concat represents the concatenation operation;

[0040] The central entity node is obtained by calculating the attention. The centrality of each node can be calculated by the weighted sum of the attention coefficients of its adjacent nodes. For entity node i, its centrality ;

[0041] in is the set of adjacent nodes of entity node i, is the attention coefficient between node i and node j;

[0042] Perform a normalization process, which includes a maximization process and a minimization process:

[0043]

[0044] in is the standardized value, and Central represents the centrality value of all entity nodes, thereby determining the central entity:

[0045]

[0046] in, The standardized centrality values ​​of all entity nodes are represented, and the node representation of the central entity is determined according to the centrality values.

[0047] Optionally, the first encoder is further used to classify the node representation of the central entity using MLP;

[0048] For the MLP layer:

[0049]

[0050] in is the output of the L-1th layer. For the first layer, , and are the weight matrix and bias vector of the Lth layer, It is a nonlinear activation function, using the ReLU activation function, and the output is:

[0051] in, is the output of the last hidden layer, and is the weight matrix and bias vector of the output layer;

[0052] The result of the MLP output layer is converted into a probability distribution using softmax and the field is determined based on the probability distribution.

[0053] Optionally, the multi-dimensional analysis module further includes domain-aspect discovery;

[0054] The domain-aspect prior matrix A is composed of preset domains and aspects corresponding to the preset domains;

[0055] Use BART's Tokenizer to encode characters, where the elements are only 0 and 1.

[0056] Optionally, the second encoder is further used to combine the text context representation of the first encoder Operate to generate the updated representation as :

[0057]

[0058] Using contextual representation As the value and key of attention, aspect information represents As a query;

[0059]

[0060] in, is the representation processed by the second encoder and is expressed as .

[0061] Optionally, the decoder is further configured to obtain a reference digest and combine a preset Token in the reference digest as input with the output of the second encoder. Perform the operation at time step t:

[0062]

[0063] The output of the decoder is , get the prediction result for the next position ;

[0064] According to the Compare with the true summary and perform loss calculation:

[0065]

[0066] Where T is the number of tokens in the target digest, is a given current decoder state Under the condition of , the predicted probability of the position.

[0067] Optionally, the decoder is further used to obtain a target text context vector output by the second encoder, and expand the target context vector according to a beam search strategy;

[0068]

[0069] Select the top-k words with the highest probability as candidates, and then update the score of each bundle;

[0070]

[0071] in, represents the cumulative score of the current bundle before selecting a new word, It means that at time step t, the probability of selecting a specific word from the probability distribution output by the decoder is calculated by the logarithmic function. This is the updated score. The k bundles with the highest scores are retained and the best bundle is selected as the generated sequence returned at the end.

[0072] In a second aspect, the present application provides an aspect-oriented multi-document summary generation method, the aspect-oriented multi-document summary generation method comprising:

[0073] The graph encoder obtains multi-document information and constructs a heterogeneous graph according to the multi-document information to generate central entity information;

[0074] The first encoder uses a multi-head cross-attention mechanism at the entity-sentence level to filter the central entity information to generate central entity related information;

[0075] The second encoder combines the central entity related information based on the aspect-sentence cross attention mechanism to determine the target text context vector;

[0076] The decoder combines the target text context vector by way of aspect detection to generate an aspect distribution analysis result, and generates target summary information according to the aspect distribution analysis result.

[0077] In summary, the present application includes the following beneficial technical effects:

[0078] This application obtains multi-document information through a graph encoder and constructs a heterogeneous graph to generate central entity information; the first encoder uses a multi-head cross-attention mechanism at the entity-sentence level to screen the central entity information to generate central entity related information; the second encoder combines the central entity related information based on the aspect-sentence cross-attention mechanism to determine the target text context vector; the decoder combines the target text context vector through aspect detection to generate aspect distribution analysis results, and generates target summary information based on the aspect distribution analysis results. The aspect detection and summary generation model ensure the aspect orientation of the summary, generate a more comprehensive and accurate summary, thereby improving the practicality of the summary, ensuring the fidelity, comprehensiveness and logic of the information, and meeting the user's needs for information from different aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a structural block diagram of the first embodiment of the aspect-oriented multi-document summary generation system of the present application;

[0080] Figure 2 This is a diagram of an aspect-oriented comprehensive multi-document summary generation framework based on multi-dimensional analysis of the first embodiment of the aspect-oriented multi-document summary generation system of the present application;

[0081] Figure 3 This is a structural diagram of a multi-dimensional analysis module of the first embodiment of the aspect-oriented multi-document summary generation system of the present application;

[0082] Figure 4 It is the aspect-oriented summary generation module of the first embodiment of the aspect-oriented multi-document summary generation system of the present application;

[0083] Figure 5 It is a flowchart of the first embodiment of the aspect-oriented multi-document summary generation method of the present application. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below through the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0085] Reference Figure 1 , Figure 1 This is a structural block diagram of the first embodiment of the aspect-oriented multi-document summary generation system of the present application.

[0086] like Figure 1 As shown, the aspect-oriented multi-document summary generation system proposed in the embodiment of the present application includes: a multi-dimensional analysis module 10 and an aspect-oriented summary generation module 20.

[0087] The multi-dimensional analysis module 10 includes a graph encoder 101 and a first encoder 102 .

[0088] The aspect-oriented summary generation module 20 includes a second encoder 201 and a decoder 202 .

[0089] The graph encoder 101 is used to obtain multi-document information and construct a heterogeneous graph based on the multi-document information to generate central entity information.

[0090] The first encoder 102 is used to adopt a multi-head cross-attention mechanism at the entity-sentence level to filter according to the central entity information to generate central entity related information.

[0091] The second encoder 201 is used to determine the target text context vector based on the aspect-sentence cross-attention mechanism and the central entity related information.

[0092] The decoder 202 is used to combine the target text context vector by aspect detection to generate an aspect distribution analysis result, and generate target summary information according to the aspect distribution analysis result.

[0093] It should be noted that aspect-based summarization aims to generate summaries from source documents that only contain content related to specific aspects. It is a technology that focuses on analyzing specific aspects (aspects) in a document given a document and generates summaries that highlight these aspects. This method does not only focus on entities and their relationships in the document, but also explores key aspects or topics in the document in more depth, such as product characteristics, the impact of events, or the results of policies. The purpose is to generate more accurate and goal-oriented summaries, with special emphasis on the factual fidelity of the aspects discussed in the original text and the accuracy of detailed expression. Its purpose is to generate aspect-related summaries, each of which contains appropriate aspect-related information. This focused summary corresponding to a specific point of interest usually meets specific information needs. It first appeared in the field of consumer feedback, where the system extracted information about product attributes and feedback sentiment from consumer reviews, and then developed into the field of multi-document summarization, especially multi-document summary generation similar to Wikipedia. Multi-document summaries have multi-topic and multi-faceted practical significance. Aspect-oriented summaries can not only convey the diverse information described in multiple documents more comprehensively, but also provide target-oriented summaries in line with user interests, so they have good application value. Most existing technologies divide aspect-oriented methods into two stages, namely, aspect discovery and identification and aspect-oriented summary generation. First, for a given source text, a pre-trained language model is used to perform context analysis to obtain relevant aspects, which can be defined as a classification problem, such as using BERT and Roberta. However, there are two types of aspect division: explicit and implicit. The explicit method is to classify based on label data to obtain displayed labels as different aspects, while the implicit method is more inclined to use topic models such as LDA to analyze different aspects. However, there are no specific labels in this method. The model will pass the implicit division aspect information to the subsequent stage. For the subsequent second stage, that is, summary generation, the acquired aspect information is usually used as guidance information for the generation task. In the summary generation process, specific mechanisms such as Copy mechanism and decoder can be used to ensure that the summary contains mentions of key aspects. In addition, aspect information can be further encoded and feature processed through graph-based methods or Transformer encoders to improve the quality and relevance of summary generation.The development of large models has also promoted the development of aspect-oriented summarization. However, there is still a lack of relevant research on aspect-oriented multi-document summary generation technology based on large models. The aspect-oriented summary technology in a single document is still limited in the application scenarios of multiple documents. Multi-document summarization will face more problems than single-document summarization. In aspect-oriented multi-document summary generation, it is necessary to conduct multi-dimensional analysis on multiple documents to divide reasonable different aspects. In summary generation, it is necessary to guide the model to learn to select texts related to different aspects in order to eliminate redundancy in the long text composed of multiple documents and improve the quality of summary generation. Similar to the writing of Wikipedia, multiple documents develop concise descriptions of different aspects of the central entity. That is, although there are large differences in the contents described in multiple documents, the multi-document summary should be developed towards the center of the whole when no target entity is provided. The center here can be regarded as the central entity, that is, the main entity of the description after multiple documents are summarized, and it is also the main element of summary writing. Different aspects should be developed around the central entity, which can be provided by the user. In the absence of user provision, it can be completed with the help of central entity analysis.

[0094] The aspect-oriented comprehensive multi-document summarization technology based on multi-dimensional analysis combines the multi-document summarization process with complex aspect analysis, providing a comprehensive method to describe topics related to multiple dimensions of the central entity. This technology aims to synthesize aspect information from multiple documents to generate a comprehensive summary covering multiple perspectives and discussion points. Such summaries not only focus on a single entity or event, but also cover multiple aspects of the topic related to the central entity. For example, Wikipedia and Baidu Encyclopedia-style entries are often based on multiple references to summarize a comprehensive description of aspects related to an entity, whether it is a person, event or concept. In the multi-document summarization task, this technology not only provides a concise description of the core content, but more importantly, it can show the complexity and diversity of the topic from multiple dimensions. It uses advanced analytical methods such as multi-dimensional aspect identification and relationship analysis to reflect all important aspects of the document collection in the summary. However, the existing technology lacks application in multi-document scenarios, and the existing technology lacks analysis on central entities, which should be the first step of multi-dimensional analysis, that is, it is necessary to carry out relevant analysis around a central entity, and the existing technology lacks the introduction of prior knowledge in realizing this task, especially in multi-dimensional analysis. This task cannot be simply modeled as a classification task because of the lack of fixed label settings. There may be different aspects for central entities in different fields. The classification task is limited by the label setting, which will limit the analysis of aspects. In addition, in summary generation, most of the existing technologies build separate models or layers, lack of integration with the existing summary generation pre-trained language model, and it is difficult to effectively use the powerful capabilities of the pre-trained language model to improve the summary generation effect. Finally, the generated summary only uses the negative log-likelihood loss to calculate the difference between the generated Token and the reference summary text, lacking the integration and judgment of aspect information. Multi-dimensional analysis in multi-document summarization needs to be combined with central entity analysis, multi-dimensional identification should be carried out around the central entity, and aspect identification, detection and judgment should be integrated into the two stages. How to effectively analyze different aspects of different fields, how to effectively integrate the identified aspect information into summary generation, how to ensure that the generated summary contains the multiple aspects we need, and ensure that the generated summary can be faithful to the facts and provide a clear and comprehensive content overview.

[0095] It is understandable that although the multi-document summary generation technology relying on pre-trained language models has made significant progress and demonstrated strong summary generation capabilities, this technology still has obvious limitations in practical applications. The existing technology mainly adopts a two-stage method to generate aspect-oriented multi-document summaries, which aims to improve the overall quality of the summary by improving the effect of aspect recognition and integrating aspect information to screen the most relevant content in the source text. However, this method often ignores the systematic analysis of the central entity and does not fully utilize the potential of multi-dimensional analysis, resulting in content dispersion and information redundancy. When conducting multi-dimensional analysis, there are different description requirements in different fields such as animals, equipment, people or cars, which puts higher requirements on the generalization ability of the model. Existing solutions usually perform aspect analysis separately in different fields, but this is too limited in practical applications in multi-document summary generation. In addition, the integration of a large amount of redundant information in the summary generation process may mislead the model learning direction and bring negative effects. The current technical methods lack a unified framework for integration with pre-trained models, usually focusing only on the effect of aspect recognition and the negative log-likelihood loss of summary generation, and lack explicit modeling of aspect-level losses. Although it helps the model understand different content and improve the sense of logical hierarchy, it fails to explicitly model to ensure that the generated text faithfully reflects all aspects. In view of these challenges, this study proposes a new comprehensive multi-document summary generation method based on multi-dimensional analysis. This method uses multi-dimensional analysis technology to deeply analyze the central entity and carry out detailed exploration of the domain and aspect around it, breaking through the limitations of existing technologies. By combining the results of central entity and aspect analysis to screen information at the sentence level, this method significantly reduces the interference of redundant information and uses the aspect detection mechanism to generate higher-quality aspect-oriented summaries. Through integration with pre-trained language models, a unified model framework is constructed, which can effectively carry out multi-dimensional analysis and realize central entity recognition, domain adaptation and aspect classification in multiple documents. This method can not only integrate entity and aspect information, but also ensure the aspect orientation of the summary through aspect detection and summary generation models, generate more comprehensive and accurate summaries, thereby improving the practicality of the summary, ensuring the fidelity, comprehensiveness and logic of the information, and meeting users' needs for information from different aspects.

[0096] In the specific implementation, Figure 2 The aspect-oriented comprehensive multi-document summary generation framework based on multi-dimensional analysis is shown, which follows the Transformer-based encoder-decoder architecture. Referring to the construction of Wikipedia, it can be found that the existence of central entities is necessary, so this technology uses a graph encoder to analyze central entities in multiple documents as the center of the entire aspect orientation. In this embodiment, the multi-dimensional analysis module 10 includes a graph encoder 101 and a first encoder 102. The first encoder is also called an encoder improved in combination with the aspect analysis task in this embodiment.

[0097] In one embodiment, the graph encoder 101 is further used to determine a sentence set and an entity set using Spacy when receiving multi-document information; encode the sentence set using a weighted pooling operation to generate a text representation; encode the entity set using a weighted pooling operation to generate an entity representation; and construct a heterogeneous graph based on the text representation and the entity representation to generate central entity information.

[0098] It should be noted that the multi-dimensional analysis module in this embodiment is composed of a graph encoder and an encoder improved by combining aspect analysis tasks, such as Figure 3 As shown in the graph coding, Spacy is first used to perform sentence segmentation and entity acquisition and coreference resolution for multiple documents, so as to obtain sentence sets and entity sets: Given a multi-document set ; After Spacy processing, we get: , that is, get the sentence set S and entity set E

[0099] in , , the entity set E is not all entities, but the top k entities in the three documents. This can reduce the computational cost and complexity. The appropriate setting of the k value will not affect the analysis effect, because the central entity is related to the frequency of occurrence to a certain extent. Then get the representation of the sentence. The input form of the sentence is:

[0100] First, we use BART's Tokenizer to get the Embedding. Here we use the comprehensive Token, Positional, and SegmentEmebdding, and sum them up to get the final input Embedding, which is expressed as follows:

[0101]

[0102] Use Longformer to build the encoder and initialize the parameters of the BART pre-trained language model. BART has good generation capabilities, and its encoder also has a strong context representation encoding capability to encode sentences and entities. It should be noted that we use parameter sharing to obtain the representation of entities in context, which is more context-sensitive than direct encoding. Then use the encoder to obtain text representation :

[0103]

[0104] Then, in order to obtain a fixed-length sentence representation, a weighted pooling operation, that is, a multi-head pooling layer, is used for encoding:

[0105]

[0106] A similar approach is used to encode entity collections. , the context representation of the entity is obtained by query, that is, the context representation at the position is obtained according to the position of the entity. However, if the entity has two or three tokens, the average representation from the start position to the end position can be used.

[0107]

[0108] Next, we build the graph: Building a heterogeneous graph , As nodes on the graph include sentence nodes and entity nodes , B represents the undirected edge between nodes. The edges in the graph are entity-entity and entity-sentence. For entity-sentence edges, these edges represent the appearance of entities in sentences and can be used to describe the distribution and context of entities in text. For each sentence and each entity , combined with the co-occurrence matrix C, the elements of which Representing Entities In the sentence The number of occurrences in the co-occurrence matrix is ​​constructed using the frequency as the value of the element. If it exists, an edge is constructed. The weight of the edge is defined as the TF-IDF score of the entity, which is calculated as follows:

[0109]

[0110] in Is Entity In the sentence The frequency in Is Entity The inverse document frequency of .

[0111] Entity-entity edges represent the relationship between different entities, usually based on their co-occurrence in the text. That is, for any pair of entities and , calculate their co-occurrence scores, the co-occurrence scores are:

[0112]

[0113] where m is the number of sentences, represents the indicator function. If two entities co-appear in a sentence, the value is 1, otherwise the value is 0. All entities and sentences are traversed to calculate the co-occurrence score. If there is a further relationship, an edge is constructed. The weight of the edge is calculated as ,in Represents the co-occurrence scores between all entities.

[0114] The above analysis can realize the construction of the graph and complete the construction of nodes and edges in the graph.

[0115] In one embodiment, the graph encoder 101 is also used to construct a heterogeneous graph ; Among them, V as a node on the heterogeneous graph contains sentence nodes and entity nodes , B represents the undirected edges between nodes, which include entity-entity and entity-sentence edges; for each sentence and each entity , combined with the co-occurrence matrix C, the elements of which Representing Entities In the sentence The number of occurrences in , using frequency as the element The value of constructs the co-occurrence matrix; the weight of the entity-sentence edge is defined as the entity occurrence The score is calculated as:

[0116]

[0117] in Is Entity In the sentence The frequency in Is Entity The inverse document frequency of ; define the weight of the entity-entity edge, in which for any pair of entities and , calculate the co-occurrence score, the co-occurrence score is:

[0118]

[0119] where m is the number of sentences, represents the indicator function, if the entity and If they appear in the sentence together, then the value is 1, otherwise the value is 0;

[0120] Iterate over all entities and sentences to calculate the co-occurrence score .

[0121] In one embodiment, the first encoder 102 is further used to update the node representation in the heterogeneous graph using a graph attention network; for each node in the heterogeneous graph, the attention coefficient of the node connected to it is calculated:

[0122]

[0123] in, is a sentence node, is an entity node, is the attention coefficient, is the learnable attention mechanism parameter, W is the weight parameter matrix in training, is the set of neighbor nodes of node i, LeakyReLU is a nonlinear activation function, and the updated node representation is:

[0124]

[0125] in, represents the updated node representation, W represents the weight parameter matrix during training, ReLU represents the activation function; a multi-head attention mechanism is used to update features:

[0126]

[0127] Concatenate the multi-head attention mechanisms to get multi-head attention:

[0128]

[0129] in, Represents multi-headed attention, Represents the attention calculated by the nth head, each head has an independent and , concat represents the concatenation operation; the central entity node is obtained by calculating the attention. The centrality of each node can be calculated by the weighted sum of the attention coefficients of its adjacent nodes. For entity node i, its centrality ;in is the set of adjacent nodes of entity node i, is the attention coefficient between node i and node j; normalization is performed, and the normalization includes maximization and minimization:

[0130]

[0131] in is the standardized value, and Central represents the centrality value of all entity nodes, thereby determining the central entity:

[0132]

[0133] in, Represents the standardized centrality values ​​of all entity nodes, and determines the node representation of the central entity based on the centrality value.

[0134] It should be noted that GATv2 is a deep learning model for graph-structured data, which uses the self-attention mechanism to update the representation of each node in the graph. The core of GATv2 is that it can assign different importance weights to each node, which are learned based on the characteristics of the nodes and their relationships with surrounding nodes. After building the graph, the graph attention network is used to update the node representation, that is, the model is used to update the node representation during the overall training process of the model. Each representation is a feature after the previous encoding, and the sentence node uses the features after sentence encoding and pooling. , entity nodes use .

[0135] Here we use and Represents the feature representation of the node. After building the graph, the GATv2 model is used to update the node. For each node in the graph, the attention coefficient of the node connected to it is calculated:

[0136]

[0137] in is the attention coefficient, is the learnable attention mechanism parameter, W is the weight parameter matrix in training, which is used to linearly transform node features, is the set of neighbor nodes of node i, and LeakyRelu is a nonlinear activation function. Update node representation:

[0138]

[0139] in, represents the updated node representation, W represents the weight parameter matrix during training, Represents the ReLU activation function; in order to capture the relationship between different types of nodes, a multi-head attention mechanism is used to update the features:

[0140]

[0141] Then the multi-head attention mechanism is spliced ​​to obtain multi-head attention:

[0142]

[0143] in, Represents multi-headed attention, Represents the attention calculated by the nth head, each head has an independent and , concat represents the concatenation operation, and the node representation output will be used as the basis for subsequent calculations;

[0144] Among them, it is necessary to calculate the central entity node based on attention. The centrality of each node can be calculated by the weighted sum of the attention coefficients of its adjacent nodes, which reflects the global importance of the node. For entity node i, its centrality ;

[0145] in is the set of adjacent nodes of entity node i, is the attention coefficient between node i and node j. Then, normalization is performed. In order to make the centrality measure comparable between different nodes, maximum and minimum normalization can be performed:

[0146]

[0147] in is the standardized value, Represents the centrality value of all entity nodes. In this way, the entity node with the largest standardized centrality level value can be regarded as the entity node with the strongest centrality, and then it is regarded as the central entity:

[0148]

[0149] in, It represents the standardized centrality values ​​of all entity nodes, and the largest one is taken as the central entity. Here, multiple entities can also be regarded as central entities and divided into fields. You only need to set a threshold or sort them by score. The node representation is obtained later, that is, the node representation of the central entity, and MLP is used to implement classification. The label here is the field to which the entity belongs. For example, drones belong to equipment, dogs belong to animals, etc. The field here can be obtained in advance as a label in Wikidata. This knowledge base provides a more comprehensive field division. The calculation at MLP can be expressed as: for the first layer:

[0150]

[0151] in is the output of the L-1th layer. For the first layer, , and are the weight matrix and bias vector of the Lth layer, It is a nonlinear activation function, using the ReLU activation function, and the output is:

[0152]

[0153] in, is the output of the last hidden layer, and is the weight matrix and bias vector of the output layer. Softmax is used to convert the results of the output layer into probability distribution. Based on the probability distribution, the domain to which it belongs can be obtained. This task and the task of generating summaries are jointly trained as multi-tasks, and the loss is:

[0154]

[0155] Among them, y is the true label of the domain to which the entity belongs, is the predicted output of MLP, and the loss of domain partitioning is , the number of labels in the preset field is N.

[0156] In one embodiment, the first encoder 102 is further used to inject entity category information into the preset rule module so that the preset rule module performs entity relationship judgment and generates a judgment result according to the ontology rules and the preset rule construction table; and determines the candidate triple set according to the judgment result.

[0157] In one embodiment, the multi-dimensional analysis module 10 is further used to classify the node representations of the central entity using MLP; layer:

[0158]

[0159] in is the output of the L-1th layer. For the first layer, , and are the weight matrix and bias vector of the Lth layer, It is a nonlinear activation function, using the ReLU activation function, and the output is:

[0160] in, is the output of the last hidden layer, and are the weight matrix and bias vector of the output layer; softmax is used to convert the results of the MLP output layer into probability distribution and determine the field according to the probability distribution.

[0161] In the specific implementation, for domain-aspect discovery, a domain-aspect prior matrix A is constructed. The matrix consists of pre-set domains and aspects corresponding to the domains. The characters are encoded using BART's Tokenizer. Only the Token number needs to be obtained, and the elements of the Token number are only 0 and 1. It means that the field contains the aspect, and vice versa, 0 means that the aspect is not included. This is the prior knowledge of aspect analysis. The analysis combined with this prior knowledge is no longer restricted by the field data. The unified training and reasoning can be performed for data in different fields, which can greatly improve the model's capabilities: .

[0162] In the specific implementation, in the aspect analysis module of this embodiment, Longformer is used to construct the encoder, and the parameters of BART are used for initialization. However, the multi-head self-attention mechanism is improved to an entity-sentence level self-attention mechanism. The main purpose is to combine the central entity information analyzed by the graph encoder, that is, the central entity analysis module, and use the encoding information at the self-attention and sentence levels to filter out information related to the central entity, which can reduce redundancy and focus on key information, reduce the impact of other information on aspect analysis, and reduce the negative impact on subsequent generation. In addition to analyzing the central entity, the graph encoder can also use its encoding of the sentences in the source text, that is, the contextualized representation. This representation can be used as the encoding of the source text and input into the subsequent aspect analysis and summary generation. In the aspect analysis encoder, the entity-sentence level cross-attention mechanism is adopted. After the central entity analysis, as part of the multi-dimensional analysis, it is crucial to screen out sentences related to the entity in the aspect analysis. Thanks to the entity relationship analysis and document information encoding of the graph encoder part and the acquisition of domain-aspect prior knowledge, the entity-sentence level self-attention mechanism constructed here can be used for document segmentation driven by central entity attention, that is, sentence screening. Sentences related to the central entity are selected according to the central entity analysis results. Sentences related to the aspect will guide the generation of higher quality aspect-based summaries. Using the representation of the central entity node output from the graph encoder , here we can To obtain, we have already obtained , sentence-level representation encoded in the graph encoder , where the entity-sentence level self-attention mechanism uses the sentence representation As the value and key of attention, the central entity represents As a query

[0163]

[0164] Cross-Attention means cross-attention calculation. Because the attention calculation here comes from different sources, it belongs to cross-attention. It can use the representation of the central entity to highlight the sentence information related to the entity. Then the calculation of other layers of the encoder is required:

[0165]

[0166] After the multi-layer encoder output, it can be expressed , the output then needs to be based on MLP to implement aspect classification, that is, to perform aspect classification based on the encoded representation and prior knowledge, and to perform calculations based on prior knowledge:

[0167] The encoder output With the domain-aspect prior matrix Used together as input to MLP and perform MLP calculations:

[0168]

[0169] Finally, the results of the aspect analysis are output, that is, the different aspects of the central entity that exist, so that the different aspects of existence can be obtained. The representation after MLP processing is the probability of different aspects. It represents the final aspect analysis result combined with prior knowledge, that is, the possible aspect prediction in the entire text. It can be regarded as a multi-label classification problem, using the binary cross entropy loss function, and the loss is calculated as:

[0170] Assume there are N aspects and there are true labels , where 1 represents the label, that is, the aspect exists, and 0 represents the absence of the aspect. The aspect loss can be calculated by the following formula:

[0171]

[0172] in Represents the predicted existence probability of each i-th aspect.

[0173] In one embodiment, the second encoder 201 is further configured to combine the text context representation of the first encoder Operate to generate the updated representation as :

[0174]

[0175] Using contextual representation As the value and key of attention, aspect information represents As a query;

[0176]

[0177] in, is the representation processed by the second encoder and expressed as .

[0178] In one embodiment, the decoder 202 is further configured to obtain a reference digest and combine the preset token in the reference digest with the output of the second encoder. Perform the operation at time step t:

[0179]

[0180]

[0181]

[0182]

[0183] The output of the decoder is , get the prediction result for the next position ;according to Compare with the true summary and perform loss calculation:

[0184]

[0185] Where T is the number of tokens in the target digest, is a given current decoder state The probability of the position predicted under the condition .

[0186] It should be noted that in the summary generation stage of this embodiment, Figure 4 The aspect-oriented summary generation module shown. This part consists of a second encoder and a decoder. The second encoder is also referred to as the encoder here. The encoder part is constructed using longformer. The weights of the BART pre-trained language model are initialized. The self-attention mechanism of the encoder part is modified to aspect-sentence cross-attention, that is, the aspect-sentence cross-attention mechanism. The attention mechanism of this part is designed to select sentences related to the aspects. It is crucial to select sentences related to the aspects. This model benefits from our previous aspect segmentation process and can be used for document segmentation with aspect-driven attention. Sentences related to the aspects are selected based on the attention scores calculated by aspect discovery. The multi-head attention score of the last encoder layer represents the correlation between aspects and sentences. Sentences related to the aspects will guide the generation of higher quality aspect-based summaries. By simply noticing the addition of the "aspect to sentence" and "transposed sentence to aspect" multi-head attention matrices, sentences related to the "aspect" are obtained according to the sorted attention scores. Different aspect representations are obtained Finally, the text context representation output by the aspect analysis encoder is combined , here, after the central entity is filtered, the central entity information will be highlighted in the context representation, and after the adjustment of the aspect analysis task, part of the aspect information can also be reflected. Through the aspect-sentence level cross-attention calculation mechanism, What we get is the text context vector calculated by the attention mechanism:

[0187]

[0188] Aspects here - sentence-level self-attention mechanism, using context representation As the value and key of attention, the aspect information represents As a query, that is, using the attention mechanism to update the sentence representation, this can help the model pay more attention to the most relevant parts of the sentence in various aspects. The updated representation is , after further processing by the encoder:

[0189]

[0190] Finally got That is, the representation after being processed by the encoder. The representation of d after passing through the multi-encoder can be written as .

[0191] In the decoder part, during the training phase, the goal is to minimize the loss between the predicted summary and the true summary. The method with teacher forcing is usually adopted. The output of the multi-layer encoding is As the input of the decoder, and the previous Token in the reference summary as the input of the current decoding step, the operation of this part can be expressed as, at time step t:

[0192]

[0193] The output of the decoder is , adapt the final decoder output to generate a prediction summary and obtain the prediction result for the next position in this state , and compared with the true summary, usually using the negative log-likelihood loss for loss calculation:

[0194]

[0195] Where T is the number of tokens in the target digest, is a given current decoder state Under the condition of , the predicted probability of the position.

[0196] In the inference phase, beam search is used for decoding. After obtaining the context representation from the encoder, the beam search strategy is initialized, usually selecting a start tag and using As the start tag, start decoding from this tag, for each time step t, iterate the following steps until each beam generates an end tag, the end tag is For each time step, expand the next possible word for each bundle:

[0197]

[0198] Select the top-k words with the highest probability as candidates, and then update the score of each bundle

[0199]

[0200] in, represents the cumulative score of the current bundle before selecting a new word, It means that at time step t, the probability of selecting a specific word from the probability distribution output by the decoder is calculated by the logarithmic function. This is the updated score, which will be used to evaluate the quality of this beam in the next step. The k beams with the highest scores are then retained, that is, k generation results. Generally, the best beam is selected as the generated sequence returned at the end.

[0201] In order to achieve aspect-oriented tasks faster, aspect loss is added in the training phase to improve the model's ability to generate aspect-oriented summaries. A detector is added at the decoder position, that is, the decoder output is used to analyze the aspects it covers. Here, a classifier needs to be added as a detector, and MLP is used to calculate the aspect score. The output for each time step t is , and then use MLP to process to get the aspect distribution, but here we still need to use prior knowledge for analysis, that is, the aspect prior matrix A, and calculate it:

[0202]

[0203] in, It is the aspect distribution of the text analysis calculated based on the predetermined domain-aspect prior knowledge matrix, and its elements can be regarded as , that is, the probability of the aspect at time step t. Next, for the generated summary, we can integrate the aspect probabilities of each time step to obtain the overall coverage of each aspect of the entire summary:

[0204]

[0205] in is the overall probability of the i-th aspect being mentioned in the entire abstract, is the probability of that aspect at time step t.

[0206] The aspect consistency loss calculates the difference between the aspect probability of the generated summary and the target aspect label, which is calculated as:

[0207]

[0208] In addition, we have two tasks, central entity analysis and aspect analysis, which are trained as multi-tasks together with the main summary task during training. The overall loss can be expressed as:

[0209]

[0210] In one embodiment, the decoder 202 is further configured to obtain a target text context vector output by the second encoder, and to expand the target context vector according to a beam search strategy;

[0211]

[0212] Select the top-k words with the highest probability as candidates, and then update the score of each bundle;

[0213]

[0214] in, represents the cumulative score of the current bundle before selecting a new word, It means that at time step t, the probability of selecting a specific word from the probability distribution output by the decoder is calculated by the logarithmic function. This is the updated score. The k bundles with the highest scores are retained and the best bundle is selected as the generated sequence returned at the end.

[0215] In the specific implementation, this embodiment realizes an aspect-oriented comprehensive multi-document summary generation based on multi-dimensional analysis, covering a graph encoder for central entity analysis, an aspect analysis encoder, and a summary generation model based on BART and Longformer, and constructs these modules into a unified framework to achieve higher-quality aspect-oriented summary generation. The main advantages and innovations of this technology include: this embodiment combines entity-sentence level attention and aspect-sentence level attention mechanism on the basis of pre-trained language model to improve the encoder, so that it can gradually filter relevant key information and reduce the redundant information impact caused by the excessively long text of multiple documents. By encoding the source text with the help of the graph encoder and updating the node representation based on the graph attention mechanism, cross-document relationships can be better modeled, central entities can be effectively analyzed, and domain-aspect prior knowledge discovery is used on the basis of central entity analysis to solve the limitation of limited domain analysis, and multiple documents in multiple fields can be processed. The analysis of the central entity also makes the aspect-oriented summary generation task more in line with actual requirements, that is, the analysis is carried out around the central entity or predefined entity to remove redundant information. The aspect analysis module uses the entity-sentence level attention mechanism to perform preliminary key information screening, so that the different aspects analyzed in the aspect analysis stage are related to the central entity, and the summary generation is more convenient. The encoder of the module adopts aspect-sentence level attention, and combines the information related to the central entity for further information identification and screening, that is, the sentence-level information related to each aspect, to achieve aspect-driven document segmentation, thereby guiding the subsequent summary generation combined with aspects. The inconsistency between the decoder part and the pre-trained language model lies in the addition of an aspect detection detector, that is, analyzing the distribution of aspects in the generated text, and constructing a joint loss to integrate aspect analysis and aspect-oriented summary generation. Overall, it realizes the joint aspect analysis and aspect-oriented summary generation tasks, constructs a complete training and reasoning framework, and effectively integrates with the pre-trained language model. When faced with a large amount of data, the model can more effectively screen out key information, better realize aspect-oriented summary generation, and achieve higher quality and comprehensive summaries. The entities and aspects can be embedded by external information, that is, the user's interests can be combined to generate targeted entity and aspect summaries.

[0216] In summary, the innovation of this technology lies in the unique application of the attention mechanism, the joint task and method innovation, and the innovative integration of central entity analysis and domain-aspect discovery with the graph encoder. These features not only make the summary generation process more intelligent and adaptable, but also significantly improve the quality and usability of the generated summaries. These innovations give this technology significant technical advantages in the field of aspect-oriented summary generation and text generation, and have good application value in scenarios such as Wikipedia generation.

[0217] This embodiment obtains multi-document information through a graph encoder and constructs a heterogeneous graph to generate central entity information; the first encoder uses a multi-head cross-attention mechanism at the entity-sentence level to filter based on the central entity information to generate central entity related information; the second encoder combines central entity related information based on the aspect-sentence cross-attention mechanism to determine the target text context vector; the decoder combines the target text context vector through aspect detection to generate aspect distribution analysis results, and generates target summary information based on the aspect distribution analysis results. The aspect detection and summary generation model ensure the aspect orientation of the summary, generate a more comprehensive and accurate summary, thereby improving the practicality of the summary, ensuring the fidelity, comprehensiveness and logic of the information, and meeting the user's needs for information from different aspects.

[0218] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present application. In specific applications, technicians in this field can make settings as needed, and the present application does not impose any limitation on this.

[0219] The present application embodiment provides a method for generating multi-document summaries based on aspect orientation. Figure 5 , Figure 5 This is a flowchart of the first embodiment of the aspect-oriented multi-document summary generation method of the present application.

[0220] In this embodiment, the aspect-oriented multi-document summary generation method includes the following steps:

[0221] Step S10: The graph encoder obtains multi-document information and constructs a heterogeneous graph based on the multi-document information to generate central entity information.

[0222] Step S20: The first encoder uses a multi-head cross-attention mechanism at the entity-sentence level to filter the central entity information to generate central entity related information.

[0223] Step S30: The second encoder combines the central entity related information based on the aspect-sentence cross attention mechanism to determine the target text context vector.

[0224] Step S40: The decoder combines the target text context vector by means of aspect detection to generate aspect distribution analysis results, and generates target summary information according to the aspect distribution analysis results.

[0225] This embodiment obtains multi-document information through a graph encoder and constructs a heterogeneous graph to generate central entity information; the first encoder uses a multi-head cross-attention mechanism at the entity-sentence level to filter based on the central entity information to generate central entity related information; the second encoder combines central entity related information based on the aspect-sentence cross-attention mechanism to determine the target text context vector; the decoder combines the target text context vector through aspect detection to generate aspect distribution analysis results, and generates target summary information based on the aspect distribution analysis results. The aspect detection and summary generation model ensure the aspect orientation of the summary, generate a more comprehensive and accurate summary, thereby improving the practicality of the summary, ensuring the fidelity, comprehensiveness and logic of the information, and meeting the user's needs for information from different aspects.

[0226] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0227] In addition, for technical details not fully described in this embodiment, please refer to the aspect-oriented multi-document summary generation method provided in any embodiment of the present application, which will not be repeated here.

[0228] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0229] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0230] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0231] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A multi-document summary generation system based on aspect orientation, characterized in that: include: Multi-dimensional analysis module and aspect-oriented summary generation module; The multi-dimensional analysis module includes a graph encoder and a first encoder; The aspect-oriented summary generation module includes a second encoder and a decoder; The graph encoder is used to obtain multi-document information and construct a heterogeneous graph according to the multi-document information to generate central entity information; The first encoder is used to adopt a multi-head cross attention mechanism at the entity-sentence level to filter according to the central entity information to generate central entity related information; The second encoder is used to combine the central entity related information based on an aspect-sentence cross attention mechanism to determine a target text context vector; The decoder is used to combine the target text context vector by aspect detection to generate an aspect distribution analysis result, and generate target summary information according to the aspect distribution analysis result; The graph encoder is further used to determine a sentence set and an entity set using Spacy when receiving multi-document information; Encoding the sentence set using a weighted pooling operation to generate a text representation; Encoding the entity set using a weighted pooling operation to generate an entity representation; Constructing a heterogeneous graph according to the text representation and the entity representation to generate central entity information; The graph encoder is also used to construct a heterogeneous graph ; Among them, V as a node on the heterogeneous graph includes sentence nodes and entity nodes , Represents undirected edges between nodes, including entity-entity and entity-sentence edges; For each sentence and each entity , combined with the co-occurrence matrix , where the elements Representing Entities In the sentence The number of occurrences in the element, using the frequency as the The values ​​of construct the co-occurrence matrix; Define the weight of the entity-sentence edge as the number of occurrences of the entity The score is calculated as: in Is Entity In the sentence The frequency in Is Entity The inverse document frequency of Define the weight of the entity-entity edge. For any pair of entities in the entity-entity edge and , calculate the co-occurrence score, the co-occurrence score is: where m is the number of sentences, represents the indicator function, if the entity and If they appear in the sentence together, then the value is 1, otherwise the value is 0; Iterate over all entities and sentences to calculate the co-occurrence score .

2. The aspect-oriented multi-document summary generation system according to claim 1, characterized in that: The first encoder is further used to update the node representation in the heterogeneous graph using a graph attention network; For each node in the heterogeneous graph, calculate the attention coefficient of the node connected to it: in, is a sentence node, is an entity node, is the attention coefficient, are learnable attention mechanism parameters, is the weight parameter matrix in training, Is a node A set of neighbor nodes, LeakyReLU is a nonlinear activation function, and the updated node representation is: in, represents the updated node representation, W represents the weight parameter matrix during training, ReLU activation function. Use multi-head attention mechanism to update features: Concatenate the multi-head attention mechanisms to get multi-head attention: in, Represents multi-headed attention, Represents the attention calculated by the nth head, each head has an independent and , concat represents the concatenation operation; The central entity node is calculated based on attention. The centrality of each node is calculated by the weighted sum of the attention coefficients of its adjacent nodes. For entity node i, its centrality ; in is the set of adjacent nodes of entity node i, is the attention coefficient between node i and node j; Perform a normalization process, which includes a maximization process and a minimization process: in is the standardized value, Represents the centrality value of all entity nodes, thereby determining the central entity: in, The standardized centrality values ​​of all entity nodes are represented, and the node representation of the central entity is determined according to the centrality values.

3. The aspect-oriented multi-document summary generation system according to claim 2, characterized in that: The first encoder is further configured to classify the node representation of the central entity using an MLP; For the MLP layer: in is the output of the L-1th layer. For the first layer, , and are the weight matrix and bias vector of the Lth layer, It is a nonlinear activation function, using the ReLU activation function, and the output is: in, is the output of the last hidden layer, and is the weight matrix and bias vector of the output layer; use The result of the MLP output layer is converted into a probability distribution and the domain is determined according to the probability distribution.

4. The aspect-oriented multi-document summary generation system according to claim 1, characterized in that: The multi-dimensional analysis module also includes domain-aspect discovery; The domain-aspect prior matrix A is composed of preset domains and aspects corresponding to the preset domains; Use BART's Tokenizer to encode characters, where the elements are only 0 and 1.

5. The aspect-oriented multi-document summary generation system according to claim 1, characterized in that: The second encoder is further used to combine the text context representation of the first encoder Operate to generate the updated representation as : Using contextual representation As the value and key of attention, aspect information represents As a query; in, is the representation processed by the second encoder and is expressed as .

6. The aspect-oriented multi-document summary generation system according to claim 5, characterized in that: The decoder is further configured to obtain a reference summary and combine the output of the second encoder with a preset Token in the reference summary as input. Perform the operation at time step t: The output of the decoder is , get the prediction result for the next position ; According to the Compare with the true summary and perform loss calculation: in is the number of tokens in the target digest, is a given current decoder state The probability of the predicted position under the condition.

7. The aspect-oriented multi-document summary generation system according to claim 6, characterized in that: The decoder is further used to obtain a target text context vector output by the second encoder, and to expand the target context vector according to a beam search strategy; Select the top-k words with the highest probability as candidates, and then update the score of each bundle; in, represents the cumulative score of the current bundle before selecting a new word, It means that at time step t, the probability of selecting a specific word from the probability distribution output by the decoder is calculated by the logarithmic function. This is the updated score, and the k bundles with the highest scores are retained to determine the final generated sequence returned.

8. A method for generating multi-document summaries based on aspect orientation, characterized in that: Executing the method executed by the system as claimed in claim 1, the aspect-oriented multi-document summary generation method comprises: The graph encoder obtains multi-document information and constructs a heterogeneous graph according to the multi-document information to generate central entity information; The first encoder uses a multi-head cross-attention mechanism at the entity-sentence level to filter the central entity information to generate central entity related information; The second encoder combines the central entity related information based on the aspect-sentence cross attention mechanism to determine the target text context vector; The decoder combines the target text context vector by way of aspect detection to generate an aspect distribution analysis result, and generates target summary information according to the aspect distribution analysis result.

Images