Abstract generation method and apparatus
By generating summaries by retrieving matching domain knowledge from a static domain knowledge base, the problem of poor summary quality caused by the introduction of external knowledge in existing technologies is solved, and higher quality summary generation is achieved.
Patent Information
- Application Number
- CN202210351256.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-02
- Publication Date
- 2026-01-20
- Estimated Expiration
- 2042-04-02
AI Technical Summary
Existing summarization techniques that incorporate external knowledge often result in poor quality during the generation process.
By obtaining the target vector representation, and in descending order of matching degree between the description vector representation in the static domain knowledge base and the target vector representation, K first vector representations are determined from the static domain knowledge base, K first domain knowledge are obtained, and a summary is generated based on the matching degree between these domain knowledge and the target text.
The quality of the generated summaries was improved by enhancing the accuracy of retrieving matching domain knowledge from a static domain knowledge base, thereby improving the quality of the target text summaries.
Smart Images
Figure CN116932741B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of text summarization technology, and specifically relates to a summarization method and apparatus. Background Technology
[0002] With the rapid growth of service calls from major enterprises, companies have provided users with channels for human customer service inquiries and complaints, primarily through online chat and telephone calls. Automatic summarization technology, applied to massive amounts of user-customer service dialogue data, is used to quickly extract the core content of the conversation, helping companies quickly understand users' core needs and improve service quality. The core idea of dialogue summarization generation technology is to convert complete dialogue data into concise text containing key information.
[0003] Existing summarization technologies that incorporate external knowledge integrate external knowledge into the generation process, but the summaries generated using existing methods are of poor quality. Summary of the Invention
[0004] This application provides a method and apparatus for generating abstracts, which can improve the quality of the generated abstracts.
[0005] To address the aforementioned technical problems, embodiments of this application provide a method for generating abstracts, including:
[0006] Obtain the target vector representation, which is a vector representation of the target text;
[0007] According to the order of matching degree between the description vector representation in the static domain knowledge base and the target vector representation from high to low, K first vector representations are determined from the description vector representations in the static domain knowledge base, where K is an integer greater than or equal to 1. The static domain knowledge base stores multiple domain knowledge and the description vector representation corresponding to each domain knowledge.
[0008] K pieces of first domain knowledge are obtained from a static domain knowledge base, and the K pieces of first domain knowledge correspond to the K first vector representations respectively;
[0009] Based on the K first domain knowledge points and the matching degree between each first domain knowledge point and the target text, a summary corresponding to the target text is generated.
[0010] Optionally, the static domain knowledge base is obtained through the following methods:
[0011] Obtain a positive sample and one or more negative samples corresponding to the positive sample. The positive sample includes second domain knowledge and description information of the second domain knowledge. The second domain knowledge matches the description information. The negative sample includes the description information and third domain knowledge. The third domain knowledge does not match the description information.
[0012] The model parameters of the dual model are iteratively updated based on the positive samples and the one or more negative samples.
[0013] The vector representation of the description information output by the updated dual model is determined as the description vector representation corresponding to the second domain knowledge;
[0014] The static knowledge base stores the knowledge of the second domain and the corresponding description vector representation of the knowledge of the second domain.
[0015] Optionally, the iterative update of the model parameters of the dual model based on the positive samples and the one or more negative samples includes:
[0016] Using a dual model, vector representations of the description information, the second domain knowledge, and the third domain knowledge are obtained respectively;
[0017] The first Euclidean distance is obtained based on the vector representation of the description information and the vector representation of the second domain knowledge;
[0018] The second Euclidean distance is obtained based on the vector representation of the description information and the vector representation of the third domain knowledge;
[0019] The contrast loss is obtained based on the first Euclidean distance and the second Euclidean distance. The model parameters of the dual model are iteratively updated with the contrast loss as the optimization objective.
[0020] Optionally, the negative samples are obtained in the following way:
[0021] Obtain the description information and N domain knowledge that do not match the description information;
[0022] Calculate the matching degree between the vector representations of the N domain knowledge and the vector representations of the description information, respectively;
[0023] According to the order of matching degree between the vector representations of the N domain knowledge and the vector representation of the description information from high to low, select M third domain knowledge from the N domain knowledge;
[0024] Each of the M third-domain knowledge and descriptive information is combined to form M negative samples;
[0025] Where M and N are positive integers, and M is less than or equal to N.
[0026] Optionally, generating a summary corresponding to the target text based on the K first domain knowledge points and the matching degree between each first domain knowledge point and the target text includes:
[0027] The K pieces of first domain knowledge are concatenated with the target text to obtain K input texts;
[0028] Input the K input texts into the encoder-decoder model to obtain K output distributions;
[0029] Based on the matching degree between each of the first vector representation and the target vector representation, and the K output distributions, the summary corresponding to the target text is determined.
[0030] Optionally, generating a summary corresponding to the target text based on the matching degree between each of the first vector representations and the target vector representation, and the K output distributions, includes:
[0031] Using the matching degree between each of the K first vector representations and the target vector representation as weights, the K output distributions are weighted and summed to obtain the output distribution of the summary of the target text;
[0032] Based on the output distribution of the target text summary, a summary corresponding to the target text is generated using a beam search method.
[0033] Optionally, the method further includes:
[0034] The encoder-decoder model is iteratively updated with the cross-entropy between the output distribution of the target text summary and the target summary as the optimization objective.
[0035] Optionally, the target text is the dialogue content, and the summary is a summary of the dialogue content.
[0036] This application also provides an abstract generation apparatus, including:
[0037] The first acquisition module is used to acquire the target vector representation, wherein the target vector representation is a vector representation of the target text;
[0038] The first determining module is used to determine K first vector representations from the description vector representations in the static domain knowledge base according to the order of matching degree between the description vector representations in the static domain knowledge base and the target vector representation from high to low, where K is an integer greater than or equal to 1, and the static domain knowledge base stores multiple domain knowledge and description vector representations corresponding to each domain knowledge;
[0039] The second acquisition module is used to acquire K pieces of first domain knowledge from a static domain knowledge base, wherein the K pieces of first domain knowledge correspond to the K first vector representations respectively;
[0040] The generation module is used to generate a summary corresponding to the target text based on the K first domain knowledge and the matching degree between each first domain knowledge and the target text.
[0041] This application also provides a summary generation apparatus, including a transceiver and a processor;
[0042] The processor is configured to: acquire a target vector representation, wherein the target vector representation is a vector representation of the target text;
[0043] According to the order of matching degree between the description vector representation in the static domain knowledge base and the target vector representation from high to low, K first vector representations are determined from the description vector representations in the static domain knowledge base, where K is an integer greater than or equal to 1. The static domain knowledge base stores multiple domain knowledge and the description vector representation corresponding to each domain knowledge.
[0044] K pieces of first domain knowledge are obtained from a static domain knowledge base, and the K pieces of first domain knowledge correspond to the K first vector representations respectively;
[0045] Based on the K first domain knowledge points and the matching degree between each first domain knowledge point and the target text, a summary corresponding to the target text is generated.
[0046] This application also provides a digest generation apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the digest generation method described above.
[0047] This application also provides a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the above-described digest generation method.
[0048] The beneficial effects of this application are:
[0049] The above scheme obtains the target vector representation, determines K first vector representations from the static domain knowledge base according to the order of matching degree between the description vector representations in the static domain knowledge base and the target vector representation from high to low, obtains K first domain knowledge from the static domain knowledge base, and generates a summary corresponding to the target text based on the K first domain knowledge and the matching degree between each first domain knowledge and the target text. By retrieving the first vector representations that match K domain knowledge in the static domain knowledge base, and then generating the summary content based on these vector representations, the accuracy of the domain knowledge matched for the target text can be improved, thereby improving the quality of the summary generated for the target text. Attached Figure Description
[0050] Figure 1 This is a flowchart illustrating the abstract generation method according to an embodiment of this application;
[0051] Figure 2 This is a schematic diagram illustrating the process of collecting and processing domain data;
[0052] Figure 3 This is a schematic diagram of domain knowledge representation learning;
[0053] Figure 4 This is a schematic diagram of the negative sample acquisition process;
[0054] Figure 5 This is a diagram illustrating the dialogue summary generation process;
[0055] Figure 6 This is a schematic diagram of the abstract generation apparatus according to an embodiment of this application;
[0056] Figure 7 This is a structural diagram of the abstract generation apparatus according to an embodiment of this application. Detailed Implementation
[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0058] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0059] It is worth noting that the technologies described in this application are not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA), and other systems. The terms "system" and "network" in this application are often used interchangeably, and the described technologies can be used not only in the systems and radio technologies mentioned above, but also in other systems and radio technologies. The following description describes New Radio (NR) systems for illustrative purposes, and NR terminology is used in most of the following description; however, these technologies can also be applied to applications beyond NR systems, such as 6th Generation (6G) communication systems.
[0060] The abstract generation method and apparatus provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.
[0061] like Figure 1 As shown, at least one embodiment of this application provides a summary generation method, including the following steps.
[0062] Step 101: Obtain the target vector representation.
[0063] It should be noted that the target vector representation is the vector representation of the target text, which refers to the text for which a summary needs to be obtained.
[0064] Step 102: Determine K first vector representations from the description vector representations of the static domain knowledge base in descending order of matching degree between the description vector representations in the static domain knowledge base and the target vector representation.
[0065] It should be noted that K is an integer greater than or equal to 1. The static domain knowledge base stores multiple domain knowledge and a description vector representation corresponding to each domain knowledge. Here, it should be noted that the description vector representation is a vector representation of the description information of the domain knowledge. The description information can be a paragraph or multiple paragraphs with specific semantics. The paragraph or multiple paragraphs can be presented in the form of text information. In some scenarios, the description information can be called the body text. The domain knowledge can be a tag or title of the description information, or it can be part of the sentences in the description information. This embodiment does not limit this.
[0066] Step 103: Obtain K pieces of first-domain knowledge from the static domain knowledge base.
[0067] It should be noted that the K first domain knowledge points correspond to the K first vector representations, respectively. In other words, after determining the K first vector representations, the domain knowledge corresponding to each first vector representation can be determined. The domain knowledge determined based on any one of the K first vector representations can be referred to as the aforementioned first domain knowledge.
[0068] Step 104: Generate a summary corresponding to the target text based on the K first domain knowledge and the matching degree between each first domain knowledge and the target text.
[0069] It should be noted that the above implementation method obtains the domain knowledge of the target text by retrieving the description vector representation corresponding to the matching domain knowledge in the static domain knowledge base, and then generates a summary based on the domain knowledge and the matching degree between the domain knowledge and the target text. This can improve the accuracy of the domain knowledge matched for the target text, thereby improving the quality of the summary generated for the target text.
[0070] Optionally, in the embodiments of this application, the target text is the dialogue content, and the summary is a summary of the dialogue content.
[0071] Optionally, in order to enable the automatic generation of summaries, the embodiments of this application may collect and process domain data. Domain data can provide the model with domain knowledge information not mentioned in the original dialogue corpus, helping the model to make more accurate inferences. Taking the e-commerce domain as an example, available domain data includes, but is not limited to, business introduction materials, product introduction materials, and training materials.
[0072] Specifically, the implementation process of collecting and processing domain data in this application can be as follows: Figure 2 As shown, it mainly includes:
[0073] Step 201: Gather domain knowledge.
[0074] In this embodiment, the domain data is collectively referred to as set G = {g i}={c i ,t i}, each piece of domain data g i All are described by information c i and tag t i (This application takes the example of domain knowledge being a label for descriptive information) where the descriptive information c i It can be a text introducing knowledge in a certain field, tagged t i This describes the domain knowledge introduced in the descriptive information. To reduce machine resource consumption, embodiments of this application limit the label t. i The text length must not exceed 100 characters.
[0075] Step 202: Determine whether the domain data contains tags.
[0076] For domain data containing tags, simply retrieve the tags from the domain data. For tags that do not contain domain data, proceed to step 203.
[0077] Step 203: Automatically construct labels based on description information.
[0078] It should be noted that, in order to meet the requirements of model training, the embodiments of this application limit each piece of domain data g. i Each has a matching tag t i Regarding the tag t i The choice of data in some fields i Naturally contains the tag t i ; and for those that do not naturally contain the tag t i domain data g i To save on manual annotation costs, based on the description information c i Based on its characteristics, the following two methods can be selected to automatically construct / tags. i :
[0079] The first method involves selecting description information c. i The first m sentences;
[0080] The second method involves selecting description information c. i The m sentences with the highest overlap with other sentences, in detail, the initial label t. i If empty, calculate the difference s between the description information and the label. i =c i -t i Choose a sentence from the set s such that the difference set s i And the tag t at this moment i The Rouge-F1 score was the highest, so it was included in the candidate tag t. iIn the process, repeat the above steps m times to obtain a label t containing m sentences. i .
[0081] After acquiring relevant domain knowledge, a static domain knowledge base can be constructed. Optionally, in at least one embodiment of this application, the specific method for acquiring the static domain knowledge base includes:
[0082] Step S11: Obtain a positive sample and one or more negative samples corresponding to the positive sample.
[0083] It should be noted that the positive sample includes second domain knowledge and its descriptive information, where the second domain knowledge matches the descriptive information; the negative sample includes the descriptive information and third domain knowledge, where the third domain knowledge does not match the descriptive information, meaning the third domain knowledge can be understood as a negative label / title that does not match the descriptive information; it should be noted that "more than one negative sample" refers to one or more negative samples. It should also be noted that for a single domain of knowledge, typically one positive sample and one or more negative samples are set for that domain; however, this application involves multiple domains of knowledge, meaning that there are multiple positive samples in this embodiment, each corresponding to one or more negative samples.
[0084] Step S12: Iteratively update the model parameters of the dual model based on the positive sample and the one or more negative samples.
[0085] Optionally, the specific implementation process of this step can be as follows:
[0086] Step S121: Using the dual model, obtain the vector representations of the description information, the second domain knowledge, and the third domain knowledge, respectively.
[0087] Step S122: Obtain the first Euclidean distance based on the vector representation of the description information and the vector representation of the second domain knowledge.
[0088] Step S123: Obtain the second Euclidean distance based on the vector representation of the description information and the vector representation of the third domain knowledge.
[0089] It should be noted that steps S122 and S123 have no specific order. S122 can be executed first and then S123, or S123 can be executed first and then S122, or both can be executed simultaneously.
[0090] Step S124: Obtain the contrast loss based on the first Euclidean distance and the second Euclidean distance, and iteratively update the model parameters of the dual model with the contrast loss as the optimization objective.
[0091] It should be noted that the embodiments of this application employ a parameter-sharing dual model. For example, constructing positive and negative samples at a ratio of 1:4 is used. Figure 3 As shown, the obtained positive and negative sample pairs are fed into the dual model to obtain the domain knowledge description information c. i A is a vector representation of a positive sample, and t is the label of the positive sample. i The vector representation B, and the labels t corresponding to the four negative samples. j(j≠i) The vector representation is E = {e1, e2, e3, e4}. Calculate the descriptive information c. i The label t corresponding to the positive sample i The Euclidean distance d(A,B) = ||AB||2; calculate the descriptive information c. i The labels t corresponding to the 4 negative samples j(j≠i) The Euclidean distance d(A,E) = ||AE||2; then contrast loss is applied. The optimization objective is defined as follows: r ∈ {0, 1}, where r = 1 for positive sample pairs and r = 0 for negative sample pairs. After the dual model has been iteratively optimized, the descriptive information c is finally stored in static form. i The vector representation A, and its corresponding label t. i .
[0092] It should also be noted that when using negative samples for dual model training, negative samples need to be obtained first. Optionally, the method for obtaining negative samples provided in at least one embodiment of this application specifically includes the following process:
[0093] Step S21: Obtain the description information and N domain knowledge that do not match the description information.
[0094] Where N is a positive integer.
[0095] Step S22: Calculate the matching degree between the vector representations of the N domain knowledge and the vector representations of the description information.
[0096] Step S23: Select M pieces of the third domain knowledge from the N domain knowledge in descending order of matching degree between the vector representations of the N domain knowledge and the vector representations of the description information.
[0097] Where M is a positive integer, and M is less than or equal to N.
[0098] Step S24: Combine the M third-domain knowledge and description information to form M negative samples.
[0099] For example, in the implementation of this application, positive and negative samples are constructed at a ratio of 1:4, and for each descriptive information c i It is related to the tag ti Matching negative samples are labels t that are similar to the descriptive information but do not match. j(j≠i) In detail, such as Figure 4 As shown, the negative sample is constructed by preparing descriptive information c. i and any number of descriptive information c i Mismatched tag t j(j≠i) The data is sequentially fed into the Bidirectional Encoder Representation from Transformers (BERT) model. Average pooling is then performed on the output vectors of each token from the last hidden layer of BERT (excluding the [CLS] symbol) to obtain the descriptive information c. i The vector representation P, and multiple labels t j(j≠i) The vector representation of Q is then used. Furthermore, matrix multiplication is employed to batch-compute the descriptive information c. i and multiple tags t j(j≠i) The vector similarity sim(P,Q) = PQ T Select the 4 tags with the highest similarity scores from them. j(j≠i) , respectively with the description information c i The group is a negative sample.
[0100] Step S13: Determine the vector representation of the description information output by the updated dual model as the description vector representation corresponding to the second domain knowledge.
[0101] Step S14: Store the second domain knowledge and the corresponding description vector representation of the second domain knowledge in the static knowledge base.
[0102] By training the dual model, obtaining the descriptive vector representation of domain knowledge based on the dual model, and storing the domain knowledge and its descriptive vector representation, a static domain knowledge base is formed.
[0103] This application embodiment obtains the vector representation of the domain knowledge description information (i.e., the aforementioned description vector representation) based on the intermediate output of the trained dual model. The domain knowledge and its description vector representation are then associated and stored in a static domain knowledge base. This allows for the matching of domain knowledge to the target text by matching the target text's vector representation with the description vector representation in subsequent processes. The acquisition of the description vector representation is achieved through training the dual model. This reduces a significant amount of manual annotation. Furthermore, because the correspondence between the description vector representation and domain knowledge is more accurate, the domain knowledge matched to the target text using this application embodiment is more accurate, further improving the quality of the final summary generated for the target text.
[0104] Optionally, in at least one embodiment of this application, step 104 is specifically implemented by the following process:
[0105] Step 1041: Concatenate the K pieces of first domain knowledge with the target text to obtain K input texts.
[0106] It should be noted that the splicing here can be understood as connecting the first domain knowledge with the target text. For example, the target text can be placed first, followed by the first domain knowledge; or the first domain knowledge can be placed first, followed by the target text.
[0107] It should be noted that the dialogue summary data used in this application embodiment can be the transcribed text of customer service-customer dialogue in a human customer service scenario, and its summary annotated by professionals. In this application embodiment, the dialogue summary data is collectively referred to as H = {h i}={x i ,y i Each dialogue summary data h i All are composed of dialogue content x i and summary y i Composition, including dialogue content x i It is a text of a dialogue, summary y i This is a summary of the key information in this dialogue. The goal of the dialogue summarization task is to: based on the given dialogue content x i Automatically generate summary y i To make the model generate a summary y i To ensure greater reliability and completeness of domain knowledge, this application matches corresponding domain knowledge from a static domain knowledge base. Specifically, it first matches the dialogue content x... i The input is fed into the dual model mentioned earlier, and the output dialogue content x is generated. i The vector representation of F is then used, and the similarity is calculated as sim(F,A) = FA. T To obtain the score, the k descriptive pieces c with the highest similarity to the vector representation F are retrieved from the static domain knowledge base. i Let A be a vector representation of A. In the static domain knowledge base, each vector representation A has a corresponding label t. i Therefore, further, we obtain the k labels t corresponding to the k vector representations A. i Finally, using "[SEP]" as the concatenation character, the k tags t are combined. i Each with the dialogue content x i By concatenating the elements, we obtain k dialogue contents x that are concatenated with domain knowledge. * i At the same time, new dialogue summary data H is obtained. *＝ {h* i}={x * i ,y i}
[0108] Step 1042: Input the K input texts into the encoder-decoder model to obtain K output distributions.
[0109] Step 1043: Based on the matching degree between each of the first vector representations and the target vector representation, and the K output distributions, determine the summary corresponding to the target text.
[0110] Optionally, in at least one embodiment of this application, step 1043 is specifically implemented as follows:
[0111] Using the matching degree between each of the K first vector representations and the target vector representation as weights, the K output distributions are weighted and summed to obtain the output distribution of the summary of the target text;
[0112] Based on the output distribution of the target text's summary, a summary corresponding to the target text is generated using a beam search method.
[0113] It should be noted that the dialogue summarization generation model selected in this application embodiment is the T5 model with an encoder-decoder structure, such as... Figure 5 As shown, the k dialogue contents x obtained in the previous step, which are concatenated with domain knowledge, are... * i The data is fed into a T5 model, resulting in k output distributions. To fuse these distributions, this application uses the k softmax-normalized retrieval scores as weights to calculate a weighted sum of the k output distributions, obtaining the final output distribution. BeamSech is then used to generate the corresponding summary text.
[0114] It should be noted that the above process can be understood as the actual process of obtaining the summary, or as the training process of the summary acquisition model. Optionally, the model training phase may also include:
[0115] The encoder-decoder model is iteratively updated with the cross-entropy between the output distribution of the target text summary and the target summary as the optimization objective.
[0116] It should be noted that during the model training phase, when inputting target text, it is necessary to manually annotate the target text with a summary so that the model can be iteratively updated using the manually annotated summary. In other words, the target summary mentioned above refers to the manually annotated summary.
[0117] It should be noted that the specific implementation process of manual annotation and summarization can be as follows: by using speech recognition technology, the text of customer service-user dialogue in the context of human customer service consultation is collected to obtain the target text. Then, with the premise of having a good grasp of the domain knowledge, the annotator completes the summary annotation of each dialogue to obtain the manually annotated summary of the target text. This manually annotated summary can be understood as the standard summary content.
[0118] It should also be noted that, with the cross-entropy between the output distribution of the summary and the manually annotated summary as the optimization objective, the model is continuously iterated and optimized. After the cross-entropy between the output distribution of the summary and the manually annotated summary reaches its optimum, the target model is selected as the summary generation model from the training models generated during the iterative update process based on the method of Recall-Oriented Understudy for GistingEvaluation-longest common subsequence (Rouge-L).
[0119] In summary, the dialogue summarization technology based on domain knowledge retrieval proposed in at least one embodiment of this application has the following advantages compared to existing technical solutions:
[0120] 1. When introducing external knowledge, complex manual annotation is unnecessary. Annotators only need to annotate the dialogue summary and select matching domain tags for the dialogue content. The domain knowledge data to be introduced can be unstructured, thus increasing the available domain knowledge data. Because the vector representation of external knowledge is stored in a static form, domain knowledge can be matched more efficiently. The method of integrating domain knowledge during the generation process is also more efficient.
[0121] 2. Customized for specific scenarios / domains, ready to use out of the box, highly applicable, and adopts dialogue summary generation technology for specific scenarios / domains.
[0122] like Figure 6 As shown, at least one embodiment of this application also provides a summary generation apparatus 600, comprising:
[0123] The first acquisition module 601 is used to acquire the target vector representation, wherein the target vector representation is a vector representation of the target text;
[0124] The first determining module 602 is used to determine K first vector representations from the description vector representations in the static domain knowledge base according to the order of matching degree between the description vector representations in the static domain knowledge base and the target vector representation from high to low, where K is an integer greater than or equal to 1, and the static domain knowledge base stores multiple domain knowledge and description vector representations corresponding to each domain knowledge;
[0125] The second acquisition module 603 is used to acquire K first domain knowledge from a static domain knowledge base, wherein the K first domain knowledge corresponds to the K first vector representations respectively;
[0126] The generation module 604 is used to generate a summary corresponding to the target text based on the K first domain knowledge and the matching degree between each first domain knowledge and the target text.
[0127] Optionally, the static domain knowledge base is obtained through the following methods:
[0128] Obtain a positive sample and one or more negative samples corresponding to the positive sample. The positive sample includes second domain knowledge and description information of the second domain knowledge. The second domain knowledge matches the description information. The negative sample includes the description information and third domain knowledge. The third domain knowledge does not match the description information.
[0129] The model parameters of the dual model are iteratively updated based on the positive samples and the one or more negative samples.
[0130] The vector representation of the description information output by the updated dual model is determined as the description vector representation corresponding to the second domain knowledge;
[0131] The static knowledge base stores the knowledge of the second domain and the corresponding description vector representation of the knowledge of the second domain.
[0132] Optionally, the implementation of iteratively updating the model parameters of the dual model based on the positive samples and the one or more negative samples includes:
[0133] Using a dual model, vector representations of the description information, the second domain knowledge, and the third domain knowledge are obtained respectively;
[0134] The first Euclidean distance is obtained based on the vector representation of the description information and the vector representation of the second domain knowledge;
[0135] The second Euclidean distance is obtained based on the vector representation of the description information and the vector representation of the third domain knowledge;
[0136] The contrast loss is obtained based on the first Euclidean distance and the second Euclidean distance. The model parameters of the dual model are iteratively updated with the contrast loss as the optimization objective.
[0137] Optionally, the negative samples are obtained in the following way:
[0138] Obtain the description information and N domain knowledge that do not match the description information;
[0139] Calculate the matching degree between the vector representations of the N domain knowledge and the vector representations of the description information, respectively;
[0140] According to the order of matching degree between the vector representations of the N domain knowledge and the vector representation of the description information from high to low, select M third domain knowledge from the N domain knowledge;
[0141] Each of the M third-domain knowledge and descriptive information is combined to form M negative samples;
[0142] Where M and N are positive integers, and M is less than or equal to N.
[0143] Optionally, the generation module 604 includes:
[0144] The concatenation unit is used to concatenate the K pieces of first domain knowledge with the target text respectively to obtain K input texts;
[0145] The acquisition unit is used to input the K input texts into the encoder-decoder model and acquire K output distributions;
[0146] The determining unit is configured to determine the summary corresponding to the target text based on the matching degree between each of the first vector representation and the target vector representation and the K output distributions.
[0147] Optionally, the determining unit is configured to:
[0148] Using the matching degree between each of the K first vector representations and the target vector representation as weights, the K output distributions are weighted and summed to obtain the output distribution of the summary of the target text;
[0149] Based on the output distribution of the target text summary, a summary corresponding to the target text is generated using a beam search method.
[0150] Optionally, the device further includes:
[0151] The update module is used to iteratively update the encoder-decoder model with the output distribution of the target text summary and the cross-entropy of the target summary as the optimization objective.
[0152] Optionally, the target text is the dialogue content, and the summary is a summary of the dialogue content.
[0153] It should be noted that if the apparatus provided in at least one embodiment of this application is an apparatus capable of performing the above-described abstract generation method, then all embodiments of the above-described abstract generation method are applicable to the apparatus and can achieve the same or similar beneficial effects.
[0154] At least one embodiment of this application also provides a summary generation apparatus, which includes a transceiver and a processor;
[0155] The processor is configured to: acquire a target vector representation, wherein the target vector representation is a vector representation of the target text;
[0156] According to the order of matching degree between the description vector representation in the static domain knowledge base and the target vector representation from high to low, K first vector representations are determined from the description vector representations in the static domain knowledge base, where K is an integer greater than or equal to 1. The static domain knowledge base stores multiple domain knowledge and the description vector representation corresponding to each domain knowledge.
[0157] K pieces of first domain knowledge are obtained from a static domain knowledge base, and the K pieces of first domain knowledge correspond to the K first vector representations respectively;
[0158] Based on the K first domain knowledge points and the matching degree between each first domain knowledge point and the target text, a summary corresponding to the target text is generated.
[0159] Optionally, the processor is configured to:
[0160] Obtain a positive sample and one or more negative samples corresponding to the positive sample. The positive sample includes second domain knowledge and description information of the second domain knowledge. The second domain knowledge matches the description information. The negative sample includes the description information and third domain knowledge. The third domain knowledge does not match the description information.
[0161] The model parameters of the dual model are iteratively updated based on the positive samples and the one or more negative samples.
[0162] The vector representation of the description information output by the updated dual model is determined as the description vector representation corresponding to the second domain knowledge;
[0163] The static knowledge base stores the knowledge of the second domain and the corresponding description vector representation of the knowledge of the second domain.
[0164] Optionally, a dual model is used to obtain vector representations of the description information, the second domain knowledge, and the third domain knowledge, respectively.
[0165] The first Euclidean distance is obtained based on the vector representation of the description information and the vector representation of the second domain knowledge;
[0166] The second Euclidean distance is obtained based on the vector representation of the description information and the vector representation of the third domain knowledge;
[0167] The contrast loss is obtained based on the first Euclidean distance and the second Euclidean distance. The model parameters of the dual model are iteratively updated with the contrast loss as the optimization objective.
[0168] Optionally, the processor is configured to:
[0169] Obtain the description information and N domain knowledge that do not match the description information;
[0170] Calculate the matching degree between the vector representations of the N domain knowledge and the vector representations of the description information, respectively;
[0171] According to the order of matching degree between the vector representations of the N domain knowledge and the vector representation of the description information from high to low, select M third domain knowledge from the N domain knowledge;
[0172] Each of the M third-domain knowledge and descriptive information is combined to form M negative samples;
[0173] Where M and N are positive integers, and M is less than or equal to N.
[0174] Optionally, the processor is configured to: concatenate the K first domain knowledge points with the target text to obtain K input texts;
[0175] Input the K input texts into the encoder-decoder model to obtain K output distributions;
[0176] Based on the matching degree between each of the first vector representation and the target vector representation, and the K output distributions, the summary corresponding to the target text is determined.
[0177] Optionally, the processor is further configured to:
[0178] Using the matching degree between each of the K first vector representations and the target vector representation as weights, the K output distributions are weighted and summed to obtain the output distribution of the summary of the target text;
[0179] Based on the output distribution of the target text summary, a summary corresponding to the target text is generated using a beam search method.
[0180] Optionally, the processor is further configured to:
[0181] The encoder-decoder model is iteratively updated with the cross-entropy between the output distribution of the target text summary and the target summary as the optimization objective.
[0182] Optionally, the target text is the dialogue content, and the summary is a summary of the dialogue content.
[0183] like Figure 7 As shown, this embodiment of the invention also provides a digest generation apparatus, including a processor 700, a transceiver 710, a memory 720, and a program stored in the memory 720 and executable on the processor 700; wherein the transceiver 710 is connected to the processor 700 and the memory 720 via a bus interface, and the processor 700 is used to read the program in the memory and execute the following processes:
[0184] Obtain the target vector representation, which is a vector representation of the target text;
[0185] According to the order of matching degree between the description vector representation in the static domain knowledge base and the target vector representation from high to low, K first vector representations are determined from the description vector representations in the static domain knowledge base, where K is an integer greater than or equal to 1. The static domain knowledge base stores multiple domain knowledge and the description vector representation corresponding to each domain knowledge.
[0186] K pieces of first domain knowledge are obtained from a static domain knowledge base, and the K pieces of first domain knowledge correspond to the K first vector representations respectively;
[0187] Based on the K first domain knowledge points and the matching degree between each first domain knowledge point and the target text, a summary corresponding to the target text is generated.
[0188] Transceiver 710 is used to receive and send data under the control of processor 700.
[0189] Among them, Figure 7 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 700 and memory represented by memory 720 together. The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 710 can be multiple components, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium, including wireless channels, wired channels, optical fibers, etc. For different user equipment, the user interface 730 can also be an interface capable of connecting external or internal devices, including but not limited to keypads, displays, speakers, microphones, joysticks, etc.
[0190] The processor 700 is responsible for managing the bus architecture and general processing, while the memory 720 can store the data used by the processor 700 during operation.
[0191] Optionally, the processor 700 can be a CPU (Central Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or CPLD (Complex Programmable Logic Device), and the processor can also adopt a multi-core architecture.
[0192] The processor executes any of the methods described in the embodiments of this application according to the obtained executable instructions by calling a computer program stored in memory. The processor and memory may also be physically separated.
[0193] Optionally, the processor 700 is used to read a program from memory and also execute the following processes:
[0194] Obtain a positive sample and one or more negative samples corresponding to the positive sample. The positive sample includes second domain knowledge and description information of the second domain knowledge. The second domain knowledge matches the description information. The negative sample includes the description information and third domain knowledge. The third domain knowledge does not match the description information.
[0195] The model parameters of the dual model are iteratively updated based on the positive samples and the one or more negative samples.
[0196] The vector representation of the description information output by the updated dual model is determined as the description vector representation corresponding to the second domain knowledge;
[0197] The static knowledge base stores the knowledge of the second domain and the corresponding description vector representation of the knowledge of the second domain.
[0198] Optionally, the processor 700 is used to read a program from memory and also execute the following processes:
[0199] Using a dual model, vector representations of the description information, the second domain knowledge, and the third domain knowledge are obtained respectively;
[0200] The first Euclidean distance is obtained based on the vector representation of the description information and the vector representation of the second domain knowledge;
[0201] The second Euclidean distance is obtained based on the vector representation of the description information and the vector representation of the third domain knowledge;
[0202] The contrast loss is obtained based on the first Euclidean distance and the second Euclidean distance. The model parameters of the dual model are iteratively updated with the contrast loss as the optimization objective.
[0203] Optionally, the processor 700 is used to read a program from memory and also execute the following processes:
[0204] Obtain the description information and N domain knowledge that do not match the description information;
[0205] Calculate the matching degree between the vector representations of the N domain knowledge and the vector representations of the description information, respectively;
[0206] According to the order of matching degree between the vector representations of the N domain knowledge and the vector representation of the description information from high to low, select M third domain knowledge from the N domain knowledge;
[0207] Each of the M third-domain knowledge and descriptive information is combined to form M negative samples;
[0208] Where M and N are positive integers, and M is less than or equal to N.
[0209] Optionally, the processor 700 is used to read a program from memory and also execute the following processes:
[0210] The K pieces of first domain knowledge are concatenated with the target text to obtain K input texts;
[0211] Input the K input texts into the encoder-decoder model to obtain K output distributions;
[0212] Based on the matching degree between each of the first vector representation and the target vector representation, and the K output distributions, the summary corresponding to the target text is determined.
[0213] Optionally, the processor 700 is used to read a program from memory and also execute the following processes:
[0214] Using the matching degree between each of the K first vector representations and the target vector representation as weights, the K output distributions are weighted and summed to obtain the output distribution of the summary of the target text;
[0215] Based on the output distribution of the target text summary, a summary corresponding to the target text is generated using a beam search method.
[0216] Optionally, the processor 700 is used to read a program from memory and also execute the following processes:
[0217] The encoder-decoder model is iteratively updated with the cross-entropy between the output distribution of the target text summary and the target summary as the optimization objective.
[0218] Optionally, the target text is the dialogue content, and the summary is a summary of the dialogue content.
[0219] At least one embodiment of this application also provides a digest generation apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the various processes in the digest generation method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.
[0220] At least one embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the various processes in the summary generation method embodiments described above and achieves the same technical effects. To avoid repetition, further details are omitted here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.
[0221] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0222] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0223] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A method for generating abstracts, characterized in that, include: Obtain the target vector representation, which is a vector representation of the target text; According to the order of matching degree between the description vector representation in the static domain knowledge base and the target vector representation from high to low, K first vector representations are determined from the description vector representations in the static domain knowledge base, where K is an integer greater than or equal to 1. The static domain knowledge base stores multiple domain knowledge and the description vector representation corresponding to each domain knowledge. K pieces of first domain knowledge are obtained from a static domain knowledge base, and the K pieces of first domain knowledge correspond to the K first vector representations respectively; Based on the K first domain knowledge points and the matching degree between each first domain knowledge point and the target text, a summary corresponding to the target text is generated; The static domain knowledge base is obtained through the following methods: Obtain a positive sample and one or more negative samples corresponding to the positive sample. The positive sample includes second domain knowledge and description information of the second domain knowledge. The second domain knowledge matches the description information. The negative sample includes the description information and third domain knowledge. The third domain knowledge does not match the description information. The model parameters of the dual model are iteratively updated based on the positive samples and the one or more negative samples. The vector representation of the description information output by the updated dual model is determined as the description vector representation corresponding to the second domain knowledge; The static domain knowledge base stores the second domain knowledge and the corresponding description vector representation of the second domain knowledge. The negative samples are obtained in the following way: Obtain the description information and N domain knowledge that do not match the description information; Calculate the matching degree between the vector representations of the N domain knowledge and the vector representations of the description information, respectively; According to the order of matching degree between the vector representations of the N domain knowledge and the vector representation of the description information from high to low, M third domain knowledge are selected from the N domain knowledge; Each of the M third-domain knowledge and descriptive information is combined to form M negative samples; Where M and N are positive integers, and M is less than or equal to N.
2. The method according to claim 1, characterized in that, The iterative update of the model parameters of the dual model based on the positive samples and the one or more negative samples includes: Using a dual model, vector representations of the description information, the second domain knowledge, and the third domain knowledge are obtained respectively; The first Euclidean distance is obtained based on the vector representation of the description information and the vector representation of the second domain knowledge; The second Euclidean distance is obtained based on the vector representation of the description information and the vector representation of the third domain knowledge; The contrast loss is obtained based on the first Euclidean distance and the second Euclidean distance. The model parameters of the dual model are iteratively updated with the contrast loss as the optimization objective.
3. The method according to any one of claims 1-2, characterized in that, The step of generating a summary corresponding to the target text based on the K first domain knowledge points and the matching degree between each first domain knowledge point and the target text includes: The K pieces of first domain knowledge are concatenated with the target text to obtain K input texts; Input the K input texts into the encoder-decoder model to obtain K output distributions; Based on the matching degree between each of the first vector representation and the target vector representation, and the K output distributions, the summary corresponding to the target text is determined.
4. The method according to claim 3, characterized in that, The step of generating a summary corresponding to the target text based on the matching degree between each of the first vector representations and the target vector representation, and the K output distributions, includes: Using the matching degree between each of the K first vector representations and the target vector representation as weights, the K output distributions are weighted and summed to obtain the output distribution of the summary of the target text; Based on the output distribution of the target text summary, a summary corresponding to the target text is generated using a beam search method.
5. The method according to claim 4, characterized in that, The method further includes: The encoder-decoder model is iteratively updated with the cross-entropy between the output distribution of the target text summary and the target summary as the optimization objective.
6. The method according to any one of claims 1-2, characterized in that, The target text is the dialogue content, and the summary is a summary of the dialogue content.
7. A summary generation apparatus, characterized in that, include: The first acquisition module is used to acquire the target vector representation, wherein the target vector representation is a vector representation of the target text; The first determining module is used to determine K first vector representations from the description vector representations in the static domain knowledge base according to the order of matching degree between the description vector representations in the static domain knowledge base and the target vector representation from high to low, where K is an integer greater than or equal to 1, and the static domain knowledge base stores multiple domain knowledge and description vector representations corresponding to each domain knowledge; The second acquisition module is used to acquire K pieces of first domain knowledge from a static domain knowledge base, wherein the K pieces of first domain knowledge correspond to the K first vector representations respectively; The generation module is used to generate a summary corresponding to the target text based on the K first domain knowledge and the matching degree between each first domain knowledge and the target text; The static domain knowledge base is obtained through the following methods: Obtain a positive sample and one or more negative samples corresponding to the positive sample. The positive sample includes second domain knowledge and description information of the second domain knowledge. The second domain knowledge matches the description information. The negative sample includes the description information and third domain knowledge. The third domain knowledge does not match the description information. The model parameters of the dual model are iteratively updated based on the positive samples and the one or more negative samples. The vector representation of the description information output by the updated dual model is determined as the description vector representation corresponding to the second domain knowledge; The static domain knowledge base stores the second domain knowledge and the corresponding description vector representation of the second domain knowledge. The negative samples are obtained in the following way: Obtain the description information and N domain knowledge that do not match the description information; Calculate the matching degree between the vector representations of the N domain knowledge and the vector representations of the description information, respectively; According to the order of matching degree between the vector representations of the N domain knowledge and the vector representation of the description information from high to low, M third domain knowledge are selected from the N domain knowledge; Each of the M third-domain knowledge and descriptive information is combined to form M negative samples; Where M and N are positive integers, and M is less than or equal to N.
8. A summary generation apparatus, characterized in that, Includes transceivers and processors; The processor is configured to: acquire a target vector representation, wherein the target vector representation is a vector representation of the target text; According to the order of matching degree between the description vector representation in the static domain knowledge base and the target vector representation from high to low, K first vector representations are determined from the description vector representations in the static domain knowledge base, where K is an integer greater than or equal to 1. The static domain knowledge base stores multiple domain knowledge and the description vector representation corresponding to each domain knowledge. K pieces of first domain knowledge are obtained from a static domain knowledge base, and the K pieces of first domain knowledge correspond to the K first vector representations respectively; Based on the K first domain knowledge points and the matching degree between each first domain knowledge point and the target text, a summary corresponding to the target text is generated; The processor is used for: Obtain a positive sample and one or more negative samples corresponding to the positive sample. The positive sample includes second domain knowledge and description information of the second domain knowledge. The second domain knowledge matches the description information. The negative sample includes the description information and third domain knowledge. The third domain knowledge does not match the description information. The model parameters of the dual model are iteratively updated based on the positive samples and the one or more negative samples. The vector representation of the description information output by the updated dual model is determined as the description vector representation corresponding to the second domain knowledge; The static domain knowledge base stores the second domain knowledge and the corresponding description vector representation of the second domain knowledge. The processor is used for: Obtain the description information and N domain knowledge that do not match the description information; Calculate the matching degree between the vector representations of the N domain knowledge and the vector representations of the description information, respectively; According to the order of matching degree between the vector representations of the N domain knowledge and the vector representation of the description information from high to low, M third domain knowledge are selected from the N domain knowledge; Each of the M third-domain knowledge and descriptive information is combined to form M negative samples; Where M and N are positive integers, and M is less than or equal to N.
9. A summary generation apparatus, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the summary generation method as described in any one of claims 1-6.
10. A readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the digest generation method as described in any one of claims 1-6.