An abstract generation method, apparatus and related device
By dividing text information into word embeddings and matching word type vectors to generate fusion vectors, the problem of key information being ignored in existing technologies is solved, and more accurate summary generation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD
- Filing Date
- 2021-11-01
- Publication Date
- 2026-07-21
AI Technical Summary
In existing technologies, due to the uneven proportion of different types of text in the input text, the lack of distinction during encoding can lead to the omission of some key information, resulting in the inability to accurately express the key content of the acquired text information when generating a summary.
The text information is divided into multiple word embeddings, and a corresponding word type vector is matched for each word embedding. The word embeddings and word type vectors are fused by addition or direct connection to generate multiple fused vectors, which are then input into the encoder-decoder generative automatic summarization model to generate a summary.
By matching and fusing word embeddings with word type vectors, we can better represent the key content of text information, ensure that low-frequency keywords are reflected in the summary, and improve the accuracy of the summary.
Smart Images
Figure CN115934934B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of deep learning technology, and in particular to a summarization method, apparatus and related equipment. Background Technology
[0002] Generative automatic summarization is a task that involves automatically generating a summary from input text. Existing technologies primarily involve an encoder reading the acquired text information, followed by a decoder decoding the encoder's output to generate a summary. However, due to the uneven proportion of different text types in the input text, indiscriminate encoding can lead to the omission of certain key information, resulting in a summary that fails to accurately represent the core content of the acquired text. Summary of the Invention
[0003] To address the problems existing in the prior art, this disclosure provides a method, apparatus, and related equipment for generating abstracts.
[0004] In a first aspect, this disclosure provides a summary generation method, comprising: dividing acquired text information into multiple word embeddings; matching each word embedding with a corresponding word type vector; fusing each word embedding with its corresponding word type vector to obtain multiple fusion vectors for representing the text information; and generating a summary based on the multiple fusion vectors.
[0005] According to the summary generation method provided in this disclosure, fusing each word embedding with its corresponding word type vector specifically includes: fusing each word embedding with its corresponding word type vector using either addition or direct connection; and the formula for the addition is:
[0006] Vector(w i ) = E i +T i
[0007] The formula for the direct connection form is:
[0008] Vector(w i ) = [E i ;T i ]
[0009] Among them, w i Represents word embedding, Vector(w i ) indicates word embedding w i The fusion vector, E i T represents the i-th column of the word vector matrix E. i Indicates word embedding w i The corresponding word type vector.
[0010] According to the summary generation method provided in this disclosure, the step of generating a summary based on the plurality of fusion vectors specifically involves: inputting the plurality of fusion vectors into an encoder-decoder generative automatic summarization model and outputting a summary; the encoder-decoder generative automatic summarization model includes an encoder and a decoder; the encoder is used to encode the fusion vectors to generate an encoder hidden layer sequence; the decoder is used to decode the encoder hidden layer sequence to generate a summary.
[0011] According to the summary generation method provided in this disclosure, the word type vector includes: a category vector, a keyword vector, and multiple preset word embedding attribute vectors.
[0012] According to the summary generation method provided in this disclosure, adjusting and determining the word type vector specifically includes: determining the correct summary of the sample text; calculating the difference between the summary generated based on the sample text and the correct summary based on a loss function; adjusting the parameters corresponding to the word type vector based on the difference; wherein the sample text is multiple texts stored in advance.
[0013] According to the summary generation method provided in this disclosure, the step of fusing each word embedding with its corresponding word type vector is: fusing each word embedding with its corresponding word type vector based on a recurrent neural network.
[0014] Secondly, this disclosure also provides a summary generation device, comprising: a word segmentation module, a matching module, a fusion module, and a summary generation module. The word segmentation module is used to divide the acquired text information into multiple word embeddings; the matching module is used to match a corresponding word type vector to each word embedding; the fusion module is used to fuse each word embedding with its corresponding word type vector to obtain multiple fused vectors representing the text information; and the summary generation module is used to generate a summary based on the multiple fused vectors.
[0015] Thirdly, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the above-described digest generation methods.
[0016] Fourthly, this disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described digest generation methods.
[0017] Fifthly, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described digest generation methods.
[0018] This disclosure provides a summarization method, apparatus, and related equipment. The method involves dividing acquired text information into multiple word embeddings; matching each word embedding with a corresponding word type vector; and fusing each word embedding with its corresponding word type vector to obtain multiple fused vectors representing the text information. Because the fused vectors obtained by matching and fusing word embeddings with their corresponding word type vectors can better represent the unique information of the acquired text information—that is, keywords in the acquired text information may be low-frequency information in a large amount of text—by fusing word embeddings with their corresponding word type vectors to obtain multiple fused vectors representing the text information, these low-frequency features, i.e., keywords in the acquired text information, can be considered when generating a summary based on these multiple fused vectors, thus better representing the key content of the acquired text information. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is one of the flowcharts illustrating the abstract generation method provided in this disclosure;
[0021] Figure 2 This is a flowchart illustrating the steps involved in training the parameters corresponding to the word type vectors in the summary generation method provided in this publication.
[0022] Figure 3 This is one of the structural schematic diagrams of the abstract generation device provided in this disclosure;
[0023] Figure 4 This is the second schematic diagram of the abstract generation device provided in this disclosure;
[0024] Figure 5 This is a schematic diagram of the configuration module in the second schematic diagram of the abstract generation device provided in this disclosure;
[0025] Figure 6 This is a flowchart illustrating a specific embodiment of the abstract generation method provided in this disclosure;
[0026] Figure 7 This is a schematic diagram of the structure of the electronic device provided in this disclosure. Detailed Implementation
[0027] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this disclosure.
[0028] Generative automatic summarization is a task that involves automatically generating a summary from input text. Existing technologies primarily involve an encoder reading the acquired text information, followed by a decoder decoding the encoder's output to generate a summary. However, due to the uneven proportion of different text types in the input text, indiscriminate encoding can lead to the omission of certain key information, resulting in a summary that fails to accurately represent the core content of the acquired text.
[0029] For example, in a specific implementation scenario, the item we want to generate a summary for is a stereo. However, during the training process of the generative automatic summarization model, the training data for televisions far exceeds that for stereos. Although the input text content is unrelated to televisions, the output summary includes televisions, or even replaces the described item with televisions, which does not match the actual meaning we want to express.
[0030] To address this problem, this embodiment provides a summary generation method. The following describes the method in conjunction with... Figure 1 Various embodiments of the summary generation method described in this disclosure are presented.
[0031] Reference Figure 1 , Figure 1 This is one of the flowcharts illustrating the abstract generation method provided in this disclosure. The abstract generation method provided in this embodiment includes:
[0032] Step S101: Divide the acquired text information into multiple word embeddings;
[0033] Step S103: Embed and match the corresponding word type vector for each word.
[0034] Step S105: Fuse each word embedding with its corresponding word type vector to obtain multiple fused vectors for representing text information;
[0035] Step S107: Generate a summary based on multiple fusion vectors.
[0036] The summary generation method provided in this disclosure divides the acquired text information into multiple word embeddings; matches each word embedding with a corresponding word type vector; and fuses each word embedding with its corresponding word type vector to obtain multiple fusion vectors for representing the text information. Because the word embeddings are matched and fused with their corresponding word type vectors, the resulting fusion vectors can better represent the unique information of the acquired text information. In other words, the keywords in the acquired text information may be low-frequency information in a large amount of text, but by fusing the word embeddings with their corresponding word type vectors to obtain multiple fusion vectors for representing the text information, these low-frequency features, i.e., the keywords in the acquired text information, can be taken into account when generating a summary based on multiple fusion vectors, thereby better representing the key content of the acquired text information.
[0037] The following is about Figure 1 The steps of the illustrated embodiment will be further explained.
[0038] Step S101: Divide the acquired text information into multiple word embeddings;
[0039] The text information obtained in step S101 is generally a document longer than 500 characters, which includes a detailed description of the item. Word embedding is a numerical representation of words, which typically maps a word to a high-dimensional vector (word vector) to represent that word.
[0040] Step S103: Embed and match the corresponding word type vector for each word.
[0041] Specifically, word type vectors are used to classify the sources of word embeddings. By matching word embeddings with their sources, the source of each word embedding is determined. Word type vectors are randomly initialized during the training of the generative automatic summarization model and are updated as the model learns.
[0042] Step S105: Fuse each word embedding with its corresponding word type vector to obtain multiple fused vectors for representing text information;
[0043] Specifically, the fusion vector includes both word embeddings and word type vectors corresponding to the word embeddings, and in some cases, position vectors.
[0044] Reference Figure 6 , Figure 6 This is a flowchart illustrating a specific embodiment of the summary generation method provided in this disclosure. The method involves fusing word embeddings with word type vectors corresponding to the word embeddings to generate a fused vector, which is then input into a recurrent neural network or a Transformer-based generative automatic summarization model to output a summary.
[0045] Step S107: Generate a summary based on multiple fusion vectors.
[0046] Specifically, summaries can be generated using generative automatic summarization models based on RNNs or Transformers, or other deep learning-based generative automatic summarization models; there are no restrictions here. Transformer-based generative automatic summarization models can also use position vectors to specify the exact order of text positions during encoding.
[0047] For low-frequency keywords of specific items in text descriptions, fused vectors can be learned by the model better than simple word embeddings, thus ensuring that low-frequency keywords are reflected in the summary.
[0048] In one alternative embodiment, for step S105, each word can be embedded and fused with its corresponding word type vector in either an additive or a direct connection manner.
[0049] The formula for fusing each word with its corresponding word type vector using a direct connection method is as follows:
[0050] Vector(w i ) = [E i ;T i ]
[0051] Among them, w i Represents word embedding, Vector(w i ) indicates word embedding w i The fusion vector, E i T represents the i-th column of the word vector matrix E. i Indicates word embedding w i The corresponding word type vector.
[0052] The formula for fusing the embedding of each word with its corresponding word type vector using an additive approach is as follows:
[0053] Vector(w i ) = E i +T i
[0054] Among them, w i Represents word embedding, Vector(w i ) indicates word embedding w i The fusion vector, E i T represents the i-th column of the word vector matrix E. i Indicates word embedding w i The corresponding word type vector.
[0055] In an optional embodiment, step S107 specifically includes: generating a summary based on multiple fusion vectors representing text information using a recurrent neural network or an encoder-decoder generative automatic summarization model; the encoder-decoder generative automatic summarization model includes an encoder and a decoder; the encoder is used to encode the fusion vectors to generate encoder hidden states; the decoder is used to decode the encoder hidden states to generate a summary.
[0056] The encoder-decoder generative automatic summarization model can be a Transformer-based generative automatic summarization model, or it can be other models evolved from the Transformer model; there are no restrictions here.
[0057] This explanation uses a Transformer-based generative automatic summarization model as an example. A Transformer model typically includes a decoder and an encoder, which are composed of multiple stacked base layers. Each base layer in the decoder consists of two sub-layers: an attention mechanism layer and a multi-layer fully connected layer. Each feedforward layer in the decoder consists of three sub-layers: an attention mechanism layer, a self-attention mechanism layer, and a multi-layer fully connected layer. Residual edges and layer normalization can be applied to each sub-layer of the decoding and encoding layers.
[0058] The core module of the Transformer-based generative automatic summarization model is the self-attention mechanism. Self-attention refers to the ability to connect information from different positions in the input sequence corresponding to the input text, calculate a certain representation of the entire sequence, and finally generate a summary based on this representation. Specifically, given a query and a key-value table, the self-attention mechanism can be the process of mapping the query to the correct input. Here, the query, key, value, and final output are all vectors. The output is usually in the form of a weighted sum, and the weights are determined by the query, key, and value.
[0059] Specifically, the fused vector is input into the Transformer-based generative automatic summarization model. First, the model encodes the fused vector to obtain the encoder hidden layer state. Here, the encoder layer hidden state is the state H of all time series of the hidden layer. The state H of all time series of the hidden layer includes the fused vector corresponding to each word embedding. The fused vector is input into the hidden layer state hi generated by the encoder of the Transformer-based generative automatic summarization model. Each hidden layer state hi is decoded to generate a summary word by word.
[0060] In one embodiment, the category vector, keyword vector, and multiple preset word embedding attribute vectors constitute a word type vector. Specifically, the multiple preset word embedding attribute vectors may include a first text vector, a second text vector, and a third text vector.
[0061] Specifically, the word type vector represents the specific source of each word embedding, and the category vector represents the specific category of the item, such as television, stereo, and mobile phone. The keyword vector represents the specific name of the object, such as portable stereo, high-definition smart TV, and women's mobile phone. The first text vector represents the text information from the title in the acquired text information. The second text vector represents the text information from the body text in the acquired text information. The third text vector represents the text information from the acquired text information describing the specifications and parameters of the item.
[0062] The parameters corresponding to all word type vectors are randomly initialized and updated during model learning to better adapt to the summary generation method.
[0063] The parameter training process for the word type vector is referred to Figure 2 , Figure 2 This is a flowchart illustrating the steps involved in training the parameters corresponding to the word type vectors in the summary generation method provided in this disclosure. The specific training process for the parameters corresponding to the word type vectors includes:
[0064] Step S201: Determine the correct summary of the sample text;
[0065] Step S202: Divide the sample text into multiple sample word embeddings;
[0066] Step S203: For each sample word embedding, match the corresponding word type vector;
[0067] Step S204: Fuse each sample word embedding with its corresponding word type vector to obtain multiple sample fusion vectors used to represent text information;
[0068] Step S205: Generate a summary based on the fusion vector of multiple samples representing textual information;
[0069] Step S206: Calculate the difference between the summary generated from the sample text and the correct summary based on the loss function;
[0070] Step S207: Adjust the parameters corresponding to the word type vector based on the differences.
[0071] The parameters corresponding to the word type vector are designed to ensure that the fusion vector of keywords representing specific information about an item can be better focused on after being input into the model. Generally, the attention weights corresponding to the keywords representing specific information about an item are increased by the parameters corresponding to the word type vector, so that the required keywords representing specific information about an item can be combined into the generated summary when the model outputs.
[0072] The following is about Figure 2 The steps of the illustrated embodiment will be further explained.
[0073] Step S201: Determine the correct summary of the sample text;
[0074] Specifically, the sample text consists of multiple pre-stored texts. Generally, the sample text is similar to the text used in normal use and may also include detailed descriptions of the item. The correct summary can be a manually written summary based on the sample text.
[0075] Step S202: Divide the sample text into multiple sample word embeddings;
[0076] Sample word embedding is a numerical representation of words or characters in sample text. Generally, a word or character is mapped to a high-dimensional vector (word vector) to represent that word or character.
[0077] Step S203: For each sample word embedding, match the corresponding word type vector;
[0078] Specifically, word type vectors are used to classify the source of sample word embeddings. The source of each sample word embedding is determined by matching it with its source, the word type vector. The word type vectors are randomly initialized during the training of the generative automatic summarization model.
[0079] Step S204: Fuse each sample word embedding with its corresponding word type vector to obtain multiple sample fusion vectors used to represent text information;
[0080] Specifically, the sample fusion vector includes both the sample word embedding and the word type vector corresponding to the sample word embedding. In some cases, it may also include the position vector.
[0081] Step S205: Generate a summary based on the fusion vector of multiple samples representing textual information;
[0082] Specifically, summaries can be generated using generative automatic summarization models based on RNNs or Transformers, or other deep learning-based generative automatic summarization models; there are no restrictions here. Transformer-based generative automatic summarization models can also use position vectors to specify the exact order of text positions during encoding.
[0083] For low-frequency keywords of specific items in text descriptions, fused vectors can be learned by the model better than simple word embeddings, thus ensuring that low-frequency keywords are reflected in the summary.
[0084] Step S206: Calculate the difference between the summary generated from the sample text and the correct summary based on the loss function;
[0085] The loss function is used to evaluate the degree to which the model's predicted values differ from the true values. Generally, the decision to stop training can be made by limiting the number of training iterations or setting a threshold for the loss function. If a threshold is set for the loss function, and the difference between the calculated summary generated from the sample text and the correct summary exceeds the set threshold, it indicates that the parameters corresponding to the word type vectors still need to be adjusted, i.e., training should continue.
[0086] Step S207: Adjust the parameters corresponding to the word type vector based on the differences.
[0087] Taking a specific application as an example, the parameter training process corresponding to the word type vector is explained. The summary generated from the sample text by the Transformer-based generative automatic summarization model at time t-1 is compared with the correct summary from the Transformer-based generative automatic summarization model at time t, i.e., the fused vector input into the Transformer-based generative automatic summarization model at time t. The difference is calculated based on the logarithmic loss function and compared with a pre-set manually defined loss function threshold. Training stops when the difference is less than the pre-set threshold.
[0088] The formula for the logarithmic loss function is:
[0089]
[0090] in The correct answer at time t is y. t-1 This is the target output at time t-1.
[0091] In one specific embodiment, the fusion of word embeddings and their corresponding word type vectors in step S105 is based on a recurrent neural network that fuses each word embedding with its corresponding word type vector.
[0092] The abstract generation apparatus provided in the embodiments of this disclosure is described below. The abstract generation apparatus described below and the abstract generation method described above can be referred to in correspondence.
[0093] This disclosure provides an abstract generation apparatus, with reference to... Figure 3 , Figure 3 One of the structural diagrams of the summary generation device provided in this disclosure includes: a word segmentation module 30, a matching module 32, a fusion module 34, and a summary generation module 36. The word segmentation module 30 is used to divide the acquired text information into multiple word embeddings; the matching module 32 is used to match a corresponding word type vector to each word embedding; the fusion module 34 is used to fuse each word embedding with its corresponding word type vector to obtain multiple fusion vectors representing the text information; and the summary generation module 36 is used to generate a summary based on the multiple fusion vectors.
[0094] The summary generation apparatus provided in this embodiment divides the acquired text information into multiple word embeddings through the word segmentation module 30; the matching module 32 matches the corresponding word type vector for each word embedding; and the fusion module 34 fuses each word embedding with its corresponding word type vector to obtain multiple fusion vectors for representing the text information. Because the word embedding is matched and fused with the corresponding word type vector, the resulting fusion vector can better represent the unique information of the acquired text information. That is, the keywords in the acquired text information may be low-frequency information in a large amount of text, but by fusing the word embedding with its corresponding word type vector to obtain multiple fusion vectors for representing the text information, the summary generation module 36 can focus on these low-frequency features, i.e., the keywords in the acquired text information, thereby better representing the key content of the acquired text information.
[0095] The following is about Figure 3 The various modules of the illustrated embodiment will be further explained.
[0096] The text information obtained by the word segmentation module 30 is generally a document with more than 500 characters, which includes a detailed description of the item. Word embedding is a numerical representation of words. Generally, a word is mapped to a high-dimensional vector (word vector) to represent the word.
[0097] For matching module 32, specifically, the word type vector is used to classify the source of word embeddings by matching each word embedding with its source. The word type vector is randomly initialized during the training of the generative automatic summarization model and is updated as the model learns.
[0098] Specifically, for the fusion module 34, the fusion vector includes both word embeddings and word type vectors corresponding to the word embeddings, and in some cases, it may also include position vectors.
[0099] For the summary generation module 36, specifically, summaries can be generated based on RNN or Transformer generative automatic summarization models, or other deep learning-based generative automatic summarization models; no restrictions are placed here. Transformer-based generative automatic summarization models can also use position vectors to specify the exact positional order of the text during encoding.
[0100] For low-frequency keywords of specific items in text descriptions, fused vectors can be learned by the model better than simple word embeddings, thus ensuring that low-frequency keywords are reflected in the summary.
[0101] In one alternative embodiment, the fusion module 34 can be fused by embedding each word into its corresponding word type vector using either an additive or a direct connection method.
[0102] The formula for fusing each word with its corresponding word type vector using a direct connection method is as follows:
[0103] Vector(w i ) = [E i ;T i ]
[0104] Among them, w i Represents word embedding, Vector(w i ) indicates word embedding w i The fusion vector, E i T represents the i-th column of the word vector matrix E. i Indicates word embedding w i The corresponding word type vector.
[0105] The formula for fusing the embedding of each word with its corresponding word type vector using an additive approach is as follows:
[0106] Vector(w i ) = E i +T i
[0107] Among them, w i Represents word embedding, Vector(w i ) indicates word embedding w i The fusion vector, E i T represents the i-th column of the word vector matrix E. i Indicates word embedding w i The corresponding word type vector.
[0108] In an optional embodiment, the summary generation module 36 is specifically used to: input multiple fusion vectors into an encoder-decoder generative automatic summarization model and output a summary; the encoder-decoder generative automatic summarization model includes an encoder and a decoder; the encoder of the encoder-decoder generative automatic summarization model is used to encode the fusion vectors to generate an encoder hidden layer sequence; the decoder of the encoder-decoder generative automatic summarization model is used to decode the encoder hidden layer sequence to generate a summary.
[0109] The encoder-decoder generative automatic summarization model can be a Transformer-based generative automatic summarization model, or it can be other models evolved from the Transformer model; there are no restrictions here.
[0110] This explanation uses a Transformer-based generative automatic summarization model as an example. A Transformer model typically includes a decoder and an encoder, which are composed of multiple stacked base layers. Each base layer in the decoder consists of two sub-layers: an attention mechanism layer and a multi-layer fully connected layer. Each feedforward layer in the decoder consists of three sub-layers: an attention mechanism layer, a self-attention mechanism layer, and a multi-layer fully connected layer. Residual edges and layer normalization can be applied to each sub-layer of the decoding and encoding layers.
[0111] The core module of the Transformer-based generative automatic summarization model is the self-attention mechanism. Self-attention refers to the ability to connect information from different positions in the input sequence corresponding to the input text, calculate a certain representation of the entire sequence, and finally generate a summary based on this representation. Specifically, given a query and a key-value table, the self-attention mechanism can be the process of mapping the query to the correct input. Here, the query, key, value, and final output are all vectors. The output is usually in the form of a weighted sum, and the weights are determined by the query, key, and value.
[0112] Specifically, the fused vector is input into the Transformer-based generative automatic summarization model. First, the model encodes the fused vector to obtain the encoder hidden layer sequence. Here, the encoder hidden layer sequence is the state H of all time series of the hidden layer. The state H of all time series of the hidden layer includes the fused vector corresponding to each word embedding. The fused vector is input into the hidden layer state hi generated by the encoder of the Transformer-based generative automatic summarization model. Each hidden layer state hi is decoded to generate a summary word by word.
[0113] In one embodiment, the category vector, keyword vector, and multiple pre-defined word embedding attribute vectors constitute the word type vector. The word embedding attribute vectors include a first text vector, a second text vector, and a third text vector.
[0114] Specifically, the word type vector represents the specific source of each word embedding, and the category vector represents the specific category of the item, such as television, stereo, and mobile phone. The keyword vector represents the specific name of the object, such as portable stereo, high-definition smart TV, and women's mobile phone. The first text vector represents the text information from the title in the acquired text information. The second text vector represents the text information from the body text in the acquired text information. The third text vector represents the text information from the acquired text information describing the specifications and parameters of the item.
[0115] In one specific embodiment, refer to Figure 4 , Figure 4 This is a second schematic diagram of the abstract generation device provided in this disclosure. The abstract generation device includes a configuration module 40, a word segmentation module 42, a matching module 44, a fusion module 46, and a abstract generation module 48. The configuration module 40 is used to configure the parameters corresponding to the word type vectors in the matching module 44, that is, to adjust and determine the parameters corresponding to the word type vectors. For example... Figure 5 As shown, the configuration module 40 specifically includes:
[0116] Unit 401 is used to determine the correct summary of the sample text;
[0117] Word segmentation unit 403 is used to divide the sample text into multiple sample word embeddings;
[0118] Matching unit 405 is used to embed the corresponding word type vector for each sample word.
[0119] The fusion unit 407 is used to fuse each sample word embedding with its corresponding word type vector to obtain multiple sample fusion vectors for representing text information.
[0120] The summarization unit 409 is used to generate a summary based on the fusion vector of multiple samples representing textual information.
[0121] Calculation unit 411 is used to calculate the difference between the summary generated from the sample text and the correct summary based on the loss function;
[0122] Adjustment unit 413 is used to adjust the parameters corresponding to the word type vector based on the differences.
[0123] The following is about Figure 5 The various module units of the illustrated embodiment will be further explained.
[0124] The sample text in unit 401 is determined to be one of several pre-stored texts. Generally, the sample text is similar to the text used in normal use and may also contain detailed descriptions of the item. The correct summary can be a summary manually written based on the sample text.
[0125] The sample word embedding in the word segmentation unit 403 is a numerical representation of words or characters in the sample text. Generally, a word or character is mapped to a high-dimensional vector (word vector) to represent the word or character.
[0126] The word type vector in matching unit 405 is used to classify the source of sample word embeddings. The source of each sample word embedding is determined by matching it with its source, i.e., the word type vector. The word type vector is randomly initialized during the training of the generative automatic summarization model.
[0127] The sample fusion vector in fusion unit 407 includes both the sample word embedding and the word type vector corresponding to the sample word embedding. In some cases, it may also include the position vector.
[0128] The summarization unit 409 can generate summaries based on RNN or Transformer generative automatic summarization models, or other deep learning-based generative automatic summarization models; no restrictions are placed here. Transformer-based generative automatic summarization models can also use position vectors to specify the exact positional order of the text during encoding.
[0129] For low-frequency keywords of specific items in text descriptions, fused vectors can be learned by the model better than simple word embeddings, thus ensuring that low-frequency keywords are reflected in the summary.
[0130] The loss function in computation unit 411 is used to evaluate the degree to which the model's predicted values differ from the true values. Generally, the number of training iterations or a threshold for the loss function can be used to determine whether training can be stopped. If a threshold for the loss function is used, and the difference between the calculated summary generated from the sample text and the correct summary is greater than the set threshold, it indicates that the parameters corresponding to the word type vector still need to be adjusted, i.e., training should continue.
[0131] Adjustment unit 413 is used to adjust the parameters corresponding to the word type vector based on the differences.
[0132] Taking a specific application as an example, the parameter training process corresponding to the word type vector is explained. The summary generated from the sample text by the Transformer-based generative automatic summarization model at time t-1 is compared with the correct summary from the Transformer-based generative automatic summarization model at time t, i.e., the fused vector input into the Transformer-based generative automatic summarization model at time t. The difference is calculated based on the logarithmic loss function and compared with a pre-set manually defined loss function threshold. Training stops when the difference is less than the pre-set threshold.
[0133] The formula for the logarithmic loss function is:
[0134]
[0135] in The correct answer at time t is y. t-1 This is the target output at time t-1.
[0136] In one specific embodiment, the fusion unit 407 specifically includes: a fusion subunit 4071, used to fuse each word embedding with its corresponding word type vector based on a recurrent neural network.
[0137] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a digest generation method.
[0138] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this disclosure, essentially, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0139] On the other hand, this disclosure also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer is able to execute the summary generation method provided by the above methods.
[0140] In another aspect, this disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the above-described digest generation methods.
[0141] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0142] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.
Claims
1. A method for generating summaries, characterized in that, include: The acquired text information is divided into multiple word embeddings; Each of the aforementioned word embeddings is matched with the corresponding word type vector; The word type vector includes: a category vector, a keyword vector, and multiple preset word embedding attribute vectors; the multiple preset word embedding attribute vectors include a first text vector, a second text vector, and a third text vector, wherein the first text vector is used to represent the text information of the title from the acquired text information; the second text vector is used to represent the text information of the body text from the acquired text information; and the third text vector is used to represent the text information describing the specifications of the item from the acquired text information. Each word embedding is fused with its corresponding word type vector to obtain multiple fused vectors for representing the text information; A summary is generated based on the multiple fusion vectors.
2. The abstract generation method according to claim 1, characterized in that, The process of fusing each word embedding with its corresponding word type vector specifically includes: Each word is embedded into its corresponding word type vector and fused using either additive or direct connection methods; and The formula for the addition is: ； The formula for the direct connection form is: ； in, Representative word embedding, Indicator word embedding The fusion vector, This represents the i-th column of the word vector matrix E. Indicator word embedding The corresponding word type vector.
3. The abstract generation method according to claim 2, characterized in that, The specific process of generating a summary based on the multiple fusion vectors is as follows: The multiple fused vectors are input into an encoder-decoder generative automatic summarization model to output a summary. The encoder-decoder generative automatic summarization model includes an encoder and a decoder. The encoder is used to encode the fused vectors to generate an encoder hidden layer sequence. The decoder is used to decode the encoder hidden layer sequence to generate a summary.
4. The abstract generation method according to claim 2, characterized in that, Adjusting and determining the word type vector specifically includes: Determine the correct summary of the sample text; Based on the loss function, the difference between the summary generated from the sample text and the correct summary is calculated. Based on the aforementioned differences, adjust the parameters corresponding to the word type vector; The sample text consists of multiple pre-stored texts.
5. The abstract generation method according to claim 1, characterized in that, The step of fusing each word embedding with its corresponding word type vector is as follows: The embedding of each word is fused with its corresponding word type vector using a recurrent neural network.
6. A summary generation apparatus, characterized in that, include: The word segmentation module is used to divide the acquired text information into multiple word embeddings; The matching module is used to embed a corresponding word type vector for each of the aforementioned words. The word type vector includes: a category vector, a keyword vector, and multiple preset word embedding attribute vectors; the multiple preset word embedding attribute vectors include a first text vector, a second text vector, and a third text vector, wherein the first text vector is used to represent the text information of the title from the acquired text information; the second text vector is used to represent the text information of the body text from the acquired text information; and the third text vector is used to represent the text information describing the specifications of the item from the acquired text information. The fusion module is used to fuse each word embedding with its corresponding word type vector to obtain multiple fusion vectors for representing the text information. The summary generation module is used to generate a summary based on the multiple fusion vectors.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the digest generation method as described in any one of claims 1 to 5.
8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the digest generation method as described in any one of claims 1 to 5.
9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the digest generation method as described in any one of claims 1 to 5.