Generate targeted summaries of text content adapted to the target audience's vocabulary

Abstract

A target summary adjusted to text content of a target audience vocabulary is generated in a digital media environment. A word generation model obtains text content and generates a target summary of the text content. During the generation of the target summary, the words of the target summary generated by the word generation model are adjusted to the target audience vocabulary using a language preference model. The language preference model is trained using machine learning of target audience training data corresponding to a corpus of text of the target audience vocabulary to learn word preferences of the target audience vocabulary between similar words (e.g., synonyms). After each word is generated using the word generation model and the language preference model, feedback about the generated words is provided back to the word generation model. The word generation model uses the feedback to generate subsequent words of the summary.

Description

Background Art

[0001] Automatic summarization of text content can be used to save time for end users by providing an overview of text content (e.g., a document or article) that can be read quickly by the user. Traditional extractive summarization techniques extract key phrases from the input text content and then select a subset of these phrases to place in the summary. However, the summaries generated by these traditional summarization techniques are usually not like humans. In addition, some traditional summarization techniques generate summaries, and then the summaries can be "adjusted" to the target audience as a post-processing step after the generation of the summary. However, adjusting the summary to the target audience after the summary is generated usually results in changing the meaning of the original text. For example, consider the sentence "the entire journey is larger than the team". Based on the language preferences of the target audience, the word "total" may be better than the word "entire", and the word "travel" may be better than the word "journey". Although both words are fine replacements for the original words, the resulting sentences formed by replacing these words, such as "the total travel is larger than the team", do not have the same meaning as the original sentence. In addition, existing summarization techniques cannot generate multiple summaries adjusted to different target audience vocabularies. Summary of the invention

[0002] In order to overcome these problems, a target summary adjusted to the text content of the target audience vocabulary is generated in a digital media environment. A word generation model obtains the text content and generates a target summary of the text content. During the generation of the target summary, the words of the target summary generated by the word generation model are adjusted to the target audience vocabulary using a language preference model. The language preference model is trained using machine learning of target audience training data corresponding to a corpus of texts of the target audience vocabulary to learn the word preferences of the target audience vocabulary between similar words (e.g., synonyms). After each word is generated using the word generation model and the language preference model, feedback about the generated words is provided back to the word generation model. The word generation model uses the feedback to generate subsequent words of the summary so that the context of the text document and the generated words of the summary are maintained.

[0003] This summary introduces some concepts in a simplified form, which will be further described in the following detailed description. Therefore, this summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The detailed description is described with reference to the accompanying drawings.

[0005] Figure 1 is an illustration of a digital media environment in an example implementation that is operable to employ techniques for generating targeted summaries tuned to target audience vocabularies described herein.

[0006] Figure 2 Depicted are systems in an example implementation in which a language preference model is trained using machine learning on target audience training data.

[0007] Figure 3 Depicted in it Figure 1 A system in an example implementation in which a word generation model generates a targeted summary of text content that is tuned to the vocabulary of a target audience.

[0008] Figure 4 Illustrated is a diagram of a neural encoder when configured as a neural encoder according to one or more implementations. Figure 1 and Figure 3 An example of a word generation model for .

[0009] Figure 5 Illustrated are examples of various summaries of input articles generated using different summarization techniques.

[0010] Figure 6 Results of metric-based evaluation of various summarization techniques are illustrated.

[0011] Figure 7 Depicts an example process for training a language preference model using machine intent and target audience training data.

[0012] Figure 8 An example process for generating a targeted summary of text content that is tuned to the vocabulary of a target audience is depicted.

[0013] Fig. 9 Additional example processes for generating targeted summaries of text content tuned to the vocabulary of a target audience are depicted.

[0014] Fig.10 Illustrated are example systems including example computing devices that represent one or more computing systems and / or devices that may implement the various techniques described herein. DETAILED DESCRIPTION

[0015] Overview

[0016] This paper describes the technology for generating a target summary of text content adjusted to target audience vocabulary in a digital media environment. The language preference model associated with the target audience vocabulary of the word generation model is utilized to generate a summary of the text content adjusted to the target audience vocabulary. The word generation model can correspond to a machine learning or rule-based summary model, which utilizes extraction and / or abstract summary technology to generate a summary of text content. The language preference model is trained using the machine learning technology of the training data about the target audience vocabulary to learn the word preferences of the target audience vocabulary between similar words (e.g., synonyms). It is noteworthy that a single word generation model can be utilized to generate multiple summaries adjusted to different audience vocabulary by utilizing different language preference models, without explicitly retraining the word generation model for each different target audience vocabulary. As described herein, the target audience vocabulary can be based on a type of words (e.g., simple or complex) or people's demographics (e.g., university education, living in the UK, income level, age or gender).

[0017] In order to generate the target summary, the word generation model obtains the text content, and uses one or more summary techniques to generate the summary of the text content. Different from the prior art that considers the language preference of the target audience as a post-processing step, the described technology uses the language preference model to adjust the words generated by the word generation model to the target audience vocabulary during the generation of the summary. After each word is generated using the word generation model and the language preference model, the feedback about the generated words is provided back to the word generation model. The word generation model can use feedback to adjust the subsequent word generation of the summary. It is worth noting that, due to the adjustment of the target audience vocabulary during the generation of the summary, the summary is generated based on the words of the generated words of the input text and the summary. In this way, the summary generated using the described technology generates a summary of the text content adjusted to the target audience vocabulary, which is more readable than the summary generated by the post-processing adjusted to the target audience. In addition, the summary of the described technology is better adjusted to the target audience than even the summary generated by humans, and the summary generated by humans is generated when there is no machine training knowledge of the target audience vocabulary.

[0018] In the following discussion, an example environment in which the techniques described herein may be employed is first described. Example implementation details and procedures that may be performed in the example environment as well as other environments are then described. Thus, execution of the example procedures is not limited to the example environment, and the example environment is not limited to execution of the example procedures.

[0019] Example Environment

[0020] Figure 11 is an illustration of a digital media environment 100 in an example implementation that is operable to employ techniques for generating targeted summaries tuned to the target audience vocabulary described herein. The illustrated environment 100 includes a service provider system 102 and a client device 104 communicatively coupled via a network 106. The computing devices implementing the service provider system 102 and the client device 104 may be configured in a variety of ways.

[0021] For example, a computing device may be configured as a desktop computer, a laptop computer, a mobile device (e.g., assuming a handheld configuration such as a tablet computer or mobile phone), etc. Thus, a computing device may range from a full resource device with large amounts of memory and processor resources (e.g., a personal computer, a game console) to a low resource device with limited memory and / or processing resources (e.g., a mobile device). Additionally, a computing device may represent multiple different devices, such as multiple servers utilized by an enterprise to perform operations "on the cloud," such as Fig.10 Described in .

[0022] The client device 104 is illustrated as including a communication module 108, which represents functionality for communicating with the service provider system 102 via the network 106, for example, using a browser, a web-enabled application, an operating system, etc. For example, the communication module 108 can form a request 110 for a summary of text content 112 adjusted to a target audience. As described herein, the text content 112 can include any type of text content, such as an article, a word processing document, text on a web page, a PDF, etc. As described herein, the target audience vocabulary 114 can be based on a type of words (e.g., simple, complex) or a person's demographics (e.g., university education, living in the UK, income level, age, gender, etc.). The request 110 can be transmitted to the service provider system 102 via the network 106. In response to the request 110, the service provider system 102 generates and returns a target summary 116 of the input text content 112 adjusted to the target audience vocabulary 114. As described herein, the target summary 116 corresponds to a shortened version of the input text content 112 that takes into account the vocabulary of the target audience. Although illustrated as being implemented remotely from the client device 104, the functionality of the illustrated service provider system 102 may also be implemented in whole or in part at the client device 104. For example, the target summary module 118 may be implemented at the client device 104 from which the request 110 was received to locally generate the target summary 116.

[0023] In order to generate the target summary 116, the service provider system 102 uses a target summary module 118, which is configured to generate the target summary 116 adjusted to the requested target audience vocabulary 114. The target summary module 118 utilizes both a word generation model 120 and a language preference model 122, the word generation model 120 is configured to generate a summary of the text content, and the language preference model 122 is trained to capture the word preferences of the target audience vocabulary 114. The word generation model 120 can correspond to any type of word generation model or algorithm that can generate a summary of the text content. In some cases, machine learning (e.g., neural network) is used to train the word generation model 120 to generate a summary of the text content. For example, the word generation model 120 can be implemented as an extraction summary algorithm, an abstract summary algorithm, a sequence to sequence neural encoder, etc.

[0024] The service provider system is further illustrated as including a language preference training module 124 ("training module 124") that trains the language preference model 122 to learn word preferences for various target audience vocabularies 114. The training module 124 trains the language preference model 122 using machine learning applied to target audience training data 126 ("training data 126") . The training data 126 corresponds to a corpus of textual content generated using the target audience vocabularies 114. The training module 124 typically trains the language preference model 122 based on the insight that among different words (e.g., synonyms) that mean the same thing, the more commonly used words are more preferred. Thus, the language preference model 122 learns word preferences, which can be defined in relative contexts in terms of target audience preferences between a word w1 and its synonym w2. Starting with the training data 126 for a particular target audience vocabulary 114, the training module 124 constructs a probability map P based on the frequency of word usage in the training data 126 together with its usage context in the training data 126. pref (w1|w2), which indicates the probability that word w1 is preferred relative to w2. Figure 2 Further discussion of training the language preference model 122 is discussed.

[0025] In one or more implementations, the term generation model 120 generates the terms of the target summary 116 using a "term-by-term" summary process. To generate each "next term" of the target summary 116, the term generation model 120 calculates a probability distribution based on a combination of the input text of the text content 112 and the text of the target summary that has already been generated. After each term of the target summary 116 is generated, the term generation model 120 receives feedback about the generated term. The feedback is used by the term generation model to generate subsequent terms of the target summary 116. Unlike existing term generation techniques, the term generation model 120 generates terms for the target summary based on both the context of the input text content 112 and the terms of the summary that has been generated so far.

[0026] In order for the target summary 116 to be adjusted to the language preferences of the target audience vocabulary 114, the word-by-word generation process employed by the word generation model 120 is constrained by the learned language preferences of the language preference model 122 during the generation of the summary. This can be achieved by replacing "less preferred" words with "more preferred" alternative words that have similar meanings to the less preferred words. However, this may result in a complete change in the meaning conveyed by the text content summary. Therefore, the target summary module 118 can adjust the target summary to the language preferences of the target audience by modifying the word generation probabilities that are consistent with the language preference model 122. Figure 3 Further discussion of utilizing the term generation model 120 and the language preference model 122 to generate a target profile tuned to the language preferences of the target audience is discussed.

[0027] The target summary module 118 can be implemented to provide a target summary for a variety of different use cases. In some instances, the target summary module 118 can be used by the content author of the text content 112 (e.g., the content author of the article, newspaper or book) to quickly and automatically generate a summary of its text content 112. The target summary module 118 enables the content author to adjust the summary to the vocabulary of a specific target audience, which can enable the author to adjust the summary to multiple different target audiences. For example, the content author may want to provide different summaries on different platforms based on different audiences who frequently use these platforms. In other instances, the target summary module 118 can be used by the customer or end consumer of the text content. In this context, the target summary module 118 can assist in providing a consumer-specific summary of a longer article or newspaper that can be easily consumed by mobile notifications or on a wearable device. In this instance, the target summary module 118 can be implemented to determine the demographic information of the end user, and then the summary is adjusted to the vocabulary corresponding to the demographic information of the user.

[0028] In general, the functions, features, and concepts described with respect to the examples above and below may be employed in the context of the example processes described in this section. In addition, the functions, features, and concepts described with respect to the different figures and examples in this document may be interchangeable with each other and are not limited to implementation in the context of a particular figure or process. In addition, the boxes associated with the different representative processes and corresponding figures herein may be applied together and / or combined in different ways. Therefore, the individual functions, features, and concepts described with respect to the different example environments, devices, components, figures, and processes herein may be used in any suitable combination and are not limited to the specific combinations represented by the examples listed in this specification.

[0029] Language preference model training

[0030] The language preference model 122 may be trained using one or more machine learning techniques to learn the word preference probabilities of the target audience vocabulary based on the frequency of word usage in the target audience vocabulary. To this end, the language preference training module 124 identifies the words in the target audience training data 126 and then identifies one or more synonyms of the words. Then, for each word-synonym pair, a word preference probability is generated for each corresponding word-synonym pair based on the corresponding frequencies of the word and the synonym in the target audience vocabulary.

[0031] In more detail, consider Figure 2 , which depicts a system 200 in an example implementation in which a language preference model is trained using machine learning on target audience training data.

[0032] In the system 200, the training module 124 obtains target audience training data 126, which corresponds to a corpus of text content of the corresponding target audience vocabulary 114. For example, the training data 126 may include a plurality of (e.g., ten thousand) pieces of text content (e.g., articles, newspapers, books) generated using the expected target audience vocabulary 114. The training module 124 uses the training data 126 to learn the word preference probabilities 202 of the language preference model 122. The word preference probabilities 202 indicate the relative preference between words (e.g., synonyms) with similar meanings in the target audience vocabulary 114. The determination of the word preference probabilities 202 can be based on the frequency of use of the words in the training data 126, so that if the first word appears more frequently in the training data 126, the first word is preferred to the second word that is a synonym of the first word. However, other metrics may also be used to determine the importance of similar words relative to each other without departing from the spirit or scope of the described technology.

[0033] In the system 200, in order to train the language preference model 122, each candidate word 204 in the training data 126 is passed to a pairing module 206 employed by the language preference training module 124. The pairing module 206 identifies all the senses 208 of a given candidate word 204 and all the synonyms 210 of the various senses 208 of the candidate word 204. In linguistics, a "sense" is one of the meanings of a word. Some words have multiple meanings, while other words have only one meaning. For example, the word "play" can have more than 50 different meanings, each with a different meaning based on the context in which the word is used in a sentence. The pairing module 206 generates a sense-synonym pair 212 for each corresponding candidate word 204. In some cases, the sense-synonym pair 212 is deformed into a form suitable for replacing the candidate word 204 based on morphological analysis.

[0034] The word sense-synonym pairs 212 are then passed to the frequency module 214 of the training module 124. The frequency module 214 determines a word sense frequency 216 and a synonym frequency 218 for each word sense 208 and synonym 210 of the word sense-synonym pairs 212, respectively. The frequencies 216 and 218 correspond to the number of occurrences of each term in the training data 126.

[0035] Next, the term preference probability module 220 of the training module 124 obtains the term sense frequency 216 and the synonym frequency 218 of each corresponding term sense-synonym pair 212 and generates a term preference probability 202 for the corresponding term sense-synonym pair 212. If the frequency of the synonym is greater than the frequency of the term in the target audience training data, the term preference probability for a given term-synonym pair indicates that the synonym is preferred over the term (or the sense of the term).

[0036] In one or more implementations, the term (w i ) than the word (w j The preferred word preference probability 202 is determined by the word preference probability module 220 using the following algorithm: pref (w i |w j ) is given by the following formula,

[0037]

[0038] In this algorithm, f wi is the word w in the training data 126 determined by the frequency module 214 i The frequency of the condition f wi ≥f wi Make sure as w j Synonyms of w iis used more frequently in the training data 126 and is therefore more preferred. The sum in the denominator only contains those with higher frequency and thus higher preference than w j Preferred word.

[0039] Once trained, the learned word preference probabilities 202 of the language preference model 122 for a given target audience vocabulary 114 can then be used by the target summary module 118 to adapt the summaries generated by the word generation model 120 to the target audience vocabulary, as described below. Figure 3 Further detailed discussion.

[0040] Generate targeted profiles tuned to the target audience's vocabulary

[0041] Figure 3 A system 300 is depicted in an example implementation, wherein Figure 1 The term generation model 120 generates a target summary 116 of the text content 112 that is adjusted to the target audience vocabulary 114.

[0042] In the system 300, the targeted summary module 118 receives a request 110 for generating a targeted summary of textual content 112 that is tuned to a target audience vocabulary 114. In some cases, the request may be received from a content author of the textual content 112, such as an author of an article, a book, etc. Alternatively, the request may be received from a consumer of the textual content 112 to generate a summary of the textual content.

[0043] The request 110 may include an indication of a target audience. For example, a content author of text content may submit a request via a user interface that enables the content author to select a target audience vocabulary for the summary. In this way, the content author may select a target audience that may, for example, correspond to an intended audience for the content. It is noteworthy that the content author may generate a plurality of different summaries using the techniques described herein so that different summaries may be generated for different target audiences. Alternatively, the target audience may be automatically determined by the target summary module 118. For example, the target audience may be determined based on demographic information associated with the end consumer requesting the summary.

[0044] The target summary module 118 provides the text content 112 to the term generation model 120 and provides the requested target audience vocabulary to the language preference model 122. In the system 300, the attention decoder 302 of the term generation model 120 receives the text content 112 as input and generates an attention distribution 304 of the terms 306 in the text content 112. The attention distribution 304 indicates a selection probability value 308 for each term 306. The selection probability value 308 indicates the probability that the corresponding term 306 is selected as the next generated term of the target summary 116. The attention decoder 302 determines the selection probability value 308 based on the terms in the input text content 112 and the terms of the target summary 116 that have been generated by the term generation model 120.

[0045] The term generation model 120 is further illustrated as including a selection module 310 configured to select a next generated term 312 for the target profile 116 based on the attention distribution 304 generated by the attention decoder 302 of the term generation model 120 and the language preference model 122 trained on the training data 126 corresponding to the target audience vocabulary 114, as described above with reference to Figure 2 described.

[0046] To encourage the generation of the most preferred terms of the target audience vocabulary 114, the selection module 310 modifies the selection probability value 308 of the attention distribution 304 using the term preference probabilities 202 of the target audience learned by the language preference model 122. Therefore, the selection module 310 selects the next generated term 312 by selecting terms from the input text content 112 based on the attention distribution 304, or by replacing terms in the input text content 112 with more preferred synonyms of the target audience as indicated by the term preference probabilities 202 of the language preference model 122. The selection module 310 can use a variety of different term generation techniques to select the next generated term 312 of the target profile 116. In this example, the selection module 310 can use the statistical replacement generation 314 or the probability reweighted generation 316 to select the next generated term 312.

[0047] In the generation technique of statistical replacement, the words generated by the word generation model are replaced with more preferred synonyms defined by the language preference model 122. To this end, the selection probability values ​​308 in the attention distribution 304 are increased for the words preferred by the target audience. This ensures that the most preferred words are used in the current context and are utilized in subsequent generation. Since the word generation model 120 uses this information for subsequent word generation, the word generation model 120 is able to generate complete sentences that make sense even after the words from the text content are replaced with more preferred words identified by the language preference model. It is worth noting that this situation does not usually occur when a complete summary is generated first, and then after the complete summary is generated, the words of the generated summary are then naively replaced with the most preferred synonyms.

[0048] To select the next generated word 312 using the generation 314 of statistical replacements, the selection module replaces the word 306 in the focus distribution 304 with a more preferred similar word as determined by the language preference model 122. In some cases, this means that the words in the focus distribution are replaced with their most preferred synonyms. In one or more implementations, the selection module 310 uses the following algorithm for the generation of statistical replacements:

[0049]

[0050] In the above algorithm, P(w) corresponds to the probability of selection 308 of a given word generated by the attention decoder 302, and P pref (w i |w k ) corresponds to the word preference probability 202 of the word learned by the language preference model 122. The word w is ranked based on its preference probability over the entire target audience vocabulary 114. i A "preference score" is generated. By weighting the resulting score with the generation probability, the scheme identifies the combination of the most preferred term with the best possible generation probability.

[0051] Although the generation of statistical substitutions 314 identifies the most preferred term replacement during term generation, it does not consider the most preferred term among the candidates for generation. To achieve this, the selection module 310 may use probability reweighted generation 316 to select the next generated term 312. In the probability reweighted generation 316, the probability of a term being selected depends on its likelihood of being used to replace a less preferred synonym, weighted by its likelihood of being used first to generate the less preferred synonym. To generate the next term 312 using probability reweighted generation 316, the following algorithm may be used:

[0052]

[0053] Note that the preference score in this case is obtained by weighting the probability of generation of the term relative to which the current candidate term is preferred. Thus, probability reweighted generation 316 selects the most preferred candidate terms relative to the entire candidate set for generation and weights them with the corresponding generation probabilities to simultaneously select the most preferred terms that are most likely to be generated in the context of the given input.

[0054] After the selection module 310 selects the next generated word 312, the generated word is added to the target summary 116. In addition, the feedback module 318 provides the generated word feedback 320 to the attention decoder 302, which can be used by the attention decoder 302 to select subsequent words of the target summary 116, thereby maintaining consistency in the generation of words of the target summary 116.

[0055] Word generation model via neural encoder

[0056] Figure 4 Illustrated is a diagram of a neural encoder when configured as a neural encoder according to one or more implementations. Figure 1 and Figure 3 An example 400 of the word generation model 120.

[0057] In example 400, the word generation model 120 is implemented as a neural encoder, which can be implemented to map a sequence of words to another sequence of words. For example, by training a large corpus of English sentences and their French translations, the word generation model 120 can learn to output the French translation of an input sentence. This concept can be applied to configure the word generation model 120 to generate a summary of text content. In order to ensure that the next generated word of the summary does not only depend on the previously generated words, the word generation model 120 can utilize a long short-term memory (LSTM) architecture. The LSTM architecture uses a gating mechanism that retains information in memory or discards previous information.

[0058] In example 400, the LSTM-based encoder receives word embedding vectors (“w i ”) as input and computes the encoder hidden state 402 (“h i ”). The final hidden state is passed to the decoder, which also calculates the decoder hidden state 404 (“s t ”) and generates a word embedding of the target word at each decoding time step (“y t ”).

[0059] When generating the words of the target summary 116, the word generation model 120 focuses more attention on some parts of the input text content 112 than other parts. For example, in the example system, the word generation model 120 is generating a target summary 116 for the input text content 112: "Germany merges victorious in a 2-0 win against Argentina". The word generation model 120 can learn to generate the word "beat" by paying higher attention to the words "victory" and "win". To do this, the word generation model 120 uses the following algorithm to calculate the attention distribution 304 on all words in the text content 112:

[0060]

[0061] a t =softmax(e t )

[0062] In the above attention distribution algorithm, machine learning techniques are used to train v and W on a corpus of text content. h , W s and b att As described above, the attention distribution 304 can be considered as a probability distribution over the words in the input text content 112 learned from the training corpus. The context vector (The weighted sum of encoder hidden states 402) is used to determine the next word of the summary to be generated based on the words already generated within the defined "context" window.

[0063] To reproduce factual segments of the input text content 112, the word generation model 120 can be extended to include a probabilistic term for deciding whether to generate words as before or to pick from the input text content 112 to replicate factual details. At each stage of the decoder, the probability p is calculated. gen , p gen Indicates whether the word is generated from the vocabulary, or copied from the source text content 112 using the attention distribution 304. This probability can be calculated using the following algorithm:

[0064]

[0065] In the above algorithm, and b gen Based on a corpus of training texts. Then you can use p gen To determine the attention distribution 304 about the words in the vocabulary given by the algorithm:

[0066]

[0067] The second term allows the word generation model 120 to select words to copy from the input text content 112 using the attention distribution 304. P(w) yields the probability of the next word being generated given the current word and context.

[0068] Summary generation comparison and results

[0069] Figure 5 Illustrated are examples 500 of various summaries of an input article 502 generated using different summarization techniques.

[0070] The human-generated summary 504 of the input article 502 includes the text: "Bafetimbi Gomis collapsed within 10 minutes of Tottenham's kick-off". The pointer-generated model summary 506 of the input article 502 includes the text: "French striker Bafetimbi Gomis, who has a history of fainting, said he is now feeling well after collapsing during Swansea's 3-2 loss to Tottenham in the Premier League on Wednesday". The summary for post-processing 508 includes the text "French striker Bafetimbi Gomis, who has a history of fainting, told CNN he is now feeling well after..."

[0071] It is noteworthy that the summary with post-processing 508 has replaced the text "history of fainting" from the input article 502 with "story of fainting" because it has determined that the word "story" is the simpler word and therefore more preferred according to the training data set. However, the second phrase of this summary does not make sense in this context. The pointer generation summary 506 has replaced the phrase "feeling good" with "looking good", which completely changes the meaning.

[0072] In contrast, the statistically replaced generated summary 510 and the probabilistically reweighted generated summary 512 of the described techniques have generated a summary that does not include the contextual errors of these conventional techniques. In contrast, the statistically replaced generated summary 510 and the probabilistically reweighted generated summary 512 of the described techniques replace the phrase "feeling good" of the input article 502 with "good" because good is more preferred according to the training data corpus text of the target vocabulary. It is worth noting that the term "good" is also meaningful in the context of the summary and the input article and does not change the meaning of the input article 502.

[0073] Figure 6Results 600 of a metric-based evaluation of various summarization techniques are illustrated. Results 600 compare summaries generated using pointer generation, post-processing, and the described PRG and SRG techniques. In this example, the language preference model 122 is trained using approximately 10,000 articles of the target audience vocabulary. It is noteworthy that the 10,000 articles used to train the language preference model 122 are different from the training data used to train the term generation model 120.

[0074] Results 600 include quality comparison 602, readability comparison 604, and preference score comparison 606. Quality comparison 602 is based on ROGUE scores, which measure the precision, recall, and f-measure of the occurrence of n-grams in the generated summary relative to a reference human-generated summary. In this example, the ROUGE-L variant is used, which looks at the longest common subsequence between the reference summary and the machine-generated summary.

[0075] Readability comparison 604 is based on Flesch reading ease, which quantifies the readability of the generated summary. The Flesch readability score uses sentence length (the number of words per sentence) and the number of syllables per word in an equation to calculate reading ease. Text with a very high Flesch reading ease score (about 100) is very easy to read. Higher scores indicate better readability and better consistency, and therefore more meaningfulness.

[0076] The preference score comparison 606 uses the average preference scores (Σ w P pref (w i |w)) to quantify the language preference captured in the profile.

[0077] As can be seen from comparing 602, 604, and 606, all of the compared summarization techniques have achieved comparable "rouge" scores indicating similar quality of the summaries generated. The statistically replaced generation ("SRG") and probabilistically reweighted generation ("PRG") techniques of the described techniques have produced summaries with higher preference scores than the baseline and human-generated summaries because they are additionally tuned to the target vocabulary. Notably, the post-processing techniques have produced similar preference scores as the described SRG and PRG techniques, however the post-processed summaries produce higher preference scores at the cost of lower readability indicating a decrease in the quality of the post-processing output. The human summaries have lower preference scores because they are generated without knowledge of the target vocabulary.

[0078] Having discussed example details of techniques for generating targeted summaries of text content tuned to the target audience's vocabulary, consider now some example procedures to illustrate additional aspects of the techniques.

[0079] Sample Program

[0080] This section describes example processes for generating targeted summaries of text content that are tuned to the vocabulary of a target audience. Aspects of the processes may be implemented in hardware, firmware, or software, or a combination thereof. The processes are shown as a collection of blocks that specify operations performed by one or more devices, and are not necessarily limited to the commands shown for performing the operations by the various boxes.

[0081] Figure 7 An example process 700 is depicted for training a language preference model using machine intent and target audience training data.

[0082] Initially, target audience training data corresponding to a corpus of textual content for a target audience vocabulary is obtained (block 702). For example, the language preference training module 124 obtains target audience training data 126 corresponding to a corpus of textual content for a corresponding target audience vocabulary 114. For example, the training data 126 may include a plurality (e.g., ten thousand) of segments of textual content (e.g., articles, papers, books) generated using the intended target audience vocabulary 114.

[0083] The language preference model is trained using one or more machine learning techniques to learn word preference probabilities for the target audience vocabulary based on the frequency of word usage in the target audience vocabulary (block 704). For example, the training module 124 uses the training data 126 to learn the word preference probabilities 202 of the language preference model 122. The word preference probabilities 202 indicate relative preferences between words (e.g., synonyms) with similar meanings in the target audience vocabulary 114. In some cases, the determination of the word preference probabilities 202 is based on the frequency of word usage in the training data 126, such that if the first word appears more frequently in the training data 126, the first word is preferred over a second word that is a synonym of the first word.

[0084] In one or more implementations, to train the language preference model 122, each candidate term 204 in the training data 126 is passed to a pairing module 206 employed by the language preference training module 124. The pairing module 206 identifies all of the word senses 208 for a given candidate term 204 and all of the synonyms 210 for each of the word senses 208 of the candidate term 204. The pairing module 206 then generates a word sense synonym pair 212 for each corresponding candidate term 204. The word sense-synonym pairs 212 are then passed to a frequency module 214 of the training module 124. The frequency module 214 determines a word sense frequency 216 and a synonym frequency 218 for each word sense 208 and synonym 210 of the word sense-synonym pairs 212, respectively. The frequencies 216 and 218 correspond to the number of occurrences of each term in the training data 126. Next, the term preference probability module 220 of the training module 124 obtains the term sense frequency 216 and the synonym frequency 218 for each corresponding term sense-synonym pair 212 , and generates the term preference probability 202 for the corresponding term sense-synonym pair 212 .

[0085] Figure 8 An example process 800 is depicted for generating a targeted summary of textual content using a term generation model and a language preference model.

[0086] Generate a target summary of text content by a word generation model (block 802 ). As an example, the target summary 116 of the text content 112 is generated by the word generation model 120 .

[0087] During generation of the target summary, the words of the target summary generated by the word generation model are adjusted to the target audience vocabulary using a language preference model, wherein the language preference model is trained using machine learning of target audience training data including data of a corpus of text of the target audience vocabulary to learn word preferences of the target audience vocabulary between similar words (block 804). As an example, during generation of the target summary 116, the words of the target summary generated by the word generation model 120 are adjusted to the target audience vocabulary 114 using a language preference model 122. The language preference model 122 is trained using machine learning of the target audience training data 126 to learn word preferences of the target audience vocabulary between similar words (e.g., synonyms).

[0088] During the generation of the summary, feedback for each generated word of the summary is provided to the word generation model, wherein the feedback utilized by the word generation model is used to generate subsequent words of the target summary (block 806). As an example, the feedback module 318 provides the generated word feedback 320 to the word generation model 120 during the generation of the target summary 116. The word generation model 120 then utilizes the feedback 320 to generate subsequent words of the target summary 116.

[0089] Fig. 9 An additional example process 900 for generating a targeted summary of text content tuned to the target audience vocabulary is depicted.

[0090] A request is received to generate a targeted summary of textual content tailored to a target audience vocabulary (block 902 ). As an example, the targeted summary module 118 receives a request 110 to generate a targeted summary of textual content 112 tailored to a target audience vocabulary 114 .

[0091] An attention distribution is generated by a word generation model, the attention distribution including words from the text content and the target audience vocabulary and a selection probability value for each word in the attention distribution, wherein the selection probability value indicates the corresponding probability of each word being selected as the next word of the target summary of the text content (box 904). As an example, the word generation model 120 generates an attention distribution 304 of words 306 in the text content 112 and the target audience vocabulary 114 together with a selection probability value 308 for each word. The selection probability value 308 indicates the corresponding probability of each word being selected as the next generated word of the target summary 116. The attention decoder 302 of the word generation model 120 can determine the selection probability value 308 based on the words in the input text content 112 and the words of the target summary 116 that have been generated by the word generation model 120.

[0092] The next word of the target profile is selected based on the attention distribution and the language preference model indicating the word preference probabilities of the target audience vocabulary (block 906). For example, the selection module 310 selects the next generated word 312 for the target profile 116 based on the attention distribution 304 generated by the attention decoder 302 of the word generation model 120 and the language preference model 122 trained on the training data 126 corresponding to the target audience vocabulary 114.

[0093] In some cases, to encourage the generation of the most preferred terms of the target audience vocabulary 114, the selection module 310 may modify the selection probability value 308 of the attention distribution 304 with the term preference probabilities 202 of the target audience learned by the language preference model 122. Thus, the selection module 310 selects the next generated term 312 by selecting terms from the input text content 112 based on the attention distribution 304, or by replacing terms from the input text content 112 with the most preferred synonyms of the target audience as indicated by the term preference probabilities 202 of the language preference model 122. The selection module 310 may select the next generated term 312 of the target profile 116 using a variety of different term generation techniques, such as generation 314 of statistical substitution or generation 316 of probability reweighting.

[0094] Feedback of the selected next word is provided to the word generation model, which causes the word generation model to modify the attention distribution for the selection of subsequent words of the target summary based on the feedback of the next selected word (box 908). For example, after the selection module 310 selects the next generated word 312, the generated word is added to the target summary 116. In addition, the feedback module 318 provides the generated word feedback 320 to the attention decoder 302, which can be used by the attention decoder 302 of the word generation model 120 to select the subsequent word of the target summary 116.

[0095] Having described example processes in terms of one or more implementations, consider now example systems and devices that can be used to implement the various techniques described herein.

[0096] Example Systems and Devices

[0097] Fig.10 An example system is illustrated generally at 1000, which includes an example computing device 1002, which represents one or more computing systems and / or devices that may implement the various techniques described herein. This is illustrated by the inclusion of a target profile module 118. The computing device 1002 may be, for example, a server of a service provider, a device associated with a client (e.g., a client device), a system on a chip, and / or any other suitable computing device or computing system.

[0098] The example computing device 1002 shown in the figure includes a processing system 1004, one or more computer-readable media 1006, and one or more I / O interfaces 1008 that are communicatively coupled to each other. Although not shown, the computing device 1002 may also include a system bus or other data and command transmission systems that couple various components to each other. The system bus may include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus that utilizes any one of a variety of bus architectures. Various other examples are also contemplated, such as control and data lines.

[0099] Processing system 1004 represents the functionality of performing one or more operations using hardware. Thus, processing system 1004 is illustrated as including hardware elements 1010 that may be configured as processors, functional blocks, and the like. This may include implementation in hardware as a dedicated integrated circuit or other logic device formed using one or more semiconductors. Hardware elements 1010 are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, a processor may include semiconductors and / or transistors (e.g., electronic integrated circuits (ICs)). In such a context, processor executable instructions may be electronic executable instructions.

[0100] Computer-readable storage media 1006 is illustrated as including memory / storage 1012. Memory / storage 1012 represents memory / storage capacity associated with one or more computer-readable media. Memory / storage 1012 may include volatile media (such as random access memory (RAM)) and / or non-volatile media (such as read-only memory (ROM), flash memory, optical disks, magnetic disks, etc.). Memory / storage 1012 may include fixed media (e.g., RAM, ROM, fixed hard drives, etc.) and removable media (e.g., flash memory, removable hard drives, optical disks, etc.). Computer-readable media 1006 may be configured in various other ways as further described below.

[0101] Input / output interface 1008 represents functionality for allowing a user to input commands and information to computing device 1002, and also allows information to be presented to the user and / or other components or devices using various input / output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone, a scanner, touch functionality (e.g., a capacitive or other sensor configured to detect physical touch), a camera (e.g., which may employ visible or invisible wavelengths such as infrared frequencies to recognize motion as gestures that do not involve touch), and the like. Examples of output devices include a display device (e.g., a monitor or projector), a speaker, a printer, a network card, a tactile response device, and the like. Thus, computing device 1002 may be configured in various ways as further described below to support reader interaction.

[0102] Various techniques may be described herein in the general context of software, hardware elements, or program modules. Typically, such modules include routines, programs, objects, elements, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The terms "module," "function," and "component" as used herein generally represent software, firmware, hardware, or a combination thereof. The features of the techniques described herein are platform-independent, meaning that the techniques can be implemented on a variety of commercial computing platforms with a variety of processors.

[0103] An implementation of the described modules and techniques may be stored on or transmitted via some form of computer-readable media. Computer-readable media may include a variety of media that can be accessed by computing device 1002. By way of example and not limitation, computer-readable media may include “computer-readable storage media” and “computer-readable signal media.”

[0104] "Computer-readable storage media" may refer to media and / or devices that enable persistent and / or non-transitory storage of information as compared to mere signal transmission, carrier waves, or signals themselves. Therefore, computer-readable storage media refers to non-signal bearing media. Computer-readable storage media include hardware such as volatile and non-volatile, removable and non-removable media and / or storage devices, which are implemented in a method or technology suitable for the storage of information such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data. Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, hard disk, cassette, magnetic tape, disk storage or other magnetic storage devices, or other storage devices, tangible media, or articles suitable for storing the desired information and accessible by a computer.

[0105] "Computer-readable signal media" may refer to signal-bearing media that is configured to send instructions to the hardware of the computing device 1002, such as via a network. Signal media may typically embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, data signal, or other transport mechanism. Signal media also includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode the information in the signal. By way of example, and not limitation, communication media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, and other wireless media.

[0106] As previously described, hardware elements 1010 and computer-readable media 1006 represent modules, programmable device logic, and / or fixed device logic implemented in hardware form, which may be employed in some embodiments to implement at least some aspects of the technology described herein, such as executing one or more instructions. Hardware may include components of an integrated circuit or system on a chip, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), and other implementations of silicon or other hardware. In this context, hardware may operate as a processing device that performs program tasks defined by instructions and / or logic implemented by hardware, and hardware (e.g., a computer-readable storage medium described previously) for storing instructions for execution.

[0107] The aforementioned combination can also be used to implement the various technologies described herein. Therefore, software, hardware or executable modules can be implemented as one or more instructions and / or logic implemented on some form of computer-readable storage medium and / or by one or more hardware elements 1010. Computing device 1002 can be configured to implement specific instructions and / or functions corresponding to software and / or hardware modules. Therefore, the implementation of a module that can be executed as software by computing device 1002 can be at least partially implemented in hardware, for example, by the use of hardware elements 1010 of computer-readable storage medium and / or processing system 1004. Instructions / or functions can be performed / operated by one or more products (e.g., one or more computing devices 1002 and / or processing system 1004) to implement the technology, modules and examples described herein.

[0108] The techniques described herein can be supported by various configurations of computing device 1002 and are not limited to the specific examples of the techniques described herein. The functionality can also be implemented in whole or in part through the use of a distributed system, such as on a "cloud" 1014 through a platform 1016 as described below.

[0109] The cloud 1014 includes and / or represents a platform 1016 for resources 1018. The platform 1016 abstracts the underlying functionality of the hardware (e.g., servers) and software resources of the cloud 1014. The resources 1018 may include applications and / or data that may be utilized when a computer process is executed on a server remote from the computing device 1002. The resources 1018 may also include services provided over the Internet and / or over a subscriber network such as a cellular or Wi-Fi network.

[0110] The platform 1016 can abstract resources and functionality to connect the computing device 1002 with other computing devices. The platform 1016 can also be used to abstract resource scaling to provide a corresponding level of scaling to the demand faced by the resources 1018 implemented via the platform 1016. Therefore, in an interconnected device embodiment, the implementation of the functionality described herein can be distributed throughout the system 1000. For example, the functionality can be implemented partially on the computing device 1002 and via the platform 1016 that abstracts the functionality of the cloud 1014.

[0111] in conclusion

[0112] Although the invention has been described in language specific to structural features and / or methodological acts, it should be understood that the invention defined in the appended claims is not necessarily limited to the specific features or acts described. Instead, the specific features and acts are disclosed as example forms of implementing the claimed invention.

Claims

1. A method implemented by at least one computing device in a digital media environment for generating a targeted summary of text content tuned to a target audience vocabulary, the method comprising: receiving, by the at least one computing device, a request to generate the targeted summary of textual content tailored to the target audience vocabulary; generating, by a term generation model, an attention distribution including terms from the text content and the target audience vocabulary, and a selection probability value for each term in the attention distribution, the selection probability value indicating a respective probability of each term being selected as the next term of the target summary of the text content; modifying the selection probability values ​​of the words in the focus distribution with corresponding word preference probabilities of a language preference model, the language preference model being trained using one or more machine learning techniques to learn the word preference probabilities of the target audience vocabulary based on word usage frequencies in target audience training data, the training of the language preference model comprising identifying synonyms of a plurality of words in the target audience training data, and generating, for word-synonym pairs, corresponding word preference probabilities based on corresponding frequencies of corresponding words and corresponding synonyms in the target audience training data; selecting the next word for the target profile based on the modified selection probability value of the attention distribution; as well as Feedback of the selected next word is provided to the word generation model, the feedback causing the word generation model to modify the attention distribution for selection of subsequent words of the target profile based on the feedback of the next generated word.

2. The method of claim 1, wherein selecting the next word for the target profile further comprises: selecting at least one word generated by the word generation model based on the attention distribution; as well as The selected at least one term is replaced with a more preferred synonym of the target audience vocabulary identified by the language preference model.

3. The method of claim 1 , wherein selecting the next word for the target profile further comprises: Words in the focus distribution are replaced with more preferred synonyms identified by the language preference model. The method of claim 1 , wherein the target audience training data comprises a corpus of the target audience vocabulary. The method of claim 4 , wherein the term preference probability indicates a relative preference between a respective term and a corresponding synonym.

6. The method of claim 1, wherein the term generation model comprises an extraction summarization algorithm or an abstraction summarization algorithm trained using one or more machine learning techniques.

7. The method of claim 6, wherein the term generation model can be utilized to generate target profiles tuned to a plurality of different target audience vocabularies by using different respective language preference models corresponding to respective target audience vocabularies without retraining the term generation model.

8. The method of claim 1, wherein the feedback is provided to the term generation model during generation of the summary.

9. A system implemented in a digital media environment for generating targeted summaries of text content tuned to a target audience vocabulary, the system comprising: at least one processor; a memory having computer readable instructions stored thereon, the computer readable instructions being executable by the processor to implement a target summary module for performing operations comprising: receiving a request for generating the target summary of textual content tailored to the target audience vocabulary; generating, by a term generation model, an attention distribution including terms from the text content and the target audience vocabulary, and a selection probability value for each term in the attention distribution, the selection probability value indicating a respective probability of each term being selected as the next term of the target summary of the text content; modifying the selection probability values ​​of the words in the focus distribution with corresponding word preference probabilities of a language preference model, the language preference model being trained using one or more machine learning techniques to learn the word preference probabilities of the target audience vocabulary based on word usage frequencies in target audience training data, the training of the language preference model comprising identifying synonyms of a plurality of words in the target audience training data, and generating, for word-synonym pairs, corresponding word preference probabilities based on corresponding frequencies of corresponding words and corresponding synonyms in the target audience training data; selecting the next word for the target profile based on the modified selection probability value of the attention distribution; and Feedback of the selected next word is provided to the word generation model, the feedback causing the word generation model to modify the attention distribution for selection of subsequent words of the target profile based on the feedback of the next generated word.

10. The system of claim 9, wherein selecting the next word for the target profile comprises: One or more terms of the distribution of interest are replaced with more preferred synonyms identified by the language preference model.

11. The system of claim 9, wherein the term preference probability indicates a relative preference between a corresponding term and a corresponding synonym.

12. The system of claim 9, wherein the term generation model comprises an extractive summarization algorithm or an abstractive summarization algorithm trained using one or more machine learning techniques.

13. The system of claim 12, wherein the term generation model can be utilized to generate target profiles tuned to a plurality of different target audience vocabularies by using different respective language preference models corresponding to respective target audience vocabularies without retraining the term generation model.

14. The system of claim 9, wherein the target audience training data comprises a corpus of the target audience vocabulary.

15. The system of claim 9, wherein the feedback is provided to the term generation model during generation of the target summary.

16. One or more non-transitory computer-readable storage media comprising instructions stored thereon that, in response to execution by a computing device, cause the computing device to perform operations comprising: receiving a request for generating a targeted summary of text content tuned to a target audience vocabulary; generating, by a term generation model, an attention distribution including terms from the text content and the target audience vocabulary, and a selection probability value for each term in the attention distribution, the selection probability value indicating a respective probability of each term being selected as the next term of the target summary of the text content; modifying the selection probability values ​​of the words in the focus distribution with corresponding word preference probabilities of a language preference model, the language preference model being trained using one or more machine learning techniques to learn the word preference probabilities of the target audience vocabulary based on word usage frequencies in target audience training data, the training of the language preference model comprising identifying synonyms of a plurality of words in the target audience training data, and generating, for word-synonym pairs, corresponding word preference probabilities based on corresponding frequencies of corresponding words and corresponding synonyms in the target audience training data; selecting the next word for the target profile based on the modified selection probability value of the attention distribution; as well as Feedback of the selected next word is provided to the word generation model, the feedback causing the word generation model to modify the attention distribution for selection of subsequent words of the target summary based on the feedback of the next generated word.

17. The one or more non-transitory computer-readable storage media of claim 16, wherein selecting the next word for the target profile comprises: One or more terms of the distribution of interest are replaced with more preferred synonyms identified by the language preference model.

18. The one or more non-transitory computer-readable storage media of claim 16, wherein the term preference probabilities indicate a relative preference between a respective term and a corresponding synonym.

19. The one or more non-transitory computer-readable storage media of claim 16, wherein the distribution of interest is generated using an extractive summarization algorithm or an abstractive summarization algorithm trained by one or more machine learning techniques.

20. The one or more non-transitory computer-readable storage media of claim 16, wherein the feedback is provided during generation of the goal summary.

Images