Dynamic construction of large language model cues

Abstract

Example solutions are disclosed for reducing the likelihood of illusion generation by a language model, such as a large language model (LLM). By injecting a sufficient range and amount of carefully selected fact data into the cues, the likelihood of hallucination by LLM may be reduced. This enables the language model to be used in a wider range of settings in which the pinching facts are problematic, while reducing the need for humans to carefully check the accuracy of the generated text. Examples include generating a transcribed summary using a summary model; extracting topic-specific data from the stored data using a scoring model; dynamically generating a language model hint using the topic-specific data and the digest; and generating output text using the language model and the language model hint.

Description

Background Art

[0001] Generative Pretrained Transformers (GPTs), such as GPT-3 and GPT-4, are a form of artificial intelligence (AI) within a group of machine learning (ML) models known as Large Language Models (LLMs). LLMs are typically deep learning models consisting of neural networks with billions of parameters, trained to predict the next word in a sentence. This allows LLMs to generate text similar to human-generated text given a given context. The current version of chatGPT is based on either GPT-3 or GPT-4. For example, GPT-4 can generate text describing an image.

[0002] However, LLMs can produce hallucinations. Hallucinations are confident responses made by an AI entity (such as a large language model) that are not corroborated by the AI ​​entity's training data and can involve fabricating alleged facts. Fabricating facts can be problematic when generating text to be consumed in commercial settings, thus preventing some applications from using LLMs. Summary of the Invention

[0003] The disclosed examples are described in detail below with reference to the drawings listed below.The following summary is intended to illustrate some of the examples disclosed herein.

[0004] An example solution for performing dynamic construction of large language model hints includes: using a summarization model to generate a summary of a transcription; using a scoring model to extract topic-specific data from stored data; using the topic-specific data and the summary to dynamically generate language model hints; and using the language model and the language model hints to generate output text. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The disclosed examples will now be described in detail with reference to the accompanying drawings listed below:

[0006] Figure 1 An example architecture that provides dynamic construction of large language model hints is shown;

[0007] Figure 2 An exemplary summary model is shown, such as may be found in Figure 1 Used in the example architecture;

[0008] Figure 3 An exemplary scoring model is shown, such as may be found in Figure 1 Used in the example architecture;

[0009] Figure 4 An exemplary dynamic prompt generator is shown, such as may be found in Figure 1 Used in the example architecture;

[0010] Figure 5A 、 Figure 5Band Figure 5C Shows the Figure 4 An exemplary progression of input to output text for a dynamic prompt generator, such as may be found in Figure 1 This happens in the example architecture of

[0011] Figure 6A 、 Figure 6B and Figure 6C Shows the Figure 4 An exemplary progression of input to output text for a dynamic prompt generator, such as may be found in Figure 1 This happens in the example architecture of

[0012] Figure 7 and Figure 8 Shown is a diagram showing how to use an example architecture such as Figure 1 a flowchart of exemplary operations that may be performed when the architecture of FIG.

[0013] Figure 9 A block diagram of an example computing device suitable for implementing some of the various examples disclosed herein is shown.

[0014] Corresponding reference numerals indicate corresponding parts throughout the drawings. DETAILED DESCRIPTION

[0015] Because large language models (LLMs) carry the risk of falsifying facts, in some settings, such as business-to-business communications, human users are either required to carefully check the accuracy of LLM-generated text or a solution is needed to reduce the risk of hallucinations. By injecting a sufficient range and amount of carefully curated factual data into the prompts provided to a language model (e.g., an LLM), the likelihood of hallucinations generated by the language model can be reduced.

[0016] This article discloses example solutions for reducing the likelihood of hallucinations generated by language models. This enables the use of language models in a wider range of settings where fabricating facts is problematic, while reducing the need for humans to carefully review the accuracy of the generated text. Examples include: using a summarization model to generate summaries of transcriptions; using a scoring model to extract topic-specific data from stored data; using topic-specific data and summaries to dynamically generate language model hints; and using a language model and language model hints to generate output text.

[0017] The example solutions described herein advantageously improve the accuracy of computational operations, such as reducing artificial intelligence (AI) hallucinations, by dynamically generating language model hints using subject-specific data (known to have sufficient accuracy) and a summary of the transcription (which provides context for the output text).

[0018] Various examples will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or similar components. References throughout this disclosure to specific examples and implementations are provided for illustrative purposes only and are not intended to be limiting of all examples unless otherwise indicated.

[0019] Figure 1 An example architecture 100 is shown that provides for dynamic construction of large language model prompts. A microphone 102 captures audio data 104 from a conversation between three speakers 106a-106c. In some examples, multiple audio capture devices are used, and the conversation may be a conversation between participants in a telephone call or video conference, such as over a public switched telephone network (PSTN), a computer network using voice over Internet Protocol (IP), or another device. In some examples, the number of participating speakers may vary.

[0020] In one scenario, speaker 106a is a sales representative, speaker 106b is a potential customer, and architecture 100 is used to generate an email message 134 that summarizes the conversation as captured in audio data 104. For example, audio data 104 includes speech signal 108 from speaker 106b, and email message 134 includes a summary of statements made by speaker 106b within speech signal 108, as well as statements made by speaker 106a and, if speaker 106c is a third participant, a statement from speaker 106c.

[0021] The automatic speech recognition (ASR) module 110 generates a transcription 112 of the audio data 104, which is provided to the summary model 200. The summary model 200 generates a summary 114 of the transcription 112. In some examples, the summary model 200 includes a machine learning (ML) model. As used herein, ML includes artificial intelligence (AI). Further details of this stage are provided in Figure 2 Shown in.

[0022] Additionally, a second model or logic component (illustrated as scoring model 300) extracts curated topic-specific data 116 from stored data 118. In some examples, scoring model 300 also includes an ML model. In some examples, stored data 118 includes graphical data, such as customer relationship management (CRM) data and enterprise suite data (e.g., calendar information and / or other data files available from an office productivity software suite). In some examples, summary 114 or an excerpt from summary 114 is used to identify speaker 106b and / or speaker 106c. Further details of this stage are provided in Figure 3 Shown in.

[0023] The summary 114 and topic-specific data 116 are combined as input 120 to a dynamic prompt generator 400, which generates customized language model prompts 122 for a language model 124. The language model prompts 122 are customized to the context of the conversation, such as the number of speakers involved and the context (e.g., the stage of the sales process). Further details of this stage are provided in Figure 4 Shown in.

[0024] The language model 124 can be an LLM and, in some examples, includes GPT-3, GPT-4, chatGPT, or an equivalent. The language model 124 generates output text 130, which can be automatically sent as an email message 134 or edited (such as by the speaker 106a or another person) using an editor 132 to refine the output text 130 into an email message 134. The email message 134 is then sent across a computer network 140 (e.g., including the Internet) to a recipient, such as the speaker 106b (e.g., a participant in the conversation) and / or another recipient who may benefit from receiving a summary of the conversation.

[0025] Figure 2 1 shows further details of summary model 200 generating summary 114 from transcription 112. Transcription 112 includes name 202a corresponding to speaker 106b, although in this example scenario, name 202a has a spelling error. Summary model 200 may output name 202a into summary 114 as it was received in transcription 112 without correcting it. However, as described below, the spelling error in name 202a will be corrected by dynamic prompt generator 400.

[0026] The summary model 200 has been trained to ingest large transcripts, covering conversations lasting tens of minutes (e.g., 30 minutes or more), and produce relevant summaries. Thus, the summary 114 includes at least one action item 204 and at least one summary highlight 206. Figures 5A to 6B Example action items and highlights are shown in . Highlights are short phrases summarizing a single subject matter aspect of the transcript 112, which might each cover a few minutes of conversation; and action items are similar to highlights, but give an anticipation of some future activity of one or more people.

[0027] Figure 3106b , which is a textual material provided by the speaker 106b at some previous point in time, or has otherwise been reviewed over time, whereas the name 202a in the transcription 112 is the result of the ASR process. Example spelling errors (discussed later in this section) Figure 5A ) is: name 202a is "Sarah" and name 202b is "Sara." It should be noted that this is merely an example, and in some examples, name 202a is correct and name 202b is incorrect.

[0028] The CRM data 302 also includes a CRM interaction history 304 associated with the speaker 106b, as represented by the names 202a and 202b, among the interaction histories of the plurality of individuals. The scoring model 300 receives participant identifications (IDs) 330 identifying the participants in the conversation captured as the audio data 104 and uses the participant IDs 330 to select relevant information within the stored data 118. In some examples, this is achieved by using a mixture of logic and heuristics to score the information within the stored data 118.

[0029] As shown, scoring model 300 generates scores 310 indicating the relevance of information within stored data 118 to participant ID 330 and / or abstract 114. In some examples, participant ID 330 is extracted from abstract 114; in some examples, abstract 114 is provided to scoring model 300; and in some examples, criteria for scoring the relevance of stored data 118 are provided in some other manner. Ranking 312 of scores 310 is used to select information downward within the stored date to generate topic-specific data 116.

[0030] The topic-specific data 116 has a subset of information from the stored data 118 that is most relevant to the summary 114. In some examples, the topic-specific data 116 includes CRM data 322 and enterprise suite data 326. The CRM data 322 and enterprise suite data 326 are select data sets because they are relevant to the summary 114 and / or the participants in the conversation, as indicated by the participant ID 330. For example, the CRM data 322 includes the name 202b and the (CRM) interaction history 324, and the enterprise suite data 326 includes calendar information 328. Figure 1In this example, where the conversation indicated in the email message 134 is a sales call, the interaction history 324 indicates whether the name 202b is associated with a contact who is a lead in the introductory phase of a sale, a prospect in the middle of a sale, or someone who is at or nearing the closing phase of a sale. The calendar information 328 is an indication of one or more potential appointments that the speaker 106b (or another intended recipient of the email message 134) has available for a subsequent meeting, call, or video conference.

[0031] Figure 4 Further details of dynamic prompt generation are shown. Input 120 is shown to include summary 114 and topic specific data 116. Dynamic prompt generator 400 has scenario specific behavior 410, such as logic and heuristics, to generate dynamic prompts specific to the context. Figure 4 106a-106c), and the CRM history of the conversation participants, as captured in the interaction history 324.

[0032] If the interaction history 324 indicates that the participants have had a previous conversation, rather than that this conversation is an initial introduction, the scenario-specific behavior 410 will generate a prompt that is different from the prompt generated for the initial introduction conversation. This dynamic approach offers advantages over traditional approaches that use static prompts (e.g., a predetermined set of prompts that are common to a group of conversations) because it prevents duplicate email messages 134 from being sent to speaker 106b at different times—even if the conversations are similar. The prompts will be different for new sales leads, won leads, and lost leads. Additionally, the prompts will be accurate and relevant.

[0033] The summary has name 202a, which was generated via ASR, while the topic-specific data 116 has name 202b, which was extracted from CRM data 302. For some names that are spelled differently but pronounced similarly, this creates the possibility of a discrepancy 402 between name 202a in summary 114 and name 202b in topic-specific data 116. In some examples, dynamic prompt generator 400 will use topic-specific data 116 as the basis for resolving discrepancy 402.

[0034] Figure 5A 、 5B 5C show an exemplary progression from input 120 to output text 130 for a two-participant conversation between speaker 106a (known as "Alex") and speaker 106b (known as "Sara"). Figure 6A 、 Figure 6B and Figure 6CAnother exemplary progression from input 120 to output text 130 is shown for a three-participant conversation between speaker 106a (as "Alex"), speaker 106b (as "John"), and speaker 106c (as "Bob").

[0035] Figure 5A A text passage 500 is shown, which may be a version of the input 120. The name 202b, the contact sales stage 502 (from the interaction history 324), the account name 504 (e.g., the contact's organization or employer), the product 506 that is the subject of the conversation, the calendar information 328 (e.g., available appointments extracted from calendar software), and the date 508 of the conversation are from the topic-specific data 116. For example, the name 202a, the contact sales stage 502, the account name 504, and the product 506 are from the CRM data 322, and the calendar information 328 is from the enterprise suite data 326. The date 508 may also be from the enterprise suite data 326 or from another source, such as a timestamp of the audio data 104.

[0036] The text passage 500 also has a highlights section 510 having the name 202a included among several highlights, and an action item section 512 identifying a plurality of action items. The highlights section 510 and the action item section 512 are from the summary 114. Figure 5A As can be seen in FIG, name 202a is "Sarah" and name 202b is "Sara".

[0037] Figure 5B A text paragraph 520 is shown, which may be a version of the language model prompt 122. The call date field 522 shows the date 508, and the sender field 524 identifies the email account and signature that should be used for the email message 134. The recipient field 526 shows the name 202b (instead of the name 202a) and identifies the individual (or other email account) to which the email message 134 is to be addressed. The account name field 528 identifies the account name 504, and the product name field 530 identifies the product 506. The meeting availability field 532 shows the calendar information 328 as available meeting times and dates.

[0038] The key points section 540 is a condensed (selected) version of the key points section 510 of the text paragraph 500 and includes the summary key points 206. The action item 204 is also included because it is a high confidence action item. Another action item in the action item section 512 of the text paragraph 500 is discarded due to low confidence or low relevance. The instruction section 544 includes a set of language models that instruct the guidance language model 124.

[0039] Instruction 546 is a customized prompt for an early sales opportunity because it prompts language model 124 to provide a product suggestion (i.e., introduction) for the recipient's organization (combining product 506 with account name 504). This is a result of the contact sales stage 502 in text paragraph 520. Instruction 548 prompts language model 124 to suggest a follow-up meeting (or other call) based on calendar information 328.

[0040] Figure 5C A text paragraph 550 is shown, which may be a version of the output text 130. The text paragraph 550 includes the name 202b, the date 508, the product 506, the account name 504, and the calendar information 328. The paragraph 552 is customized for the initial contact sales phase and may be based on how the language model 124 reacts to the instruction 546.

[0041] Figure 6A A text paragraph 600 is shown, which may be a version of input 120. Name 202b, contact sales stage 602, account name 604, product 606, calendar information 328, and date 608 are from subject-specific data 116 and are Figure 5A Name 202c is also included because text paragraph 600 is a scene of a three-participant conversation. Text paragraph 600 also has a focus section 610 and an action item section 612 for identifying multiple action items, which correspond to focus section 510 and action item section 512, respectively.

[0042] Figure 6B A text paragraph 620 is shown, which may be a version of the language model prompt 122. The call date field 622 shows the date 608, and the sender field 624 identifies the email account and signature that should be used for the email message 134. The recipient field 626 shows the name 202b and identifies the individual (or other email account) to which the email message 134 is to be addressed. The account name field 628 identifies the account name 604, and the product name field 630 identifies the product 606. The meeting availability field 632 shows the calendar information 328 as available meeting times and dates.

[0043] The key points section 640 is a condensed (selected) version of the key points section 610 of the text paragraph 600 and includes the summary key points 206. The action item 204 in the action item section 642 is also included because it is a high confidence action item. The instruction section 644 includes a set of language models that instruct the guidance language model 124.

[0044] Instruction 646 is a custom prompt for a mid-term sales opportunity because it prompts language model 124 to provide language other than an introduction to the product. This is a result of the contact sales stage 602 in text paragraph 620. Instruction 648 prompts language model 124 to suggest a follow-up meeting (or other call) based on calendar information 328.

[0045] Figure 6C A text paragraph 650 is shown, which may be a version of the output text 130. The text paragraph 650 includes the names 202b and 202c, the date 608, the product 606, the account name 604, and the calendar information 328. The paragraph 652 is customized for the mid-term sales opportunity and may be based on how the language model 124 reacts to the instruction 646.

[0046] Figure 7 A flowchart 700 illustrating exemplary operations that may be performed by the architecture 100 for dynamic construction of large language model hints is shown. In some examples, the operations described with respect to the flowchart 700 are performed by Figure 9 The flowchart 700 begins by capturing or otherwise receiving audio data 104 in operation 702. Operation 704 uses ASR on the audio data 104 to generate a transcription 112. Figure 7 In the example shown in , the audio data 104 includes a speech signal 108 corresponding to the name 202 b in the CRM data 322 .

[0047] Operation 706 generates a summary 114 for the transcript 112 using the summary model 200. In some examples, the summary 114 includes the action items 204 extracted from the transcript 112. In some examples, the summary 114 also includes summary highlights 206. In some examples, the summary model 200 includes an ATS model and / or an ML model.

[0048] Operation 708 uses the scoring model 300 to extract the subject-specific data 116 from the stored data 118. In some examples, the stored data 118 includes graph data. In some examples, the stored data 118 includes uncurated CRM data 302 and / or uncurated enterprise suite data 306. In some examples, the subject-specific data 116 includes curated CRM data 322 and / or curated enterprise suite data 326. The CRM data 322 includes the name 202b. In some examples, the enterprise suite data 326 includes calendar information 328. In some examples, the scoring model 300 includes an ML model.

[0049] In some examples, operation 708 is performed using operations 710 and 712. Operation 710 scores the relevance of stored data 118 to transcript 112 using scoring model 300. Operation 712 selects curated enterprise suite data 326 and curated CRM data 322 based at least on the scores.

[0050] Operation 714 dynamically generates a language model hint 122 using the topic-specific data 116 and the summary 114. In some examples, the language model hint 122 is based on at least the count of speakers 106a-106c identified in the summary 114. In some examples, the language model hint 122 is based on at least the interaction history 324 associated with the name 202a identified in the summary 114. In some examples, the language model hint 122 is based on at least the calendar information 328. In some examples, as part of operation 714, operation 716 resolves the differences 402 between the summary 114 and the topic-specific data 116 based on the topic-specific data 116. In some examples, the differences 402 between the summary 114 and the topic-specific data 116 include name differences.

[0051] Operation 718 generates output text 130 using language model 124 and language model hint 122. In some examples, language model 124 includes GPT, such as GPT-3, GPT-4, or chatGPT. In some examples, output text 130 is edited in operation 720 and then sent as an email message 134. Operation 722 sends output text 130 as email message 134 across computer network 140.

[0052] Figure 8 A flowchart 800 is shown that illustrates exemplary operations that may be performed by the architecture 100. In some examples, the operations described with respect to the flowchart 800 are performed by Figure 9 The flowchart 800 begins with operation 802, which includes generating a summary of the transcription using a summary model.

[0053] Operation 804 includes extracting topic-specific data from the stored data using the scoring model. Operation 806 includes dynamically generating language model hints using the topic-specific data and the summary. Operation 808 includes generating output text using the language model and the language model hints.

[0054] Additional Examples

[0055] An example system includes: a processor; and a computer-readable medium storing instructions that, when executed by the processor, are operable to: generate a summary of a transcription using a summarization model; extract topic-specific data from stored data using a scoring model; dynamically generate language model hints using the topic-specific data and the summary; and generate output text using the language model and the language model hints.

[0056] An example computer-implemented method includes: using a summarization model to generate a summary of a transcription of audio data; using a scoring model to extract topic-specific data from stored data; using the topic-specific data and the summary to dynamically generate language model hints; and using the language model and the language model hints to generate output text.

[0057] One or more example computer storage devices have computer-executable instructions stored thereon that, when executed by a computer, cause the computer to perform operations comprising: generating a summary of a transcription using a summarization model; extracting topic-specific data from stored data using a scoring model, wherein the stored data includes graph data; dynamically generating language model hints using the topic-specific data and the summary; and generating output text using the language model and the language model hints.

[0058] Alternatively, or in addition to other examples described herein, examples include any combination of the following:

[0059] - Sending output text as an email message across a computer network;

[0060] - Subject-specific data includes curated enterprise suite data and curated CRM data;

[0061] -The summary includes action items extracted from the transcription;

[0062] - Use ASR on audio data to generate transcriptions;

[0063] - language model hints are based at least on the counts of speakers identified in the summary;

[0064] - a language model hint based on at least the interaction history associated with the name identified in the summary;

[0065] -Enterprise Suite data including calendar information;

[0066] - Language model hints are based at least on calendar information;

[0067] - The subject-specific data will prevail to resolve discrepancies between the abstract and the subject-specific data;

[0068] - Capture audio data;

[0069] - receiving audio data;

[0070] - the audio data contains a speech signal corresponding to the name in the CRM data;

[0071] -The abstract also includes the key points;

[0072] -Summary models include automatic text summarization (ATS) models;

[0073] -Summary models include ML models;

[0074] -Stored data includes graphic data;

[0075] - The stored data includes uncurated enterprise suite data;

[0076] - The stored data includes uncurated CRM data;

[0077] -Enterprise Suite data including calendar information;

[0078] - Use a scoring model to score the relevance of the stored data to the transcript;

[0079] -Selection of at least select enterprise suite data and select CRM data based on the ratings;

[0080] -CRM data includes name;

[0081] - Scoring models include ML models;

[0082] - discrepancies between the abstract and the subject-specific data, including name discrepancies;

[0083] - Language models include GPT;

[0084] - Language models include GPT-3, GPT-4, or chatGPT; and

[0085] - Edit output text before sending it as an email message.

[0086] Although aspects of the present disclosure have been described in terms of various examples and their associated operations, those skilled in the art will understand that combinations of operations from any number of different examples are also within the scope of aspects of the present disclosure.

[0087] Sample operating environment

[0088] Figure 9is a block diagram of an example computing device 900 (e.g., a computer storage device) for implementing various aspects disclosed herein, and is collectively designated as computing device 900. In some examples, one or more computing devices 900 are provided for local computing solutions. In some examples, one or more computing devices 900 are provided as cloud computing solutions. In some examples, a combination of local and cloud computing solutions is used. Computing device 900 is merely one example of a suitable computing environment and is not intended to imply any limitation on the scope of use or functionality of the examples disclosed herein, whether used alone or as part of a larger collection.

[0089] Computing device 900 should not be interpreted as having any dependency or requirement with any one or combination of the components / modules shown. The examples disclosed herein can be described in the general context of computer code or machine-usable instructions (including computer-executable instructions, such as program components), which are executed by computers or other machines (such as personal data assistants or other handheld devices). Generally, program components including routines, programs, objects, components, data structures, etc. refer to the code that performs specific tasks or realizes specific abstract data types. The examples disclosed herein can be put into practice in various system configurations, including personal computers, laptop computers, smart phones, mobile tablet computers, handheld devices, consumer electronics, special computing devices, etc. The examples disclosed herein can also be put into practice in a distributed computing environment when tasks are performed by a remote processing device connected through a communication network.

[0090] The computing device 900 includes a bus 910 that directly or indirectly couples the following devices: computer storage memory 912, one or more processors 914, one or more presentation components 916, input / output (I / O) ports 918, I / O components 920, a power supply 922, and a network component 924. Although the computing device 900 is depicted as appearing to be a single device, multiple computing devices 900 can work together and share the depicted device resources. For example, the memory 912 can be distributed across multiple devices, and the processor(s) 914 can be located with different devices.

[0091] Bus 910 represents one or more buses (such as an address bus, a data bus, or a combination thereof). Although for clarity, Figure 9 The various blocks are represented by lines, but alternative representations can be used to depict the various components. For example, in some examples, presentation components such as display devices are I / O components, and some examples of processors have their own memory. There is no distinction between categories such as "workstation", "server", "laptop", "handheld device", as they are all foreseen in Figure 9and within the scope of references to "computing devices" herein. Memory 912 may take the form of computer storage media as described below and is operable to provide storage for computer-readable instructions, data structures, program modules, and other data for computing device 900. In some examples, memory 912 stores one or more of an operating system, a general-purpose application platform, or other program modules and program data. Thus, memory 912 is capable of storing and accessing data 912a and instructions 912b, which are executable by processor 914 and configured to perform the various operations disclosed herein.

[0092] In some examples, memory 912 includes computer storage media. Memory 912 may include any number of memories associated with or accessible by computing device 900. Memory 912 may be located internal to computing device 900 (e.g., Figure 9 ), external to computing device 900 (not shown), or both (not shown). Additionally, or alternatively, memory 912 can be distributed across multiple computing devices 900, for example, in a virtualized environment where instruction processing is performed on multiple computing devices 900. For purposes of this disclosure, "computer storage media," "computer storage memory," "memory," and "memory device" are synonymous with memory 912, and none of these terms include carrier waves or propagated signaling.

[0093] (Multiple) processors 914 can include any number of processing units that read data from various entities (such as memory 912 or I / O components 920). Specifically, (multiple) processors 914 are programmed to execute computer-executable instructions for implementing various aspects of the present disclosure. These instructions can be executed by a processor, by multiple processors within the computing device 900, or by a processor external to the client computing device 900. In some examples, (multiple) processors 914 are programmed to execute instructions, such as the instructions in the flowcharts discussed below and depicted in the accompanying drawings. In addition, in some examples, (multiple) processors 914 represent implementations of analog technologies for performing the operations described herein. For example, these operations can be performed by an analog client computing device 900 and / or a digital client computing device 900. (Multiple) presentation components 916 present data indications to a user or other device. Exemplary presentation components include a display device, a speaker, a printing component, a vibration component, and the like. Those skilled in the art will understand and appreciate that computer data can be presented in a variety of ways, such as visually in a graphical user interface (GUI), audibly through a speaker, wirelessly between computing devices 900, across a wired connection, or otherwise. I / O ports 918 allow computing device 900 to be logically coupled to other devices, including I / O components 920, some of which may be built-in. Example I / O components 920 include, but are not limited to, microphones, joysticks, game controllers, satellite dishes, scanners, printers, wireless devices, and the like.

[0094] The computing device 900 can operate in a networked environment via a network component 924 that uses logical connections to one or more remote computers. In some examples, the network component 924 includes a network interface card and / or computer-executable instructions (e.g., a driver) for operating a network interface card. Communications between the computing device 900 and other devices can be performed over any wired or wireless connection using any protocol or mechanism. In some examples, the network component 924 is operable to communicate using a transport protocol over a public, private, or hybrid (public and private) network, or using short-range communication technologies such as near field communication (NFC), Bluetooth, or the like. TM Brand communication, etc.) or a combination thereof wirelessly transmits data between devices. Network component 924 communicates with remote resources 928 (e.g., cloud resources) across network 930 via wireless communication links 926 and / or wired communication links 926a. Various different examples of communication links 926 and 926a include wireless connections, wired connections, and / or dedicated links, and in some examples, at least a portion is routed through the Internet.

[0095] Although described in conjunction with the example computing device 900, the examples of the present disclosure can be implemented by many other general or special computing system environments, configurations or devices. Examples of well-known computing systems, environments and / or configurations that may be applicable to aspects of the present disclosure include, but are not limited to, smartphones, mobile tablets, mobile computing devices, personal computers, server computers, handheld or laptop devices, multiprocessor systems, game consoles, microprocessor-based systems, set-top boxes, programmable consumer electronics, mobile phones, mobile computing and / or communication devices in wearable or accessory form factors (e.g., watches, glasses, headphones or earphones), network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, virtual reality (VR) devices, augmented reality (AR) devices, mixed reality devices, holographic devices, etc. Such systems or devices can accept input from a user in any manner, including from an input device such as a keyboard or pointing device, via gesture input, proximity input (such as hovering) and / or via voice input.

[0096] Examples of the present disclosure can be described in the general context of computer-executable instructions (such as program modules), which are executed by one or more computers or other devices in the form of software, firmware, hardware or a combination thereof. Computer-executable instructions can be organized into one or more computer-executable components or modules. Typically, program modules include but are not limited to routines, programs, objects, components and data structures that perform specific tasks or implement specific abstract data types. Aspects of the present disclosure can be implemented using such components or modules of any number and organization. For example, aspects of the present disclosure are not limited to the specific computer-executable instructions or specific components or modules shown in the figures and described herein. Other examples of the present disclosure may include different computer-executable instructions or components with more or less functions than shown and described herein. In examples involving general-purpose computers, aspects of the present disclosure convert the general-purpose computer into a special-purpose computing device when it is configured to execute the instructions described herein.

[0097] As an example and not limitation, computer-readable media include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable memory implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, etc. Computer storage media are tangible and mutually exclusive with communication media. Computer storage media are implemented in hardware and do not include carrier waves and propagation signals. For the purposes of this disclosure, computer storage media are not the signal itself. Exemplary computer storage media include hard disks, flash drives, solid-state memories, phase change random access memories (PRAMs), static random access memories (SRAMs), dynamic random access memories (DRAMs), other types of random access memories (RAMs), read-only memories (ROMs), electrically erasable programmable read-only memories (EEPROMs), flash memories or other storage technologies, compact disc read-only memories (CD-ROMs), digital versatile discs (DVDs) or other optical storage, cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information for access by a computing device. In contrast, communication media typically embodies computer-readable instructions, data structures, program modules, etc. in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0098] The execution order of the operations in the examples of the present disclosure shown and described herein is not necessary and can be performed in different sequential ways in each example. For example, it is anticipated that running or performing a specific operation before, simultaneously with, or after another operation is within the scope of the present disclosure. When introducing the elements of aspects of the present disclosure or its examples, the words "one", "a", "the" and "said" are intended to mean that there are one or more of these elements. The terms "comprise", "comprising" and "having" are intended to be inclusive and mean that additional elements may also be present in addition to the elements listed. The term "exemplary" is intended to mean "an example of...". The phrase "one or more of the following: A, B and C" means "at least one of A and / or at least one of B and / or at least one of C".

[0099] Having described aspects of the present disclosure in detail, it will be apparent that modifications and variations may be made without departing from the scope of the present disclosure as defined in the appended claims. As various changes may be made in the above-described structures, products, and methods without departing from the scope of the present disclosure, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.

Claims

1. A system comprising: Processor (914); as well as A computer-readable medium (912) storing instructions (912b) that, when executed by the processor, are operable to: generating a summary (114) of the transcription (112) using the summary model (200); using a scoring model (300) to extract subject-specific data (116) from stored data (118); dynamically generating language model hints (122) using the topic-specific data and the summary; and Output text (130) is generated using the language model (124) and the language model prompts.

2. The system of claim 1 , wherein the instructions are further operable to: The output text is sent across a computer network as an electronic mail (email) message.

3. The system of claim 1, wherein the subject-specific data comprises selected enterprise suite data and selected customer relationship management (CRM) data, and wherein the summary comprises action items extracted from the transcription.

4. The system of claim 1 , wherein the instructions are further operable to: Automatic speech recognition (ASR) is used on the audio data to generate the transcription.

5. The system of claim 1, wherein the language model hint is based at least on a count of speakers identified in the summary and / or an interaction history associated with names identified in the summary.

6. The system of claim 1, wherein the topic-specific data includes calendar information, and the language model prompt is based at least on the calendar information.

7. The system of claim 1 , wherein the instructions are further operable to: Any discrepancies between the abstract and the subject-specific data shall prevail.

8. A computer-implemented method comprising: generating a summary (114) of the transcription (112) of the audio data using the summary model (200); using a scoring model (300) to extract subject-specific data (116) from stored data (118); dynamically generating language model hints (122) using the topic-specific data and the summary; and Output text (130) is generated using the language model (124) and the language model prompts.

9. The computer-implemented method of claim 8, further comprising: The output text is sent across a computer network as an electronic mail (email) message.

10. The computer-implemented method of claim 8, wherein the subject-specific data comprises select enterprise suite data and select customer relationship management (CRM) data, and wherein the summary comprises action items extracted from the transcription.

11. The computer-implemented method of claim 8, further comprising: Automatic speech recognition (ASR) is used on the audio data to generate the transcription.

12. The computer-implemented method of claim 8, wherein the language model hint is based at least on a count of speakers identified in the summary and / or an interaction history associated with names identified in the summary.

13. The computer-implemented method of claim 8, wherein the topic-specific data includes calendar information, and the language model prompt is based at least on the calendar information.

14. The computer-implemented method of claim 8, further comprising: Any discrepancies between the abstract and the subject-specific data shall prevail.

15. A computer storage device having computer-executable instructions stored thereon, the instructions, when executed by a computer, causing the computer to perform operations comprising: generating a summary (114) of the transcription (112) using the summary model (200); extracting topic-specific data (116) from stored data (118) using a scoring model (300), wherein the stored data includes graphical data; dynamically generating language model hints (122) using the topic-specific data and the summary; and Output text (130) is generated using the language model (124) and the language model prompts.

16. The computer storage device of claim 15, wherein the operations further comprise: The output text is sent across a computer network as an electronic mail (email) message.

17. The computer storage device of claim 15, wherein the subject-specific data comprises select enterprise suite data and select customer relationship management (CRM) data, and wherein the summary comprises action items extracted from the transcription.

18. The computer storage device of claim 15, wherein the operations further comprise: Automatic speech recognition (ASR) is used on the audio data to generate the transcription.

19. The computer storage device of claim 15, wherein the language model hint is based at least on a count of speakers identified in the summary and / or an interaction history associated with names identified in the summary, and wherein the topic-specific data includes calendar information, and the language model hint is based at least on the calendar information.

20. The computer storage device of claim 15, wherein the operations further comprise: Any discrepancies between the abstract and the subject-specific data shall prevail.

Images