Machine learning methods to determine likelihood for event to occur through sentiment analysis of digital conversations

Abstract

To provide a method, a storage medium, and a server system that predicts likelihood for an event to occur through sentiment analysis of digital conversations.SOLUTION: A computer implementation method comprises the steps of: accessing a trained learning machine; evaluating, using a machine learning model, the transcript to output a first sentiment score related to a first party in a unique domain; accessing digital engagement data representing engagement of the first party with digital assets associated with a second party; evaluating one or more sentiment score values and the digital engagement data to output a value indicative of a likelihood of the first party to take a particular action; and determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computer device associated with the second party.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit under 35 U.S.C. § 119 of Indian Patent Application No. 202211014184, filed on March 16, 2022, entitled "Machine Learning Methods to Determine a Likelihood for an Event to Occur through Sentiment Analysis of Digital Conversations" (Attorney Docket No. 089496.0188), the entire contents of which are incorporated herein by reference as if fully set forth herein for all purposes. Copyright Notice

[0002] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyrights. (c)2021-2022 Treasure Data, Inc.

[0003] One technical field of this disclosure is natural language processing, which includes machine analysis of transcripts of conversations. Another technical field is machine learning, which includes classifiers that predict the likelihood of an event occurring through sentiment analysis of digital conversations. [Background technology]

[0004] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued, and thus, unless expressly indicated otherwise, it should not be assumed that any approach described in this section qualifies as prior art by virtue of its inclusion in this section.

[0005] Companies have large databases of customer and potential customer data. In some fields, companies may have complex data related to the geographic distribution of customers and potential customers, the rules and regulations associated with each of their customers and potential customers, the habits of their customers or potential customers in using specific products of the company, and records of interactions between company representatives and their customers or potential customers. Examples of interactions may include transcripts of conversations or records of email exchanges. All of this data may be stored in tables organized by a database or in objects created by the company's servers. However, the scale of the data prevents companies from efficiently determining product supply chains to maximize the efficiency of their operations.

[0006] Therefore, an automated method is needed to examine this data as it updates and determine whether an event is likely to occur. For example, a company's representatives may have multiple meetings with multiple customers or potential customers throughout the day. The records of these interactions would be impossible for the company to categorize and respond to predictively. Therefore, an automated method is needed to analyze these records to predict events and respond accordingly in order to improve the operations of the company and its supply chain.

[0007] A particular challenge is emerging in the pharmaceutical industry, where face-to-face meetings between pharmaceutical sales teams and healthcare professionals (HCPs) have shifted to mostly digital online meetings over video platforms like Zoom and Microsoft Teams. For example, a long-term trend in the pharmaceutical industry over the past decade has been a shortening of face-to-face meetings between sales teams and healthcare professionals (HCPs), both in duration and frequency. Furthermore, such meetings have become even shorter under stressful conditions, such as pandemic situations, with the average length of such meetings dropping to 8–14 minutes during the COVID-19 pandemic. At the same time, the impact of digital meetings on prescribing of pharmaceutical compositions has become unpredictable and, in many cases, is reduced or ineffective compared to face-to-face meetings. Therefore, there is a dire need in the pharmaceutical industry to find ways to change prescribing behavior by analyzing digital interview artifacts (e.g., transcripts) and, based on sentiment analysis of these artifacts, predict prescribing behavior or determine what changes are needed in digital interviews. Summary of the Invention [Problem to be solved by the invention]

[0008] The present invention has been made to solve the problems in the prior art described above. [Means for solving the problem]

[0009] The appended claims serve as a summary of the invention.

[0010] The drawings are as follows: [Brief explanation of the drawings]

[0011] [Figure 1] 1 illustrates the use and major functional elements of a distributed computer system in which one embodiment may be implemented. [Figure 2] The system of FIG. 1 is illustrated with a focus on exploratory data analysis instructions and database tables in one embodiment. [Figure 3] 1 illustrates an example of a computer-implemented process or algorithm for generating metadata for database tables useful for exploratory data analysis. [Figure 4] 1 illustrates a computer system upon which one embodiment may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0012] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be understood that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.

[0013] The text of this disclosure, in combination with the drawings, is intended to set forth in prose the algorithms necessary to program a computer to implement the claimed invention, and is intended to be as detailed as one skilled in the art to which this disclosure pertains would communicate about each programming aspect, such as programmed functions, inputs, conversions, outputs, etc. That is, the level of detail described in this disclosure is comparable to the level of detail that one skilled in the art would normally use to describe the structure and function of a programmed algorithm or program to implement the invention claimed herein.

[0014] In the following sections, the embodiments will be described according to the following items: 1. Overall Overview 2. Overview of structure and function 2.1 Examples of Distributed System Architectures 2.2 Example of how to determine the likelihood of an event based on recorded data 2.3 Advantages and Improvements 3. Example - Hardware Overview *

[0015] 1. Overall Overview

[0016] Embodiments can enable data scientists, data engineers, and machine language team members to convert raw transcript data into sentiment scores and then use these sentiment scores, along with other engagement data, to predict and act on the likelihood of an action or event. One specific application is inferring a healthcare professional's propensity to prescribe a pharmaceutical composition after conducting a digital interview with a pharmaceutical company representative who manufactures and / or sells the pharmaceutical composition. Embodiments can be programmed to infer a sentiment score for a healthcare professional (HCP) based on machine analysis of the interview transcript data from the digital interview by analyzing the text to infer sentiment or sentiment. In some embodiments, the sentiment or sentiment is classified according to six innate human emotions (sadness, anger, contempt, disgust, surprise, and fear) as defined in the Ekman Taxonomy of Universal Emotions, as well as additional states such as agreement. In some embodiments, the sentiment score may represent the overall state of the interview sentiment on a spectrum from negative (e.g., unhappy or unlikely to take action) to positive (e.g., happy or likely to take action) (on a corresponding scale of -1 to 1). By combining additional digital engagement data related to the HCP's digital advertisements, website, or other computer-based communications with the sentiment score, it is possible to predict via a machine learning model the HCP's propensity to prescribe pharmaceutical compositions after each digital interview and / or whether that propensity is increasing or decreasing. In some embodiments, the additional engagement data may include data or digital artifacts (e.g., transcriptions) from previous interviews that are analyzed relative to and / or compared to the sentiment score to understand the overall sentiment of each party. In some embodiments, the output of the machine learning model may classify, predict, or output the next best recommendation to the sales team to take action (e.g., setting up a face-to-face interview).

[0017] In one embodiment, a computer-implemented method includes accessing a trained machine learning model, the machine learning model being trained on domain data specific to a specific domain, the machine learning model being trained to accept transcription data as input, predict or classify an emotional component of one or more portions of the transcription related to a first party, and output an emotional score, the emotional score representing a likelihood that the first party will take an action; establishing a programmatic connection between the first computer and a second computer; and, at the second computer, accessing the trained machine learning model using the programmatic connection. receiving a natural language transcription of a conversation between a first party and a second party from a computer system; evaluating the transcription using a machine learning model to output a first sentiment score associated with the first party in a unique domain; accessing digital engagement data representing the first party's engagement with a digital asset associated with the second party; evaluating the one or more sentiment score values ​​and the digital engagement data to output a value representing a likelihood that the first party will take a particular action; and determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computing device associated with the second party.

[0018] In some embodiments, the method also includes automatically submitting an order to a second computer if the value is above a threshold, the order specifying that a product associated with the action or event be shipped to the first party. The method also includes receiving, at the second computer from the first computer, a second natural language transcription of a second conversation between the first party and the second party.

[0019] In one embodiment, the method also includes using a machine learning model to evaluate the second natural language transcription to output a second emotion score related to the first party in a domain associated with the second party, automatically updating the value with the second emotion score, and determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computing device associated with the second party. Alternatively or additionally, the method may include building a trained machine learning model by selecting domain data from a database, selecting a machine learning type based on information about the first party and the second party, and training the selected machine learning type with the domain data to build the machine learning model.

[0020] In some embodiments, the domain data is selected based on at least a field associated with the second party, a geographic location of the first party, and a geographic location of the second party. Alternatively or additionally, the domain data may include words and phrases each having an associated flag indicating emotion data or classifications of those words and phrases. In various embodiments, the emotion data or classifications include information related to categories including sadness, anger, contempt, disgust, surprise, fear, and agreeableness.

[0021] Additionally, the method may include filtering the natural language transcription to exclude data relating to portions of the natural language transcription that represent what the second party is saying. Alternatively or additionally, the first party may be a healthcare professional, and the action may include the first party writing a prescription for a particular pharmaceutical composition.

[0022] In another embodiment, one or more non-transitory computer-readable storage media store one or more sequences of program instructions that, when executed by one or more processors, perform a programmatic analysis including: accessing a trained machine learning model, the machine learning model being trained on domain data specific to a specific domain, the machine learning model being trained to accept transcription data as input, predict or classify an emotional component of one or more portions of the transcription associated with a first party, and outputting an emotional score, the emotional score representing a likelihood that the first party will take an action; establishing a programmatic connection between the first computer and a second computer; and The computer is configured to cause one or more processors to perform the following steps: receive a natural language transcription of a conversation between a first party and a second party from a first computer using a programmatic connection; evaluate the transcription using a machine learning model to output a first emotion score associated with the first party in a unique domain; access digital engagement data representing the first party's engagement with a digital asset associated with the second party; evaluate the one or more emotion score values ​​and the digital engagement data to output a value representing a likelihood that the first party will take a particular action; and determine whether the value exceeds a threshold, and if so, automatically send a notification to a computing device associated with the second party.

[0023] In some embodiments, the storage medium may also include a sequence of program instructions that, when executed on the one or more processors, cause the one or more processors to perform the step of automatically submitting an order to a second computer if the value exceeds a threshold, the order configured to ship a product associated with the action or event to the first party.

[0024] In some embodiments, the storage medium may also include a sequence of program instructions that, when executed on the one or more processors, cause the one or more processors to perform the following steps: receive, at the second computer from the first computer, a second natural language transcription of a second conversation between the first party and the second party, or evaluate the second natural language transcription using a machine learning model to output a second emotion score related to the first party in a domain associated with the second party; automatically updating the value with the second emotion score; and determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computing device associated with the second party.

[0025] In some embodiments, the storage medium may also include a sequence of program instructions that, when executed on the one or more processors, cause the one or more processors to build a machine learning model by selecting domain data from a database, selecting a machine learning type based on information about the first party and the second party, and training the selected machine learning type with the domain data to build the machine learning model. In one embodiment, the domain data is selected based on at least a field associated with the second party, a geographic location of the first party, and a geographic location of the second party. Alternatively or additionally, the machine learning model may be domain-specific to the first party and the second party, and the emotion data or classification may include information related to categories including sadness, anger, contempt, disgust, surprise, fear, and agreeableness. In various embodiments, the action includes the first party purchasing or prescribing a particular product of the second party.

[0026] In another embodiment, a server system of a company may include a network interface, one or more processors coupled to the network interface, and one or more memory devices coupled to the one or more processors, the one or more memory devices including a database configured to store information about customers of the company, information about potential customers of the company, and information about a domain of the company. The one or more memory devices further include, when executed on the one or more processors, accessing a trained machine learning model, the machine learning model being trained on domain data specific to a specific domain, the machine learning model being trained to accept transcription data as input, predict or classify an emotional component of one or more portions of the transcription associated with a first party, and outputting an emotional score, the emotional score representing a likelihood that the first party will take an action; establishing a programmatic connection between the first computer and a second computer; and, at the second computer, transmitting, from the first computer, an emotional response between the first party and the second party using the programmatic connection. receiving a natural language transcription of a conversation between the first and second parties using a machine learning model to output a first sentiment score associated with the first party in a unique domain; accessing digital engagement data representing the first party's engagement with a digital asset associated with the second party; evaluating the one or more sentiment score values ​​and the digital engagement data to output a value representing the likelihood that the first party will take a particular action; and determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computing device associated with the second party.

[0027] The above-described embodiments, features, and aspects are examples of the subject matter of the present disclosure; other embodiments, features, and aspects will become apparent from other sections of the present disclosure.

[0028] 2. Overview of structure and function

[0029] 2.1 Examples of Distributed System Architectures

[0030] FIG. 1 illustrates a distributed computer system in which one embodiment may be implemented, showing its usage and major functional elements.

[0031] In one embodiment, computer system 100 includes components that are implemented at least in part by hardware (e.g., one or more hardware processors executing stored program instructions stored in one or more memories to perform the functions described herein) in one or more computing devices. In other words, all functions described herein represent acts that, in various embodiments, are performed by programming on a special purpose computer or a general purpose computer. FIG. 1 illustrates only one of many possible configurations of components that may be configured to carry out the programming described herein. Other configurations may include fewer or different components, and the division of labor between components may vary depending on the configuration.

[0032] 1 and the other drawings, as well as the description and claims in this disclosure, are all intended to present, disclose, and claim a technical system and method in which specially programmed computers with dedicated distributed computer system designs perform functions not previously available, providing a practical application of computer technology to the problem of developing, validating, and deploying machine learning models. As such, this disclosure presents a technical solution to a technical problem, and any interpretation that this disclosure or the claims encompass any judicial exceptions to patent eligibility (e.g., abstract ideas, mental processes, methods of organizing human activity, or mathematical algorithms) is unsupported by this disclosure and is incorrect.

[0033] In one embodiment, multiple user computers 102 a, 102 b, an administrator computer 105, a teleconferencing computing system 109, and a network 130 are communicatively coupled to an enterprise computing system 106. Each of the user computers 102 a, 102 b and the administrator computer 105 may be a desktop computer, laptop computer, tablet computer, smartphone, or other computing device, and may be directly coupled or indirectly coupled via one or more network links. The user computers 102 a, 102 b may be associated with end users who interact with programs provided by the teleconferencing computing system 109 and / or the enterprise computing system 106 to generate natural language transcription data and other engagement data as described herein. The administrator computer 105 may be associated with other end users responsible for configuring, operating, or managing the enterprise computing system 106.

[0034] In one embodiment, network 130 may be one or more local area networks, wide area networks, or internetworks using data links, either wired or wireless, terrestrial or satellite. In one embodiment, enterprise computing system 106 and teleconferencing computing system 109 include networked computers that are capable of dispatching communications with user computers 102 a, 102 b, or other entities, by call or command, as described in other sections herein.

[0035] The enterprise computing system 106 includes a record analysis engine 160, profile management instructions 161, a network interface 162, a customer database 163, a lead database 164, and / or an engagement data database 165. Commercially available examples of the enterprise computing system 106 include the TREASURE DATA Customer Data Platform (CDP) from Treasure Data, Inc. and Treasure Data KK. The network interface 162 is a device configured to enable the enterprise computing system 106 and associated components to communicate with other devices over a network. For example, the network interface 162 is coupled to the engines, instructions, and databases described with respect to the enterprise computing system 106 and communicatively couples the enterprise computing system to the network 130. Functionally, the network interface 162 provides a means for integrating the enterprise computing system 106 with other systems (e.g., the teleconferencing computing system 109).

[0036] Record analysis engine 160 includes a sequence of executable stored instructions organized as functional units, packages, and elements that are executed to perform the operations and steps described herein. In one embodiment, record analysis engine 160 includes machine learning models 170, record analysis instructions 171, build instructions 173, and / or domain data 172.

[0037] Machine learning model 170 may be, for example, a trained machine learning model built using build instructions 173 and accessible from enterprise computing system 170 to generate sentiment data associated with natural language text. For example, machine learning model 170 may be FLARE, described when it was written on the World Wide Web at the "flare" subdomain of the "readthedocs.io" domain, or the spaCy natural language processing library integrated with the PYTHON® environment. FLARE is suitable for use with Japanese transcription. Alternatively, machine learning model 170 may be any of the transformer-based machine learning models (e.g., BERT, RoBERTa, or ClinicalBERT models). Furthermore, machine learning model 170 may be specific to a company, to a company and a particular customer, entity, or other party, or to a particular product of a company and / or a particular party.

[0038] In some embodiments, the machine learning model 170 is unique because it is trained on domain data 172 that is specific to a product, company, and / or a particular party. For example, scientific, medical, or technical terms specific to different medical or pharmaceutical fields may be represented in a training dataset to train the machine learning model 170. Experiments have shown that general data, such as from Wikipedia or other general public sources, is not as effective as domain-specific training data, particularly data representing conversations in that domain. The training dataset may include labeled conversations in the same or different domains, and may use data from CDPs obtained from sources other than transcriptions. Furthermore, with a sufficient training dataset, a single machine learning model 170 can output highly accurate predictions that take into account all six innate human emotions defined in the Ekman taxonomy of universal emotions, as well as additional states such as agreeableness. In embodiments, the use of the Ekman taxonomy is not required; it is sufficient to simply classify the change in emotion compared to the emotion from the previous interview as a trend from negative to positive using a real number scale from -1 to +1. In some embodiments, machine learning models 170 may include one or more machine learning models. For example, machine learning models 170 may include a trained machine learning model and a machine learning classifier.

[0039] Recording analysis instructions 171 are instructions that, when executed, enable recording analysis engine 160 to analyze natural language transcriptions to generate transcription data that can be evaluated by machine learning model 170. For example, recording analysis instructions 171 may identify each person and / or associated party in the transcription. In some embodiments, recording analysis instructions 171 may filter out portions of the transcription, such as by excluding portions of the transcription associated with a company or other selected party's representative. In some embodiments, recording analysis instructions 171 may determine or select from memory a particular machine learning model 170 to use to accurately evaluate the transcription data. For example, recording analysis instructions 171 may identify users appearing in the transcription, identify parties, organizations, or companies associated with each of those users appearing in the transcription, and select a trained machine learning model 170 specific to those parties, organizations, companies, or specific users from a list of pre-built and stored machine learning models. In some embodiments, for example, if the recording analysis instructions 171 are unable to identify a trained machine learning model 170 that is suitable for a particular transcription, the recording analysis instructions 171 may request the model to be built from an external computing device or may cause build instructions 173 to be executed to build an appropriate machine learning model 170.

[0040] The build instructions 173, when executed by one or more processors, include instructions that cause the enterprise computing system 106 to build a particular machine learning model 170. The build instructions 173 include instructions for identifying and selecting a particular type of machine learning model, identifying and selecting domain data 172 for the particular machine learning model (which may be based on the transcription data and / or the parties or organizations within the transcription), and training the selected machine learning model using the selected domain data 172. For example, the build instructions 173 may identify and select a type of machine learning model from among those specified herein based on the transcription data, a first party associated with the transcription data, a second party associated with the transcription data, and / or both the first and second parties. In some embodiments, the build instructions 173 select a type of machine learning model from a list of machine learning models by referencing a lookup table indicating the best types of machine learning models for the first and / or second parties. The build instructions 173 may identify domain data 173 based on the transcription, the language of the transcription, the identity of the first party, the geographic location of the first party, the identity of the second party, the geographic location of the second party, and / or the identity of a particular user or person appearing in the transcription. For example, the build instructions 173 may identify information in the transcription according to one or more predetermined rules and use a lookup table to select data elements from the domain data 173 to be used to train or build a machine learning model 170 specific to the identified information. Examples of identifiable information include language, parties, and party locations. In some embodiments, the build instructions 173 fine-tune or train the model to build a trained model. For example, the build instructions 173 can fine-tune the model by selecting domain data 172 specific to the application for which the model is being deployed.In some embodiments, the application may be a medical pharmaceutical environment, where unique features (e.g., words, phrases, etc.) obtained from understanding conversations between pharmaceutical distributors and healthcare professionals, including selected domain data, are used to fine-tune or train a machine learning model, resulting in a model that is specific to the application for which the model is being deployed and focused on accuracy and efficiency. In some embodiments, the machine learning model may be built and stored as a trained machine learning model 173 for various modalities. For example, the machine learning model may be trained or fine-tuned with application-specific training for various digital conversation modalities, such as chatbots, real-time speech recognition, transcription data, etc.

[0041] Domain data 172 is a repository that includes multiple variables and associated flags. In some embodiments, domain data 172 may be structured as a data table. Each of the multiple variables may include a word or phrase and one or more associated scores. The one or more associated scores may be specific to each category of emotional or categorical data. For example, domain data 172 may include a first variable stored as a string containing the phrase "I like that," which may be associated with a happiness score, a sadness score, an agreeableness score, or any other category of emotional data determined to be relevant. In some embodiments, each variable in domain data 172 may be associated with or indicated as belonging to a particular language, a particular party, a particular geographic location, or the like, which build instructions can use to select the appropriate domain data 172 for a particular build. In some embodiments, domain data 172 is created or uploaded by an administrator. In some embodiments, domain data 172 is created or updated using an inductive learning machine model.

[0042] Customer database 163 and potential customer database 164 are data repositories that contain information about lists of customers and potential customers, respectively. For example, the databases may store table- or object-structured information that associates customers or potential customers with information related to them. For example, in some embodiments, the information may include geographic location, a list of employees or agents, language, a list of physical locations, history with the company, including past purchases, and / or samples previously sent to each customer and potential customer.

[0043] The engagement data database 165 is a data repository containing information about the history of interactions between specific parties and the company's digital assets. The specific parties may be customers or potential customers. For example, the engagement data may include document records, such as emails from or to each party; calendar records specifying users, accounts, or parties in meetings; information about interactions between the specific parties' computing systems and the company's computing systems; information about the length of time users from the specific parties spent on websites associated with the company; the number of times the specific parties requested information and the type of information provided; and so on. The engagement data allows the company to update information about each customer or potential customer based on the engagement of one or more users associated with the customer or potential customer with digital assets such as websites, webinars, and email lists. In some embodiments, the engagement data database 165 may include engagement data flagged for various digital conversation modalities. For example, some objects in the database may be specific to abbreviations or slang specific to a particular context and (for example) a chat box modality.

[0044] Profile management instructions 161, when executed by one or more processors, include instructions that cause enterprise computing system 106 to manage profiles associated with enterprises, customers, and / or potential customers. For example, profile management instructions 161 may receive or identify engagement data based on interactions with the enterprise's digital assets, associate the engagement data with particular customers or potential customers, and update engagement data database 165, customer database 163, or potential customer database 164 accordingly. In some embodiments, profile management instructions 161 may receive records, such as emails, associate the records with particular parties, and automatically update engagement data database 165 to include the records.

[0045] The teleconferencing computing system 109 includes a teleconferencing engine 190 that includes machine-readable instructions that, when executed on one or more processors of the teleconferencing computing system 109, enable the teleconferencing computing system 109 to communicatively couple a first user computing device of the user computing devices 102a to a second user computing device of the user computing devices 102b. For example, the teleconferencing engine 190 can connect the first user computing device to the second user computing device via the network 130. In some embodiments, the first user computing device 102a may display a web application, mobile application, or other type of computer application page that enables a first user on the first user computing device to virtually conference with a second user on a second user computing device. In some embodiments, the teleconferencing engine 190 is configured to enable multiple computing devices and users to participate in a virtual chat room. Examples of teleconferencing computer systems 109 include the commercial services ZOOM, MICROSOFT TEAMS, BLUEJEANS, GOOGLE MEET, and computers supporting functionally equivalent systems.

[0046] In some embodiments, the teleconference computing system 109 includes a transcription engine 191 that generates a natural language transcription of the conversations between users in a virtual conference. For example, the transcription engine 191 can identify portions of the conference associated with each user by monitoring noise data coming from each user computing device 102a, 102b, associating the noise data with each user computing device 102a, 102b (based on the credentials used to access the teleconference) and therefore with the user, and running machine learning algorithms on the noise to convert the noise into natural language text. The transcription engine 191 can create a record of the virtual teleconference by storing the natural language transcription (or portions thereof) of a particular virtual conference in a transcription database 192. In some embodiments, a transcription 193 is created for each virtual conference and stored in the transcription database 192. The transcription 193 may be created in real time as the noise data from the virtual meeting is received by the teleconferencing computing system 109, or may be created by feeding the audio recording data of the virtual meeting into the transcription engine 191 in a post-processing step.

[0047] The foregoing is a generalized, high-level description of the operation of enterprise computing system 106 in one embodiment. A complete description of all possible operations and uses of enterprise computing system 106 is beyond the scope of this disclosure and would obscure the focus of this disclosure.

[0048] 2.2 Example of how to determine the likelihood of an event based on recorded data

[0049] Figure 2 illustrates an example of a computer-implemented process or algorithm for determining the likelihood of an event based on recorded data. Figure 2 and the other flow diagrams herein are intended to illustrate at a functional level the interaction that one skilled in the art to which this disclosure pertains may engage in programming and implementing the algorithm. In one embodiment, the event is a specific action taken by a first party or an agent of the first party. In some embodiments, the specific action may be associated with a company's product or the first party's propensity to use or prescribe the product. In some embodiments, the product is a pharmaceutical product.

[0050] The flow diagrams are not intended to depict every instruction, method object, or substep that may be required to program every aspect of a working program, but are provided at a functional level illustration commonly used at a high level of skill in the art to convey a basis for developing a working program.

[0051] In the example of FIG. 2 , computer-implemented process 200 begins execution at block 201, where a trained machine learning model is accessed. It should be appreciated that while FIG. 2 is described with respect to transcriptions and transcription data, alternative embodiments may implement other digital conversation artifacts or modalities. For example, a second computer, such as enterprise computing system 106, may be programmed to select a trained machine learning model specific to a domain of which a particular transcription is a part. In some embodiments, the selected trained machine learning model is specific to a domain associated with the enterprise and / or its customers or potential customers. The trained machine learning model is trained to accept as input transcription data containing information about a first party and a second party, predict or classify the emotional content of one or more portions of the transcript associated with one party, and output an emotional score, where the emotional score represents the likelihood that the first party will take a particular action. In some embodiments, the trained machine learning model is selected from a database containing multiple trained machine learning models. In some embodiments, the trained machine learning model is selected based on the identities of the first party and the second party, or their representatives. In some embodiments, the trained machine learning model may be built, for example, as described with respect to FIG.

[0052] At block 202, a programmatic connection between the first computer and the second computer is established. For example, block 202 may be programmed to establish a connection between the teleconferencing computer system 109 and the enterprise computing system 106. In some embodiments, the process connection is made over a network, for example, using an application protocol interface (API), an application-specific protocol, or parameterized HTTP calls. The specific means of the programmatic connection is not critical, so long as the enterprise computing system 106 has a means to electronically request and receive data from the teleconferencing computer system 109.

[0053] In block 203, the second computer retrieves or receives a natural language transcription between the first and second parties. For example, the second computer retrieves or receives a natural language transcription between one or more representatives of a company and one or more representatives of another company. The transcription may be, for example, a digitally stored electronic transcription of an audiovisual or teleconference call between one or more HCP representatives and one or more representatives of a company that manufactures or distributes the pharmaceutical composition. The transcription may be generated automatically by the teleconference computing system 109 using speech-to-text technology and stored in an electronic digital format. In some embodiments, the second computer requests the natural language transcription via a specific program call function, which includes an identification of the requested transcription, an identification of the first party, and / or an identification of the second party. In some embodiments, the second computer may retrieve the transcription directly from a database on the first computer. In some embodiments, the natural language context includes additional metadata, such as usernames or identities of users appearing in the transcript, timestamps, duration of the virtual meeting, etc.

[0054] At block 204, the trained machine learning model is used to evaluate the transcription and output a first sentiment score associated with the first party in a domain associated with the second party. For example, the second computer may convert the transcription into discrete structured data elements and / or filter data from the transcription. In one example, the second computer may filter out all transcript data associated with the second party or a particular user. In another example, the second computer may parse the transcript data and break it down into structured data elements, each associated with the first party or the second party and including the parsed and identified data.

[0055] The transcript data is evaluated using a trained machine learning model, and the trained machine learning model outputs a first emotion score based on predicted or classified emotion components associated with the transcript data. For example, the predicted or classified emotion components may include scores for each emotion category, such as anger, sadness, agreeableness, contempt, disgust, surprise, and fear. The trained machine learning model may evaluate all the transcript data (e.g., via single-shot detection (SSD)) and automatically output a first emotion score representing the likelihood that the first party will perform a specific action. For example, in some embodiments, the specific action may be the purchase of a specific product associated with the second party, the tendency of the second party to prescribe, distribute, or recommend the specific product associated with the second party, or the tendency of the second party not to recommend the specific product.

[0056] In some embodiments, the trained machine learning model may output multiple sentiment scores. For example, in the context of prescribing pharmaceutical compositions, HCPs tend to be conservative, so a pharmaceutical manufacturer or distributor may need multiple interviews to provide the HCP with enough information to persuade the HCP to change their prescription-writing actions. For example, healthcare professionals may initially be conservative with new drugs because they value patient health and safety. Therefore, multiple interviews may be necessary to provide additional information, such as scientific findings, field clinical data, and recommendations or outcomes from other healthcare professionals, to address concerns or questions. Multiple sentiment scores corresponding to multiple transcriptions may indicate a tendency to write a desired prescription over time. In these embodiments, the input to the trained machine learning model in block 204 may include one or more sentiment score values ​​that were output after evaluation of one or more other transcriptions of one or more other interviews.

[0057] Additionally, in some embodiments, other data points from the CDP may be used as inputs to the evaluation of the machine learning model. For example, a predicted score value already calculated in the CDP, representing the propensity to purchase an item or service that applies to the same entity or the same pharmaceutical composition manufacturer or distributor, may be added as an input during the evaluation stage. In some embodiments, a "recency" value, representing the length of time since the interview occurred, as represented in the transcript may be used as an input. For example, more recent transcripts may be weighted more highly in the evaluation of the model.

[0058] At block 205, other digital engagement data representing the first party's engagement with the second party's digital asset is accessed. As described above, the engagement data includes records, statistics, and / or metadata regarding interactions in which the first party or an agent of the first party interacts with the second party's digital asset. Examples of digital assets may include contact center transcriptions, chat data or chat transcriptions, websites, webinars, or other online tools where conversations are captured. The engagement data may be programmatically obtained by calling the CDP. The engagement data may be collected over time and stored in a database where the engagement data or records are associated with the first party. The second computer may then select the digital engagement data by querying the database for the first party's identity. In some embodiments, the filtering or selection of engagement data may be based on more variables (e.g., the particular domain of the transcription, the particular language of the transcription, etc.) or may be based on the particular action being predictively analyzed.

[0059] At block 206, a machine learning classifier is used to evaluate the first sentiment score and the engagement data to output a value representing the likelihood that the first party will take a particular action. The machine learning classifier may be used, for example, by a second computer, to analyze the engagement data and the first sentiment score and output a value. For example, the machine learning classifier may analyze each piece of data to determine the likelihood that the first party will take an action. In some embodiments, the machine learning classifier may output a binary value of "yes" or "no." In some embodiments, the machine learning classifier may classify the sentiment score and the engagement data to generate a value.

[0060] At decision block 207, the value is compared to a predetermined threshold. If the value does not exceed the predetermined threshold, at block 209, the value may be stored in the first party's profile in a database. If the value exceeds the predetermined threshold, at block 208, a notification is sent to a user computing device associated with the second party. In some embodiments, the predetermined threshold may be set manually by an administrator or may be automatically updated based on rules associated with the second party. For example, the threshold may be adjusted based on the number of products associated with a particular action that the second party has in stock, the price of the particular product, the number of samples available, etc. The notification may be in the form of a push notification, an alarm, an email, a text message, etc. The notification may be configured to be sent to personnel who may take the particular action. In some embodiments, if the value exceeds the threshold, the second computer may automatically submit an order to the second computer, the order specifying that the product associated with the event be shipped to the first party.

[0061] The trend values, scores, or other outputs of the machine learning models generated as described above may be used in and / or integrated with other systems for several purposes. The trend values ​​may be displayed in a graphical user interface or dashboard directed to a user who is a representative of an entity interested in the results (e.g., a pharmaceutical company representative). The trend values ​​may be used to drive other actions. For example, a high trend value indicates a high likelihood of prescribing a pharmaceutical composition or taking other action, and the enterprise computing system 106 may be programmed to modify the sample item delivery plan or reschedule the sample item for delivery sooner. Conversely, a low trend value indicates a low likelihood of prescribing a pharmaceutical composition or taking other action, and the enterprise computing system 106 may be programmed to cancel the sample item delivery or cancel the schedule to delay the sample item for delivery. Priority or ordering values ​​may also be altered.

[0062] FIG. 3 illustrates an example of a computer-implemented process or algorithm for building or training a machine learning model using domain data. In the example of FIG. 3, a computer-implemented process 300 begins execution at block 301, where a domain is determined. In some embodiments, the domain may be selected based on information associated with the natural language transcription being analyzed. In some embodiments, the domain may be determined based on the identity of a first party, a second party, or a specific person appearing in the transcription. In some embodiments, the domain may be determined based on a specific action being analyzed. In some embodiments, the domain may be selected based on the geographic location of the parties, the language of the transcription, or the industry associated with the discussion in the transcription.

[0063] At block 302, domain data is selected based on the determined domain. For example, the domain data may be selected by querying a database of domain data to select all data elements specific to or associated with the determined domain. For example, the domain data may be selected based on common terminology in an industry (e.g., an industry related to either the first party or transcription). In some embodiments, the domain data may be selected based on the language used by the first party or an agent of the first party. In various embodiments, the domain data may be selected based on multiple variables such that the selected data is specific to the domain and / or specific action being analyzed. The domain data may include information including words, phrases, or audio data, including translation data and associated flags. For example, each object of the domain data may include a flag indicating an emotional category and / or emotional score associated with that information.

[0064] At block 303, a machine learning model or type is selected based on the domain and domain data. The machine learning model or type may be selected from a list of available engines stored in a database or accessible over a network. In some embodiments, the second computer analyzes the domain and domain data and selects the machine learning model or type by consulting a lookup table of the best machine learning model engines or types. In some embodiments, the selection of the machine learning model is based on the identity of the first party or the second party.

[0065] At block 304, a unique machine learning model is built using the domain data and the machine learning type. For example, the domain data may be used to train a selected machine learning model to output a unique trained machine learning model that can accept transcription data, predict or classify the emotional content of one or more portions of the transcription related to the first party, and output an emotional score. For example, flags in the domain data may train a machine learning model or type to identify an emotional state or emotional classification associated with particular audio data, words, or phrases, and aggregate and / or convolve multiple data elements of the transcription data to output a range of one or more emotional scores. In some embodiments, the emotional score may be "1" for high emotion, "-1" for low emotion, or "0" for neutral emotion.

[0066] 2.3 Advantages and Improvements

[0067] Embodiments of the present disclosure offer many advantages and improvements over conventional approaches. The disclosed approach is highly scalable compared to custom scripts or other manual programming approaches. Workflow behavior is easily parameterized and customized through various parameters in configuration files. Embodiments of a computer system allow companies to create or build their own models to determine the likelihood that another company or person will take a particular action. This possibility allows companies to better manage their resources and better direct products through more intelligent supply chain flows. Furthermore, the embodiments described herein improve a company's computer systems and technology environment by enabling companies to create or build their own trained models to convert large amounts of raw data into score data, which in turn is usable data for computing systems and administrators to implement new processes or take actions.

[0068] Additionally, past practice has relied on human intuition or heuristics based on memory or mood to determine HCPs' propensity to prescribe pharmaceutical compositions and / or determine the next best action to take after each digital encounter. Using embodiments of the present disclosure, a machine learning model can be used to evaluate evidence present in the interview transcript data and a data-driven sentiment score based on the HCP's digital engagement behavior to more objectively and accurately predict the propensity (whether increasing or decreasing) and / or next best action to be taken associated with a particular digital encounter.

[0069] 3. Example - Hardware Overview

[0070] According to one embodiment, the techniques described herein are implemented on at least one computing device. The techniques may be implemented using a combination of at least one server computer and / or other computing devices coupled, in whole or in part, over a network, such as a packet data network. The computing device may be hardwired to implement the techniques, or may include a digital electronic device, such as at least one application-specific integrated circuit (ASIC) or field-programmable gate array (FPGA), permanently programmed to implement the techniques, or may include at least one general-purpose hardware processor programmed to implement the techniques according to program instructions in firmware, memory, other storage, or a combination thereof. Such a computing device may also combine custom hardwired logic, ASIC, or FPGA with custom programming to accomplish the techniques described herein. A computing device may be a server computer, a workstation, a personal computer, a portable computer system, a handheld device, a mobile computing device, a wearable device, a body-worn or implantable device, a smartphone, a smart appliance, an internetworking device, an autonomous or semi-autonomous device (e.g., a robot or unmanned ground vehicle or unmanned aerial vehicle), or any other electronic device incorporating hardwired logic and / or program logic for implementing the techniques described herein, one or more virtual computing machines or instances in a data center, and / or a network of server computers and / or personal computers.

[0071] Figure 4 illustrates a computer system on which one embodiment can be implemented. The example of Figure 4 schematically depicts a computer system 800 and instructions for implementing the techniques of this disclosure in hardware, software, or a combination of hardware and software, using, for example, boxes and circles with a level of detail typically used by those skilled in the art to which this disclosure pertains when communicating computer architecture and computer system implementations.

[0072] Computer system 800 includes an input / output (I / O) subsystem 802, which may include buses and / or other communication mechanisms for communicating information and / or instructions between components of computer system 800 via electronic signal paths. I / O subsystem 802 may include an I / O controller, a memory controller, and at least one I / O port. Electronic signal paths are represented schematically in the drawings, for example, by lines, single-headed arrows, or double-headed arrows.

[0073] At least one hardware processor 804 is coupled to the I / O subsystem 802 for processing information and instructions. The hardware processor 804 may include, for example, a general-purpose microprocessor or microcontroller and / or a special-purpose microprocessor, such as an embedded system or a graphics processing unit (GPU) or a digital signal processor or an ARM processor. The processor 804 may include an integrated arithmetic logic unit (ALU) or may be coupled with a separate ALU.

[0074] Computer system 800 includes one or more memory units 806 (e.g., main memory), coupled to I / O subsystem 802, for electronically and digitally storing data and one or more sequences of instructions executed by processor 804. Memory 806 may include volatile memory, such as various forms of random access memory (RAM) or other dynamic storage devices. Memory 806 may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by processor 804. When such instructions are stored on a non-transitory computer-readable storage medium accessible by processor 804, computer system 800 may be a special-purpose machine customized to perform the operations specified in the instructions.

[0075] Computer system 800 further includes non-volatile memory, such as read-only memory (ROM) 808 or other static storage device coupled to I / O subsystem 802 for storing information and instructions for processor 804. ROM 808 may include various forms of programmable ROM (PROM), such as erasable programmable ROM (EPROM) or electrically erasable programmable ROM (EEPROM). Persistent storage 810 may include various forms of non-volatile RAM (NVRAM), such as flash memory (i.e., solid-state storage), magnetic disks, or optical disks (such as CD-ROM or DVD-ROM), and may be coupled to I / O subsystem 802 for storing information and instructions. Storage 810 is an example of one or more non-transitory computer-readable storage media that may be used to store one or more sequences of instructions and data that, when executed by processor 804, cause computer-implemented methods that perform the techniques described herein to be performed.

[0076] The instructions stored in memory 806, ROM 808, or storage device 810 may include one or more sets of instructions organized as modules, methods, objects, functions, routines, or calls. The instructions may be organized as one or more computer programs, operating system services, or application programs (including mobile applications). The instructions may include operating system and / or system software; one or more libraries supporting multimedia, programming, or other functionality; data protocol instructions or stacks implementing TCP / IP, HTTP, or other communications protocols; file formatting instructions for parsing or rendering files coded using HTML, XML, JPEG, MPEG, or PNG; user interface instructions for rendering or interpreting commands for a graphical user interface (GUI), a command line interface, or a text user interface; and application software (e.g., an office suite, an Internet access application, a design and manufacturing application, a graphics application, an audio application, a software engineering application, an educational application, a game, or miscellaneous applications). The instructions may implement a web server, a web application server, or a web client. The instructions may be organized as a presentation layer, an application layer, or a data storage layer (e.g., a relational database system with or without SQL, an object store, a graph database, a flat file system, or other data store).

[0077] Computer system 800 may be coupled to at least one output device 812 via I / O subsystem 802. In one embodiment, output device 812 is a digital computer display. Examples of displays that may be used in various embodiments include a touchscreen display, a light-emitting diode (LED) display, a liquid crystal display (LCD), or an electronic paper display. Computer system 800 may also include other types of output device(s) 812 as an alternative or in addition to a display device. Examples of other output device(s) 812 include a printer, a ticket printer, a plotter, a projector, a sound or video card, a speaker, a buzzer or piezoelectric device or other audible device, a lamp or LED or LCD indicator, a haptic device, an actuator, or a servo.

[0078] At least one input device 814 is coupled to the I / O subsystem 802 for communicating signals, data, command selections, or gestures to the processor 804. Examples of input device 814 include a touchscreen, a microphone, a still and video digital camera, alphanumeric and other keys, a keypad, a keyboard, a graphics tablet, an image scanner, a joystick, a clock, switches, buttons, dials, sliders, and / or various sensors (e.g., force sensors, motion sensors, thermal sensors, accelerometers, gyroscopes, and inertial measurement unit (IMU) sensors), and / or various transceivers (e.g., wireless (e.g., cellular or Wi-Fi, radio frequency (RF), or infrared (IR)) transceivers and global positioning system (GPS) transceivers).

[0079] Another type of input device is the control device 816, which may perform cursor control functions or other automated control functions (e.g., navigating a graphical interface on a display screen) instead of or in addition to input functions. The control device 816 may be a touchpad, mouse, trackball, or cursor direction keys for communicating directional information and command selections to the processor 804 and for controlling cursor movement on the display 812. The input device may have at least two degrees of freedom in two axes (a first axis (e.g., x-axis) and a second axis (e.g., y-axis)) that allow the device to specify a position on a plane. Another type of input device is a wired, wireless, or optical control device, such as a joystick, wand, console, steering wheel, pedals, gear shift mechanism, or other type of control device. The input device 814 may include a combination of different input devices (e.g., a video camera and a depth sensor).

[0080] In another embodiment, computer system 800 may include an Internet of Things (IoT) device, in which case one or more of output device(s) 812, input device(s) 814, and controller(s) 816 are omitted. Alternatively, in such an embodiment, input device(s) 814 may include one or more cameras, motion detectors, thermometers, microphones, earthquake detectors, other sensors or detectors, measurement devices, or encoders, and output device(s) 812 may include a dedicated display (e.g., an LED or LCD single-line display), one or more indicators, display panels, meters, valves, solenoids, actuators, or servos.

[0081] If computer system 800 is a mobile computing device, input device 814 may include a global positioning system (GPS) receiver coupled with a GPS module capable of triangulating with multiple GPS satellites to calculate and generate geolocation data (e.g., latitude and longitude values ​​for the geophysical location of computer system 800). Output device 812 may include hardware, software, firmware, and interfaces to generate location report packets, notifications, pulse or heartbeat signals, or other repetitive data transmissions to host 824 or server 830 that identify the location of computer system 800, alone or in combination with other application-specific data.

[0082] Computer system 800 may implement the techniques described herein using customized hardwired logic, at least one ASIC or FPGA, firmware, and / or program instructions or logic that, when loaded and used or executed in combination with the computer system, cause (i.e., program) the computer system to operate as a special-purpose machine. According to one embodiment, the techniques described herein are performed by computer system 800 in response to processor 804 executing at least one sequence of at least one instruction stored in main memory 806. Such instructions may be read into main memory 806 from another storage medium, such as storage device 810. Execution of the sequences of instructions stored in main memory 806 causes processor 804 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0083] The term "storage medium" as used herein refers to any non-transitory medium that stores data and / or instructions that cause a machine to operate in a specific fashion. Such storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks (such as storage device 810). Volatile media include dynamic memory (such as memory 806). Example forms of storage media include, for example, hard disks, solid-state drives, flash drives, magnetic data storage media, any optical or physical data storage media, memory chips, etc.

[0084] Storage media are distinct from but may be used in combination with transmission media. Transmission media involves transferring information between storage media. Transmission media include, for example, coaxial cables, copper wire, and fiber optics, including the wires that comprise a bus in I / O subsystem 802. Transmission media may also be in the form of acoustic or light waves, such as those generated during radio wave or infrared data communications.

[0085] Various forms of media may be involved in carrying at least one sequence of at least one instructions to processor 804 for execution. For example, the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions over a communications link (e.g., fiber optic or coaxial cable or a telephone line) using a modem. A modem or router local to computer system 800 may receive the data on the communications link and convert the data into a format readable by computer system 800. For example, a receiver such as a radio frequency antenna or infrared detector may receive the data carried in a radio or optical signal and appropriate circuitry may pass the data to I / O subsystem 802 (e.g., place the data on a bus). I / O subsystem 802 conveys the data to memory 806, from which processor 804 retrieves and executes the instructions. The instructions received by memory 806 may optionally be stored on storage device 810 either before or after execution by processor 804.

[0086] Computer system 800 also includes a communication interface 818 coupled to bus 802. Communication interface 818 provides a two-way data communication coupling to a network link 820, which is directly or indirectly connected to at least one communications network (e.g., network 822 or a public or private cloud over the Internet). For example, communication interface 818 may be an Ethernet networking interface, an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem providing a data communication connection to a corresponding type of communications line (e.g., an Ethernet cable or any type of metallic or fiber optic cable or telephone line). Network 822 broadly represents a local area network (LAN), a wide area network (WAN), a private network, an internetwork, or any combination thereof. Communication interface 818 may include a LAN card providing a data communication connection to a compatible LAN, or a cellular radiotelephone interface wired to transmit or receive cellular data in accordance with cellular radiotelephone wireless networking standards, or a satellite radio interface wired to transmit or receive digital data in accordance with satellite wireless networking standards. In any such implementation, communication interface 818 sends and receives electrical, electromagnetic or optical signals over signal paths that carry digital data streams representing various types of information.

[0087] Network link 820 typically provides electrical, electromagnetic, or optical data communication with other data devices, directly or through at least one network (e.g., using satellite, cellular, Wi-Fi, or Bluetooth technology). For example, network link 820 may provide a connection through network 822 to a host computer 824.

[0088] Further, network link 820 may provide connectivity through network 822 or to other computing devices through internetworking equipment and / or computers operated by an Internet Service Provider (ISP) 826. ISP 826 provides data communication services through a worldwide packet data communication network illustrated as Internet 828. A server computer 830 may be coupled to Internet 828. Server 830 broadly represents any computer, data center, virtual machine, or virtual computing instance with or without a hypervisor, or a computer running a containerized programming system such as DOCKER or KUBERNETES. Server 830 may represent an electronic digital service, implemented using two or more computers or instances, accessed and used by sending web service requests, uniform resource locator (URL) strings with parameters in an HTTP payload, API calls, application service calls, or other service calls. Computer system 800 and server 830 may form elements of a distributed computing system that may include other computers, processing clusters, server farms, or other organizations of computers that cooperate to perform tasks or run applications or services. Server 830 may include one or more sets of instructions organized as modules, methods, objects, functions, routines, or calls. The instructions may be organized as one or more computer programs, operating system services, or application programs (including mobile applications).The instructions may include operating system and / or system software, one or more libraries supporting multimedia, programming, or other functionality, data protocol instructions or stacks implementing TCP / IP, HTTP, or other communication protocols, file formatting instructions for parsing or rendering files coded using HTML, XML, JPEG, MPEG, or PNG, user interface instructions for rendering or interpreting commands for a graphical user interface (GUI), a command line interface, or a text user interface, and application software (e.g., an office suite, an Internet access application, a design and manufacturing application, a graphics application, an audio application, a software engineering application, an educational application, a game, or miscellaneous applications). Server 830 may include a web application server that hosts a presentation layer, an application layer, and a data storage layer (e.g., a relational database system with or without structured query language (SQL), an object store, a graph database, a flat file system, or other data storage).

[0089] Computer system 800 can send messages and receive data and instructions (including program code) through the network(s), network link 820 and communication interface 818. In the Internet example, a server 830 might transmit a requested code for an application program through Internet 828, ISP 826, local network 822 and communication interface 818. The received code may be executed by processor 804 as it is received, and / or stored in storage device 810, or other non-volatile storage for later execution.

[0090] Executing instructions as described in this section may result in the implementation of a process in the form of an instance of a computer program, which is running and consists of program code and its current activity. Depending on the operating system (OS), a process may consist of multiple threads of execution executing multiple instructions in parallel. In this context, a computer program may be a passive collection of instructions, while a process may be the actual execution of those instructions. Several processes may be associated with the same program. For example, opening several instances of the same program often means that two or more processes are running. Multitasking may be implemented to allow multiple processes to share the processor 804. Although each processor 804 or each core of a processor executes only one task at a time, the computer system 800 may be programmed to implement multitasking to allow each processor to switch between running tasks without having to wait for each task to finish. In one embodiment, switching may occur when a task performs an I / O operation, when a task indicates that it can switch, or when there is a hardware interrupt. To enable fast response of interactive user applications, time-sharing may be implemented by performing rapid context switching to make the parallel execution of multiple processes appear to occur simultaneously. In one embodiment, for security and reliability, the operating system may disable direct communication between independent processes, thereby providing tightly mediated and managed inter-process communication functionality.

[0091] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from embodiment to embodiment. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive. The sole and exclusive indication of the scope of the invention, and what Applicant intends to be the scope of the invention, is the literal and equivalent scope of the set of claims issued hereby, in the specific form in which such claims are issued, including any later amendments.

Claims

1. accessing a trained machine learning model, the machine learning model being trained on domain data specific to a specific domain, the machine learning model being trained to accept transcription data as input, predict or classify an emotional component of one or more portions of the transcription related to a first party, and output an emotional score, the emotional score representing a likelihood that the first party will take action; establishing a programmatic connection between a first computer and a second computer; receiving, at the second computer, from the first computer using the programmatic connection, a natural language transcription of the conversation between the first and second parties; evaluating the transcription using the machine learning model to output a first sentiment score associated with the first party in the distinctive domain; accessing digital engagement data representing the first party's engagement with a digital asset associated with the second party; evaluating the one or more sentiment score values ​​and the digital engagement data to output a value representing the likelihood that the first party will take a particular action; determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computing device associated with the second party; 20. A computer-implemented method comprising:

2. 2. The method of claim 1, further comprising automatically submitting an order to the second computer if the value exceeds the threshold, the order specifying that a product associated with the event be shipped to the first party.

3. receiving, at the second computer from the first computer, a second natural language transcription of a second conversation between a first party and a second party; using the machine learning model to evaluate the second natural language transcription to output a second sentiment score related to the first party in a domain associated with the second party; automatically updating the value with the second emotion score; determining whether the value exceeds the threshold, and if so, automatically sending a notification to a computing device associated with the second party; The method of claim 1 further comprising:

4. The method of claim 1, further comprising the step of building the machine learning model by selecting the domain data from a database, selecting a machine learning type based on information about the first party and the second party, and training the selected machine learning type with the domain data to build the machine learning model.

5. The method of claim 4, wherein the domain data is selected based on at least a field related to the second party, the geographic location of the first party, and the geographic location of the second party.

6. The method of claim 5, wherein the domain data includes words and phrases each having an associated flag, the flag indicating sentiment data or classification of the words and phrases.

7. The method of claim 1, further comprising a step of filtering the natural language transcription so as to exclude data relating to portions of the natural language transcription that represent what the second party is saying.

8. One or more non-transitory computer-readable storage media, the storage media storing one or more sequences of program instructions, the one or more sequences of program instructions, when executed by one or more processors, accessing a trained machine learning model, the machine learning model being trained on domain data specific to a specific domain, the machine learning model being trained to accept transcription data as input, predict or classify an emotional component of one or more portions of the transcription related to a first party, and output an emotional score, the emotional score representing a likelihood that the first party will take action; establishing a programmatic connection between a first computer and a second computer; receiving, at the second computer, from the first computer using the programmatic connection, a natural language transcription of the conversation between the first and second parties; evaluating the transcription using the machine learning model to output a first sentiment score associated with the first party in the distinctive domain; accessing digital engagement data representing the first party's engagement with a digital asset associated with the second party; evaluating the one or more sentiment score values ​​and the digital engagement data to output a value representing the likelihood that the first party will take a particular action; determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computing device associated with the second party; causing the one or more processors to execute The storage medium.

9. The storage medium of claim 8, further comprising a sequence of program instructions that, when executed on the one or more processors, cause the one or more processors to perform the step of automatically submitting an order to the second computer if the value exceeds the threshold, the order being configured to ship a product associated with the action to the first party.

10. When executed on said one or more processors, receiving, at the second computer from the first computer, a second natural language transcription of a second conversation between a first party and a second party; using the machine learning model to evaluate the second natural language transcription to output a second sentiment score related to the first party in a domain associated with the second party; automatically updating the value with the second emotion score; determining whether the value exceeds the threshold, and if so, automatically sending a notification to a computing device associated with the second party; and further comprising a sequence of program instructions for causing the one or more processors to execute: The storage medium according to claim 8.

11. The storage medium of claim 8, further comprising a sequence of program instructions that, when executed on the one or more processors, cause the one or more processors to perform the steps of building the machine learning model by selecting the domain data from a database, selecting a machine learning type based on information about the first party and the second party, and training the selected machine learning type with the domain data to build the machine learning model.

12. The storage medium described in claim 11, wherein the domain data is selected based on at least a field related to the second party, a geographic location of the first party, and a geographic location of the second party.

13. A storage medium as described in claim 8, wherein the machine learning model is specific to a domain specialized to the first party and the second party.

14. A corporate server system, comprising: A network interface; one or more processors coupled to the network interface; one or more memory devices coupled to the one or more processors, the one or more memory devices including a database configured to store information about customers of the company, information about potential customers of the company, and information about domains of the company; The server system, The one or more memory devices are further configured to store one or more sequences of program instructions, the one or more sequences of program instructions, when executed by one or more processors, accessing a trained machine learning model, the machine learning model being trained on domain data specific to a specific domain, the machine learning model being trained to accept transcription data as input, predict or classify an emotional component of one or more portions of the transcription related to a first party, and output an emotional score, the emotional score representing a likelihood that the first party will take action; establishing a programmatic connection between a first computer and a second computer; receiving, at the second computer, from the first computer using the programmatic connection, a natural language transcription of the conversation between the first and second parties; evaluating the transcription using the machine learning model to output a first sentiment score associated with the first party in the distinctive domain; accessing digital engagement data representing the first party's engagement with a digital asset associated with the second party; evaluating the one or more sentiment score values ​​and the digital engagement data to output a value representing the likelihood that the first party will take a particular action; determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computing device associated with the second party; causing the one or more processors to execute The server system.

Images