A machine learning method that uses sentiment analysis of digital conversations to determine the likelihood of an event occurring.

Abstract

To provide a method, a storage medium, and a server system that predicts likelihood for an event to occur through sentiment analysis of digital conversations.SOLUTION: A computer implementation method comprises the steps of: accessing a trained learning machine; evaluating, using a machine learning model, the transcript to output a first sentiment score related to a first party in a unique domain; accessing digital engagement data representing engagement of the first party with digital assets associated with a second party; evaluating one or more sentiment score values and the digital engagement data to output a value indicative of a likelihood of the first party to take a particular action; and determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computer device associated with the second party.SELECTED DRAWING: Figure 2

Description

TECHNICAL FIELD

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit under 35 U.S.C. § 119 of Indian Patent Application No. 202211014184, filed on March 16, 2022, entitled "Machine Learning Methods to Determine a Likelihood for an Event to Occur through Sentiment Analysis of Digital Conversations" (Attorney Docket No. 089496.0188), the entire contents of which are incorporated herein by reference for all purposes as if fully set forth in this specification. COPYRIGHT NOTICE

[0002] Portions of the disclosure of this patent document contain subject matter that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the patent and trademark office patent files or records, but otherwise reserves all copyright rights whatsoever. (c) 2021-2022 Treasure Data, Inc.

[0003] One technical field of the present disclosure is natural language processing, including machine analysis of conversation transcripts. Another technical field is machine learning, including classifiers that predict the likelihood of an event occurring through sentiment analysis of digital conversations.

BACKGROUND ART

[0004] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, none of the approaches described in this section should be considered as prior art merely by virtue of their inclusion in this section.

[0005] Companies possess large databases of customer and potential customer data. In some areas, companies may have complex data relating to the geographical distribution of customers and potential customers, the rules and regulations associated with each customer and potential customer, the habits of customers or potential customers who use the company's specific products, and records of interactions between the company's employees and customers or potential customers. Examples of interactions may include transcripts of conversations and records of email exchanges. All of this data may be stored in tables organized by the database, or in objects created on the company's servers. However, due to the sheer volume of data, companies are unable to efficiently determine the product supply chain in order to maximize the efficiency of their business operations.

[0006] Therefore, an automated method is needed to inspect and update this data to determine whether an event is likely to occur. For example, a company's representative may have multiple meetings with multiple customers or potential customers in a single day. It would be impossible for a company to categorize and predictively respond to the records of these interactions. Therefore, an automated method is needed to analyze these records, predict events, and respond accordingly in order to improve the operation of the company and its supply chain.

[0007] The pharmaceutical industry is facing a specific challenge. In this industry, meetings between pharmaceutical sales teams and healthcare professionals are shifting to almost entirely digital online meetings on video platforms such as Zoom and Microsoft Teams. For example, a long-term trend in the pharmaceutical industry over the past decade has shown that the duration and frequency of in-person meetings between sales teams and healthcare professionals (HCPs) have decreased. Furthermore, such meetings become even shorter under stressful conditions such as pandemics, with the average duration of such meetings dropping to 8-14 minutes during the COVID-19 pandemic. At the same time, the impact of digital meetings on drug formulation prescriptions has become unpredictable, often being lower or less effective than in-person meetings. Therefore, in the pharmaceutical sector, there is a pressing need to find ways to change prescribing behavior by analyzing digital meeting artifacts (such as transcripts) and predicting prescribing behavior or determining what changes are needed in digital meetings based on the emotional analysis of these artifacts. [Overview of the project] [Problems that the invention aims to solve]

[0008] This invention was made to solve the problems of the above-mentioned conventional technology. [Means for solving the problem]

[0009] The attached patent claims serve as an overview of the present invention.

[0010] The drawing is as follows. [Brief explanation of the drawing]

[0011] [Figure 1] This shows the usage and main functional elements of a distributed computer system capable of implementing one embodiment. [Figure 2] The system in Figure 1 is shown, focusing on the exploratory data analysis command and database table of one embodiment. [Figure 3] This document presents an example of a computer-based process or algorithm for generating metadata for database tables useful for exploratory data analysis. [Figure 4] This document describes a computer system capable of implementing one embodiment of the system. [Modes for carrying out the invention]

[0012] The following description includes various specific details for illustrative purposes to ensure that the present invention is fully understood. However, it goes without saying that the present invention can be implemented even without these specific details. Furthermore, well-known structures and devices are shown in block diagram form, for example, to prevent the present invention from becoming unnecessarily ambiguous.

[0013] The text of this disclosure, in conjunction with the drawings, is intended to provide a detailed, prose-like description of the algorithms necessary to program a computer to implement the claimed invention, with the same level of detail as a person skilled in the art would typically use to describe the programming aspects, such as the programmed functions, inputs, conversions, and outputs. In other words, the level of detail in the description in this disclosure is the same as the level of detail typically used by a person skilled in the art to describe the programmed algorithms, program structure, and functions for implementing the claimed invention.

[0014] The following sections will describe embodiments according to the following items. 1. Overall Overview 2. Overview of Structure and Function 2.1 Examples of Distributed System Architectures 2.2 Example of a method for determining the likelihood of an event based on recorded data 2.3 Advantages and Improvements 3. Examples - Hardware Overview *

[0015] 1. Overall Overview

[0016] The embodiments enable data scientists, data engineers, and members of machine language teams to convert raw transcript data into sentiment scores and then use these sentiment scores in conjunction with other engagement data to predict and act on the likelihood of actions or events. One specific application is for healthcare professionals to predict their tendency to prescribe a drug composition after conducting a digital interview with a representative of a pharmaceutical company that manufactures and / or sells the drug composition. The embodiments can be programmed to predict the sentiment score of a healthcare professional (HCP) based on machine analysis of the interview transcript data of the digital interview by analyzing the text to predict emotions or states of mind. In some embodiments, states of mind or emotions are classified according to the six innate human emotions (sadness, anger, contempt, disgust, surprise, and fear) as defined in the Ekman Taxonomy of Universal Emotions, as well as further states such as consent. In some embodiments, the sentiment score may represent the overall state of interview sentiment on a spectrum (corresponding to a scale from -1 to 1) ranging from negative (e.g., unhappy or unlikely to take action) to positive (e.g., happy or likely to take action). By linking additional digital engagement data related to the HCP's digital advertising, website, or other computer-based communications with the sentiment score, it is possible to predict, via a machine learning model, the HCP's tendency to prescribe a drug composition after each digital interview, and / or whether that tendency is strengthening or weakening. In some embodiments, the additional engagement data may include data from previous interviews or digital artifacts (e.g., transcripts) which are analyzed relative to and / or compared to the sentiment score. This is to understand the overall sentiment of each party. In some embodiments, the output of the machine learning model may classify, predict, or output the next best recommendation for a sales team taking action (e.g., setting up an in-person interview).

[0017] In one embodiment, a computer implementation includes the steps of accessing a trained machine learning model, the machine learning model being trained on domain data specific to a specific domain, the machine learning model being trained to accept transcript data as input, predict or classify the sentiment component of one or more parts of the transcript relating to a first party, and output a sentiment score, the sentiment score representing the likelihood of the first party taking action; establishing a program connection between a first computer and a second computer; and on the second computer, using the program connection, accessing the first computer. The process includes: receiving a natural language transcript of a conversation between a first party and a second party from a data source; evaluating the transcript using a machine learning model to output a first sentiment score related to the first party in a specific domain; accessing digital engagement data representing the first party's engagement with digital assets associated with the second party; evaluating one or more sentiment score values ​​and digital engagement data to output a value representing the likelihood that the first party will take a specific action; and determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computer device associated with the second party.

[0018] In some embodiments, this method also applies when the value exceeds a threshold. to, the 2 computers by means to the computer device associated with the second party The method includes a step of automatically submitting an order, wherein the order specifies that the products associated with the action or event should be shipped to the first party. The method also includes a step of receiving a second natural language transcript of a second conversation between the first party and the second party from the first computer on a second computer.

[0019] In one embodiment, the method further comprises the steps of: evaluating a second natural language transcription using a machine learning model to output a second sentiment score associated with the first party in a domain associated with a second party; automatically updating a value with the second sentiment score; determining whether the value exceeds a threshold, and in response to the value exceeding the threshold, automatically transmitting a notification to a computer device associated with the second party. Alternatively or additionally, the method may comprise the step of building a trained machine learning model by selecting domain data from a database, selecting a machine learning type based on information about the first party and the second party, and training the selected machine learning type with the domain data to build the machine learning model.

[0020] In some embodiments, the domain data is selected based at least on a field associated with the second party, a geographic location of the first party, and a geographic location of the second party. Alternatively or additionally, the domain data may include words and phrases each having an associated flag, where the flags indicate sentiment data or classification of those words and phrases. In various embodiments, the sentiment data or classification includes information related to categories including sadness, anger, contempt, disgust, surprise, fear, and agreement.

[0021] Further, the method may comprise the step of filtering the natural language transcription such that data related to portions representing speech by the second party is excluded from the natural language transcription. Alternatively or additionally, the first party may be a healthcare worker, and the action includes the first party writing a prescription for a specific pharmaceutical composition.

[0022] In another embodiment, one or more non-temporary computer-readable storage media store one or more sequences of program instructions, which, when executed by one or more processors, provide access to a trained machine learning model, the machine learning model being trained on domain data specific to a particular domain, the machine learning model being trained to accept transcribed data as input, predict or classify the sentiment components of one or more portions of the transcribed data relating to a first party, and output a sentiment score, the sentiment score representing the likelihood of the first party taking action; and establishing a program connection between a first computer and a second computer. In a computer, the computer has one or more processors perform the following steps: receiving a natural language transcript of a conversation between a first party and a second party from a first computer using a program connection; evaluating the transcript using a machine learning model and outputting a first sentiment score related to the first party in a specific domain; accessing digital engagement data representing the first party's engagement with digital assets associated with the second party; evaluating one or more sentiment score values ​​and digital engagement data and outputting a value representing the likelihood that the first party will take a specific action; and determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computer device associated with the second party.

[0023] In some embodiments, the storage medium is also executed by one or more processors if the value exceeds a threshold. to the computer device associated with the second party A step of automatically submitting an order, which may include a sequence of program instructions causing one or more processors to perform a step of automatically submitting an order, wherein the order is configured to ship the product associated with that action or event to a first party.

[0024] In some embodiments, the storage medium may also include a sequence of program instructions that, when executed on one or more of its processors, cause one or more of its processors to perform the following steps: receiving a second natural language transcript of a second conversation between a first party and a second party from a first computer in a second computer; evaluating the second natural language transcript using a machine learning model and outputting a second sentiment score related to the first party in a domain associated with the second party; automatically updating the value with the second sentiment score; determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computer device associated with the second party.

[0025] In some embodiments, the storage medium may also include a sequence of program instructions that, when executed by one or more processors, cause one or more processors to perform the steps of building a machine learning model by selecting domain data from a database, selecting machine learning types based on information about the first and second parties, and training the selected machine learning types with the domain data to build a machine learning model. In one embodiment, the domain data is selected based on at least a field related to the second party, the geographical location of the first party, and the geographical location of the second party. Alternatively or additionally, the machine learning model may be specific to domains dedicated to the first and second parties, and the sentiment data or classification may include information related to categories including sadness, anger, contempt, disgust, surprise, fear, and agreement. In various embodiments, the action includes the first party purchasing or prescribing a specific product of the second party.

[0026] In another embodiment, the enterprise server system may include a network interface, one or more processors coupled to the network interface, and one or more memory devices coupled to the one or more processors, the memory devices including a database configured to store information about the enterprise's customers, information about the enterprise's potential customers, and information about the enterprise's domain. The one or more memory devices may further include the step of accessing a trained machine learning model, which, when executed by one or more processors, is trained on domain data specific to a particular domain, and is trained to accept transcribed data as input, predict or classify sentiment components of one or more portions of the transcribed data relating to a first party, and output a sentiment score, the sentiment score representing the likelihood that the first party will take action; the step of establishing a program connection between a first computer and a second computer; and in the second computer, using the program connection, accessing the first computer from the first computer to access the first party and the second party. The system is configured to store one or more sequences of program instructions that cause one or more of its processors to execute the following steps: receiving a natural language transcript of a conversation between the parties; evaluating the transcript using a machine learning model and outputting a first sentiment score related to the first party in a specific domain; accessing digital engagement data representing the first party's engagement with digital assets associated with the second party; evaluating one or more sentiment score values ​​and digital engagement data and outputting a value representing the likelihood that the first party will take a specific action; and determining whether the value exceeds a threshold, and if so, automatically sending a notification to a computer device associated with the second party.

[0027] The embodiments, features, and aspects described above are examples of those covered by this disclosure, and other embodiments, features, and aspects will be apparent from other sections of this disclosure.

[0028] 2. Overview of Structure and Function

[0029] 2.1 Examples of Distributed System Architectures

[0030] Figure 1 shows a distributed computer system that can implement one embodiment, illustrating its usage and main functional elements.

[0031] In one embodiment, the computer system 100 includes components that are at least partially implemented by hardware in one or more computing devices (for example, one or more hardware processors that execute stored program instructions stored in one or more memories in order to perform the functions described herein). In other words, all functions described herein represent operations that are performed by programming on a dedicated computer or a general-purpose computer in various embodiments. Figure 1 shows only one of many possible component configurations configured to perform the programming described herein. Other configurations may include fewer or different components, and the division of labor between components may vary depending on the configuration.

[0032] Figure 1, and other drawings, as well as all descriptions and claims in this disclosure, are intended to present, disclose, and claim a technical system and technical method that provides a practical application of computer technology to the problems of developing, validating, and deploying machine learning models by specially programmed computers through a dedicated distributed computer system design that performs functions not previously available. Thus, this disclosure presents a technical solution to a technical problem, and any interpretation that this disclosure or the claims encompass any case law exclusions for patent eligibility (e.g., abstract ideas, mental processes, methods of organizing human activity, or mathematical algorithms) is unsupported by this disclosure and is incorrect.

[0033] In one embodiment, a plurality of user computers 102a, 102b, an administrator computer 105, a remote conferencing computing system 109, and a network 130 are communicatively connected to a corporate computing system 106. Each of the user computers 102a, 102b, and the administrator computer 105 is a desktop computer, laptop computer, tablet computer, smartphone, or other computing device, which may be directly connected or indirectly connected via one or more network links. User computers 102 may be associated with an end user who interacts with programs provided by the remote conferencing computing system 109 and / or the corporate computing system 106 to generate natural language transcription data and other engagement data as described herein. The administrator computer 105 may be associated with another end user responsible for configuring, operating, or managing the corporate computer system 106.

[0034] In one embodiment, the network 130 may be one or more local area networks, wide area networks, or internetworks using wired or wireless, terrestrial or satellite data links. In one embodiment, the enterprise computing system 106 and the teleconferencing computing system 109 include networked computers that can dispatch communications with user computers 102a, 102b, or other entities by call or command as described in other sections of this specification.

[0035] The enterprise computing system 106 includes a record analysis engine 160, a profile management command 161, a network interface 162, a customer database 163, a potential customer database 164, and / or an engagement data database 165. A commercially available example of the enterprise computing system 106 is the TREASURE DATA Customer Data Platform (CDP) from Treasure Data, Inc. and Treasure Data KK. The network interface 162 is a device configured to enable the enterprise computing system 106 and its associated components to communicate with other devices over a network. For example, the network interface 162 is coupled with the engine, command, and database described with respect to the enterprise computing system 106, and connects the enterprise computing system to network 130 in a communicative manner. Functionally, the network interface 162 provides a means for integrating the enterprise computing system 106 with other systems (e.g., a teleconferencing computing system 109).

[0036] The recording analysis engine 160 includes a sequence of executable storage instructions organized as functional units, packages, and elements, which are executed to perform the operations and steps described herein. In one embodiment, the recording analysis engine 160 includes a machine learning model 170, a recording analysis instruction 171, a build instruction 173, and / or domain data 172.

[0037] The machine learning model 170 may be, for example, a trained machine learning model built using build instruction 173, and accessible from the enterprise computing system 170 for generating sentiment data related to natural language text. For example, the machine learning model 170 may be the FLARE or the spaCy natural language processing library integrated with the PYTHON® environment, as described when it was written to the "flare" subdomain of the "readthedocs.io" domain on the World Wide Web. FLARE is suitable for use with Japanese transcription. Alternatively, the machine learning model may be any of the transformer-based machine learning models (e.g., BERT, RoBERTa model, or ClinicalBERT model). Furthermore, the machine learning model 170 may be specific to an enterprise, specific to an enterprise and a specific customer, entity, or other party, or specific to an enterprise's specific product and / or specific party.

[0038] In some embodiments, the machine learning model 170 is unique because it is trained on domain data 172 specific to a product, company, and / or specific party. For example, the machine learning model 170 may be trained on a training dataset that represents scientific, medical, or technical terminology specific to different fields of medicine or pharmaceuticals. Experiments have shown that general data from Wikipedia or other common public sources is not as effective as domain-specific training data (especially data representing conversations in that domain). The training dataset may include labeled conversations in the same or different domains and may use data from CDPs obtained from sources other than transcription. Furthermore, with sufficient training datasets, a single machine learning model 170 can output highly accurate predictions that take into account all six innate human emotions as defined in the Ekman classification of universal sentiment, as well as additional states such as agreeability. In embodiments, the use of the Ekman classification is not mandatory; the change in sentiment compared to the sentiment of the previous interview may simply be classified as a trend from negative to positive using a real number scale of -1 to +1. In some embodiments, the machine learning model 170 may include one or more machine learning models. For example, the machine learning model 170 may include a trained machine learning model and a machine learning classifier.

[0039] The recording analysis command 171, when executed, is a command that enables the recording analysis engine 160 to analyze the natural language transcript and generate transcript data that can be evaluated by a machine learning model 170. For example, the recording analysis command 171 may identify each person and / or related party in the transcript. In some embodiments, the recording analysis command 171 may filter out parts of the transcript, which is done, for example, by excluding parts of the transcript that are associated with a company or an agent of another selected party. In some embodiments, the recording analysis command 171 may determine or select from memory a specific machine learning model 170 to be used to accurately evaluate the transcript data. For example, the recording analysis command 171 may identify users appearing in the transcript, identify the parties, organizations, or companies associated with each of those users appearing in the transcript, and select a pre-trained machine learning model 170 specific to those parties, organizations, companies, or specific users from a list of pre-built and stored machine learning models. In some embodiments, for example, if the recording analysis command 171 cannot identify a trained machine learning model 170 suitable for a particular transcription, the recording analysis command 171 may request an external computing device to build a model to be built in order to build an appropriate machine learning model 170, or may cause a build command 173 to be executed.

[0040] The build instruction 173, when executed by one or more processors, includes instructions to cause a specific machine learning model 170 to be built in the enterprise computing system 106. The build instruction 173 includes instructions to identify and select a specific type of machine learning model, identify and select domain data 172 for that specific machine learning model (which may be based on transcribed data and / or parties or organizations in the transcribed data), and train the selected machine learning model using the selected domain data 172. For example, the build instruction 173 may identify and select a type of machine learning model from among those specified herein based on transcribed data, a first party associated with the transcribed data, a second party associated with the transcribed data, and / or both the first and second parties. In some embodiments, the build instruction 173 selects a type of machine learning model from a list of machine learning models by referring to a lookup table that shows the best types of machine learning models for the first and / or second parties. The build instruction 173 may identify domain data 173 based on the transcript, the language of the transcript, the identification information of a first party, the geographical location of a first party, the identification information of a second party, the geographical location of a second party, and / or the identification information of a specific user or person appearing in the transcript. For example, the build instruction 173 may identify information in the transcript according to one or more predetermined rules and use a lookup table to select data elements from the domain data 173 that will be used to train or build a machine learning model 170 specific to the identified information. Examples of identifiable information include language, parties, and the locations of the parties. In some embodiments, the build instruction 173 builds a trained model by fine-tuning or training the model. For example, the build instruction 173 may fine-tune the model by selecting domain data 172 specific to the application in which the model is being deployed.In some embodiments, the application may be a medical pharmaceutical environment, where unique features (e.g., words, phrases, etc.) derived from understanding conversations between pharmaceutical distributors and healthcare professionals, along with selected domain data, are used to fine-tune or train a machine learning model, thereby obtaining a unique model focused on accuracy and efficiency for the application in which the model is being deployed. In some embodiments, the machine learning model may be built and stored as a machine learning model 173 trained on a variety of modalities. For example, the machine learning model may be trained or fine-tuned by application-specific training on various digital conversation modalities such as chatbots, real-time speech recognition, and transcription data.

[0041] Domain data 172 is a repository containing multiple variables and associated flags. In some embodiments, domain data 172 may be structured as a data table. Multiple variables may each contain a word or phrase and one or more associated scores. One or more associated scores may be specific to each category of sentiment data or classification data. For example, domain data 172 may contain a first variable stored as a string containing the phrase "I like that," which may be associated with a happiness score, sadness score, agreement score, or any other applicable category of sentiment data. In some embodiments, each variable of domain data 172 may be associated with, or indicated as belonging to, a specific language, a specific party, a specific geographical location, or the like, which can be used by build instructions to select the appropriate domain data 172 for a particular build. In some embodiments, domain data 172 is created or uploaded by an administrator. In some embodiments, domain data 172 is created or updated using an inductive learning machine model.

[0042] The customer database 163 and the potential customer database 164 are data repositories that contain information about lists of customers and potential customers, respectively. For example, the databases may store information in a tabular or object structure that associates customers or potential customers with information related to them. For example, in some embodiments, the information may include geographical location, a list of staff or agents, language, a list of physical locations, a history with the company including past purchases, and / or samples previously sent to each of the customers and potential customers.

[0043] The engagement data database 165 is a data repository containing information about the history of interactions between a specific party and the company's digital assets. The specific party may be a customer or a potential customer. For example, the engagement data may include documentary records, such as emails from or to each party, calendar records specifying users, accounts, or parties in meetings, information about interactions between a specific party's computing system and the company's computing system, information from a specific party about the length of time a user stayed on a website associated with the company, the number of times a specific party requested information, and the type of information provided, and so on. The engagement data enables the company to update information about each customer or potential customer based on their engagement with digital assets such as websites, webinars, and email lists. In some embodiments, the engagement data database 165 may include engagement data flagged with respect to various digital conversation modalities. For example, some objects in the database may be specific to abbreviations or slang specific to a particular context and (e.g.) a chatbox modality.

[0044] The profile management instruction 161, when executed by one or more processors, includes instructions that cause the enterprise computing system 106 to manage profiles associated with enterprises, customers, and / or potential customers. For example, the profile management instruction 161 can receive or identify engagement data based on interactions with the enterprise's digital assets, associate this engagement data with a specific customer or potential customer, and update the engagement data database 165, customer database 163, or potential customer database 164 accordingly. In some embodiments, the profile management instruction 161 can receive records such as emails, associate those records with a specific party, and automatically update the engagement data database 165 to include those records.

[0045] The remote conferencing computing system 109 includes a remote conferencing engine 190, which, when executed on one or more processors of the remote conferencing computing system 109, includes machine-readable instructions that enable the remote conferencing computing system 109 to communicatively couple a first user computing device of user computing device 102a to a second user computing device of user computing device 102b. For example, the remote conferencing engine 190 can connect the first user computing device to the second user computing device via a network 130. In some embodiments, the first user computing device 102a may display a web application, mobile application, or other type of computer application page that enables a first user on the first user computing device to virtually confer with a second user on the second user computing device. In some embodiments, the remote conferencing engine 190 is configured to enable multiple computing devices and users to join a virtual chat room. Examples of remote conferencing computer systems 109 include computers that support commercial services such as ZOOM, MICROSOFT TEAMS, BLUEJEANS, GOOGLE MEET, and functionally equivalent systems.

[0046] In some embodiments, the remote conferencing computing system 109 includes a transcription engine 191 that transcribes conversations between users in a virtual conference into natural language. For example, the transcription engine 191 is capable of identifying portions of a conference associated with each user, which is done by monitoring noise data coming from each user computing device 102a, 102b, associating the noise data with each user computing device 102a, 102b, and therefore with the user (based on the credentials used to access the remote conference), and by applying a machine learning algorithm to the noise to convert it into natural language text. The transcription engine 191 is capable of creating a record of the virtual remote conference, which is done by storing the natural language transcript (or a portion thereof) of a particular virtual conference in a transcription database 192. In some embodiments, a transcript 193 is created for each virtual conference and stored in the transcription database 192. The transcript 193 may be created in real time when the remote conferencing computing system 109 receives noise data from the virtual conference, or it may be created in a post-processing step by feeding the audio recording data of the virtual conference to the transcription engine 191.

[0047] The above description is a generalized and broad overview of the operation of the enterprise computing system 106 in one embodiment. A complete description of all possible operations and uses of the enterprise computing system 106 would be beyond the scope of this disclosure and would obscure the focus of this disclosure.

[0048] 2.2 Example of a method for determining the likelihood of an event based on recorded data

[0049] Figure 2 shows an example of a computer-implemented process or algorithm for determining the likelihood of an event based on recorded data. Figure 2 and each other flowchart in this specification illustrate the functional level of interaction that persons skilled in the art in the field relating to this disclosure would have when programming and implementing an algorithm. In one embodiment, an event is a specific action taken by a first party or an agent of a first party. In some embodiments, the specific action may be associated with a company's product or the first party's tendency to use or prescribe that product. In some embodiments, the product is a pharmaceutical product.

[0050] Flowcharts are not intended to show all instructions, method objects, or substeps that might be necessary to program all aspects of a working program, but rather are provided as examples of functional levels commonly used at a high level of technical expertise in order to convey the basis for developing a working program.

[0051] In the example in Figure 2, the computer execution process 200 begins execution from block 201, where it accesses a trained machine learning model. Naturally, the explanation in Figure 2 concerns transcription and transcribed data, but in alternative embodiments, other artifacts or modalities of digital conversation may be implemented. For example, a second computer, such as an enterprise computing system 106, may be programmed to select a trained machine learning model specific to the domain to which a particular transcription is part. In some embodiments, the selected trained machine learning model is specific to a domain associated with an enterprise and / or an enterprise's customers or potential customers. The trained machine learning model is trained to accept transcribed data containing information about a first and second party as input, predict or classify the sentiment component of one or more portions of the transcription related to one of the parties, and output a sentiment score, which represents the likelihood that the first party will take a particular action. In some embodiments, the trained machine learning model is selected from a database containing multiple trained machine learning models. In some embodiments, the trained machine learning model is selected based on the identification information of the first and second parties, or their representatives. In some embodiments, the trained machine learning model may be built as described with respect to Figure 3, for example.

[0052] In block 202, a program connection is established between the first computer and the second computer. For example, block 202 may be programmed to establish a connection between the remote conferencing computer system 109 and the enterprise computing system 106. In some embodiments, the process connection is made over a network, for example, using an application protocol interface (API), a purpose-specific protocol, or a parameterized HTTP call. The specific means of the program connection are not critical, as long as the enterprise computing system 106 has means to electronically request and receive data from the remote conferencing computer system 109.

[0053] In block 203, the second computer retrieves or receives natural language transcripts between the first and second parties. For example, the second computer retrieves or receives natural language transcripts between one or more agents of a company and one or more agents of another company. The transcripts may be, for example, digitally stored electronic transcripts of audiovisual calls or teleconferencing calls between one or more representatives of an HCP and one or more representatives of a pharmaceutical composition manufacturer or distributor. The transcripts may be generated naturally and automatically by the teleconferencing computing system 109 using speech-to-text technology and stored in electronic digital format. In some embodiments, the second computer requests a natural language transcript via a specific program call function, which includes identification information for the requested transcript, identification information for the first parties, and / or distinguished names for the second parties. In some embodiments, the second computer may retrieve the transcript directly from the database of the first computer. In some embodiments, the natural language context includes additional metadata, such as the username or identification information of users appearing in the transcript, a timestamp, and the duration of the virtual meeting.

[0054] In block 204, the trained machine learning model is used to evaluate the transcript and output a first sentiment score related to the first party in domains associated with the second party. For example, the second computer may convert the transcript into discrete structured data elements and / or filtered data from the transcript. In one example, the second computer may filter out all transcript data associated with the second party or a specific user. In another example, the second computer may analyze the transcript data and break down the information into structured data elements, each associated with the first or second party and containing analyzed and identified data.

[0055] The transcribed data is evaluated using a trained machine learning model, which outputs a first sentiment score based on the predicted or classified sentiment components associated with the transcribed data. For example, the predicted or classified sentiment components may include scores for each sentiment category, such as anger, sadness, agreement, contempt, disgust, surprise, and fear. The trained machine learning model may evaluate all the transcribed data (e.g., by single-shot detection (SSD)) and automatically output a first sentiment score representing the likelihood that the first party will take a specific action. For example, in some embodiments, the specific action may be the purchase of a specific product associated with the second party, the tendency of the second party to prescribe, distribute, or recommend a specific product associated with the second party, or the tendency of the second party not to recommend that specific product.

[0056] In some embodiments, the trained machine learning model may output multiple sentiment scores. For example, in the context of prescribing pharmaceutical compositions, HCPs tend to be conservative, so multiple meetings may be required for the pharmaceutical manufacturer or distributor to provide the HCP with enough information to change their prescribing actions. For example, healthcare professionals may be initially conservative with new drugs because they prioritize patient health and safety. Therefore, multiple meetings may be required to explain scientific findings, clinical data from the field, and recommendations or results from other healthcare professionals to address their concerns and questions. Multiple sentiment scores corresponding to multiple transcripts may indicate that the tendency to write the desired prescription emerges over time. In these embodiments, the input to the trained machine learning model in block 204 may include one or more sentiment score values ​​output after the evaluation of one or more other transcripts from one or more other meetings.

[0057] Furthermore, in some embodiments, other data points from the CDP may be used as input to the evaluation of the machine learning model. For example, a predictive score value already calculated in the CDP, representing the tendency to purchase a certain item or service for the same entity or the same pharmaceutical composition manufacturer or distributor, may be added as input during the evaluation stage. In some embodiments, a "recency" value, representing the length of time since the interview took place, which appears in the transcribed data, may be used as input. For example, more recent transcribed data may be given a higher weight in the evaluation of the model.

[0058] In Block 205, other digital engagement data representing the first party's engagement with the second party's digital assets is accessed. As described above, the engagement data includes records, statistics, and / or metadata relating to interactions in which the first party or its agent interacts with the second party's digital assets. Examples of digital assets may include contact center transcripts, chat data or chat transcripts, websites, webinars, or other online tools in which conversations are captured. The engagement data may be retrieved programmatically by calling the CDP. The engagement data may be collected over time and stored in a database in which the engagement data or records are associated with the first party. The second computer may then select the digital engagement data by querying the database for the identification of the first party. In some embodiments, the filtering or selection of engagement data may be based on a number of variables (e.g., specific domains of transcription, specific languages ​​of transcription, etc.) or on specific actions that are being predictively analyzed.

[0059] In block 206, a machine learning classifier is used to evaluate the first sentiment score and engagement data, and outputs a value representing the likelihood that the first party will take a specific action. The machine learning classifier is used, for example, by a second computer to analyze the engagement data and the first sentiment score and output a value. For example, the machine learning classifier may analyze each piece of data to determine the likelihood that the first party will take an action. In some embodiments, the machine learning classifier may output a binary value of "yes" or "no". In some embodiments, the machine learning classifier may classify the sentiment score and also classify the engagement data to generate a value.

[0060] In decision block 207, the value is compared to a predetermined threshold. If the value does not exceed the predetermined threshold, in block 209, the value may be stored in the first party's profile in the database. If the value exceeds the predetermined threshold, in block 208, a notification is sent to the user computing device associated with the second party. In some embodiments, the predetermined threshold may be manually set by an administrator or automatically updated based on rules associated with the second party. For example, the threshold may be adjusted based on the number of products associated with a particular action that the second party has in stock, the price of a particular product, the number of available samples, etc. The notification may be in the form of a push notification, alarm, email, text message, etc. The notification is configured to be sent to an employee who may take a particular action. In some embodiments, if the value exceeds the threshold, the second computer Computer device associated with the second party Orders may be submitted automatically, and the orders specify that the products associated with that event should be shipped to the first party.

[0061] The trend values, score values, or other outputs of the machine learning model generated as described above may be used in and / or integrated with other systems for several purposes. The trend values ​​may be displayed in a graphical user interface or dashboard directed to a user who is a representative of an entity interested in the results (e.g., a pharmaceutical company representative). The trend values ​​may be used to facilitate other actions. For example, a high trend value indicates a high probability of prescribing a pharmaceutical composition or taking other actions, in which case the enterprise computing system 106 may be programmed to change the sample item delivery plan or to change the schedule to expedite the delivery of the sample items. Conversely, a low trend value indicates a low probability of prescribing a pharmaceutical composition or taking other actions, in which case the enterprise computing system 106 may be programmed to cancel the delivery of the sample items or to cancel the schedule to delay the delivery of the sample items. Priority or ordering values ​​can also be changed.

[0062] Figure 3 shows an example of a computer execution process or algorithm for building or training a machine learning model using domain data. In the example in Figure 3, the computer execution process 300 starts execution from block 301, where the domain is determined. In some embodiments, the domain may be selected based on information associated with the natural language transcript being analyzed. In some embodiments, the domain may be determined based on the identification of a first party, a second party, or a specific person appearing in the transcript. In some embodiments, the domain may be determined based on a specific action being analyzed. In some embodiments, the domain may be selected based on the geographical location of the parties, the language of the transcript, or the industry associated with the discussion in the transcript.

[0063] In block 302, domain data is selected based on the determined domain. For example, the selection of domain data may be performed by querying a database of domain data to select all data elements specific to or associated with the determined domain. For example, the selection of domain data may be performed based on common terminology in an industry (e.g., an industry related to either parties or transcription). In some embodiments, the selection of domain data may be performed based on the language used by the first party or the agent of the first party. In various embodiments, the selection of domain data may be performed based on multiple variables such that the selected data is specific to the domain being analyzed and / or a particular action. The domain data may include words, phrases, or audio data and may include information including translated data and associated flags. For example, each object of domain data may include flags indicating a sentiment category and / or sentiment score associated with that information.

[0064] In block 303, a machine learning model or type is selected based on the domain and domain data. The machine learning model or type may be selected from a list of available engines stored in a database or accessible over a network. In some embodiments, a second computer analyzes the domain and domain data and selects a machine learning model or type by referring to a lookup table of the best machine learning model engine or type. In some embodiments, the selection of a machine learning model is based on the identification of the first or second party.

[0065] In block 304, a unique machine learning model is built using the domain data and machine learning type. For example, the domain data may be used to train a selected machine learning model to output a unique pre-trained machine learning model that accepts transcript data, predicts or classifies the sentiment component of one or more parts of the transcript related to a first party, and outputs a sentiment score. For example, the domain data flags may be used to train a machine learning model or type to identify sentiment states or sentiment classifications associated with specific audio data, words, or phrases, and to aggregate and / or convolve multiple data elements of the transcript data to output one or more sentiment scores within a range. In some embodiments, the sentiment score may be "1" for high sentiment, "-1" for low sentiment, and "0" for neutral sentiment.

[0066] 2.3 Advantages and Improvements

[0067] Embodiments of this disclosure offer many advantages and improvements over conventional approaches. The methods of this disclosure are highly scalable compared to custom scripts or other manual programming methods. The operation of the workflow is easily parameterized and customizable through various parameters in a configuration file. Embodiments of the computer system enable an enterprise to create or build a unique model to determine the likelihood that another enterprise or individual will take a specific action. This likelihood enables the enterprise to better manage its resources and better direct its products through a more intelligent supply chain flow. Furthermore, embodiments described herein improve an enterprise's computer systems and technology environment by enabling an enterprise to create or build a unique trained model to transform large amounts of raw data into score data, and further into data that can be used by computing systems and administrators to implement new processes or take actions.

[0068] Furthermore, in past practices, human intuition or heuristics based on memory and mood have been used to determine the tendencies of HCPs when prescribing drug compositions and / or deciding the best next action to take after each digital interview. By using embodiments of the present disclosure, it is possible to more objectively and accurately predict tendencies (whether to strengthen or weaken) and / or the best next action to take associated with a particular digital interview by evaluating the evidence revealed in interview transcript data and data-driven sentiment scores based on the HCP's digital engagement behavior using a machine learning model.

[0069] 3. Examples - Hardware Overview

[0070] According to one embodiment, the technique described herein is implemented on at least one computing device. The technique may be implemented using a combination of at least one server computer and / or other computing devices, all or in part, connected by a network such as a packet data network. The computing device may include a digital electronic device, such as at least one application-specific integrated circuit (ASIC) or field-programmable gate array (FPGA), that is hardwired or permanently programmed to implement the technique, or at least one general-purpose hardware processor programmed to implement the technique according to program instructions in firmware, memory, other storage devices, or a combination thereof. Such a computing device may also combine custom hardwired logic, ASICs, or FPGAs with custom programming to achieve the technique described herein. Computing devices may be server computers, workstations, personal computers, portable computer systems, handheld devices, mobile computing devices, wearable devices, devices attached to or implantable in the body, smartphones, smart appliances, internetworking devices, autonomous or semi-autonomous devices (e.g., robots or unmanned ground vehicles or unmanned aerial vehicles), or any other electronic devices incorporating hardwired logic and / or programmable logic for implementing the techniques described herein, one or more virtual computing machines or instances in a data center, and / or a network of server computers and / or personal computers.

[0071] Figure 4 shows a computer system capable of implementing one embodiment. The example in Figure 4 schematically represents a computer system 800 and instructions for implementing the technology of the present disclosure in the form of hardware, software, or a combination of hardware and software, and is represented with the same level of detail as commonly used by those skilled in the art in the relevant field when communicating about computer architectures and embodiments of computer systems.

[0072] The computer system 800 includes an input / output (I / O) subsystem 802, which may include buses and / or other communication mechanisms for transmitting information and / or instructions between components of the computer system 800 via electronic signal paths. The I / O subsystem 802 may include an I / O controller, a memory controller, and at least one I / O port. The electronic signal paths are schematically represented in the drawings, for example, by lines, one-way arrows, or two-way arrows.

[0073] To process information and instructions, at least one hardware processor 804 is coupled with the I / O subsystem 802. The hardware processor 804 may include, for example, a general-purpose microprocessor or microcontroller, and / or a dedicated microprocessor, for example, an embedded system or graphics processing unit (GPU), or a digital signal processor or an ARM processor. The processor 804 may include an integrated logic unit (ALU), or may be coupled with an independent ALU.

[0074] The computer system 800 includes one or more memory units 806 (e.g., main memory), which are coupled with an I / O subsystem 802 for electronically digitally storing data and one or more instruction sequences executed by the processor 804. Memory 806 may include volatile memory, such as various forms of random-access memory (RAM) or other dynamic storage devices. Memory 806 may also be used to store temporary variables or other intermediate information during the execution of instructions by the processor 804. If such instructions are stored in a non-temporary computer-readable storage medium accessible from the processor 804, the computer system 800 can be made a dedicated machine customized to perform the operations specified in the instructions.

[0075] The computer system 800 further includes non-volatile memory, for example, read-only memory (ROM) 808 or other static storage devices coupled to the I / O subsystem 802 for storing information and instructions for the processor 804. The ROM 808 may include various forms of programmable ROM (PROM), for example, erasable PROM (EPROM) or electrically erasable PROM (EEPROM). The persistent storage device 810 may include various forms of non-volatile RAM (NVRAM), for example, flash memory (i.e., solid-state memory), magnetic disks, or optical disks (such as CD-ROMs or DVD-ROMs), and may be coupled to the I / O subsystem 802 for storing information and instructions. The storage device 810 is an example of one or more non-temporary computer-readable storage media that may be used to store one or more instruction sequences and data so that, when executed by the processor 804, a computer implementation method performing the techniques described herein is carried out.

[0076] Instructions stored in memory 806, ROM 808, or storage device 810 may include one or more instruction sets organized as modules, methods, objects, functions, routines, or calls. Instructions may be organized as one or more computer programs, operating system services, or application programs (including mobile applications). Instructions may include an operating system and / or system software, one or more libraries supporting multimedia, programming, or other functions, data protocol instructions or stacks implementing TCP / IP, HTTP, or other communication protocols, file format processing instructions for parsing or representing files coated using HTML, XML, JPEG, MPEG, or PNG, user interface instructions for representing or interpreting commands relating to a graphical user interface (GUI), command-line interface, or text user interface, and application software (e.g., office suites, internet access applications, design and manufacturing applications, graphics applications, audio applications, software engineering applications, educational applications, games, or miscellaneous applications). Instructions may implement a web server, web application server, or web client. Instructions may be organized as a presentation layer, application layer, or data storage layer (e.g., a relational database system, object store, graph database, flat file system, or other data storage, with or without SQL).

[0077] The computer system 800 may be coupled to at least one output device 812 via an I / O subsystem 802. In one embodiment, the output device 812 is a digital computer display. Examples of displays that can be used in various embodiments include a touchscreen display or a light-emitting diode (LED) display or a liquid crystal display (LCD) or an electronic paper display. The computer system 800 may also include other types of output devices 812 as alternatives or additions to the display device. Examples of other output devices 812 include a printer, a ticket printer, a plotter, a projector, a sound card or video card, a speaker, a buzzer or piezoelectric device or other audible device, a lamp or LED or LCD indicator, a haptic device, an actuator, or a servo.

[0078] At least one input device 814 is coupled to the I / O subsystem 802 to transmit signals, data, command selections, or gestures to the processor 804. Examples of input devices 814 include touchscreens, microphones, still and video digital cameras, alphanumeric and other keys, keypads, keyboards, graphics tablets, image scanners, joysticks, clocks, switches, buttons, dials, sliders, and / or various sensors (e.g., force sensors, motion sensors, thermal sensors, accelerometers, gyroscopes, and inertial measuring unit (IMU) sensors), and / or various transceivers (e.g., wireless (e.g., cellular or Wi-Fi, radio frequency (RF), or infrared (IR)) transceivers and Global Positioning System (GPS) transceivers).

[0079] Another type of input device is the control device 816, which may perform cursor control or other automatic control functions (e.g., navigation of a graphical interface on a display screen) as an alternative to or addition to the input function. The control device 816 may be a touchpad, mouse, trackball, or cursor directional keys for transmitting directional information and command selection to the processor 804 and for controlling the movement of a cursor on the display 812. The input device may have at least 2 degrees of freedom on two axes (a first axis (e.g., x-axis) and a second axis (e.g., y-axis)) that allow the device to specify a position on a plane. Another type of input device is a wired control device, a wireless control device, or an optical control device, e.g., a joystick, wand, console, steering wheel, pedals, gear shift mechanism, or other type of control device. The input device 814 may include a combination of several different input devices (e.g., a video camera and a depth sensor).

[0080] In another embodiment, the computer system 800 may include an Internet of Things (IoT) device, in which case one or more of the output device 812, input device 814, and control device 816 may be omitted. Alternatively, in such an embodiment, the input device 814 may be one or more cameras, motion detectors, thermometers, microphones, seismic detectors, other sensors or detectors, measuring devices or encoders, and the output device 812 may be a dedicated display (e.g., a single-line LED or LCD display), one or more indicators, a display panel, a meter, a valve, a solenoid, an actuator, or a servo.

[0081] If the computer system 800 is a mobile computing device, the input device 814 may include a GPS receiver coupled with a Global Positioning System (GPS) module, the GPS module capable of generating geolocation data (e.g., latitude and longitude values ​​of the geophysical position of the computer system 800) by calculation in a triangle with multiple GPS satellites. The output device 812 may include hardware, software, firmware, and interfaces for generating location reporting packets, notifications, pulse signals or heartbeat signals, or other repetitive data transmissions that identify the position of the computer system 800, either alone or in combination with other application-specific data, to the host 824 or server 830.

[0082] The computer system 800 can implement the techniques described herein using customized hardwired logic, at least one ASIC or FPGA, firmware and / or program instructions or logic, which, when loaded and used or executed in combination with the computer system, cause the computer system to operate as a dedicated machine (i.e., program the computer system to do so). According to one embodiment, the techniques described herein are implemented by the computer system 800 in response to the execution of at least one sequence of at least one instruction stored in main memory 806 by the processor 804. Such instructions can be read into main memory 806 from another storage medium, such as a storage device 810. Once the sequence of instructions stored in main memory 806 is executed, the processor 804 implements the process steps described herein. In alternative embodiments, hardwired circuits may be used instead of, or in combination with, software instructions.

[0083] The term "storage medium" as used herein means any non-temporary medium that stores data and / or instructions for operating a machine in a particular manner. Such storage mediums may include non-volatile and / or volatile media. Examples of non-volatile media include optical disks or magnetic disks (e.g., storage device 810). Examples of volatile media include hard disks, solid-state drives, flash drives, magnetic data storage media, any optical or physical data storage media, memory chips, etc.

[0084] A storage medium is different from a transmission medium, but may be used in combination with a transmission medium. The transmission medium is involved in transferring information between storage mediums. Examples of transmission media include coaxial cables, copper wires, and optical fibers, which include wires containing the bus of the I / O subsystem 802. The transmission medium may also be in the form of sound waves or light waves, for example, sound waves or light waves generated during radio wave or infrared data communication.

[0085] Various forms of media may be involved in transporting at least one sequence of at least one instruction to processor 804 for execution. For example, the instruction may first be held on a magnetic disk or solid-state drive of a remote computer. The remote computer may load the instruction into its dynamic memory and transmit the instruction via a modem-based communication link (e.g., optical fiber, coaxial cable, or telephone line). A modem or router local to computer system 800 can receive data on the communication link and convert the data into a format readable by computer system 800. For example, a receiver such as a radio frequency antenna or infrared detector can receive data transported by radio or optical signals, and the appropriate circuit can pass the data to I / O subsystem 802 (e.g., put the data on a bus). I / O subsystem 802 transports the data to memory 806, and processor 804 retrieves the instruction from memory 806 and executes it. The instruction received by memory 806 may optionally be stored in storage device 810 before or after execution by processor 804.

[0086] The computer system 800 also includes a communication interface 818 coupled to bus 802. The communication interface 818 provides a two-way data communication coupling with network link 820, which is directly or indirectly connected to at least one communication network (e.g., network 822, or a public or private cloud on the Internet). For example, the communication interface 818 may be an Ethernet networking interface, an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem that provides data communication connectivity to a corresponding type of communication line (e.g., an Ethernet cable or any type of metal cable or fiber optic line or telephone line). Network 822 broadly represents a local area network (LAN), a wide area network (WAN), a private branch network, an internetwork, or any combination thereof. The communication interface 818 may include a LAN card that provides data communication connectivity to a compatible LAN, or a cellular radiophone interface wired to transmit or receive cellular data in accordance with cellular radiophone radio networking standards, or a satellite radio interface wired to transmit or receive digital data in accordance with satellite radio networking standards. In any such embodiment, the communication interface 818 transmits and receives electrical, electromagnetic, or optical signals on a signal path that carries digital data streams representing various types of information.

[0087] Network link 820 typically provides electrical, electromagnetic, or optical data communication with other data devices directly or via at least one network (e.g., using satellite technology, cellular technology, Wi-Fi technology, or Bluetooth technology). For example, network link 820 can provide a connection to host computer 824 via network 822.

[0088] Furthermore, network link 820 can provide connections through network 822, or connections to other computer devices via internetworking equipment and / or computers operated by an Internet Service Provider (ISP) 826. ISP 826 provides data communication services through a worldwide packet data communication network, designated as the Internet 828. Server computer 830 may be coupled to the Internet 828. Server 830 broadly represents any computer, data center, virtual machine, or virtual computing instance with or without a hypervisor, or a computer running a containerized programming system such as Docker or Kubernetes. Server 830 may represent an electronic digital service, which is implemented using two or more computers or instances and accessed and used by sending web service requests, uniform resource locator (URL) strings with parameters in an HTTP payload, API calls, application service calls, or other service calls. Computer system 800 and server 830 may form elements of a distributed computing system, which includes other computers, processing clusters, server farms, or other computer formations that cooperate to perform tasks or run applications or services. Server 830 may include one or more instruction sets organized as modules, methods, objects, functions, routines, or calls. Instructions may be organized as one or more computer programs, operating system services, or application programs (including mobile applications).The instructions may include an operating system and / or system software, one or more libraries supporting multimedia, programming, or other functions, data protocol instructions or stacks implementing TCP / IP, HTTP, or other communication protocols, file format processing instructions for parsing or representing files coated using HTML, XML, JPEG, MPEG, or PNG, user interface instructions for representing or interpreting commands relating to a graphical user interface (GUI), command-line interface, or text user interface, and application software (e.g., office suites, internet access applications, design and manufacturing applications, graphics applications, audio applications, software engineering applications, educational applications, games, or a variety of other applications). Server 830 may include a web application server hosting a presentation layer, an application layer, and a data storage layer (e.g., a relational database system, object store, graph database, flat file system, or other data storage, with or without structured query language (SQL)).

[0089] The computer system 800 is capable of sending messages and receiving data and instructions (including program code) through a network, network link 820, and communication interface 818. In the example of the internet, server 830 can send requested code for an application program through the internet 828, ISP 826, local network 822, and communication interface 818. The received code may be executed by processor 804 upon receipt and / or stored in storage device 810 or other non-volatile storage device for later execution.

[0090] By executing instructions as described in this section, it is possible to implement a process in the form of an instance of a computer program, which is running and consists of program code and its current activity. Depending on the operating system (OS), a process may consist of multiple execution threads that execute multiple instructions in parallel. In this context, a computer program is a passive collection of instructions, while a process is the actual execution of those instructions. Several processes may be associated with the same program. For example, opening several instances of the same program often means that two or more processes are running. Multitasking may be implemented to allow multiple processes to share processor 804. Although each processor 804 or each core of a processor executes only one task at a time, the computer system 800 may be programmed to implement multitasking so that each processor can switch between running tasks without having to wait for each task to finish. In one embodiment, switching may occur when a task performs an I / O operation, when a task indicates that it is switchable, or when there is a hardware interrupt. Time-sharing may be implemented to enable fast response times for interactive user applications, which is achieved by rapidly switching contexts to make it appear as if multiple processes are running concurrently. In one embodiment, for security and reliability reasons, the operating system may prevent direct communication between independent processes, thereby enabling strictly mediated and controlled inter-process communication functionality.

[0091] The above specification has described embodiments of the present invention with reference to many specific details that may differ from one embodiment to another. Therefore, this specification and the drawings should be considered illustrative rather than restrictive. The sole and exclusive indicator of the scope of the present invention, and what the applicant intends to be the scope of the present invention, is the literal and equivalent scope of the set of claims issued by this application, in the particular form in which such claims are issued (including any subsequent amendments). [Note 1] A step of accessing a trained machine learning model, wherein the machine learning model is trained on domain data specific to a specific domain, the machine learning model is trained to accept transcript data as input, predict or classify the sentiment component of one or more portions of the transcript related to a first party, and output a sentiment score, the sentiment score representing the likelihood that the first party will take action, and The steps include establishing a program connection between the first computer and the second computer, The second computer receives a natural language transcript of a conversation between the first party and the second party from the first computer using the program connection, The steps include evaluating the transcription using the machine learning model and outputting a first sentiment score related to the first party in the specific domain, The steps include accessing digital engagement data representing the engagement of the first party with digital assets associated with the second party, The steps include evaluating the one or more emotion score values ​​and the digital engagement data to output a value representing the likelihood that the first party will take a specific action, The steps include determining whether the value exceeds a threshold, and if it does, automatically sending a notification to the computer device associated with the second party, A computer implementation method including [Note 2] The method according to Appendix 1, further comprising the step of automatically submitting an order to the second computer if the value exceeds the threshold, wherein the order specifies that the product associated with the event be shipped to the first party. [Note 3] The method according to Appendix 1, further comprising the step of receiving a second natural language transcript of a second conversation between a first party and a second party from the first computer in the second computer. [Note 4] The steps include: evaluating the second natural language transcription using the machine learning model and outputting a second sentiment score related to the first party in the domain associated with the second party; The steps include automatically updating the aforementioned value with the second sentiment score, The steps include determining whether the value exceeds the threshold, and if it does, automatically sending a notification to the computer device associated with the second party, The method described in Appendix 3, further including the above. [Note 5] The method according to Appendix 1, further comprising the steps of selecting the domain data from a database, selecting a machine learning type based on information relating to the first and second parties, and building the machine learning model by training the selected machine learning type with the domain data. [Note 6] The method according to Appendix 5, wherein the domain data is selected based on at least the field relating to the second party, the geographical location of the first party, and the geographical location of the second party. [Note 7] The method according to Appendix 6, wherein the domain data includes words and phrases, each having an associated flag, and the flag indicates sentiment data or classification of the words and phrases. [Note 8] The method according to Appendix 1, wherein the emotion data or classification includes information related to categories including sadness, anger, contempt, disgust, surprise, fear, and agreement. [Note 9] The method according to Appendix 1, further comprising the step of filtering the natural language transcript such that data relating to the portion representing what the second party is saying is removed from the natural language transcript. [Note 10] The method described in Appendix 1, wherein the first party is a healthcare professional, and the event includes the first party writing a prescription for a particular pharmaceutical composition. [Note 11] One or more non-temporary computer-readable storage media, the storage media storing one or more sequences of program instructions, and the one or more sequences of program instructions, when executed by one or more processors, A step of accessing a trained machine learning model, wherein the machine learning model is trained on domain data specific to a specific domain, the machine learning model is trained to accept transcript data as input, predict or classify the sentiment component of one or more portions of the transcript related to a first party, and output a sentiment score, the sentiment score representing the likelihood that the first party will take action, and The steps include establishing a program connection between the first computer and the second computer, The second computer receives a natural language transcript of a conversation between the first party and the second party from the first computer using the program connection, The steps include evaluating the transcription using the machine learning model and outputting a first sentiment score related to the first party in the specific domain, The steps include accessing digital engagement data representing the engagement of the first party with digital assets associated with the second party, The steps include evaluating the one or more emotion score values ​​and the digital engagement data to output a value representing the likelihood that the first party will take a specific action, The steps include determining whether the value exceeds a threshold, and if it does, automatically sending a notification to the computer device associated with the second party, To cause one or more of the above processors to execute The aforementioned storage medium. [Note 12] The storage medium according to Appendix 11, further comprising a sequence of program instructions causing the one or more processors to perform the step of automatically submitting the order, where the value exceeds the threshold, the order is configured to ship the product associated with the action to the first party. [Note 13] The storage medium according to Appendix 11, further comprising a sequence of program instructions that, when executed by one or more processors, causes one or more processors to perform the step of receiving a second natural language transcription of a second conversation between a first party and a second party from the first computer. [Note 14] When executed on one or more of the aforementioned processors, The steps include: evaluating the second natural language transcription using the machine learning model and outputting a second sentiment score related to the first party in the domain associated with the second party; The steps include automatically updating the aforementioned value with the second sentiment score, The steps include determining whether the value exceeds the threshold, and if it does, automatically sending a notification to the computer device associated with the second party, A storage medium according to Appendix 13, further comprising a sequence of program instructions that causes one or more processors to execute the above. [Note 15] The storage medium according to Appendix 11, further comprising a sequence of program instructions that, when executed on one or more processors, causes one or more processors to perform the steps of building a machine learning model by selecting the domain data from a database, selecting a machine learning type based on information about the first and second parties, and training the selected machine learning type with the domain data to build the machine learning model. [Note 16] The domain data is selected based on at least the field relating to the second party, the geographical location of the first party, and the geographical location of the second party, in the storage medium described in Appendix 15. [Note 17] The machine learning model is specific to the domains of the first and second parties, as described in Appendix 11. [Note 18] The storage medium described in Appendix 11 includes information relating to categories such as sadness, anger, contempt, disgust, surprise, fear, and agreement, which are the emotional data or classifications. [Note 19] The storage medium described in Appendix 11, wherein the aforementioned action includes the first party purchasing or prescribing a specific product of the second party. [Note 20] It is a corporate server system, Network interface and One or more processors coupled to the aforementioned network interface, One or more memory devices coupled with the one or more processors, the one or more memory devices including a database configured to store information about the company's customers, information about the company's potential customers, and information about the company's domain, The server system includes, The one or more memory devices are further configured to store one or more sequences of program instructions, and when the one or more sequences of program instructions are executed by one or more processors, A step of accessing a trained machine learning model, wherein the machine learning model is trained on domain data specific to a specific domain, the machine learning model is trained to accept transcript data as input, predict or classify the sentiment component of one or more portions of the transcript related to a first party, and output a sentiment score, the sentiment score representing the likelihood that the first party will take action, and The steps include establishing a program connection between the first computer and the second computer, The second computer receives a natural language transcript of a conversation between the first party and the second party from the first computer using the program connection, The steps include evaluating the transcription using the machine learning model and outputting a first sentiment score related to the first party in the specific domain, The steps include accessing digital engagement data representing the engagement of the first party with digital assets associated with the second party, The steps include evaluating the one or more emotion score values ​​and the digital engagement data to output a value representing the likelihood that the first party will take a specific action, The steps include determining whether the value exceeds a threshold, and if it does, automatically sending a notification to the computer device associated with the second party, To cause one or more of the above processors to execute The aforementioned server system.

Claims

1. A computer implementation method performed by a second computer, The step of accessing a trained machine learning model by the second computer, wherein the machine learning model is trained on domain data specific to a specific domain dedicated to a product, company, or particular party, or is trained on domain data including specialized scientific, medical, or technical terminology, the machine learning model is trained to accept transcribed data as input, predict or classify the sentiment component of one or more portions of the transcribed data relating to a first party, and output a sentiment score, the sentiment score representing the likelihood that the first party will take action, and the trained machine learning model is built by selecting the domain data from a database, selecting a machine learning type based on information about the first and second parties, and training the selected machine learning type on the domain data. The steps include establishing a program connection between the first computer and the second computer using the second computer, The second computer receives, using the program connection, a natural language transcript of a conversation between the first party and the second party from the first computer; The steps include: using the second computer to evaluate the transcription using the machine learning model and outputting a first sentiment score related to the first party in the specific domain; The steps include: using the second computer to access digital engagement data representing the first party's engagement with digital assets associated with the second party; The second computer evaluates the one or more emotion score values ​​and the digital engagement data and outputs a value representing the likelihood that the first party will take a specific action. The second computer determines whether the value exceeds a threshold, and if it does, the second computer automatically sends a notification to the computer device associated with the second party. A computer implementation method including

2. The method according to claim 1, further comprising the step of automatically submitting an order by the second computer to the computer device associated with the second party if the value exceeds the threshold, the order specifying that the order is to ship the product associated with the action of the first party.

3. The second computer receives from the first computer a second natural language transcript of a second conversation between the first party and the second party, The steps include: using the machine learning model, the second computer evaluates the second natural language transcription and outputs a second sentiment score related to the first party in the domain associated with the second party; The second computer automatically updates the value with the second sentiment score, The second computer determines whether the value exceeds the threshold, and if it does, the second computer automatically sends a notification to the computer device associated with the second party. The method according to claim 1, further comprising:

4. The method according to claim 1, wherein the domain data is selected based on at least a field relating to the second party, the geographical location of the first party, and the geographical location of the second party.

5. The method according to claim 4, wherein the domain data includes words and phrases, each having a related flag, and the flag indicates sentiment data or classification of the words and phrases.

6. The method according to claim 1, further comprising the step of filtering the natural language transcript by the second computer such that data relating to portions representing what the second party is saying is removed from the natural language transcript.

7. One or more non-temporary computer-readable storage media, the storage media storing one or more sequences of program instructions, the one or more sequences of program instructions being executed by one or more processors of a second computer, A step of accessing a trained machine learning model, wherein the machine learning model is trained on domain data specific to a specific domain, the machine learning model is trained to accept transcript data as input, predict or classify the sentiment component of one or more portions of the transcript relating to a first party, and output a sentiment score, the sentiment score representing the likelihood that the first party will take action, and the trained machine learning model is built by selecting the domain data from a database, selecting a machine learning type based on information about the first and second parties, and training the selected machine learning type on the domain data, The steps include establishing a program connection between the first computer and the second computer, The second computer receives, using the program connection, a natural language transcript of a conversation between the first party and the second party from the first computer; The steps include evaluating the transcription using the machine learning model and outputting a first sentiment score related to the first party in the specific domain, The steps include accessing digital engagement data representing the engagement of the first party with digital assets associated with the second party, The steps include evaluating the one or more emotion score values ​​and the digital engagement data to output a value representing the likelihood that the first party will take a specific action, The steps include determining whether the value exceeds a threshold, and if it does, automatically sending a notification to the computer device associated with the second party, To cause one or more processors to execute this, The aforementioned storage medium.

8. The storage medium according to claim 7, further comprising a sequence of program instructions causing one or more processors to perform the step of automatically submitting an order to the computer device associated with the second party, where the order is configured to ship the product associated with the action of the first party, if the value exceeds the threshold.

9. When executed by one or more processors, The second computer receives from the first computer a second natural language transcript of a second conversation between the first party and the second party, The steps include: evaluating the second natural language transcription using the machine learning model and outputting a second sentiment score related to the first party in the domain associated with the second party; The steps include automatically updating the aforementioned value with the second sentiment score, The steps include determining whether the value exceeds the threshold, and if it does, automatically sending a notification to the computer device associated with the second party, The program instructions further include a sequence of program instructions that cause one or more processors to execute the following: The storage medium according to claim 7.

10. The storage medium according to claim 7, wherein the domain data is selected based on at least a field relating to the second party, the geographical location of the first party, and the geographical location of the second party.

11. The storage medium according to claim 7, wherein the machine learning model is specific to domains dedicated to the first party and the second party.

12. A server system for an enterprise, Network interface and One or more processors coupled to the aforementioned network interface, One or more memory devices coupled with the one or more processors, the one or more memory devices including a database configured to store information about the company's customers, information about the company's potential customers, and information about the company's domain, The server system includes, The one or more memory devices are further configured to store one or more sequences of program instructions, and when the one or more sequences of program instructions are executed by one or more processors, A step of accessing a trained machine learning model, wherein the machine learning model is trained on domain data specific to a specific domain, the machine learning model is trained to accept transcript data as input, predict or classify the sentiment component of one or more portions of the transcript relating to a first party, and output a sentiment score, the sentiment score representing the likelihood that the first party will take action, and the trained machine learning model is built by selecting the domain data from a database, selecting a machine learning type based on information about the first and second parties, and training the selected machine learning type on the domain data, The steps include establishing a program connection between the first computer and the second computer, The second computer receives, using the program connection, a natural language transcript of a conversation between the first party and the second party from the first computer; The steps include evaluating the transcription using the machine learning model and outputting a first sentiment score related to the first party in the specific domain, The steps include accessing digital engagement data representing the engagement of the first party with digital assets associated with the second party, The steps include evaluating the one or more emotion score values ​​and the digital engagement data to output a value representing the likelihood that the first party will take a specific action, The steps include determining whether the value exceeds a threshold, and if it does, automatically sending a notification to the computer device associated with the second party, To cause one or more processors to execute this, The aforementioned server system.

Images