Adaptive interactive healthcare-related sales training system

Abstract

A method for interacting with a user on a computer system includes generating contextual data about the user, generating recommendation data based on the contextual data, and receiving a first interaction data from the user. The method also includes generating a second interaction data in response to the first interaction data, wherein the second interaction data is generated as a function of feedback data from a user profile associated with the user, live data based on an analysis of the first interaction data, and one or more attributes of a character generated by the computer system. Additionally, the method generates live feedback data as a function of the interaction, wherein the live feedback data includes one or more insights.

Description

[0156372.0787014NP2] - 1 - ADAPTIVE INTERACTIVE RELATED SALES TRAINING SYSTEM CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of and priority to U.S. Provisional Application No.63 / 644,837, filed May 09, 2024, which is incorporated herein by reference in its entirety. BACKGROUND

[0001] Training paradigms in pharmaceutical sales, while recognized as essential fororganizational effectiveness, often rely on rigid instructional structures and generally prescribed best practices. These traditional methods fail to provide customized analytics, real-time assessments, and adaptable educational environments, focusing instead on static and decontextualized scenarios. Attempts have been made to improve pharmaceutical sales training. However, the methods employed, including those using current AI-enhanced sales training models, lack a personalization aspect afforded through a variety of data.

[0002] While a variety of systems for training pharmaceutical sales representatives havebeen made and used, it is believed that no one prior to the inventors has made or used an invention as described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The accompanying drawings, which are incorporated in and constitute a part of thisspecification, illustrate embodiments of the invention, and, together with the general description of the invention given above, and the detailed description of the embodiments given below, serve to explain the principles of the present invention.

[0004] FIG. 1 is a diagrammatic view of an exemplary operating environment including asystem of the present invention in communication with a computing device via a network.

[0005] FIG. 2 is a diagrammatic view of an exemplary operating computer system of FIG.[0156372.0787014NP2] - 2 -

[0006] FIG. 3 is a graphical view of dashboard for a user of the system ofFIG.1.

[0007] FIG. 4 is a graphical view of an exemplary selection of a healthcare providercharacter from the system of FIG.1.

[0008] FIG. 5 is a graphical view of an exemplary customization dashboard for ahealthcare provider character from the system of FIG.1.

[0009] FIG. 6 is a graphical view of an exemplary feedback dashboard for a user that hascompleted a role play training session with the system of FIG.1.

[0010] FIG. 7 depicts a flowchart of a process for interacting with a sales representativethat may be performed by the system of FIG 1.

[0011] FIG. 8 is a graphical view of an exemplary real-time dashboard for a user that isundergoing a role play training session with the system of FIG.1.

[0012] FIG. 9 depicts a flowchart of a second process for interacting with a salesrepresentative that may be performed by the system of FIG 1.

[0013] FIG. 10 depicts a flowchart of exemplary training and inference pipelines formachine learning in accord with some embodiments under the present disclosure.

[0014] FIG. 11 is a diagrammatic view of an exemplary neural network under the presentdisclosure.

[0015] The drawings are not intended to be limiting in any way, and it is contemplated thatvarious embodiments of the invention may be carried out in a variety of other ways, including those not necessarily depicted in the drawings. The accompanying drawings incorporated in and forming a part of the specification illustrate several aspects of the present invention, and together with the description serve to explain the principles of the invention; it being understood, however, that this invention is not limited to the precise arrangements shown.[0156372.0787014NP2] - 3 -

[0016] The following description of certain examples of the invention should not be usedto limit the scope of the present invention. Other examples, features, aspects, embodiments, and advantages of the invention will become apparent to those skilled in the art from the following description, which is by way of illustration, one of the best modes contemplated for carrying out the invention. As will be realized, the invention is capable of other different and obvious aspects, all without departing from the invention. Accordingly, the drawings and descriptions should be regarded as illustrative in nature and not restrictive.

[0017] As discussed herein, the terms avatar and character may be used interchangeably.

[0018] Referring now to FIG. 1, an operating environment 10 is a diagrammatic view ofan exemplary operating environment including a system of the present invention in communication with a computing device 22 via a network 24 to have a role play training session for a sales representative that is most similar to the actual interaction between the sales representative and a healthcare provider (HCP) in the field. The system allows for a sales representative to have on- the-go, real-time intelligent and tailored sales training at their convenience to improve their product messaging prior to interacting with customers. The multi-modal learning environment provided by the system includes web and mobile environments. In some embodiments, the sales training may be rendered visually within augmented / virtual reality-based solutions. The dynamic calibration allowed by the system facilitates continuous learning and improvement for a sales representative. The AI-powered system enhances sales representative development through immediate and adaptive coaching and feedback, behavioral analysis, and tailored training scenarios. The AI-powered system also analyzes interactions between a sales representative and a customer to identify areas for improvement and offer coaching to optimize sales representative performance. The AI-powered system may also track salesperson behavior to provide insights for personalized training and role-play scenarios. Revenue losses due to out-of-territory training can be avoided as can the associated training costs, all while developing the ability to have impactful interactions with customers. Assuring compliance with brand strategy and brand messaging is an additional benefit to the pharmaceutical company using the device.[0156372.0787014NP2] - 4 -

[0019] The operating environment include user data 11, a big data source 12, afield coaching report (FCR) data source 13, a customer data source 14, a large language model (LLM) system 16, a voice system 18, a personalization and recommendation system 19, and an avatar 20. In some versions of operating environment 10, user data 11, big data source 12, FCR data source 13, customer data source 14, LLM system 16, voice system 18, personalization and recommendation system 19, and avatar 20 may send and receive communications between one another directly. Alternatively, in other versions of operating environment 10, user data 11, big data source 12, FCR data source 13, customer data source 14, LLM system 16, voice system 18, personalization and recommendation system 19, and avatar 20 may communicate with each other through network 24. Network 24 may include one or more private or public networks (e.g. the Internet) that enable the exchange of data.

[0020] The user data 11 comprises information related to a user (sales representative) of asystem of the present invention. The information in user data 11 may include calendar data related to client meetings scheduled with the user, including the meeting location, location (e.g., latitude / longitude coordinates) information related to the user’s device, historical role play data, and data sourced from a customer relationship management (CRM) system (e.g., historical sales call data not otherwise available in big data source 12, client profile data). Big data source 12 may include qualitative interview data from interviews between healthcare providers (HCP) and sales representatives, as well as a repository of diverse sale scenarios. By way of example, the interviews may comprise questions about disease states and / or branded medications on the market with the aim of gaining an understanding of what a prescribing algorithm for an HCP is and / or how they create it for their patients. Field coaching report (FCR) data source 13 may include data captured during field coaching events, such as a report created by a training manager regarding a sales representative’s performance in the field. Customer data source 14 may include customer (e.g., pharmaceutical company A) specific data such as marketing data, sales data, and organizational compliance data, as well as key performance indicators. Stated differently, any marketing (e.g., marketing insights), sales (e.g., sales messaging), and compliance (e.g., guidelines of what sales representatives can and cannot say to HCPs) data that the customer may have, including data specific to a HCP. In some embodiments, user data 11, big data source 12, FCR data source 13,[0156372.0787014NP2] - 5 - and customer data source 14 may be one or more databases that are either stand alone or a component of the LLM system 16.

[0021] The large language model (LLM) system 16 comprises a language model that maybe trained using one or more of the user data 11, big data source 12, FCR data source 13, and customer data source 14. In some embodiments, one or more of the user data 11, big data source 12, FCR data source 13, and customer data source 14 may be used to generate one or more summaries, which may be consolidated into question and / or answer pairs to train the LLM system 16. During a role play training session, the LLM system 16 receives input derived from a user and outputs a response. The effect is an ongoing dialogue, or an interaction, with the user. The LLM system 16 may also, in some embodiments, analyze the interaction to generate feedback that will be presented to the user. The feedback may be presented in real time, or at the end of the role playing training session, and may include information regarding the sales representative’s performance during the role playing training session. Feedback is further discussed with FIG. 6 herein.

[0022] The voice system 18 may include voice recognition / automatic speech recognitiontechnology that enables a user’s speech to be transformed into an input that can be sent to the LLM system 16 and / or personalization and recommendation system 19. In an embodiment, a commercial off the shelf product such as OpenAI’s Whisper neural network can be used. The Avatar 20 is used as a proxy to interact with a user during a role play training session and may be, in some embodiments, presented through the computing device 22. Additionally, the Avatar 20 may be representative of the type of healthcare provider (HCP) to be used for the role play training session. The representation may include a variety of attributes that may be exchanged with the LLM system 16 to alter how the LLM system 16 outputs a response to received input. The voice system 18 may also analyze speech to detect persuasion techniques using, for example, voice modulation and natural language processing (NLP). Further discussion of the attributes is made with reference to FIGS.4 and 5 herein.

[0023] The personalization and recommendation system 19 comprises a language modelthat may be trained using one or more of the user data 11, big data source 12, FCR data source 13,[0156372.0787014NP2] - 6 - and customer data source 14 to generate simulated training profiles and targeted training recommendations. In some embodiments the personalization and recommendation system 19 may be a part of the LLM system 16 rather than a standalone component. In some embodiments, one or more of the user data 11, big data source 12, FCR data source 13, and customer data source 14 may be used to generate recommendations, dynamic role play experiences, and personalized training modules. The personalized training modules may conduct behavior analysis (leveraging, for example, long short-term memory modules) to analyze speech cadence, hesitation, confidence, and engagement levels. The personalized training modules may also provide user-specific customization through the adaptation of training content based on the identification of knowledge gaps, and the recommendation of learning paths based on prior performance. The system 19 may comprise a light-mobile client that facilitates the capture of user data 11 – such as real-time location data from a user’s device – and uses the captured data along with other user data 11 (e.g., calendar data) to, for example, identify potential client meetings scheduled for the day in the user’s location and generate a training plan.

[0024] During a role play training session, the personalization and recommendation system19 may provide real-time (just-in-time) recommendations to training session participants. The recommendations may relate to, for example, the quality of the conversation as it pertains to brand strategy, talking points, voice modulation, use of phrases, and compliance. Additionally, in some embodiments, the personalization and recommendation system 19 may use historical role play data, historical sales call data, and client profile data to generate recommendations on areas of improvements, strengths, to include practice pathway / training recommendations. The personalization and recommendation system 19 may also, in some embodiments, use historical role play data and historical sales call data to dynamically alter in session prompts for a role play training session. In an embodiment, during a role play training session, sentiment analysis methodologies, including use of long short-term memory models, may be utilized to assess a sales representative’s confidence and engagement levels. In another embodiment, speech analytics techniques, including use of long short-term memory models, may be applied to identify hesitation, interruptions, and filler word utilization to refine coaching recommendations. In yet another embodiment, historical CRM interaction logs are analyzed to synthesize real-world training[0156372.0787014NP2] - 7 - exemplars. In another embodiment, response for the system of the operational environment 10 is generated through implementation of a transformer-based AI model fine-tuned on sales-specific data.

[0025] Referring now to FIG. 2, user data 11, big data source 12, FCR data source 13,customer data source 14, LLM system 16, voice system 18, personalization and recommendation system 19, avatar 20, computing device 22, and network 24 of operating environment 10 may be implemented on one or more computing devices or systems, such as an exemplary computer system 26. Computer system 26 may include a processor 28, a memory 30, a mass storage memory device 32, an input / output (I / O) interface 34, and a Human Machine Interface (HMI) 36. Computer system 26 may also be operatively coupled to one or more external resources 38 via network 24 or I / O interface 34. External resources may include, but are not limited to, servers, databases, mass storage devices, peripheral devices, augmented / virtual reality devices, cloud-based network services, or any other suitable computer resource, such as a artificial intelligence / machine learning system (AI / ML), that may used by computer system 26.

[0026] Processor 28 may include one or more devices selected from microprocessors,micro-controllers, digital signal processors, microcomputers, central processing units, field programmable gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other devices that manipulate signals (analog or digital) based on operational instructions that are stored in memory 30. Memory 30 may include a single memory device or a plurality of memory devices including, but not limited, to read-only memory (ROM), random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache memory, or any other device capable of storing information. Mass storage memory device 32 may include data storage devices such as a hard drive, optical drive, tape drive, non-volatile solid state device, or any other device capable of storing information.

[0027] Processor 28 may operate under the control of an operating system 40 that residesin memory 30. Operating system 40 may manage computer resources so that computer program code embodied as one or more computer software applications, such as an application 42 residing[0156372.0787014NP2] - 8 - in memory 30, may have instructions by processor 28. In an alternative embodiment, processor 28 may execute the application 42 directly, in which case operating system 40 may be omitted. One or more data structures 44 may also reside in memory 30, and may be used by processor 28, operating system 40, or application 42 to store or manipulate data.

[0028] I / O interface 34 may provide a machine interface that operatively couples processor28 to other devices and systems, such as network 24 or external resource 38. Application 42 may thereby work cooperatively with network 24 or external resource 38 by communicating via I / O interface 34 to provide the various features, functions, applications, processes, or modules comprising embodiments of the invention. Application 42 may also have program code that is executed by one or more external resources 38, or otherwise rely on functions or signals provided by other system or network components external to computer system 26. Indeed, given the nearly endless hardware and software configurations possible, persons having ordinary skill in the art will understand that embodiments of the invention may include applications that are located externally to computer system 26, distributed among multiple computers or other external resources 38, or provided by computing resources (hardware and software) that are provided as a service over network 24, such as a cloud computing service.

[0029] HMI 36 may be operatively coupled to processor 28 of computer system 26 in aknown manner to allow a user to interact directly with computer system 26. HMI 36 may include video or alphanumeric displays, a touch screen, a speaker, and any other suitable audio and visual indicators capable of providing data to the user. HMI 36 may also include input devices and controls such as an alphanumeric keyboard, a pointing device, keypads, pushbuttons, control knobs, microphones, etc., capable of accepting commands or input from the user and transmitting the entered input to processor 28.

[0030] A database 46 may reside on mass storage memory device 32, and may be used tocollect and organize data used by the various systems and modules described herein. Database 46 may include data and supporting data structures that store and organize the data. In particular, database 46 may be arranged with any database organization or structure including, but not limited to, a relational database, a hierarchical database, a network database, or combinations thereof. A[0156372.0787014NP2] - 9 - database management system in the form a computer software application executing as instructions on processor 28 may be used to access the information or data stored in records of database 46 in response to a query, where a query may be dynamically determined and executed by operating system 40, other applications 42, or one or more modules.

[0031] FIG. 3 is a graphical view of an exemplary dashboard for a user of the system ofFIG.1. Components of the dashboard may be stored in the system under a user profile for the sales representative and categorized by a profile data type (cumulative insight, cumulative metric, session history, daily challenge). The dashboard may include one or more cumulative insights 50, a cumulative metric 52, a session history 54, a daily challenge dashboard 56, and a role play session activation button 58. The one or more cumulative insights 50 may include one or more insight categories, each of which may be represented by a cumulative rating. The rating depicts a user’s performance in the particular insight category across all role playing training sessions for the user. For example, and as shown in FIG.3, a Brand Strategy Alignment insight category is shown with a cumulative rating of 39% along with nine additional insight categories each with their respective cumulative rating. In an embodiment, the one or more cumulative ratings are calculated for a user by the LLM system 16. A non-limiting list of insight categories include: Brand Strategy Alignment, Vocabulary and Terminology, Grammer and Syntax, Challenger Sale Techniques, Active Listening Skills, Response Time, Empathy and Personalization, Questioning Techniques, Compliance, and Tone. The cumulative metric 52 is a reflection of at least the one or more cumulative insights 50 for a user. In one embodiment, the cumulative metric 52 is represented with a five-point scale, where the average of the cumulative ratings of the one or more cumulative insights 50 is translated to the five-point scale. The session history 54 may list the latest role playing training sessions a user has done. The daily challenge dashboard 56 may represent a training gamification, where a user is challenged to engage in a number of role play training sessions per day and is only presented with the number of role play training sessions completed, as shown in FIG.3. In addition, the exemplary dashboard may include a way to initiate a new role play training session, shown in FIG.3 with role play session activation button 58.[0156372.0787014NP2] - 10 -

[0032] FIG. 4 is a graphical view exemplary selection of a healthcare providercharacter. The character selector 60 may enable a user to select a character 62 to be used in a role play training session. Each character 62 may resemble an HCP, and may also include one or more attributes that, when interpreted by the LLM system 16, alters how the LLM system 16 responds to input derived from a user during a role play training session. For example, a character 62 with a Health Care Practitioner Type attribute set to Pulmonology may respond with confusion during a role play training session if a user discusses a medication aimed at alleviating constipation. Attributes are discussed further with reference to FIG. 5 below. The character 62 may be a pre- existing character, such as the one shown in FIG.4, or a new customized character that is generated using the character customization dashboard 70.

[0033] FIG. 5 is a graphical view of an exemplary customization dashboard, charactercustomization dashboard 70, for a healthcare provider character. The character customization dashboard 70 may comprise one or more attributes which a user can select. A list of non-limiting examples of attributes and their respective options include: practitioner type 71 (internal medicine, family medicine, gastroenterology, pulmonology), level of objection 72 (high, medium, low), personality type 74 (data driven, skeptical, story-telling, agreeable, busy, to-the-point), available time 76 (selectable minute increments), prescribing pattern (not shown), and mentality 78 (product, patient). As discussed with respect to FIG.4, the attributes can be collectively interpreted by the LLM system 16 to alter its response to user derived input. For example, generating a character 62 with the level of objection 72 set to “high” and the personality type 74 set to at least “data driven” and “skeptical” may prompt the LLM system 16 to respond with a strong aversion to the pharmaceutical being discussed by the sales representative unless the sales representative provides data-driven and / or verifiable facts about the pharmaceutical. If, in a similar example, the level of objection 72 is set to “low,” the LLM system 16 may respond more favorably to the pharmaceutical, such as by stating a willingness to try the pharmaceutical on an existing patient.

[0034] FIG. 6 is a graphical view of an exemplary feedback dashboard, feedbackdashboard 80, for a user that has completed a role play training session with the system of FIG.1. The feedback dashboard 80 may be presented to the user upon completion of the role play training[0156372.0787014NP2] - 11 - session in an embodiment, or the 80 or any of its components may be presented during the role play training session. The feedback dashboard 80 may comprise feedback data of various feedback data types, including one or more insights 82, a session overview 84, session details 86 which include a metric 88, one or more field coaching reports (FCRs)(not shown), and a session transcript 89. The feedback data may leverage information from one or more of the big data source 12, FCR data source 13, and customer data source 14. The one or more insights 82 may include one or more insight categories, each of which may be represented by a rating. The rating depicts a user’s performance in the particular insight category for the completed role playing training session. For example, and as shown in FIG. 6, a Language Clarity and Precision category is shown with a rating of 39% along with seven additional insight categories each with their respective rating. In an embodiment, the one or more ratings are calculated for a user by the LLM system 16. A non-limiting list of insight categories include: Language Clarity and Precision, Vocabulary and Terminology, Brand Strategy Alignment, Challenger Sale Techniques, Response Time, Questioning Techniques, Compliance, and Tone. Insight categories may be fact-driven, such as whether a sales representative’s interaction with the character 62 during the role play training session included messaging that was consistent with brand strategy for the pharmaceutical or compliant with specific claims. In addition, insight categories may be behavioral / soft skill-driven, such as whether the sales representative incorporated active listening techniques or practiced emotional learning. In some embodiments, one or more components of the feedback dashboard 80 may be generated by AI / ML.

[0035] The session overview 84 may summarize the role play training session, as shownin FIG. 6, and may be generated, in some embodiments, by the LLM system 16. The summarization may be achieved, in some embodiments, by analyzing the dialogue from the role play training session and generating information targeted to the sales representative that completed the role play training session. The information may be based on, for example, data from customer data source 14, such as key performance indicators. The session details 86 may include a variety of details regarding the completed role play training session including, but not limited to: session duration, rating for the sentiment of the session, duration of session that is deemed filler interactions, questions exchanged, and links to review the session by watching, reading, or[0156372.0787014NP2] - 12 - listening to it. The metric 88, which may included in session details 86, is calculated as a function of at least the one or more insights 82 from the role play training session for a user. In one embodiment, the metric 88 is represented with a five-point scale, where the average of the ratings of the one or more insights 82 is translated to the five-point scale. The session transcript 89 includes a transcript of the completed role play training session and may, in some embodiments, be used as additional data to train the LLM system 16. The session transcript 89 may also include analysis and feedback of the completed role play training session.

[0036] FIG.7 is a flowchart of a process 100 for interacting with a sales representative thatmay be performed by the system of FIG 1. This process may be initiated 102, in some embodiments, when selecting the role play session activation button 58 depicted in FIG.3. In step 104, a healthcare provider (HCP) character is generated by either selecting from existing HCP characters having set attributes or customizing a new HCP character and its attributes, as discussed with reference to FIGS.4 and 5 above.

[0037] In step 106, a role play training session is conducted. The HCP character selectedmay manifest itself as avatar 20 to a user through the computing device 22, and the LLM system 16 may be made aware of the attributes of the selected HCP character in order to account for them when generating responses. An interaction may then ensue, with the system receiving input derived from the sales representative and generating a response to the received input. FIG. 8, described below, depicts an exemplary interaction in real-time. The interaction may fall under one or more of the following non-limited interaction types: prescribing algorithm, recommended dose, efficacy, drug interactions, safety profile, dosage titration, side effects, expanding to other disease states / indications (e.g., depression, schizophrenia), patient identification (identifying new patients that may be appropriate for a pharmaceutical), soft skills (convincing / persuasive). During the interaction, input derived from the sales representative may include numerous types of sentences, including questions. A non-limiting list of example questions that may be received include: Connecting Questions ^I was curious, what was it about pharmaceutical A that attracted your attention?[0156372.0787014NP2] - 13 - ^Anything else that attracted your to pharmaceutical A?^ Have you found that pharmaceutical A has been safe and effective, or are you stilllooking for something more? Situation Questions ^What got you involved with community mental health?^ What are you using now to impact your patients suffering from depression?Problem Awareness Questions ^What do you like or don't like about pharmaceutical A?^ Why do you or don't you like pharmaceutical A?Two Truths Questions ^It sounds like things are going fairly well for your patients, is there anything you wouldchange about it if you could? ^Why would you change that?^ Why is that important to you now?^ Has that had an impact on you? In what way?Solution Awareness Questions ^When you solve this problem for your patients, how would it be different than it is nowfor you? How would that make you feel? ^What prevented you in the past from changing your prescribing habits?^ What prevented you from making a change in the past?Consequence Questions ^What if you don’t do anything about this problem and patients continue to get worse?^ What are you going to do if nothing changes for this patient?

[0038] In step 108, the role play session is terminated. This may occur by receiving atermination request from the user (e.g., the user selecting a session termination button [not shown]). The termination may also occur via natural conversation termination means, such as receiving input derived from the user that states, by way of non-limiting examples: (1) “Sounds good. I will follow up and make sure everything went smoothly…,” (2) “See you next time Dr….,” or “Thanks again for your time.”[0156372.0787014NP2] - 14 -

[0039] In step 110, feedback data In one embodiment, generated feedbackdata may resemble one or more of the feedback data types described with respect to FIG.6 herein. Feedback data may be generated by the LLM system 16, or any other part of the system, and presented in a dashboard, such as feedback dashboard 80.

[0040] In step 112, a user profile, comprising data of profile data types as that describedwith respect to FIG. 3 herein, for the sales representative that completed the role play session is updated. The update may include feedback generated in step 110. For example, if step 110 generated in part an insight 82 related to response time with a value of 78%, the cumulative insight 50 related to response time of the user profile would be updated by incorporating the 78% value. As another example, the metric 88 generated in step 110 may be used to update the cumulative metric 52 of the user profile.

[0041] FIG. 8 is a graphical view of an exemplary real-time dashboard, real-timedashboard 90, for a user that is undergoing a role play training session with the system of FIG.1. The real-time dashboard 90 may comprise real-time data of various real-time data types, including contextual information 92, one or more recommendations 94, a live transcript 96, and one or more insights 98. The real-time data may leverage information from one or more of the user data 11, big data source 12, FCR data source 13, customer data source 14, and personalization and recommendation system 19. For example, contextual information 92 may comprise information related to the user for the day based on their location and integrated calendar data. The personalization and recommendation system 19 generates the one or more recommendations 94 so that they are available to the sales representative just-in-time during the training session. The recommendations may be tailored to the specific context of an appointment identified from calendar data, focusing on skills and scenarios directly relevant to the location and potential client profile. A non-limiting list of the one or more recommendations 94 includes: Brand Strategy, Talking Points, Voice Modulation, Use of Phrases, and Compliance. The recommendations may be achieved, in some embodiments, by analyzing the dialogue of the ongoing role play training session and / or information from any of the data sources of the operating environment 10 and generating information targeted to the sales representative in that training session. The information[0156372.0787014NP2] - 15 - may be based on, for example, data from data source 14, such as key performance indicators. The live transcript 96 includes an ongoing transcript of the role play training session and may, in some embodiments, be used as additional data to train the personalization and recommendation system 19 or LLM system 16. The real-time transcript 96 may also include analysis and feedback of the role play training session. AI / ML, such as reinforcement learning- based decision trees, may be used dynamically adapt the training session, and hence alter what is being generated by the system in the live transcript 96, or other sections of the real-time dashboard 90 (e.g., one or more recommendations 94).

[0042] The one or more insights 98 may include one or more insight categories, each ofwhich may be represented by a rating that may be adjusted in real time during the training session. The rating depicts a user’s performance in the particular insight category for the ongoing role playing training session. For example, and as shown in FIG.8, a Language Clarity and Precision category is shown with a rating of 39% along with seven additional insight categories each with their respective rating. In an embodiment, the one or more ratings are calculated for a user by the personalization and recommendation system 19. A non-limiting list of insight categories include: Language Clarity and Precision, Vocabulary and Terminology, Brand Strategy Alignment, Challenger Sale Techniques, Response Time, Questioning Techniques, Compliance, and Tone. Insight categories may be fact-driven, such as whether a sales representative’s interaction with the character 62 during the role play training session includes messaging that is consistent with brand strategy for the pharmaceutical or compliant with specific claims. In addition, insight categories may be behavioral / soft skill-driven, such as whether the sales representative is incorporating active listening techniques or practicing emotional learning. In some embodiments, one or more components of the real-time dashboard 90 may be generated by AI / ML.

[0043] FIG. 9 is a flowchart of a process 200 for interacting with a user (e.g., salesrepresentative) that may be performed by the system of FIG 1. Process 200 may be carried out during an interactive role play training session and used to generate the system’s portion of the interaction based on, amongst other things, input provided by the user. This process may be initiated, in some embodiments, when selecting the role play session activation button 58 depicted[0156372.0787014NP2] - 16 - in FIG.3. Process 200 may also be a of step 106 of process 100, aside from being a standalone process.

[0044] In step 202, contextual data is generated about the user. The contextual data may,in some embodiments, include a user’s location that is generated from a device associated with the user. The contextual data may also include one or more appointments for the user that are identified based on the user’s location and scheduling data (e.g., via a calendar integration) associated with the user. The one or more appointments may be associated with customers of the user.

[0045] In step 204, recommendation data is generated based on the contextual data. Therecommendation data may, in some embodiments, be generated at least in part as a function of any of the contextual data (e.g., data about the customer sourced from customer data source 14) and / or interaction data from the user. Non-limiting examples of categories of recommendation data include: a brand strategy category, a talking points category, a voice modulation category, a use of phrases category, and a compliance category.

[0046] In step 206, interaction data is received from the user. In some embodiments, theinteraction data may be a voice input based on the user’s speech. Voice system 18 may be used to generate the voice input using methods, such as, natural language processing. Additional interaction data may then be generated, in step 208, in response to the interaction data of step 206.

[0047] The interaction data (i.e., training data) of step 208 may be generated, in someembodiments, using one or more AI / ML models as a function of one or more of: feedback data from a user profile associated with the user, live data based on an analysis of the interaction data of step 206, and one or more attributes of the healthcare provider (HCP) character generated by the computer system in step 104 of process 100. The one or more AI / ML models may include, in an embodiment, one or more of a transformer-based AI model, a reinforcement learning-based decision tree, and a long short-term memory model. The one or more AI / ML models may be associated with one or more AI / ML engines (e.g., simulation engine, personalization engine, recommendation engine), and may be trained using the methods discussed herein. However, other methods for training the one or more AI / ML models are possible. Interaction data generated in step 208 may resemble dialogue in response to the user’s speech provided in the interaction data of step 206, and may be altered in complexity and / or difficulty by the one or more AI / ML models as a function of the feedback data. The feedback data may be based on historical interactions by[0156372.0787014NP2] - 17 - the user, live data, or the live feedback data in step 210. Live data may include, in some embodiments, measurements of speech cadence, hesitation, confidence, interruptions, filler word utilization, and engagement levels.

[0048] In step 210, live feedback data may be generated using AI / ML as a function of theinteraction enabled by process 200. The live feedback data may be based on one or more insights, such as Language Clarity and Precision, Vocabulary and Terminology, Brand Strategy Alignment, Challenger Sale Techniques, Response Time, Questioning Techniques, Compliance, and Tone.

[0049] Various embodiments under the present disclosure can incorporate AI / MLfunctionality. For example, for purposes of the present disclosure, LLM system 16, personalization and recommendation system 19, and computing device 22 of FIG.1 can be said to comprise one or more AI / ML engines, either separately or together. Each component may comprise a separate instance of an identical AI / ML engine, or a different AI / ML engine. Or a “central” AI / ML engine could be running at any location, such as LLM system 16, and others of the foregoing systems could function like an output / input interface to the central AI / ML engine, allowing user input, data collection, user interface for a user, etc.

[0050] It should be understood that AI / ML engine can comprise one or more AI / MLengines. Commonly the terms machine learning engine or machine learning algorithm are used to refer to a specific algorithm. The term artificial intelligence commonly is used to refer to an entire system that achieves intelligence-like outcomes while using multiple sub-systems, such as multiple machine learning algorithms. But both ML and AI have been used to identify a variety of functionalities or types of systems that utilize various combinations of specific ML algorithms. As used herein, AI / ML engine is intended to denote a variety of AI / ML functionalities that fall under the category of AI or ML algorithms and systems that utilize such functionalities. Examples of AI / ML engine can comprise any one or more of e.g.: supervised learning, reinforcement learning, reinforcement learning-based decision trees, natural language processing such as LLMs, neural networks, transformer-based AI, recurrent neural network, long short-term memory models, computer vision, facial recognition, chatbots, virtual assistants, unsupervised learning, generative AI, other AI or ML models, and / or combinations of any of the foregoing.[0156372.0787014NP2] - 18 -

[0051] Building an AI / ML engine several development steps where the actualtraining of a ML model or algorithm is just one step in a training pipeline. An important part in AI / ML development is AI / ML model lifecycle management. One embodiment of a model lifecycle management procedure 2700 is illustrated in FIG. 10. The model lifecycle management can in some embodiments comprise two pipelines: a training pipeline 2705 and an inference pipeline 2750.

[0052] At 2710 in the training pipeline 2705, data ingestion 2710 occurs, which includesgathering raw (training) data from a data storage. After data ingestion 2710, there may also be a step that controls the validity of the gathered data. At 2715 data pre-processing occurs, which can include feature engineering applied to the gathered data. This may involve, e.g., data normalization or data formatting or transformation required for the input data to the AI / ML model. After the ML model’s architecture is fixed, it should be trained on one or more datasets. At 2720 model training is performed in which the AI / ML model is trained with the raw training data. To achieve good performance during live operation in a system (the so-called inference phase), the training datasets should be representative of actual data the ML model will encounter during live operation. The training process often involves numerically tuning the ML model’s trainable parameters (e.g., the weights and biases of the underlying neural network (NN)) to minimize a loss function on the training datasets. The loss function may be, for example, based on maximizing sales performance, maximizing a user’s strengths, minimizing a user’s weaknesses, or other metrics. The purpose of the loss function is to meaningfully quantify the reconstruction error for the particular use case at hand. At 2725 model evaluation can be performed where the performance is benchmarked to some baseline. Model training 2720 and evaluation 2725 can be iterated until an acceptable level of performance is achieved. At 2730 model registration occurs, in which the AI / ML model is registered with any corresponding data on how the AI / ML model was developed, and e.g., AI / ML model evaluation data. At 2735 model deployment occurs, wherein the trained / re-trained AI / ML model is implemented in the inference pipeline 2750.

[0053] Data ingestion 2755 in the inference pipeline 2750 refers to gathering raw(inference) data from a data source. Data pre-processing 2760 can be essentially identical / similar to the data pre-processing 2715 of the training pipeline 2705. At 2765, the operational model received from the training pipeline 2705 is used to process new data received during operation of[0156372.0787014NP2] - 19 - the system of operating environment 10 or thereof. At 2770 data and model monitoring is performed. Here the inference data is analyzed to determine whether the inference data are from a distribution that aligns with the training data, as well as monitoring model outputs for detecting any performance, or operational, variance or drifts. The variance or drift is used at 2745 (drift detection) to update the AI / ML model registration.

[0054] The training process is typically based on some variant of a gradient descentalgorithm, which, at its core, typically comprises three components: a feedforward step, a back propagation step, and a parameter optimization step. These steps can be described using a dense ML model (i.e., a dense NN with a bottleneck layer) as an example.

[0055] Feedforward: A batch of training data, such as a mini-batch, (e.g., severaldownlink-channel estimates) is pushed through the ML model, from the input to the output. The loss function is used to compute the reconstruction loss for all training samples in the batch. The reconstruction loss may be an average reconstruction loss for all training samples in the batch.

[0056] Back propagation (BP): The gradients (partial derivatives of the loss function, L,with respect to each trainable parameter in the ML model) are computed. The back propagation algorithm sequentially works backwards from the ML model output, layer-by-layer, back through the ML model to the input. The back propagation algorithm is built around the chain rule for differentiation: When computing the gradients for layer n in the ML model, it uses the gradients for layer n + 1.

[0057] Parameter optimization: The gradients computed in the back propagation step areused to update the ML model’s trainable parameters. An approach is to use the gradient descent method with a learning rate hyperparameter (α) that scales the gradients of the weights and biases. It is preferred to make small adjustments to each parameter with the aim of reducing the average loss over the (mini) batch. It is common to use special optimizers to update the ML model’s trainable parameters using gradient information. The following optimizers are widely used to reduce training time and improving overall performance: adaptive sub-gradient methods (AdaGrad), RMSProp, and adaptive moment estimation (ADAM).

[0058] The above process (feedforward, back propagation, parameter optimization) can berepeated many times until an acceptable level of performance is achieved on the training dataset. An acceptable level of performance may refer to the ML model achieving a pre-defined average[0156372.0787014NP2] - 20 - reconstruction error over the training normalized MSE of the reconstruction error over the training dataset is less than, say, 0.1). Alternatively, it may refer to the ML model achieving a pre-defined value chosen by a user.

[0059] In some implementations, a function F(⋅) may be generated by a ML process, suchas, for example, supervised learning, reinforcement learning, and / or unsupervised learning. It should further be understood that supervised learning may be done in various ways, such as, for example, using random forests, support vector machines, neural networks, and the like. By way of non-limiting example, any of the following types of neural networks that may be utilized, including, deep neural networks (DNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs), or any other known or future neural network that satisfies the needs of the system. In an implementation using supervised learning the neural networks may be easily integrated into the hardware described in the operating environment 10 of FIG.1 (e.g., in the form of simple vector-matrix multiplications).

[0060] Referring now to FIG. 11, an example NN 2900 (e.g., DNN) is shown. In someimplementations, and as shown, the neural network 2900 may include two hidden layers represented by dashed boxes 2901 and 2902. In one implementation, the inputs 2903 may be fed into the NN 2900. Next, the inputs 2403 may go through a set of hidden layers (e.g., 2901 and / or 2902). Once the inputs 2903 pass though the hidden layers 2901 and / or 2902, they may be output (e.g., as an output layer) as outputs 2904, 2905. Outputs 2904, 2905 could be, e.g., feedback based on training scenario; recommendations; personalized training exercises; or another output value. Possible inputs can include e.g.: data from simulated role play scenarios, data from actual sales calls captured by CRM systems, training scenario difficulty measurements, training scenario complexity measurements; calendar data, location data, or other variables.

[0061] As should be understood by one of ordinary skill in the art, in order for the NN2900 to output proper a proper analysis, it should be trained properly (e.g., with a collection of samples) to accurately extract the likelihood values. If not trained properly, overfitting (e.g., when the NN memorizes the structure of the preambles but is unable to generalize to unseen preamble characteristics) or underfitting (e.g., when the NN is unable to learn a proper function even on the data that it was trained on) may happen. Thus, implementations may exist that prevent overfitting[0156372.0787014NP2] - 21 - or underfitting, involving a set of well- features that must be extracted from the preamble characteristics.

[0062] Having shown and described various embodiments of the present invention, furtheradaptations of the methods and systems described herein may be accomplished by appropriate modifications by one of ordinary skill in the art without departing from the scope of the present invention. Several of such potential modifications have been mentioned, and others will be apparent to those skilled in the art. For instance, the examples, embodiments, geometrics, materials, dimensions, ratios, steps, and the like discussed above are illustrative and are not required. Accordingly, the scope of the present invention should be considered in terms of any claims that may be presented and is understood not to be limited to the details of structure and operation shown and described in the specification and drawings.

Claims

[0156372.0787014NP2] - 22 - What is claimed:

1. A method for interacting with a user on a computer system comprising:generating contextual data about the user; generating recommendation data based on the contextual data; receiving a first interaction data from the user; generating a second interaction data in response to the first interaction data, wherein the second interaction data is generated as a function of: feedback data from a user profile associated with the user, live data based on an analysis of the first interaction data, and one or more attributes of a character generated by the computer system; and generating, live feedback data as a function of the interaction, wherein the live feedback data comprises one or more insights.

2. The method of claim 1, wherein generating the contextual data comprises:generating a location of the user from a device associated with the user; and identifying one or more appointments for the user as a function of the location and scheduling data associated with the user, each of the one or more appointments associated with a customer of the user.

3. The method of claim 2, wherein a calendar integration is used to source the scheduling data.

4. The method of claim 1, wherein the recommendation data comprises a category from a groupconsisting of: a brand strategy category;[0156372.0787014NP2] - 23 - a talking points category; a voice modulation category; a use of phrases category; and a compliance category.

5. The method of claim 4, wherein the recommendation data is generated at least in part as afunction of the contextual data.

6. The method of claim 4, wherein the recommendation data is generated at least in part as afunction of the first interaction data.

7. The method of claim 1, wherein the second interaction data is generated at least in part usingone or more AI / ML models.

8. The method of claim 7, wherein one of the one or more AI / ML models is a transformer-basedAI model or a reinforcement learning-based decision tree.

9. The method of claim 7, wherein the live data is analyzed using a long short-term memorymodel.

10. The method of claim 7, wherein the one or more AI / ML models alter the complexity anddifficulty of information in the second interaction data based on the feedback data.

11. The method of claim 1, wherein the feedback data is based on historical interactions by theuser.[0156372.0787014NP2] - 24 -12. The method of claim 1, wherein the live data comprises measurements of speech cadence,hesitation, confidence, interruptions, filler word utilization, and engagement levels.

13. The method of claim 1, wherein the character is generated in part using AI / ML and the oneor more attributes are selected from a group consisting of: a practitioner type, a level of objection, a personality type, an available time, a prescribing pattern, and a mentality.

14. The method of claim 1, wherein the live feedback data is generated using AI / ML.

15. The method of claim 14, wherein the one or more insights are insight categories selected froma group consisting of: Language Clarity and Precision, Vocabulary and Terminology, Brand Strategy Alignment, Challenger Sale Techniques, Response Time, Questioning Techniques, Compliance, and Tone.

16. A system for interacting with a user comprising:[0156372.0787014NP2] - 25 - a processor; and a memory including instructions that, when executed by the processor, cause the system to: generate contextual data about the user, generate recommendation data based on the contextual data, receive a first interaction data from the user, generate a second interaction data in response to the first interaction data, wherein the second interaction data is generated as a function of: feedback data from a user profile associated with the user, live data based on an analysis of the first interaction data, and one or more attributes of a character generated by the computer system, and generate, live feedback data as a function of the interaction, wherein the live feedback data comprises one or more insights.

17. The system of claim 16, wherein generating the contextual data comprises:generating a location of the user from a device associated with the user; and identifying one or more appointments for the user as a function of the location and scheduling data associated with the user, each of the one or more appointments associated with a customer of the user.

18. The system of claim 16, wherein the second interaction data is generated at least in part usingone or more AI / ML models, wherein one of the one or more AI / ML models is a transformer-based AI model or a reinforcement learning-based decision tree.[0156372.0787014NP2] - 26 -19. The system of claim 16, wherein the data is generated using AI / ML and theone or more insights are insight categories selected from a group consisting of: Language Clarity and Precision, Vocabulary and Terminology, Brand Strategy Alignment, Challenger Sale Techniques, Response Time, Questioning Techniques, Compliance, and Tone.

20. A system for simulating interactive sales training comprising:a voice system configured to generate a voice input based on a user’s speech; a simulation AI / ML engine configured to generate training data; a personalization AI / ML engine configured to: analyze the voice input according to speech cadence, hesitation, confidence, and engagement levels; adjust the difficulty and complexity of the training data generated by the simulation AI / ML engine based on one or more of: feedback data from a user profile associated with the user, and the analyzed voice input; and a recommendation AI / ML engine configured to: generate contextual data about the user; generate recommendation data based on the contextual data and the feedback data;[0156372.0787014NP2] - 27 - wherein the training data is generated in part as a function of the personalization AI / ML engine and the recommendation AI / ML engine.

Images