Method and system for adaptive performance evaluation through scenario simulation

Abstract

Disclosed are a system and a method for adaptive performance evaluation of a trainee through scenario-based communication simulation. The system includes a persona configurator module hosting a virtual persona agent configured by a first large language model (LLM) to engage the trainee in dialog, and a feedback module hosting an evaluation agent configured by a second LLM to analyze the dialog. An orchestrator module extracts persona parameters, scenario parameters and evaluator parameters from a configuration command and generates separate configuration commands for the persona LLM and the evaluation LLM. The persona agent replicates professional identity and behavioral traits under a scenario context, while the evaluation agent applies skill-testing attributes and evaluation rubrics to assess the trainee's performance. The evaluation output includes scores, strengths, weaknesses and improvement recommendations. The system prevents circular bias, ensures objective evaluation, and enables holistic training of communication and professional skills.

Description

[0001] Ref: IN202421070079-PCT

[0002] METHOD AND SYSTEM FOR ADAPTIVE PERFORMANCE EVALUATION THROUGH SCENARIO SIMULATION

[0003] FIELD OF THE DISCLOSURE

[0004] [1] The present disclosure relates to digital training systems, and more particularly to a method and system for training personnel on communication skills by simulating a real-time communication and evaluating the performance of the personnel.

[0005] BACKGROUND OF THE DISCLOSURE

[0006] [2] There are several professional fields that involve heavy dependence on strong interpersonal skills of the professional. For instance, a sales representative, besides being articulative and knowledgeable of the products he / she is showcasing to a potential client, must additionally be smart to understand the peculiar personality traits of the potential client to adapt their sales pitch for maximizing the chances of a successful business deal. Quite often, this is heavily dependent on the quality of experience gained by the professional during the course of his professional career. The key to gain quality experience is having a multitude of conversational engagements with a diverse quality of potential or existing clients. This is practically not possible all times as an organization cannot guarantee that the professionals employed by it are given diverse opportunities for gaining the required experience. The conventional process of training, through on the job interactions with potential / existing clients, is not efficient as it also depends heavily on the professional’s learning capabilities and the capacity to grasp the nuances of learnings from the interaction.

[0007] [3] With the advent of artificial intelligence (Al), several solutions have come to the fore that use advanced Al based communication models to train a target user, such as a sales representative, by engaging him / her in virtual communicative sessions replicating real-time scenarios, such as that relating to a sales pitch. One such solution is recited by published patent application WO2022 / 240918A1 that provides a chatbot that is adapted to engage with a trainee through communicative dialogue in respect of a specific subject matter. The communicative dialogue is enabled through a skills module configured Ref: IN202421070079-PCT within the proposed system. A trainer creates, removes or updates the sill module with pertinent training skills which the trainee is desirous of acquiring. The described solution also envisages that the trainee’s interaction is monitored by a human administrator who is also tasked with evaluating the trainee’s interaction and provide guidance to the trainee. To enable this functionality, the described solution provides an evaluator that is configured to determine a communication performance of the trainee based on the trainee’s responses in respect of a subject area, a facial expression of the trainee detected during the training session, a gesture of the trainee detected during the training session, and the like. Based on the evaluation, a manager of the trainee may provide feedback to the trainee to help him / her improve his / her skills. The prior art, while providing a solution to the problems in the conventional training systems, does have an inherent drawback to the extent that the feedback is being provided through human intervention. Thus, the system is limited in its usability since the feedback will be subjective based on the skills and experience of the person providing the feedback.

[0008] [4] JP2023171705 recites another solution that may solve the inherent problem of the above cited document. Specifically, JP2023171705 provides a communication capability training chatbot system that is adapted to simulate a specific communication situation by initiating a role-playing communication exchange with a user, and analyzing the exchange to evaluate it quantitatively and qualitatively. The user is evaluated in respect of the clarity, relevance, politeness, appropriateness, persuasive power of the user’s responses during the exchange. The user is thereafter provided feedback, such as on the strength and weaknesses of the user’s abilities, areas of improvement and the like. This document, however, also has several disadvantages. Firstly, the engagement work and evaluation work is carried out by the same Al model, and this may limit the quality of evaluation because of the bias and circularity of the Al model due to which it will adjudge the responses based on the same logic it used for conversing with the user. Consequently, the evaluation may not be independent. Additionally, the Al model may lack objectivity as the user is being tested against the Al’s own conversational style. Thus, the Al model is severely constrained in providing a holistic and an unbiased evaluation, and hence the capabilities of the proposed system are very limited. Ref: IN202421070079-PCT

[0009] [5] In view of the above, there is a need for a solution that enables a holistic training of a trainee, such as for a professional skill by engaging the trainee in a rolebased conversation. There further exists a need for a solution for a holistic and an unbiased evaluation of the trainee based on the responses provided by the trainee during the conversation.

[0010] SUMMARY OF THE DISCLOSURE

[0011] [6] In order to provide a holistic solution for training and evaluation of trainees while circumventing the aspect of inherent bias of the Al module, as encountered in some of the known solutions, the present invention envisages to provide a distributed training and evaluation system. The training system and the evaluation system while working collaboratively maintain their independence in their assigned tasks of testing the trainee by engaging him / her in a communicative exchange and evaluating the responses of the trainee during the said exchange, respectively.

[0012] [7] An object of the present disclosure is to enable holistic testing an evaluation of a trainee.

[0013] [8] Another object of the present disclosure is to ensure that the evaluation is not limited by the inherent biasness and circularity of the evaluation system.

[0014] [9] Another object of the present disclosure is to deploy two independent Al based systems that perform the tasks of training and evaluation, respectively, while working collaboratively to ensure that the trainee is provided holistic feedback.

[0015]

[0010] According to an embodiment of the disclosure there is provided a system for adaptive performance evaluation of a trainee. The system includes a user device pertaining to the trainee and a persona configurator module communicably coupled to the user device and hosting a virtual persona agent operating in a pre-defined scenario. The virtual persona agent is adapted to engage in a communicative dialog with the trainee. The system further includes a feedback module communicably coupled to the user device and the persona configurator module. The feedback module hosts an evaluation agent adapted to access the communicative dialog and evaluate the same for conducting the performance evaluation. The system additionally includes an Al engine module communicably coupled Ref: IN202421070079-PCT to the persona configurator module and the feedback module. The Al engine module includes a persona large language model (LLM) adapted to configure the virtual persona agent hosted on the persona configurator module, and an evaluation LLM adapted to configure the evaluation agent hosted on the feedback module. The system also includes an orchestrator module communicably coupled to the Al engine module. The orchestrator module is an LLM configured to transmit a first parameter configurator command to the persona LLM for configuring the virtual persona agent in the pre-defined scenario and a second parameter configurator command to the evaluation LLM for configuring the evaluation agent based on a configuration command. The present invention envisages that the virtual persona agent is defined by persona parameters including at least one of identity attributes, behavioral control attributes, information architecture attributes and challenge presentation attributes. Moreover, the pre-defined scenario is defined by scenario parameters comprising at least one of scenario context attributes and calibration control attributes, and the evaluation agent is defined by evaluator parameters including at least one of skill-testing attributes and evaluation-criteria attributes. In addition, the first parameter configurator command defines the persona parameter and the scenario parameter for the virtual persona agent in the pre-defined scenario, and the second parameter configurator command defines the evaluator parameters and the contextual scenario parameters for the evaluation agent. Also, the evaluation agent is adapted to evaluate the communicative dialog by identifying relevant attributes of the scenario and the persona parameters considered by the trainee in the communicative dialog and correlating the same to its evaluator parameters and the contextual scenario parameters.

[0016]

[0011] In an embodiment of the present invention, the configuration command identifies the attributes of the persona parameters, the scenario parameters and the evaluator parameters for which weights are to be assigned in the first parameter configurator command and the second parameter configurator command.

[0017]

[0012] In another embodiment of the present invention, the identity attributes define professional identity and professional background of the virtual persona agent, behavioral control attributes define the emotional and psychological behavior of the virtual persona agent, information architecture attributes define knowledge base of the virtual Ref: IN202421070079-PCT persona agent and knowledge sharing conditions, and challenge presentation attributes define obstacles and objections that the virtual persona agent presents to the trainee.

[0018]

[0013] In yet another embodiment of the present invention, the scenario context attributes define business context for the virtual persona agent to engage in the communicative dialog and calibration control attributes define a level of difficulty and challenge of the communicative dialog.

[0019]

[0014] In still another embodiment of the present invention, the skill-testing attributes define criteria for identifying professional competencies to be evaluated during the communicative dialog and evaluation-criteria attributes define a rubric framework for evaluation of the trainee based on engagement in communicative dialog.

[0020]

[0015] In still another embodiment of the present invention, the configuration command is received from an administrative user in natural language.

[0021]

[0016] In still another embodiment of the present invention, the orchestration module performs a parameter extraction in the configuration command to identify and segregate relevant attributes of at least one of the persona parameters, the scenario parameters and the evaluator parameters intended for the communicative dialog.

[0022]

[0017] In still another embodiment of the present invention, the persona LLM assigns weights to pertinent attributes of the persona parameters and the scenario parameters based on the first parameter configurator command for configuring the virtual persona agent in the pre-defined scenario.

[0023]

[0018] In still another embodiment of the present invention, the evaluation LLM 132 assigns weights to pertinent attributes of the evaluator parameters and the contextual scenario parameters based on the second parameter configurator command for configuring the evaluation agent.

[0024]

[0019] In still another embodiment of the present invention, the orchestrator module is configured to ensure that persona parameters are not transmitted to the evaluation LLM and the evaluator parameters are not transmitted to the persona LLM, thereby preventing circular bias between the persona LLM and the evaluation LLM. Ref: IN202421070079-PCT

[0025]

[0020] In still another embodiment of the present invention, the persona LLM and evaluation LLM are pre-trained and fine-tuned on annotated conversation datasets tagged with persona parameters, scenario parameters, evaluator parameters, and scoring rationales, such that the persona LLM maintains role consistency and the evaluation LLM performs structured rubric -based evaluation.

[0026]

[0021] In another embodiment of the present invention, disclosed is a method for adaptive performance evaluation of a trainee. The method includes act of receiving, by an orchestrator module, a configuration command specifying persona parameters, scenario parameters, and evaluator parameters; act of extracting, by the orchestrator module, the persona parameters, the scenario parameters, and the evaluator parameters from the configuration command; act of formulating, by the orchestrator module, a first parameter configurator command for a persona LLM and a second parameter configurator command for an evaluation LLM by defining relevant attributes of the persona parameters, the scenario parameters, and the evaluator parameters extracted from the configuration command; act of transmitting, by the orchestrator module, the first parameter configurator command to the persona LLM for configuring a virtual persona agent operating in a predefined scenario, and the second parameter configurator command to the evaluation LLM to configure an evaluation agent; act of configuring, by the persona LLM, the virtual persona agent and the evaluation agent by the evaluation LLM; act of hosting the virtual persona agent at a persona configurator module to engage in a communicative dialog with a trainee on a user device, and the evaluation agent at a feedback module to independently assess the communicative dialog; act of engaging, by the virtual persona agent, in the communicative dialog with the trainee via a user device; act of analyzing, by the evaluation agent, the communicative dialog to identify relevant attributes of the scenario and the persona parameters therein and assigning points based on scenario parameters; and act of generating, by the evaluation agent, feedback comprising at least one of evaluation scores, strengths, weaknesses, and suggested improvements for the trainee. The virtual persona agent is defined by the persona parameters comprising at least one of identity attributes, behavioral control attributes, information architecture attributes and challenge presentation attributes. The pre-defined scenario is defined by the scenario parameters comprising at least one of scenario context attributes and calibration control attributes, and the evaluation Ref: IN202421070079-PCT agent 133 is defined by evaluator parameters including at least one of skill-testing attributes and evaluation-criteria attributes. Further, the first parameter configurator command defines the persona parameter and the scenario parameter for the virtual persona agent in the pre-defined scenario, and the second parameter configurator command defines the evaluator parameters and contextual scenario parameters for the evaluation agent.

[0027]

[0022] The afore-mentioned objectives and additional aspects of the embodiments herein will be better understood when read in conjunction with the following description and accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. This section is intended only to introduce certain objects and aspects of the present disclosure, and is therefore, not intended to define key features or scope of the subject matter of the present disclosure

[0028] BRIEF DESCRIPTION OF THE DRAWINGS

[0029]

[0023] The figures mentioned in this section are intended to disclose exemplary embodiments of the claimed system and method. Further, the components / modules and steps of a process are assigned reference numerals that are used throughout the description to indicate the respective components and steps. Other objects, features, and advantages of the present disclosure will be apparent from the following description when read with reference to the accompanying drawings.

[0030]

[0024] Figure 1 is a block diagram of system for adaptive performance evaluation of a trainee through scenario simulation, in accordance with an embodiment of the present invention;

[0031]

[0025] Figure 2 is a block diagram of an Al engine module implemented din the system of Fig. 1;

[0032]

[0026] Figure 3 is a flow chart of a method implemented in the system of Fig. 1.

[0033]

[0027] Like reference numerals refer to like parts throughout the description of several views of the drawings. Ref: IN202421070079-PCT

[0034] DETAILED DESCRIPTION OF THE DISCLOSURE

[0035]

[0028] This section is intended to provide explanation and description of various possible embodiments of the present disclosure. The embodiments used herein, and various features and advantageous details thereof are explained more fully with reference to non-limiting embodiments illustrated in the accompanying drawings and detailed in the following description. The examples used herein are intended only to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable the person skilled in the art to practice the embodiments used herein. Also, the examples / embodiments described herein should not be construed as limiting the scope of the embodiments herein. Corresponding reference numerals indicate corresponding parts throughout the drawings.

[0036]

[0029] The present disclosure relates to a system and a method holistically testing and evaluating a trainee for a professional skill that the trainee is desired to exhibit for efficient fulfillment of his professional duties. The trainee may be a person employed in or working in a professional field that requires strong interpersonal and communication skills. Few non-limiting examples of such persons could be a sales representative for a product / services company, a student counsellor, a negotiator for licenses, a customer service representative, and the like. Particularly, the present invention is directed towards training of personnel that are required to interact with different target persons, such as potential clients for their organization, as a part of their professional duties and their interactions with the target persons remain the core activity for serving the target persons. The present invention has been conceived in view of the realization that the trainees’ success depends on the effectiveness of their training and the feedback that is provided to them. The working of the present invention will be evident from the ensuing description of the envisaged system and the method employed therein.

[0037]

[0030] Fig. 1 illustrates a system 100 for adaptive performance evaluation of a trainee through scenario simulation, in accordance with an embodiment of the present invention. As shown, at least one trainee 105 undertakes a testing and evaluation activity using a virtual environment enabled through his / her user device 110, such as a phone, a tablet, a notebook, a personal computer and the like. The trainee 105 is configured to be communicably connected to a persona configurator module 120 through his / her user device Ref: IN202421070079-PCT

[0038] 105 over a wired or a wireless link. The persona configurator module 120 is configured to engage the trainee 105 in a communicative dialog in respect of one or more professional skills that the trainee is desired to possess by replicating a real- world persona, such as that of a potential client who wants to buy a product, a licensee, a potential or existing client and the like, by configuring a virtual persona agent 121 operating in a specific pre-defined scenario, such as situation where the potential client is evaluating competitive products, or a licensee is seeking a software license on an alternative software service while currently using another service and the like, to help in assessing the trainee’s current capabilities in respect of the desired one or more professional skills, such as sales, negotiation and likes thereof. The pre-defined scenario, in the present context, refers to a specific operational limitation or constrains under which the communicative dialog is to be carried out. The persona configurator module 120 is adapted to deliver training content in the form of the communicative dialog, including responses from the trainee 105 and counter-responses from the virtual persona agent 121, which replicates a real-world negotiation / discussion, such as between the sales representative and the potential client, as will be explained in the ensuing description. Both of the user device 110 and the persona configurator module 120 may have audio-video communication capabilities to enable the communicative dialog to be textual, verbal or a combination thereof. The system 100 also includes a feedback module 125, communicably coupled to the persona configurator module 120 and, optionally, the trainee’s device 110. The feedback module 125 is configured to assess, at least, the communicative dialog between the persona configurator module 120 and the trainee 105 and, in particular, the responses of the trainee 105 during the communicative dialog in context to the counter-responses from the virtual persona agent 121 to evaluate the competency level of the trainee 105 in respect of the desired one or more professional skills. Based on the evaluation, the feedback module 125 is adaptive to provide constructive feedback to the trainee 105 about his / her competency level, thereby enabling the trainee 105 achieve the desired competency level in the one or more professional skills. It may be noted that the persona configurator module 120 and the feedback module 125 are communicably coupled to each other such that the communicative dialog between the trainee 105 and the persona configurator module 120 is accessible to the feedback module Ref: IN202421070079-PCT

[0039] 125. However, both the persona configurator module 120 and the feedback module 125 are independent systems which work with each other in a collaborative manner when required.

[0040]

[0031] In addition to the afore-said modules, the system 100 also includes an Al engine module 130 communicably coupled to the persona configurator module 120 and the feedback module 125 to control the operation of both the said modules. Particularly, the Al engine module 130 includes a persona large language model (LLM) 131 that is trained to configure the desired virtual persona agent for the specified scenario to engage in the communicative dialog with the trainee by hosting the virtual persona agent 121 on the persona configurator module 120. In addition, the Al engine module 130 includes an evaluation LLM 132 that enables the feedback module 125 to analyze the communicative dialog between the configured virtual persona agent 121, hosted on the persona configurator module 120, and the trainee, and evaluate the performance thereof. Additionally, the system 100 includes a data storage and management module 135, which is a central repository for all types of data used for the different modules. Another important module of the system 100 is an orchestrator module 140 that is communicably coupled to the Al engine module 130 and is adapted to provide relevant information to the Al engine module 130 to enable the same to configure the virtual persona agent 121 for the specified scenario, and also to instruct the feedback module 125 to evaluate the performance of the trainee 105 in relation to the specified scenario. The broad purpose of the system 100 may be evident from exemplary use case. In particular, if the trainee 105, an accounts manager in a product company, is to be trained for making effective sales pitches to clients, the persona configurator module 120 will host the virtual person agent 121, configured by the persona LLM 131 of the Al engine module 130, of an uninterested potential client in a scenario where the potential client is already using a rival product that is priced lower. The persona configurator module 120 will engage in a communicative dialog with the trainee 105 to replicate a usual conversation pattern of a potential client in the specified scenario. The trainee 105 engages in the communicative dialog by providing responses, based on his current proficiency level, to the virtual persona agent’s counter-responses. The entire communicative dialog is accessed by the feedback module 125, which independently analyzes the same and evaluates the current proficiency level, such as beginner, intermediate or highly skilled, of the trainee 105. The feedback module 125 then provides Ref: IN202421070079-PCT a rubric matrix to the trainee 105 based on his performance as feedback on the improvements that are desired. The detailed working of the present invention will be evident from the ensuing description of all the modules and a detailed explanation of interdependent working thereof.

[0041]

[0032] For the purpose of the working of the present invention, it is envisaged that the virtual persona agent 121 needs to digitally replicate a real-world human having a characteristic persona and behavior owing to his / her professional knowledge, experience and emotional background. To enable this replication, the virtual persona agent 121, being a digital replica, is defined based on one or more of persona parameters including, but not limited to, identity attributes, behavioral control attributes, information architecture attributes and challenge presentation attributes. The identity attributes pertain to a plurality of identity elements that define the professional identity and background of the virtual persona agent 121 akin to those of real-world professionals in different industries. A first identity element of the plurality of identity elements defines a professional role for the virtual persona agent, such as, but not limited to, specific job title, hierarchical position within an organization, years of experience in the current position, scope of responsibilities, reporting relationships, and the like that control how the persona interacts during the communicative dialog. A second identity element of the plurality of identity elements defines an industry context for the virtual persona agent 121, such as, but not limited to, sector-specific knowledge, terminology usage pattern, awareness of industry trends and challenges, competitive landscape understanding, regulatory concerns and the like, that control the limits of the virtual persona agent’s knowledge of the surrounding professional environment he is operating in. A third identity element of the plurality of identity elements defines the authority level for the virtual persona agent 121, such as, but not limited to, decision making power, budget approval limits, need for consensus or higher approval, influence over organizational decisions and the like. A fourth identity element of the plurality of identity elements defines the cultural background for the virtual persona agent 121, such, but not limited to, geographic origin, language patterns and accent influences, business etiquette preferences, communication style norms, cultural sensitivities and the like that control the interaction pattern of the virtual persona agent 121. Ref: IN202421070079-PCT

[0042]

[0033] Further, the behavioral control attributes include a plurality of control elements that define the emotional and psychological behavior of the virtual persona agent 121, akin to a real-world person, in any type of communication. A first control element of the plurality of control elements defines the emotional state for the virtual persona agent 121, such as, but not limited to, starting mood (positive / neutral / negative), emotional volatility defining mood changes, triggers that cause emotional shifts, recovery patterns after negative interactions and the like. A second control element of the plurality of control elements defines the patience thresholds for the virtual persona agent 121, such as, but not limited to, initial patience level, degradation rate based on poor response or repetition, specific triggers that accelerate patience loss, minimum patience floor below which conversation may end and the like. A third control element of the plurality of the control elements defines the trust building patterns for the virtual persona agent 121, such as, but not limited to, starting trust level (which is low for new relationships), trust building velocity based on trainee effectiveness, specific actions that build or destroy trust and the like, that controls information disclosure. A fourth control element of the plurality of control elements defines the resistance levels exhibited by the virtual persona agent 121, such as, but not limited to, degree of pushback against proposals, skepticism towards claims, openness to new ideas, conditions under which resistance decrease or increases during conversation and the like. A fifth control element of the plurality of control elements define response timing for the virtual persona agent 121, such as, but not limited to, natural conversation pacing, thinking pauses for complex questions, quick responses for emotional reactions, silence usage as a tactical tool and the like.

[0043]

[0034] Furthermore, the information architecture attributes include a plurality of ontology elements that define the knowledge base for the virtual persona agent 121 and the conditions under which the knowledge is shared. A first ontology element of the plurality of ontology elements defines the public information knowledge base, such as, but not limited to, facts the persona can share freely like company name, general role description, obvious market conditions, widely known challenges and the like, without requiring any effort or trust to obtain. A second ontology element of the plurality of ontology elements defines gated information layer for the virtual persona agent 121, such as, but not limited to, actual budgets requiring high trust level, decision timelines needing business case Ref: IN202421070079-PCT establishment, competitive situations requiring reciprocal disclosure and the like, pertaining to valuable information that requires specific conditions to be shared. A third ontology element of the plurality of ontology elements defines hidden information layer for the virtual persona agent 121, such as, but not limited to, internal political dynamics, personal career concerns, undisclosed stakeholder influences, confidential strategic plans and the like, pertaining to critical facts the virtual persona agent 121 knows but cannot disclose under any circumstances or approach. A fourth ontology element of the plurality of ontology elements defines false information layer pertaining to the virtual persona agent 121, such as, but not limited to, deliberate misinformation used to test trainee’s diligence, such as incorrect competitor prices to see if trainee verifies, false timeline pressures to test pushbacks, exaggerated problems to test discovery skills and the like. A fifth ontology element of the plurality of ontology elements defines disclosure triggers for the virtual persona agent 121, such as, but not limited to, specific conditions that cause information revelation, including trust threshold, time pressure, value demonstration requirements, reciprocal information exchange, tactical advantages and the like.

[0044]

[0035] Additionally, the challenge presentation attributes include a plurality of challenge elements that define the orchestrated obstacles, objections and difficulties the virtual persona agent 121 may present to test specific trainee skills. A first challenge element of the plurality of challenge elements defines objection types for the virtual persona agent 121, such as, but not limited to, price-related concerns about cost and value, timeline challenges about implementation speed, authority questions about decision power, trust issues about vendor credibility, technical concerns about solution fit and the like. A second challenge element of the plurality of challenge elements defines predetermined order of objection presentation, rules for when each objection surfaces, escalation patterns if objections aren't addressed, resolution conditions for each challenge and the like. A third challenge element of the plurality of challenge elements defines intensity modulation for the virtual persona agent 121, such as, but not limited to, starting intensity level for each objection, factors that increase or decrease intensity, persona emotional state influence on delivery, maximum intensity caps to maintain realism and the like. A fourth challenge element of the plurality of challenge elements defines recovery opportunities for the virtual persona agent 121, such as, but not limited to, windows where trainee can address concerns, Ref: IN202421070079-PCT partial credit for incomplete responses, second chance triggers after initial failure, coaching moments where persona provides hints and the like. A fifth challenge element of the plurality of challenge elements defines success conditions for the virtual persona agent 121 , such as, but not limited to, specific responses that resolve objections, evidence required for persona satisfaction, partial resolution possibilities, permanent failure triggers that end opportunities and the like.

[0045]

[0036] The present invention envisages that the virtual persona agent 121 is selectively configurable to have desired professional, behavioural, and personal attributes to replicate traits of any type of real-world person, such as a difficult to please client, a rude negotiator, an easy-going customer and the like, operating in different industry sectors and scenarios. For this, the present invention envisages to define the scenario using one or more scenario parameters including, but not limited to, scenario context attributes and calibration control attributes. The scenario context attributes include a plurality of situation elements pertaining to environmental and situational factors that establish the business context and realism of a situation surrounding the communicative dialog. A first situation element of the plurality of situation elements defines the business situation under which the communicative dialog needs to happen, such as, but not limited to, deal type (like new sale, renewal, upsell), relationship stage (like cold, warm, existing), urgency factors (like end of quarter, budget deadline), stakes involved (like strategic account, competitive situation) and the like. A second situation element of the plurality of situation elements defines the market conditions surrounding the communicative dialog, such as, but not limited to, industry growth or declining trends, competitive pressure intensity, pricing environment (like premium, commodity), regulatory factors affecting decisions, economic climate impact and the like. A third situation element of the plurality of situation elements defines the organizational context, such as, but not limited to, company size and type, industry vertical specifics, geographic and cultural factors, technological maturity level, business model characteristics and the like. A fourth situation element of the plurality of situation elements defines time pressures in any communicative dialog, such as, but limited to, decision timeline constraints, budget cycle considerations, competitive bid deadlines, implementation requirements, seasonal factors affecting urgency. A fifth situation element of the plurality of situation elements defines the success implications in a communication, Ref: IN202421070079-PCT such as, but not limited to, business impact of success or failure, career implications for persona, organizational consequences, market position effects, relationship outcomes and the like.

[0046]

[0037] Further, the calibration control attributes include a plurality of calibration control elements that are used to adjust the overall difficulty, complexity and challenge level of the communicative dialog. A first calibration control element of the plurality of calibration control elements defines a global difficulty setting for the communicative dialog, such as, but not limited to, overall challenge level (from beginner to expert) affecting all other parameters proportionally, determining base resistance levels, setting complexity thresholds and the like. A second calibration control element of the plurality of calibration control elements defines experience level adjustments of the communicative dialog, such as, but not limited to, modifications based on trainee seniority, prior training performance, role-specific expectations, industry experience factors, learning curve positioning and the like. A third calibration control element of the plurality of calibration control elements defines the complexity modulation levels of the communicative dialog, such as, but not limited to, number of simultaneous challenges, information complexity levels, stakeholder involvement, technical depth requirements, political dynamics intensity and the like. A fourth calibration control element of the plurality of calibration control elements defines support made available to the trainee during the communicative dialog, such as, but not limited to hint frequency and clarity, coaching intervention thresholds, error tolerance levels, recovery opportunity frequency, guidance explicitness, and the like. A fifth calibration control element of the plurality of calibration control elements defines the time constraints applicable to the communicative dialog, such as, but not limited to, conversation duration limits, response time pressures, decision deadlines within scenario, scenario phase transitions, pacing expectations in communication and the like.

[0047] [ 381 As already mentioned earlier, the virtual persona agent 121 is configurable by the Al engine module 130. Fig. 2 illustrates a block diagram of the Al engine module 130 that further includes the persona LLM 131 and the evaluation LLM 132. It may be noted that the present invention envisages that both the persona LLM 131 and the evaluation LLM 132 are independently configured and operable to achieve the aim of preventing circular bias creeping in the predictions made by the two LLMs. The present Ref: IN202421070079-PCT invention envisages that the persona LLM 131 is trained on training data pertaining to the persona parameters and the scenario parameters so as to configure the virtual persona agent 121 that can authentically converse like a real-world person. The persona LLM 131 is expected to maintain character consistency during the communicative dialog while exhibiting realistic human behavioral pattern. Accordingly, the persona LLM 131 is firstly pre-trained on professional recorded conversations that are annotated with tags identifying the persona parameters and scenario parameters. For instance, each training conversation may be tagged with specific identity attributes (such as "procurement manager" or "technical evaluator"), industry context markers (such as "enterprise software" or "manufacturing"), hierarchical position indicators (such as "senior level" or "C-suite"), and cultural background markers that influenced the conversation dynamics. The conversation will also be tagged with behavioral control attributes to identify the moment -by-moment emotional state labels showing mood transitions, patience degradation markers indicating where frustration increased, trust evolution indicators showing relationship development, resistance pattern tags identifying pushback moments, and response timing data capturing natural conversation flow. In addition, the conversations will be tagged with information architecture attributes, like with disclosure types (public, gated, or hidden), trigger events that caused revelation, trust levels at disclosure moments, successful discovery techniques used, and instances of strategic information withholding. Moreover, the conversations will be tagged with challenge presentation attributes to identify the objections and obstacles encountered during the conversation, like objection type classifications, intensity level measurements, escalation or de-escalation points, resolution techniques that worked, and failure points where challenges weren't overcome.

[0048]

[0039] Based on the training, the persona LLM 131 creates persona template and systematically organizes the identity and behavioral control attributes into reusable configurations. In an embodiment, the persona templates may be tagged with plurality of identity elements, such as different role taxonomies covering hundreds of professional positions, each with associated authority levels, typical responsibilities, common concerns, and standard interaction patterns; comprehensive industry-specific attributes for mapping industry verticals including typical vocabulary, common pain points, regulatory awareness, competitive dynamics, and business cycle patterns; organizational level definitions of Ref: IN202421070079-PCT hierarchical positions from individual contributor through C-suite, with associated decision-making patterns, approval requirements, political considerations, communication styles; experience level indicators like sophistication scales based on years in role, industry exposure, situation familiarity, negotiation expertise, and professional maturity; and cultural behavior patterns like regional and cultural communication preferences, business etiquette expectations, relationship building approaches, trust development patterns, and conflict resolution styles. Additionally, the persona templates may be tagged with the plurality of control elements, such as emotional state machines like complex emotional models with multiple states, transition triggers between states, intensity modulation within states, recovery pathways from negative states, and personality-influenced baselines; patience degradation models like mathematical formulas for patience reduction, acceleration factors based on triggers, floor values preventing unrealistic behavior, recovery mechanisms allowing patience restoration, and personality modifiers affecting rates; trust building algorithms like trust calculation formulas based on multiple inputs, positive and negative trust events, trust momentum effects, trust gates for information disclosure, and relationship stage influences; resistance pattern libraries like collections of pushback behaviors, escalation sequences when challenged, de-escalation triggers for cooperation, personality-based resistance styles, and situational resistance modifiers; response timing patterns like natural conversation pacing models, thinking pause insertion algorithms, emotional response acceleration, strategic silence usage, and personality- influenced timing.

[0049]

[0040] The training of persona LLM 131 is taken a step further by fine tuning the same in several phases. In the first phase, the model is trained to learn fundamental character embodiment through exposure to millions of conversation examples. The training is carried out with an emphasis on maintaining consistent identity throughout extended dialogues with special focus on expressing personality through vocabulary, syntax, and conversational patterns. The training also involves development of state management capabilities for tracking emotional and mental conditions and establishment of baseline behavioral patterns that can be modified by configuration. The next phase of training involves learning of the model to embody diverse professional roles authentically, maintaining role-appropriate knowledge and concerns, expressing authority levels Ref: IN202421070079-PCT appropriately, and reflecting cultural backgrounds naturally. The model also trains on managing behavioral traits such as mastering emotional state transitions, implementing patience degradation realistically, building trust through conversational actions, exhibiting resistance patterns naturally, and timing responses appropriately. This phase also involves training about trait of sharing information like strategic information disclosure, implementing gating conditions properly, maintaining information consistency, deploying misdirection tactically, and protecting hidden information. Additionally, the model is also trained on putting forth challenges in a conversation like presenting objections naturally within conversation flow, escalating intensity based on context, providing recovery opportunities appropriately, and maintaining challenge consistency. The final phase involves training about how the scenario parameters may influence the behavior of the virtual persona agent 121 and also about calibration of difficulty levels in a conversation using calibration control attributes of the scenario parameters.

[0050]

[0041] Moving ahead, as already described, the Al engine module 130 includes an evaluation LLM 132 that is configured to enable the feedback module 125 to analyze the communicative dialog between the virtual persona agent 121 hosted on the persona configurator module 120 and the trainee 105. In an embodiment of the present invention, the evaluation LLM 132 configures an evaluation agent 133, which is hosted on the feedback module 125 to carry out the analysis of the communicative dialog with specific context to the pre-defined scenario. This is important because the trainee is expected to respond differently in different real-world scenarios, and thus the evaluation strategy should have a contextual reference to the pre-defined scenario. To enable this, the present invention envisages that the evaluation agent 133 is defined by evaluator parameters including at least one of skill-testing attributes and evaluation-criteria attributes, as will be described now. The skill-testing attributes include a plurality of skill elements that define the criteria for identification and specification of exact professional competencies being developed and measured during the communicative dialog. A first skill element of the plurality of skill elements defines dimensions for primary skill being evaluated from the communicative dialog and may include, but not limited to, core competencies that are the main focus of training, such as negotiation for sales scenarios, empathy for customer service, leadership for management training, or technical explanation for engineering roles Ref: IN202421070079-PCT and the like. A second skill element of the plurality of skill elements defines dimensions for secondary skills that support competencies that enhance primary skills, such as active listening supporting negotiation, rapport building supporting sales, or clear communication supporting technical discussions and the like. A third skill element of the plurality of skill elements defines observable behaviors of the trainee during the communicative dialog and may include specific actions that demonstrate skill mastery, such as paraphrasing for active listening, value reframing for negotiation, emotional acknowledgment for empathy, or structured explanation for technical communication and the like. A fourth skill element of the plurality of skill elements defines the learning objective expectations from the evaluation, such as measurable outcomes the training aims to achieve, improvement targets from baseline performance, specific behavior changes desired, competency level advancement goals. A fifth skill element of the plurality of skill elements defines the supportive interconnections between different skills, combined skill demonstrations, skill prerequisite relationships, and synergistic skill effects on outcomes and the like.

[0051]

[0042] Further, the evaluation-criteria attributes include a plurality of evaluation elements that defines a comprehensive rubric framework defining how trainee performance is measured, scored, and translated into actionable feedback. A first evaluation element of the plurality of evaluation elements defines rubric specifications, such as, but not limited to, detailed scoring guides for each skill dimension, performance level descriptors from failing to expert, specific indicators for each performance level, and partial credit rules for incomplete demonstrations. A second evaluation element of the plurality of evaluation elements defines evidence requirements for evaluation, such as, but not limited to, minimum number of observations needed per skill, quality thresholds for valid evidence, types of evidence accepted (verbal, strategic, behavioral), and evidence weighting based on clarity and relevance, and the like. A third evaluation element of the plurality of evaluation elements defines scoring mechanisms for evaluation such as, but not limited to, mathematical formulas for score calculation, weight distribution across skill dimensions, bonus point opportunities for exceptional performance, penalty applications for critical errors, and the like. A fourth evaluation element of the plurality of evaluation elements defines performance benchmarks, such as, but not limited to, industry-standard performance expectations, role-specific baseline requirements, experience-adjusted Ref: IN202421070079-PCT standards, and percentile rankings against peer groups. A fifth evaluation element of the plurality of evaluation elements defines feedback templates for providing evaluation outcome, such as, but not limited to, structured formats for delivering assessment results, language complexity based on trainee level, balance between criticism and encouragement, specific improvement recommendations and the like.

[0052]

[0043] To enable the evaluation LLM 132 to configure the evaluation agent to perform the evaluation of the communicative dialog, it needs to be pre-trained with training data including conversations that are tagged with comprehensive expert annotations. In particular, the conversations are tagged by experts such that different instances of skill demonstration are annotated with appropriate skill-testing attributes, quality ratings for each skill demonstration, timing information, context information pertaining to different scenarios, partial credit justifications and missed opportunity identifications. Additionally, the conversations are also tagged with evaluation-criteria attributes to define the complete scoring rationales showing how experts applied rubrics, evidence citations supporting each score, weight distribution decisions, contextual adjustment explanations, and feedback priority determinations. Thus, the evaluation LLM 132 is trained to evaluate conversations happening under different scenarios. Consequent to the pre-training step, the evaluation LLM 132 builds a comprehensive rubric library for providing guidance about structured implementation of the evaluator parameters for carrying out the evaluation under different pre-defined scenarios. The rubric library has a defined framework for skill testing mappings including definitions for skill dimension having clear, observable definitions for each measurable skill, behavioral indicators that demonstrate proficiency, anti-patterns that indicate skill absence, progression markers showing improvement, and mastery indicators for expertise; observable behavior catalogs having comprehensive lists of actions that indicate skill presence, verbal patterns suggesting competency, strategic choices revealing understanding, timing indicators showing sophistication, and recovery behaviors demonstrating resilience; demonstration requirements having minimum observation counts for confidence, quality thresholds for valid evidence, context requirements for fair assessment, consistency expectations across conversation, and duration requirements for sustained demonstration; and competency progression models outlining skill development pathways from novice to expert, intermediate milestone definitions, plateau identification Ref: IN202421070079-PCT patterns, breakthrough indicators, and regression warning signs. The rubric library additionally includes specifications for evaluation criteria including performance level descriptors, such as detailed behavioral descriptions for each performance tier, clear differentiation between adjacent levels, specific examples for each level, common misconceptions at each tier, and transition indicators between levels; evidence quality standards, such as criteria for high-quality evidence, ambiguous evidence handling rules, contradictory evidence resolution, evidence weighting principles, and confidence threshold requirements; scoring calculation methods defining the mathematical formulas for dimensional scores, aggregation methods for overall scores, bonus and penalty applications, normalization procedures, and rounding rules; contextual adjustment frameworks defining the industry-specific performance expectations, role -based standard modifications, experience-level calibrations, situational difficulty factors, and cultural consideration adjustments.

[0053]

[0044] After the rubric library has been built, the evaluation LLM 132 is fine tuned in several phases. In the first phase, the LLM 132 learns to observe the communication without interfering. In particular, the model learns to process conversations as continuous evidence streams and training on identifying skill demonstrations in natural conversation. The model develops understanding of pattern recognition for complex multi -turn behaviors and learns to maintain provisional assessments throughout interaction while mastering the separation between observation and intervention. In the second phase, the model is trained on element specific evaluation and specifically on skill recognition by learning to identify diverse skill demonstrations accurately, distinguishing between similar skills, recognizing partial demonstrations, tracking skill progression, and identifying missed opportunities. The model also learns mastering consistent rubric interpretation, applying scoring rules uniformly, calculating partial credit fairly, weighing evidence appropriately, and generating scores transparently. The final phase of the fine tuning includes contextual calibration by training on how the scenario parameters and the scenario context affects evaluation standards. The model also learns how the calibration-control attributes affect scoring. The model additionally learns to understand context-appropriate performance expectations, to master the balance between absolute and relative assessment and to develop nuanced feedback based on calibration factors. Further, for configuring the Ref: IN202421070079-PCT evaluation agent 133, the evaluation LLM 132 defines the evaluator parameters with specific context to the pre-defined scenario. This means that the pre-defined scenario is same for the virtual persona agent and the evaluation agent, the context of the pre-defined scenario will differently configure the virtual persona agent and the evaluation agent 133. This means that the virtual persona agent and the evaluation agent 133 may have some common attributes, albeit with different weights, for the scenario parameters, the evaluation LLM 132 may additionally consider certain other attributes of the scenario parameters based on the pre-defined scenario while configuring the evaluation agent 133. Thus, the scenario parameters for the evaluation LLM 132 will be referred to as contextual scenario parameters. The term contextual scenario parameters is defined as a subset or transformation of the scenario parameters. The contextual scenario parameters may exclude persona parameters and evaluator-inapplicable fields, and may normalize, map, or redact scenario fields to align with the scoring rubric and evidence extraction

[0054]

[0045] Further, as already disclosed, the virtual persona agent (operable in a specific scenario) and the evaluation agent 133 are assigned their roles by persona LLM 131 and the evaluation LLM 132, respectively. However, the instruction for configuration of the virtual persona agent and the evaluation agent is generated by the orchestrator module 140. The orchestrator module 140 is another LLM that identifies the correct combination of persona and scenario parameters for the persona LLM 131 to configure the virtual persona for evaluating the trainee 105, and evaluator parameters and contextual scenario parameters for the evaluation LLM 132 to evaluate the trainee 105 based on the communicative dialog. To enable this functionality, the orchestrator module 140 may receive a natural language input from an administrative / authorized user of the system 100 to configure a testing environment through the communicative dialog by deploying a specific type of virtual persona agent working in a specific scenario, and to evaluate the said communicative dialog by a specifically configured evaluation agent 133. In an exemplary and non-limiting embodiment, the administrative user may provide the input, referred to as a configuration command, as “Create a scenario where a skeptical fleet manager with 200 vehicles needs to be convinced to switch from their current provider. They're concerned about price and reliability. Test negotiation and relationship building skills of the trainee. ” The present invention envisages that the orchestration module 140 is Ref: IN202421070079-PCT configured to identify and segregate the persona parameters and scenario parameters, as also the evaluator parameters, for providing appropriate instructions to the persona LLM 131 and the evaluation LLM 132. In particular, persona LLM 131 receives the persona and scenario parameters identified from the configuration command. Further, the evaluation LLM 132 receives the evaluator parameters and the scenario parameters from the configuration command. While both the LLMs receive the scenario parameters, the orchestrator module 140 ensures that the context under which the said parameters are to be used by both the LLMs is different, as will be explained below. The present invention envisages that for segregation, the orchestrator module 140 adheres to the following rules:

[0055] 1. All of the persona parameters, scenario parameters and the evaluator parameters must be identified from the configuration command

[0056] 2. The relevant parameters for each of the persona and evaluation LLMs must be shared with the respective LLMs in an error-free manner

[0057] 3. The scenario context attributes would have a behavioral context for virtual persona agent and performance context for evaluation agent 133.

[0058] 4. The calibration control attributes are used for defining the complexity / resistance level for the virtual persona agent but for the evaluation agent 133, those attributes relate to the strictness / normalization for evaluation.

[0059] 5. Evaluator context. To score scenario-aware competencies, the evaluation agent receives contextual scenario parameters — a need-to-know subset or transformation of the scenario parameters used by the persona agent. These parameters provide just enough context to interpret the rubric while preserving parameter separation. The contextual scenario parameters include: (i) task objective and success criteria; (ii) constraints (time, tone, compliance, allowed / forbidden tools); (iii) domain metadata (industry, locale, risk class, difficulty); and (iv) reference facts or answer keys required for correctness checks. They exclude persona parameters (style, tactics, prompt wording, seed examples) and any fields that could steer the persona or leak evaluator strategy. When a scenario detail could bias evaluation (e.g., “counterparty is uncooperative”), it is conveyed as a normalized tag (e.g., counterparty_resistance=high) rather than raw persona instructions. Ref: IN202421070079-PCT

[0060]

[0046] To be able to achieve this, the orchestrator module 140 is trained on thousands of complete examples of complex statements defining real-world professional engagements. Few non-limiting examples of training data include statements representing industry variations (examples from technology, healthcare, finance, manufacturing, retail, and more) each with unique identity and scenario context attributes; role variations ranging from individual contributors to C-suite, each with different identity attributes and skill testing attributes; skill variations covering all professional skills from negotiation to leadership, each with unique evaluator parameters; difficulty variations from beginner to expert scenarios, showing how calibration control attribute affects both the virtual persona agent and the evaluation agent 133; cultural variations having global examples showing how cultural context affects identity attributes, behavioral control attributes and scenario context attributes. In the first phase of the training, the orchestrator module 140 learns to identify the persona, the scenario and the evaluator parameters reliably. This is done by identifying recognition patterns, such as by learning hundreds of ways people describe personas (e.g. "experienced buyer," "technical evaluator," "C-suite executive," "procurement specialist"); learning behavioral descriptors (e.g. "impatient," "analytical," "relationship-focused," "skeptical," "data-driven"); learning information descriptions (e.g., “hidden budget", "undisclosed timeline," "confidential requirements"); learning challenge descriptions (e.g., "will object to," "concerns about," "needs proof of," "questions regarding"); learning skill descriptions (e.g., "improve negotiation," "develop discovery," "enhance closing"); learning evaluation descriptions (e.g., "assess ability," "score on," "provide feedback about"); learning context descriptions (e.g., "enterprise deal," "competitive market," "regulated industry"); learning calibration descriptions (e.g., "make it challenging," "appropriate for seniors," "high difficulty”). In the second phase of training, the orchestrator module 140 learns to extract elements / parameters consistently even when expressed differently. The orchestrator module 140 learns to identify implied parameters, e.g. if no behavioral control attributes have been referred to in the configuration command, then same is implied from identity attributes and scenario context attributes in the command. The orchestrator module 140 additionally learns to parse compound elements so as to identify multiple parameters contained in the configuration command. For instance, phrase "Complex negotiation in competitive market" should be identified to have skill- Ref: IN202421070079-PCT testing attributes, scenario context attributes, and calibration control attributes. In the third phase of the testing, the orchestrator module 140 learns critical splitting rules through contrastive examples, such as by using thousands of properly split configurations where the persona parameters create challenges and evaluator parameters evaluate responses. The orchestrator module 140 is also trained on examples of situations relating to what not to do, such as not sending skill-testing attributes to the persona LLM 131 for configuring virtual persona agent. The orchestrator module 140 is also trained on examples related to handling scenarios where parameters are intertwined in the configuration command but must be properly segregated. In the fourth phase of training, the orchestrator module 140 is trained to transform the scenario parameters differently for configuring the virtual persona agent and the evaluation agent 133. For instance, “very challenging” may mean high resistance, minimal initial cooperation, complex objections for configuring the virtual persona agent while it may mean strict scoring, reduced partial credit, higher performance expectations for configuring the evaluation agent 133. In the final stage, the orchestration module 140 learns to verify its own output. For this, the orchestration module 140 is trained in completeness checking for ensuring all parameters are addressed in output; alignment verification for confirming both configurations address the same scenario from their perspectives; separation validation for verifying no inappropriate element crossover; and transformation confirmation for ensuring elements transform appropriately for their destination. Each training example is also validated by human intervention for element completeness to ensure that all attributes are present and correctly identified and split correctness to ascertain that the identified attributes are routed to the relevant configuration, i.e., the virtual persona agent and the evaluation agent 133, whichever is relevant. It is also validated if each attribute is transformed appropriately for the configuration it is catering to, and the scenario remains coherent across both configurations. Moreover, it is validated that the resulting configurations would create realistic training experiences.

[0061]

[0047] Fig. 3 illustrates a method 300 implemented in the system 100 for evaluating the trainee 105. In use, at 305, the configuration command is first received at the orchestration module 140. At 310, the orchestration module 140 performs an act of parameter extraction at a very primitive level with the aim to identify the attributes and the respective elements thereof. In particular, the orchestration module 140 segregates the Ref: IN202421070079-PCT different attributes pertaining to the one or more of persona parameters the scenario parameters and the evaluator parameters present in the configuration command. For instance, in a non-limiting example, the orchestrator module 140 may identify the plurality of identity elements from the configuration command by scanning for role descriptors (such as "manager," "director," "analyst," or "specialist") to determine professional identity. The orchestrator module 140 identifies industry context through sector mentions, company type references, or market descriptions. The orchestrator module 140 detects authority indicators through decision-making references, approval requirements, or hierarchy mentions and extracting experience markers from seniority references, years of experience mentions, or sophistication indicators. The orchestrator module 140 finds cultural elements through geographic references, language mentions, or business culture descriptions. In a similar manner, the orchestrator module 140 extracts the behavioral control attributes, information architecture attributes, challenge presentation attributes, scenario context attributes, calibration control attributes, skill-testing attributes and evaluation-criteria attributes. For the sake of brevity, the detailed description of the process, which is similar to that for identity attribute described above, is not being repeated. After all the relevant attributes have been extracted from the configuration command, the orchestration module 140 at 315 formulates a first parameter configurator command for the persona LLM 131 and a second parameter configurator command for the evaluation LLM 132. Both of the first parameter configurator command and the second parameter configurator command have all the relevant attributes and respective elements thereof defined with specificity to enable specific configuration of the virtual persona agent and the evaluation agent 133. This will be explained with reference to the example of plurality of identity elements described above. Particularly, the orchestration module 140 converts the extracted role descriptions into specific character configurations including title, level, department, and function. Also, the industry context is translated into detailed industry knowledge, terminology sets, and concern profiles. The authority indicators are defined by transforming hierarchy indicators into decision-making capabilities, approval requirements, and influence levels. Further, the orchestrator module 140 converts sophistication markers into behavioral complexity, knowledge depth, and interaction sophistication and identified cultural parameters and translates cultural elements into Ref: IN202421070079-PCT communication styles, relationship approaches, and business etiquette. In a similar manner, the behavioral control attributes, information architecture attributes, challenge presentation attributes, scenario context attributes, calibration control attributes, skill-testing attributes and evaluation-criteria attributes, and their respective elements are defined with specificity.

[0062]

[0048] At 320, the orchestration module 140 transmits the first parameter configurator command to the persona LLM 131 and the second parameter configurator command to the evaluation LLM 132. At 325, the persona LLM 131 configures the virtual persona agent based on the attributes (related to persona parameters and the scenario parameters) defined with specificity in the first parameter configurator command. Particularly, specificity means that the boundary conditions for the virtual persona agent are specified to define what the persona can do and cannot do by assigning weight to the different attributes identified by the orchestrator module 140. Simultaneously, the evaluation LLM 132 configures the evaluation agent 133 based on the attributes (related to evaluator and contextual scenario parameters). At 330, the virtual persona agent is hosted at the persona configurator module 120 to engage in the communicative dialog with the trainee 105 and the evaluation agent 133 is hosted on the feedback module 125 to independently evaluate the communicative dialog. In particular, the virtual persona agent being hosted has all the background information pertaining to the persona parameters, and particularly its components, viz. identity attributes, behavioral control attributes, information architecture attributes and challenge presentation attributes, and their corresponding ingredient elements. In an embodiment of the present invention, the virtual persona agent includes all role-specific knowledge, authority parameters, and cultural characteristics; communication style including vocabulary selection, sentence construction patterns, and conversational mannerisms; industry-specific information, competitive awareness, and situational understanding. Also, the virtual persona agent has emotional state set to configured baseline, with volatility parameters and trigger sensitivities activated; the patience counter is initialized at starting value with degradation algorithm loaded and triggers armed; the trust meter starts at configured level (typically low) with building rate parameters and gate thresholds set; the resistance patterns are load with intensity settings and de-escalation conditions ready; response timing parameters are configured, including natural delays, thinking pauses, and strategic silence capabilities. Ref: IN202421070079-PCT

[0063] Additionally, all persona knowledge is sorted into public, gated, hidden, and false categories with clear boundaries; each piece of gated information is linked to specific disclosure conditions, creating complex revelation logic; disclosure triggers arm for activation when conditions are met during conversation; false information and testing elements are prepared for strategic deployment; it is verified information architecture has no conflicts or logical impossibilities. Moreover, complete objection library is loaded with all scripted challenges and variations ready for use; challenge presentation order is set according to choreography rules with timing parameters; each objection's intensity level is calibrated based on difficulty settings and persona emotional state; windows for trainee recovery is configured with partial credit possibilities and coaching moment triggers; escalation pathway is prepared for activation if trainee responses are inadequate.

[0064]

[0049] Also, the virtual persona agent is provided context of the scenario parameters, such as the scenario's business context is embedded into all persona responses and concerns; competitive landscape and economic factors that influence persona behavior and objections are provided; company- specific elements are integrated into persona knowledge and decision criteria; urgency factors affecting persona patience and decision timeline expectations are provided. Moreover, the virtual persona agent is provided details about overall challenge level that modifies all behavioral parameters proportionally; adjustment of persona’s sophistication and expectation levels based on trainee experience level; additional layers of complexity to be activated based on calibration settings; availability and coaching frequency adjust per difficulty configuration; time limits and other constraints activate according to calibration parameters.

[0065]

[0050] On similar lines, the evaluation agent 133 being hosted at the feedback module 125 has all background and contextual information regarding the scenario and the evaluation parameters, and particularly their respective attributes. In an embodiment of the present invention, the evaluation agent 133 has details that includes pattern recognition systems for main competencies with sensitivity thresholds configured; skill observation systems to capture complementary capabilities; observable action catalogs for identifying skill demonstrations in conversation; systems to monitor achievement of specific learning goals; compound skill recognition systems to identify combined competency demonstrations. Moreover, the evaluation agent 133 has scoring guides for each skill Ref: IN202421070079-PCT dimension with all performance level descriptors; mechanisms for capturing and cataloging proof of skill demonstration; mathematical scoring engines with formulas, weights, and normalization factors; appropriate feedback formats based on trainee profile and learning objectives; performance comparison standards for percentile and peer comparison calculations. Besides, the evaluation agent 133 has sector-specific performance expectations configured for contextual assessment; scenario difficulty factors that should influence scoring activate; market conditions and business situation factors that affect evaluation standards; regional and cultural factors that influence performance expectations; and defined performance standards based on role and organizational level. Additionally, the evaluation agent 133 has details of scoring modifications based on configured challenge level applying to all rubrics; performance expectations adjustments based on trainee seniority and experience; recognition systems for handling multi-faceted challenges; Adjustments for limited hint availability or coaching support; Scoring modifications for time-constrained scenarios.

[0066]

[0051] At 335, the virtual persona agent engages in the communicative dialog with the trainee replicating a real-world situation such that the virtual persona agent communicates based one or more of the defined attributes relating to, but not limited to, industry knowledge, professional experience, professional responsibilities, mood shifts, emotional traits, patience levels, resistance in communication, openness to disclose information, and the like. The communicative dialog proceeds between the trainee 105 and the virtual persona agent and is continuously shared with the evaluation agent 133.

[0067]

[0052] At 340, the evaluation agent 133 analyzes the communicative dialog to identify whether the trainee conducted the communicative dialog in recognition of the relevant attributes of the scenario and the persona parameters and evaluates the same by co-relating the identified attributes to the evaluator parameters and contextual scenario parameters. In an embodiment of the present invention, the evaluation agent 133 assesses the trainee’s recognition of the virtual persona agent’s identity attributes and his / her engagement with the virtual persona agent 121 in recognition thereof. In a non-limiting example, it may be assessed whether the trainee recognizes and appropriately addresses the virtual persona agent's professional role, understands decision-making limitations and requirements thereof., whether the trainee speaks the persona's industry language and Ref: IN202421070079-PCT address sector concerns, whether the trainee has appropriate cultural awareness and communication style matching. Based on this assessment, the evaluation agent 133 may map the trainee’s performance to skill testing attributes, such as communication and relationship building skills, of the evaluator parameters and awards points for appropriate role recognition and professional interaction in accordance with appropriate evaluationcriteria attributes of the evaluator parameters. Additionally, the evaluation agent 133 assesses the trainee’s response and evaluates the same with specific context to the trainee’s response based on recognition of behavioral control attributes of the persona parameters. In a non-limiting example, the evaluation agent 133 may assess and evaluate whether trainee recognized and adapted to virtual persona agent’s mood changes and declining patience. It may also be assessed and measured if the trainee exhibited effective rapport building and trust development strategies and how well the trainee handled pushback and skepticism. Based on this assessment, the evaluation agent 122 maps the performance to appropriate skill testing attributes, such as emotional intelligence, adaptability, and relationship building competencies, and assigns points calculated based on behavioral adaptation effectiveness in accordance with the appropriate evaluation-criteria attributes of the evaluator parameters. Moreover, the evaluation agent 133 may assess the trainee’s response and evaluates the same with specific context to recognition of information architecture attributes of the persona parameters. In another non-limiting example, the trainee may be assessed and evaluated based on the questioning strategies and information gathering approaches employed by the trainee, whether the trainee builds necessary conditions for information disclosure, whether the trainee validates information rather than accepting blindly, and trainee's own information management and reciprocal disclosure. Based on this assessment, the evaluation agent 133 maps the performance to appropriate skill testing attributes, such critical thinking, and strategic communication skills, and awards points for effective information management in accordance with the appropriate evaluation-criteria attributes of the evaluator parameters.

[0068]

[0053] In another embodiment of the present invention, the evaluation agent 133 assesses the trainee’s response and evaluates the same with specific context to the trainee’s response based on recognition of the challenge presentation attributes of the persona parameters. In a non-limiting example, the trainee may be assessed and evaluated based on Ref: IN202421070079-PCT whether trainee properly recognizes and validates concerns, uses different techniques to address objections and challenges, handles initial failure and uses second chances, exhibits appropriate persistence without being pushy and the like. Based on this analysis, the evaluation agent 133 maps the performance to appropriate skill testing attributes, such as objection handling and negotiation skills, and assigns points based on objection handling effectiveness. Further, the evaluation agent adjusts the assigned points based on an assessment of the communicative dialog based on the scenario context attributes and the calibration control attributes. In a non-limiting example, the points may be adjusted based on assessment of difficulty level in the pre-defined scenario and the points are normalized ensuring fairness regardless of challenge level. In this process, the evaluation agent 133 maintains evaluation objectivity while accounting for configured complexity.

[0069]

[0054] At 345, after the communicative dialog concludes, the evaluation agent 133 performs a comprehensive analysis of the normalized points to generate a feedback for the trainee. In particular, all the collected evidence, i.e., the assessed conversations and points assigned to attributes pursuant to the assessment, is sorted based on the skill-testing attributes identifiable in the evidence for systematic evaluation. Each piece of the collected evidence is liked to evaluation-criteria attributes with quality ratings. Any contradictory evidence is resolved through contextual analysis and weighting and a confidence score is calculated based on evidence quantity and quality. Each skill-testing attributes, identified in the evidence, is provided an individual score based on the evaluation-criteria attributes and relevant weights, signifying the importance of each skill-testing attribute, are applied. The scores are modify based on the pertinent scenario context attributes and calibrated based on calibration control attributes to account for difficulty levels relevant for the skills. A weighted score aggregate is then generated.

[0070]

[0055] At 345, based on the generated weighted score aggregate, the evaluation agent generates a feedback in which the top performing skills, based on the generated scores, are identified with specific evidence citations. Also, key development areas are highlighted with concrete examples and the trainee is provided specific inputs regarding the specific actions the trainee should take to improve performance in specific areas. Also, there may be recommendations provided for personalized development plan based on identified gaps level. Ref: IN202421070079-PCT

[0071]

[0056] From the foregoing, it may be evident that the present invention enables an adaptive performance evaluation of trainees by creating a virtual persona agent operating in a pre-defined scenario replicating a real-world person, that engages in the communicative dialog replicating real-world negotiations / discussions. The communicative dialog is independently evaluated by the evaluation agent 133 to avoid the problem of circular bias creeping in the evaluation process. The evaluation is carried out to assess the trainee’s performance in continuing the communicative dialog in recognition of and response to the virtual persona agent’s professional experience and behavioral traits. Thus, the evaluation comprehensively evaluates the trainee’s performance by considering all relevant factors under which the trainee handled the dialog. Based on the evaluation, the present invention envisages to generate feedback for the trainee to help him / her improve his performance in areas where improvement is required. As describes, the system 100 is adaptable by creating different types of virtual persona agents operating in different scenarios to cater the training process to any specific requirements and different types of trainee.

[0072]

[0057] Further, while the elements of the present invention have been described to highlight the best modes of carrying out the invention, several modifications may exist, which would still lie within the scope of the present invention. Firstly, both the persona LLM 131 and the evaluation LLM 132 may be pre-trained and fine tuned off-the-shelf LLM models. Additionally, the configuration command may be generated by the orchestrator module 140 by accessing the Human Resources Management System or any other system of an organization where the recommendations for training requirements for different personnel in the organization may have been specified or there are specific performance parameters being stored, such as sales or service performance numbers. The orchestrator module 140 is configured to obtain such details, identify the training requirements and configure the configuration command accordingly.

[0073]

[0058] The term exemplary is used herein to mean serving as an example. Any embodiment or implementation described as exemplary is not necessarily to be construed as preferred or advantageous over other embodiments or implementations. Further, the use of terms such as including, comprising, having, containing and variations thereof, is meant Ref: IN202421070079-PCT to encompass the items / components / process listed thereafter and equivalents thereof as well as additional items / components / process.

[0074]

[0059] It will be apparent that various aspects of the present disclosure may be implemented as a software and hardware (such as a processing unit) in the implementations illustrated in the figures. Additionally, it should be appreciated that according to one aspect of this embodiment, one or more computer programs that when executed perform methods of the present disclosure need not reside on a single computer or processor but may be distributed in a modular fashion amongst a number of different computers or processors to implement various aspects of the present disclosure.

[0075]

[0060] Although the subject matter is described in language specific to structural features and / or acts, it is to be understood that the subject matter defined in the claims is not necessarily limited to the specific features or process as described above. In fact, the specific features and acts described above are disclosed as mere examples of implementing the claims and other equivalent features and processes which are intended to be within the scope of the claims.

Claims

Ref: IN202421070079-PCTWhat is claimed is:

1. A system for adaptive performance evaluation of a trainee, the system comprises: a user device pertaining to the trainee; a persona configurator module 120 communicably coupled to the user device, the persona configurator module 120 configured to host a virtual persona agent operating in a predefined scenario, the virtual persona agent adapted to engage in a communicative dialog with the trainee; a feedback module 125 communicably coupled to the user device 110 and the persona configurator module 120, the feedback module 125 hosting an evaluation agent 133 adapted to access the communicative dialog and evaluate the same for conducting the performance evaluation; an Al engine module 130 communicably coupled to the persona configurator module 120 and the feedback module 125, the Al engine module comprising: a persona large language model (LLM) 131 adapted to configure the virtual persona agent hosted on the persona configurator module 120, and an evaluation LLM 131 adapted to configure the evaluation agent 133 hosted on the feedback module 125; and an orchestrator module 140 communicably coupled to the Al engine module 130, the orchestrator module 140 being an LLM configured to transmit a first parameter configurator command to the persona LLM 131 for configuring the virtual persona agent in the pre-defined scenario and a second parameter configurator command to the evaluation LLM 132 for configuring the evaluation agent 133 based on a configuration command, wherein the virtual persona agent is defined by persona parameters comprising at least one of identity attributes, behavioral control attributes, information architecture attributes and challenge presentation attributes, and wherein the pre-defined scenario is defined by scenario parameters comprising at least one of scenario context attributes and calibration control attributes, andRef: IN202421070079-PCT wherein the evaluation agent 133 is defined by evaluator parameters comprising at least one of skill-testing attributes and evaluation-criteria attributes, and wherein the first parameter configurator command defines the persona parameter and the scenario parameter for the virtual persona agent in the pre-defined scenario, and the second parameter configurator command defines the evaluator parameters and the contextual scenario parameters for the evaluation agent 133, and wherein the evaluation agent 133 evaluates the communicative dialog by identifying relevant attributes of the scenario and the persona parameters considered by the trainee in the communicative dialog and correlating the same to its evaluator parameters and the contextual scenario parameters.

2. The system as claimed in claim 1, wherein the configuration command identifies the attributes of the persona parameters, the scenario parameters and the evaluator parameters for which weights are to be assigned in the first parameter configurator command and the second parameter configurator command.

3. The system as claimed in claim 1, wherein the identity attributes define at least professional identity and professional background of the virtual persona agent, behavioral control attributes define the emotional and psychological behavior of the virtual persona agent 121, information architecture attributes define knowledge base of the virtual persona agent 121 and knowledge sharing conditions, and challenge presentation attributes define obstacles and objections that the virtual persona agent presents to the trainee.

4. The system as claimed in claim 1 , wherein the scenario context attributes define business context for the virtual persona agent to engage in the communicative dialog and calibration control attributes define a level of difficulty and challenge of the communicative dialog.Ref: IN202421070079-PCT5. The system as claimed in claim 1, wherein the skill-testing attributes define criteria for identifying professional competencies to be evaluated during the communicative dialog and evaluation-criteria attributes define a rubric framework for evaluation of the trainee based on engagement in communicative dialog.

6. The system as claimed in claim 1 , wherein the configuration command is received from an administrative user in natural language.

7. The system as claimed in claim 1, wherein the orchestration module performs a parameter extraction in the configuration command to identify and segregate relevant attributes of at least one of the persona parameters, the scenario parameters and the evaluator parameters intended for the communicative dialog.

8. The system as claimed in claim 1, wherein the persona LLM 131 assigns weights to pertinent attributes of the persona parameters and the scenario parameters based on the first parameter configurator command for configuring the virtual persona agent in the predefined scenario.

9. The system as claimed in claim 1, wherein the evaluation LLM 132 assigns weights to pertinent attributes of the evaluator parameters and the contextual scenario parameters based on the second parameter configurator command for configuring the evaluation agent.

10. The system as claimed in claim 1, wherein the orchestrator module (140) is configured to ensure that persona parameters are not transmitted to the evaluation LLM (132) and the evaluator parameters are not transmitted to the persona LLM (131), thereby preventing circular bias between the persona LLM (131) and the evaluation LLM (132).Ref: IN202421070079-PCT11. The system a claimed in claim 1, wherein the persona LLM (131) and evaluation LLM (132) are pre-trained and fine-tuned on annotated conversation datasets tagged with persona parameters, scenario parameters, evaluator parameters, and scoring rationales, such that the persona LLM maintains role consistency and the evaluation LLM performs structured rubric-based evaluation.

12. A method for adaptive performance evaluation of a trainee, the method comprising: receiving, by an orchestrator module (140), a configuration command specifying persona parameters, scenario parameters, and evaluator parameters; extracting, by the orchestrator module (140), the persona parameters, the scenario parameters, and the evaluator parameters from the configuration command; formulating, by the orchestrator module (140), a first parameter configurator command for a persona LLM (131) and a second parameter configurator command for an evaluation LLM (132) by defining relevant attributes of the persona parameters, the scenario parameters, and the evaluator parameters extracted from the configuration command; transmitting, by the orchestrator module (140), the first parameter configurator command to the persona LLM (131) for configuring a virtual persona agent (121) operating in a predefined scenario, and the second parameter configurator command to the evaluation LLM(132) to configure an evaluation agent (133); configuring, by the persona LLM (131), the virtual persona agent (121) and the evaluation agent (133) by the evaluation LLM (132); hosting the virtual persona agent (121) at a persona configurator module (120) to engage in a communicative dialog with a trainee on a user device (110, and the evaluation agent(133) at a feedback module (125) to independently assess the communicative dialog; engaging, by the virtual persona agent, in the communicative dialog with the trainee via a user device;Ref: IN202421070079-PCT analyzing, by the evaluation agent (133), the communicative dialog to identify relevant attributes of the scenario and the persona parameters therein and assigning points based on scenario parameters; and generating, by the evaluation agent, feedback comprising at least one of evaluation scores, strengths, weaknesses, and suggested improvements for the trainee, and wherein the virtual persona agent is defined by the persona parameters comprising at least one of identity attributes, behavioral control attributes, information architecture attributes and challenge presentation attributes, and wherein the pre-defined scenario is defined by the scenario parameters comprising at least one of scenario context attributes and calibration control attributes, and wherein the evaluation agent 133 is defined by evaluator parameters comprising at least one of skill-testing attributes and evaluation-criteria attributes, and wherein the first parameter configurator command defines the persona parameter and the scenario parameter for the virtual persona agent in the pre-defined scenario, and the second parameter configurator command defines the evaluator parameters and contextual scenario parameters for the evaluation agent 133.

13. The method as claimed in claim 12, wherein the identity attributes define at least professional identity and professional background of the virtual persona agent, behavioral control attributes define the emotional and psychological behavior of the virtual persona agent 121, information architecture attributes define knowledge base of the virtual persona agent 121 and knowledge sharing conditions, and challenge presentation attributes define obstacles and objections that the virtual persona agent presents to the trainee.

14. The method as claimed in claim 12, wherein the scenario context attributes define business context for the virtual persona agent to engage in the communicative dialog and calibration control attributes define a level of difficulty and challenge of the communicative dialog.Ref: IN202421070079-PCT15. The method as claimed in claim 12, wherein the skill-testing attributes define criteria for identifying professional competencies to be evaluated during the communicative dialog and evaluation-criteria attributes define a rubric framework for evaluation of the trainee based on engagement in communicative dialog.

16. The method as claimed in claim 12, wherein the persona LLM 131 assigns weights to pertinent attributes of the persona parameters and the scenario parameters based on the first parameter configurator command for configuring the virtual persona agent in the predefined scenario.

17. The system as claimed in claim 12, wherein the evaluation LLM 132 assigns weights to pertinent attributes of the evaluator parameters and the contextual scenario parameters based on the second parameter configurator command for configuring the evaluation agent 133.

18. The method as claimed in claim 12, wherein the orchestrator module (140) is configured to ensure that persona parameters are not transmitted to the evaluation LLM (132) and the evaluator parameters are not transmitted to the persona LLM (131), thereby preventing circular bias between the persona LLM (131) and the evaluation LLM (132).

Images