Big language model-based accompanying training method and device, electronic equipment and storage medium

Abstract

This application discloses a training method, device, electronic device, and storage medium based on a large language model. The method includes pre-configuring training modes, which at least include open-ended training and restricted training. The open-ended training and restricted training can run independently or be switched on demand. Responding to the input of the user to be trained, the method selects the training mode and simultaneously conducts training based on a fallback process, outputting the final training evaluation result. The fallback process ensures the continuity of the large language model training process. This application provides a training method including open-ended training, process tree restricted training, and a fallback process. The open-ended training integrates a knowledge base and a large evaluation model; the process tree restricted training includes configurable nodes and eight core functions; and the fallback process has model monitoring, self-interruption, and dialogue switching capabilities, achieving integrated training with flexible interaction, standardized control, and stable operation.

Description

Technical Field

[0001] This application relates to the field of intelligent tutoring technology, and in particular to a tutoring method, device, electronic device, and storage medium based on a large language model. Background Technology

[0002] In the context of intensifying competition in the financial industry, the communication skills and professional competence of bank account managers directly impact transaction completion rates and customer satisfaction, making efficient training a core requirement for bank operations. However, traditional training methods and existing technological solutions have significant limitations:

[0003] (1) Traditional training technology solutions commonly used in the industry

[0004] Traditional methods primarily rely on "human mentoring + offline centralized training," such as the "mentor-apprentice" training program for new client managers at a state-owned bank. Mentors simulate scenarios where clients inquire about financial products, and then provide feedback based on their personal experience. The drawbacks of this traditional approach are: 1. High resource dependence; the scale of a single training session is limited, making it unsuitable for large-scale, high-frequency training. 2. Limited scenario coverage; it only covers a few common scenarios (such as savings inquiries and credit card applications), failing to simulate complex scenarios such as high-net-worth client asset allocation and dispute resolution. 3. Subjective evaluation; there is no unified scoring standard, feedback is delayed, and client managers cannot make timely adjustments. Summary of the Invention

[0005] This application provides a training method, device, electronic device, and storage medium based on a large language model, which can meet the flexibility of real business scenarios while ensuring training quality and system stability.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a tutoring method based on a large language model, applied to a server, the method comprising:

[0008] The training mode is pre-configured, which includes at least open training and restricted training. The open training and restricted training can run independently or be switched as needed. Both the open training and the restricted training are built based on a large language model.

[0009] In response to the input of the user seeking coaching, the coaching mode is selected, and coaching is conducted based on a fallback process, which outputs the final coaching evaluation result. The fallback process is used to ensure the continuity of the coaching process for large language models.

[0010] In some embodiments, responding to the input of the user seeking coaching, selecting the coaching mode, and simultaneously conducting coaching based on a fallback process and outputting the final coaching evaluation result includes:

[0011] The target coaching mode is determined based on the input content of the user to be coached;

[0012] After the target training mode is activated, the fallback process detects one or more scenarios, including abnormal responses during the training process, training gaps after training interruption, and abnormal process connection after interruption, and ensures the continuity of the training process in the target training mode.

[0013] In some embodiments, the fallback process includes an agent self-interruption mechanism, which is used to:

[0014] Monitor the response time of each round of large language model and / or knowledge base retrieval small model;

[0015] When the response time exceeds the preset threshold, the agent internally suspends the current large language model or the retrieval of small models from the knowledge base, and records the training status of the user to be trained at the time of the interruption.

[0016] In some embodiments, the fallback process includes a fallback script library invocation mechanism, which is used for:

[0017] A backup script library is constructed, and backup scripts are pre-set according to the functional categories of actual business scenarios. The backup scripts cover the entire training process and support variable replacement to ensure the continuity of the scripts with the current training process.

[0018] Establish a dialogue matching logic. After the intelligent agent is interrupted, it reads the business function scenario, process tree node or historical dialogue keywords at the time of the interruption, and matches the optimal dialogue content from the fallback dialogue library.

[0019] In some embodiments, the fallback process includes a recovery strategy, which is used to:

[0020] The system implements a continuous backup mode, responding to the user's instruction to prioritize completing the coaching session, maintaining the backup dialogue interaction until the coaching session ends, and generating supplementary evaluations after the coaching session ends.

[0021] The system executes dynamic recovery mode in response to the user's instruction to wait for recovery. After a preset number of interaction rounds, it detects the model's running status. When the model is back to normal and usable, it switches back to intelligent coaching mode and continues the original dialogue from the interruption point, while supplementing key information in the fallback phase.

[0022] When executing the continuous fallback mode or the dynamic recovery mode, the dialogue data of the fallback phase is synchronized to the evaluation unit, which is used to generate the training evaluation results so that the final comprehensive evaluation generated when the large language model is available is based on the complete training process data.

[0023] In some embodiments, the pre-configured coaching mode includes at least open coaching and restricted coaching, wherein the open coaching and the restricted coaching can operate independently or be switched on demand, including:

[0024] The coaching mode is pre-configured as an open coaching mode, and product information from business lines is collected, organized and constructed into a knowledge base according to product name, product function, and applicable scenarios.

[0025] Using a retrieval optimization strategy, when the large language model needs to generate customer scripts or verify customer manager answers, relevant fragments from the knowledge base are retrieved as prompt words and input into the large language model.

[0026] Virtual customer roles are set up based on a large language model. The virtual customer roles include role attribute definitions and role prompt word generation. The role attribute definitions are determined based on the bank's real customer profiles. The role prompt word generation is achieved by converting the role attributes into natural language descriptions and concatenating them into role prompt words.

[0027] Manage dialogue interaction and dialogue memory by designing dialogue logic, controlling casual conversation content, and optimizing contextual memory;

[0028] The training results in an open-ended training mode are obtained by evaluating and scoring the training sessions using a large language model according to preset evaluation dimensions.

[0029] In some embodiments, the pre-configured coaching mode includes at least open coaching and restricted coaching, wherein the open coaching and the restricted coaching can operate independently or be switched on demand, including:

[0030] The coaching mode is pre-configured as restricted coaching, and key parameters of the process tree are set. These key parameters include at least the following parameter types:

[0031] Customer intent is used to define the core objectives of the node.

[0032] Node navigation is used to control the direction of the training process;

[0033] The coach's tips are designed to help account managers answer questions accurately.

[0034] Intent keywords are used to identify customer intent and the key points of the account manager's response;

[0035] The standard answer is used as the benchmark for scoring during practice sessions.

[0036] By reusing the node attribute placeholders of the key parameters of the process tree, a restricted coaching mode is obtained.

[0037] Secondly, embodiments of this application also provide a tutoring device based on a large language model, applied on a server side, the device comprising:

[0038] A configuration module is used to pre-configure coaching modes, which include at least open coaching and restricted coaching modes. The open coaching and restricted coaching modes can run independently or be switched as needed.

[0039] The selection execution module is used to respond to the input of the user to be trained, select the training mode, and conduct training based on the fallback process and output the final training evaluation result. The fallback process is used to ensure the continuity of the training process.

[0040] Thirdly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the above-described method.

[0041] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the above-described method.

[0042] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: Pre-configure a tutoring mode, which includes at least open tutoring and restricted tutoring. The open tutoring and restricted tutoring can run independently or be switched on demand. Both open tutoring and restricted tutoring are built based on a large language model. Responding to the input of the user to be tutored, the tutoring mode is selected. Simultaneously, tutoring is conducted based on a fallback process, and the final tutoring evaluation result is output. The fallback process is used to ensure the continuity of the large language model tutoring process. Through the above method, a "dual-mode + fallback" three-layer architecture design is adopted: innovatively combining open and flexible interaction with standardized process tree control, and superimposing a fallback emergency layer, it not only meets the flexibility of real business scenarios but also ensures training quality and system stability, filling the gap in existing technologies where "flexibility and standardization cannot be simultaneously achieved." Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0044] Figure 1 This is a schematic diagram illustrating the principle of the tutoring method based on a large language model in the embodiments of this application;

[0045] Figure 2This is a flowchart illustrating the tutoring method based on a large language model in an embodiment of this application;

[0046] Figure 3 This is a schematic diagram of the structure of the tutoring device based on a large language model in the embodiments of this application;

[0047] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] 1. Limitations of training resources and scenarios: Traditional manual training has a limited scale and a narrow scope of scenarios, which can only meet basic training needs.

[0050] 2. Training process and quality out of control: Traditional manual evaluation relies on personal experience, which is highly subjective and prone to errors; although there are real-time evaluation modules in related technical solutions, they lack a strong constraint mechanism on the training direction, cannot proactively bring the conversation back from the core business, and the evaluation dimensions do not cover the bank's compliance requirements, nor can they assess responsiveness. Account managers find it difficult to correct deviations in a timely manner, resulting in slow improvement in training effectiveness.

[0051] 3. System interruption risk: None of the existing solutions are designed with an emergency handling mechanism. In concurrent scenarios where bank customer managers are training online at the same time on a large scale, if problems such as large model response timeout or knowledge base retrieval anomalies occur, there is no backup plan for switching scripts or connecting processes, which may easily lead to training interruption and seriously affect the training rhythm and experience.

[0052] 4. Poor scalability: Traditional training requires retraining human trainers for new scenarios, resulting in a long preparation period; when adding banking business scenarios to patented solutions, the product knowledge base and question-and-answer base relationship needs to be rebuilt, there are no templated configuration tools, and it relies on technical personnel for configuration, making it difficult to adapt to the rapid iteration needs of banking business.

[0053] To address the aforementioned shortcomings, the core objective of this application's embodiments is to provide an intelligent coaching method that is resource-efficient, process-controllable, continuous, stable, and flexibly scalable. Specifically, it addresses the following:

[0054] 1. Limited resources and narrow application scenarios: By combining a dual-mode architecture with a bank-specific knowledge base, it supports large-scale online training simultaneously, covering most business scenarios such as corporate banking, wealth management, and lending, and meeting the training needs of the entire business line.

[0055] 2. Process out of control and poor evaluation: Strong constraints are imposed on the direction of the training to avoid deviating from the core; combined with the "real-time scoring and evaluation model for single sentences", accurate evaluation is achieved from dimensions such as compliance, professionalism, expression, and responsiveness, and timely feedback is provided to help account managers make rapid improvements.

[0056] 3. System Interruption: The system is designed with a fallback process layer to monitor the model's running status in real time. When an anomaly occurs, the intelligent agent will automatically interrupt itself and switch to the fallback script to ensure continuous training and significantly reduce the risk of interruption.

[0057] 4. Poor scalability: The process tree template design allows new scenarios to be created by simply replacing product information and scripts. Business personnel can complete the configuration independently without technical involvement, enabling rapid adaptation to business iterations.

[0058] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0059] like Figure 1 As shown, the left side is the process tree-structured functional module architecture for coaching, including the functional boundaries and hierarchical relationships of each module:

[0060] 1. Configuration management module, used as the basic settings layer, for defining and managing the rules and environment of coaching:

[0061] Core parameter configuration: Set global rules for coaching, such as difficulty level, time limit, scoring weight, etc.

[0062] Flow tree configuration: Construct the logical framework of the dialogue, specifying the branching paths and key nodes of the dialogue.

[0063] Placeholder reuse: Allows the reuse of preset variables or dialogue fragments in the process tree, improving configuration efficiency.

[0064] 2. Core functional modules, serving as the business processing layer, carry the main interaction and processing logic during the coaching process:

[0065] Customer role-playing: AI simulates a specific customer role and engages in dialogue with the user.

[0066] Real-time scoring for each sentence: Immediate quality assessment of every user's reply.

[0067] Coaching script tips: Provide guiding script suggestions when the user is stuck.

[0068] Handling small talk: Identify and properly address conversations that stray from the topic, guiding them back on track.

[0069] Script polishing: Optimize the user's expression and provide a more professional way of speaking.

[0070] Forced redirection: When the conversation deviates significantly from the flow, it will be forcibly pulled back to a preset key node.

[0071] 3. Evaluation output module, serving as the result output layer, is responsible for summarizing and presenting the entire coaching process:

[0072] Comprehensive assessment: Based on individual sentence scores and overall performance, a comprehensive ability assessment report is generated.

[0073] Report generation: Organize the evaluation results into a structured document to facilitate review and improvement.

[0074] like Figure 1 As shown, the right side illustrates the core workflow of virtual client coaching, presenting the complete coaching execution steps from start to finish in a linear fashion:

[0075] 1. The starting point of the process: the user starts the coaching system.

[0076] 2. Knowledge base construction: The system loads or updates professional knowledge, product information, industry terminology, etc., to support the dialogue.

[0077] 3. Virtual Customer Role Setting: Based on the training objectives, select or create a specific virtual customer profile (such as "a picky purchasing manager").

[0078] 4. Dialogue Interaction and Memory Management: Users engage in real-time dialogues with virtual clients, and the system records the dialogue history to maintain contextual coherence.

[0079] 5. Large Model Evaluation and Scoring: During or after the conversation, the large model scores the user's performance from multiple dimensions.

[0080] 6. Output evaluation and recommendation report: The system generates a detailed evaluation report, including score, strengths, weaknesses and improvement suggestions.

[0081] The dual-mode architecture in this application embodiment refers to the "open-ended training mode" and the "process tree-restricted training mode" included in this system. These two modes can operate independently or be switched on demand, balancing training flexibility and standardization. The open-ended training mode in this application embodiment uses an external bank-specific knowledge base. The large language model acts as a virtual customer based on the knowledge base, and the dialogue is completely open based on preset logic. After training, the evaluation model outputs multi-dimensional scores and improvement suggestions. The process tree-restricted training mode in this application embodiment uses a configurable process tree as its core framework, integrating eight core functions such as customer role-playing and real-time single-sentence scoring to achieve a standardized and controllable training mode. The fallback process layer in this application embodiment monitors the running status of the large language model and the knowledge base retrieval model in real time. When anomalies such as response timeouts occur, the intelligent agent is triggered to automatically interrupt and switch to a fallback script, ensuring an emergency level of training continuity. The process tree template extension in this application embodiment solidifies the core structure of the process tree into a template. New scenarios only require replacing the product name, script, etc., to quickly generate new training processes, enabling a batch creation extension method for intelligent agents.

[0082] This application provides a tutoring method based on a large language model, such as... Figure 2 The diagram shows a flowchart of a tutoring method based on a large language model in an embodiment of this application. The method includes at least the following steps S210 to S220:

[0083] Step S210: Pre-configure the coaching mode. The coaching mode includes at least open coaching and restricted coaching. The open coaching and the restricted coaching can run independently or be switched as needed. Both the open coaching and the restricted coaching are built based on a large language model.

[0084] The open-ended coaching service on the server side is based on a "hybrid retrieval + context optimization" approach. It uses a combination of vector retrieval and inverted index to recall knowledge base information and optimizes the model context by "retaining recent dialogue text" to balance interaction coherence and response speed.

[0085] The "parameterized configuration + dynamic role adaptation" method of the process tree-restricted coaching on the server side: supports configurable parameters such as customer intent and node jump, dynamically replaces the script and jump conditions through role tags, and realizes real-time scoring of single sentences and forced jump correction.

[0086] Step S220: In response to the input of the user to be trained, the training mode is selected, and training is carried out based on the fallback process and the final training evaluation result is output. The fallback process is used to ensure the continuity of the training process of the large language model.

[0087] On the server side, a fallback process is adopted based on the "indicator monitoring - script matching - dual-strategy recovery" method: monitoring multi-dimensional indicators such as model resources, response time, and content matching degree; matching variable fallback scripts according to scenarios and nodes; and providing two strategies, continuous fallback and dynamic recovery, to ensure the continuity of training.

[0088] Unlike traditional human-assisted training solutions, the above method significantly improves resource utilization efficiency: it supports large-scale simultaneous online training without the need for human intervention in coaching and evaluation, greatly reducing labor costs. Furthermore, training frequency is no longer limited by human resources, meeting the high-frequency training needs of banks. The method also significantly expands the scope of application scenarios: covering most business scenarios such as corporate banking, wealth management, and lending, and simulating complex scenarios such as high-net-worth client asset allocation and SME loan complaints, far exceeding the basic scenarios that can only be covered by human intervention. Finally, the method improves the objectivity and timeliness of evaluation: standardized multi-dimensional scoring avoids subjective biases in human evaluation, and real-time feedback helps account managers make immediate adjustments, resulting in a more significant improvement in training effectiveness.

[0089] Unlike solutions in related technologies, the training process and quality of the above method are controllable: in addition to real-time scoring and analysis of individual sentences and coaching prompts, it also provides strong constraints on the direction of training and timely correction. The training capabilities are comprehensively expanded through this method: the templated process tree design allows business personnel to generate new scenarios simply by designing dialogues according to the templates, enabling rapid adaptation to the high-frequency iteration needs of banking business. System stability is enhanced through this method: a new fallback mechanism ensures that even in multi-user concurrent scenarios, training interruptions can be avoided through script switching and mode adjustment, guaranteeing a smooth training experience.

[0090] In one embodiment of this application, the step of responding to the input of the user to be trained, selecting the training mode, and simultaneously conducting training based on a fallback process and outputting the final training evaluation result includes: determining the target training mode based on the input of the user to be trained; after the target training mode is started, detecting any one or more scenarios such as abnormal responses during the training process, training gaps after training interruption, and abnormal process connection after interruption through the fallback process, and ensuring the continuity of the training process of the target training mode.

[0091] The fallback process layer is mainly used to monitor the running status of the large language model and knowledge base retrieval model in real time. When anomalies such as response timeout occur, the intelligent agent is triggered to automatically interrupt and switch the fallback script to ensure the emergency level of continuous training.

[0092] Specifically, it is used for agents to automatically interrupt themselves, solving the problem of abnormal unresponsiveness in models.

[0093] Used as a fallback script library to solve the problem of training gaps after interruptions.

[0094] Used for recovery strategies to address the issue of process continuity after interruption.

[0095] In one embodiment of this application, the fallback process includes an agent self-interruption mechanism, which is used to: monitor the response time of each round of the large language model and / or the knowledge base retrieval small model; when the response time exceeds a preset threshold, the agent internally suspends the current call to the large language model or the knowledge base retrieval small model, and records the training status of the user to be trained at the time of the interruption.

[0096] The fallback mechanism includes, but is not limited to, agent self-interruption, primarily addressing the issue of abnormal model unresponsiveness.

[0097] This mainly includes interruption triggering logic: monitoring the response time of the large model and the knowledge base retrieval small model in each round. If the response time exceeds the limit threshold, the agent immediately suspends the current large model or knowledge base retrieval small model call, and records the training status at the time of interruption to prepare for subsequent connection.

[0098] In one embodiment of this application, the fallback process includes a fallback dialogue script library invocation mechanism, which is used to: construct a fallback dialogue script library, pre-set fallback dialogue scripts according to the functional classification of actual business scenarios, the fallback dialogue scripts cover the entire training process and support variable replacement to ensure the continuity of the dialogue scripts with the current training process; establish dialogue script matching logic, after the intelligent agent is interrupted, reads the business function scenario, process tree node or historical dialogue keywords at the time of interruption, and matches the optimal dialogue script content from the fallback dialogue script library.

[0099] The fallback process mechanism includes, but is not limited to, calling up the fallback script library, mainly to solve the problem of training gaps after interruption.

[0100] First, the script library is built: pre-set backup scripts according to business scenarios and functions, covering the entire training process, and supporting variable replacement (such as {{current_product}}{{current_node}}) to ensure that the scripts are consistent with the current training process.

[0101] Secondly, the dialogue matching logic: After the intelligent agent is interrupted, it automatically reads the "business function scenario", "process tree node" or "historical dialogue keywords" at the time of the interruption, and matches the most relevant content from the backup dialogue library to avoid obvious gaps in training.

[0102] In one embodiment of this application, the fallback process includes a recovery strategy, which is used to: execute a continuous fallback mode, in response to the user's instruction to prioritize completing the coaching session, maintain the fallback dialogue interaction until the coaching session ends, and generate a supplementary evaluation after the coaching session ends; execute a dynamic recovery mode, in response to the user's instruction to wait for recovery, detect the model's operating status after a preset number of interaction rounds, and when the model is back to normal and usable, switch back to the intelligent coaching mode and continue the original dialogue from the interruption point, while supplementing the key information of the fallback stage; when executing the continuous fallback mode or the dynamic recovery mode, synchronize the dialogue data of the fallback stage to the evaluation unit, which is used to generate the coaching evaluation result so that the final comprehensive evaluation generated when the large language model is available is based on the complete training process data.

[0103] The fallback mechanism includes, but is not limited to, recovery strategies, which are mainly used to solve the problem of process continuity after interruption.

[0104] First, regarding the dual-strategy selection: Strategy 1: Continuous backup mode. If the account manager selects "Prioritize completing training," the system maintains the backup dialogue interaction until the training session ends, generating a supplementary evaluation afterward. Strategy 2: Dynamic recovery mode. If the account manager selects "Wait for system recovery," the system detects model usage after a specific number of interaction rounds. When it returns to normal usability, it automatically switches back to intelligent coaching mode, resuming the original dialogue from the interruption point and supplementing key information from the backup phase.

[0105] Secondly, data synchronization is guaranteed: regardless of the strategy chosen, the dialogue data in the fallback phase will be synchronized to the evaluation module to ensure that the final comprehensive evaluation is based on the complete training process when the large model is available.

[0106] The above method employs contextual memory optimization and evaluation model design to balance dialogue coherence and model response speed. Combined with example guidance and logical chain prompts, it achieves multi-dimensional and accurate evaluation, which is superior to existing simple keyword matching evaluation methods.

[0107] In one embodiment of this application, the pre-configured coaching mode includes at least open coaching and restricted coaching, which can operate independently or be switched on demand. The process includes: pre-configuring the coaching mode as open coaching; collecting business line product information; organizing and constructing a knowledge base according to product name, product function, and applicable scenarios; employing a retrieval optimization strategy, when the large language model needs to generate customer dialogue or verify customer manager responses, retrieving relevant fragments from the knowledge base as prompt words and inputting them into the large language model; setting virtual customer roles based on the large language model, whereby the virtual customer roles include role attribute definitions and role prompt word generation. The role attribute definitions are determined based on real bank customer profiles, and the role prompt word generation is achieved by converting role attributes into natural language descriptions and concatenating them into role prompt words; managing dialogue interaction and dialogue memory by designing dialogue logic, controlling casual conversation content, and optimizing contextual memory; and evaluating and scoring the coaching session according to preset evaluation dimensions using the large language model to obtain the coaching result for the open coaching mode.

[0108] First, the knowledge base construction mainly addresses the issue of narrow scenario coverage.

[0109] 1. Data Collection and Structuring: Collect product information from business lines such as corporate banking, wealth management, and lending, and organize it according to the structure of "product name - product function - applicable scenario" to ensure information completeness;

[0110] 2. Retrieval Optimization: A hybrid retrieval mechanism combining vector retrieval and inverted index is adopted. When the large language model needs to generate customer scripts or verify customer manager answers, relevant fragments from the knowledge base are automatically retrieved as prompt words to input into the model, ensuring script accuracy and retrieval response speed, and meeting the needs of real-time interaction.

[0111] Secondly, the virtual customer role setting mainly addresses the issue of insufficient scene realism.

[0112] 1. Role Attribute Definition: Based on real customer profiles of the bank, define the core attributes of the role: age (e.g., young white-collar workers, high-net-worth clients), occupation (e.g., business owners, retired employees), risk preference (e.g., conservative, aggressive), and communication style (rational inquiry type, impulsive complaint type, approachable consultation type, critical questioning type); fields

[0113] 2. Role prompt generation: Convert attributes into natural language descriptions and concatenate them into system prompts. For example: "You are now playing the role of a small business owner with a stable risk appetite, a direct communication style, and a focus on business loan costs and approval efficiency. Questions should revolve around loan amount, interest rate, and repayment period." This enables diverse role simulations that match the characteristics of real customers.

[0114] Furthermore, dialogue interaction and memory management primarily address the issue of poor dialogue coherence.

[0115] 1. Dialogue Logic Design: Organize the dialogue according to the practical logic of "account manager identifying pain points → account manager introducing products → customer asking multiple rounds of detailed follow-up questions," for example:

[0116] Account Manager: "Is your company's current financing need for expanding production or for working capital turnover?" (Identifying pain points);

[0117] Account Manager: "To meet your cash flow needs, I recommend our 'Business Quick Loan,' which offers competitive credit limits and interest rates." (Introducing the product)

[0118] Virtual customer: "What collateral is required for this loan? How long does the approval process take?" (Follow-up questions for details);

[0119] 2. Casual Conversation Control: When the account manager engages in casual conversation (such as "The weather is nice today") or gives irrelevant answers (the account manager talks about financial management when the customer asks about loan interest rates), the virtual customer will automatically steer the conversation back on track, for example: "The weather is indeed nice, but I'm more concerned about the loan interest rate you just mentioned. Could you elaborate?"

[0120] 3. Context memory optimization: Only recent dialogue text is retained to avoid model context overload, ensuring dialogue coherence while guaranteeing model response speed to meet real-time interaction requirements.

[0121] Finally, the large-scale model evaluation and scoring mainly addresses the issue of inaccurate evaluation.

[0122] 1. Evaluation Dimensions and Standards: Four core evaluation dimensions are defined, each with clear standards:

[0123] Compliance capability: Whether it complies with financial regulatory requirements (e.g., whether it discloses investment risks);

[0124] Communication skills: Is the speech fluent and the logic clear?

[0125] Professional competence: Is the product knowledge accurate? Is the demand matching reasonable?

[0126] Adaptability: Whether one can handle follow-up questions from customers and correct any deviations in one's own answers;

[0127] 2. Optimized prompts: A Few-Shot (example-guided) + Chain-of-Thought (logical chain) prompting method is used to guide the evaluation model to output objective results according to standards. For example:

[0128] "Based on the following dialogue, please score it from four dimensions: compliance ability, communication ability, professional ability, and adaptability. Output structured results and improvement suggestions. Example: The account manager did not mention the investment risks in the dialogue, so compliance ability should be deducted; the answer regarding interest rates is accurate, so professional ability scores highly. Dialogue content: [Account Manager: 'This investment has good returns and is very suitable for you'; Customer: 'Is there any risk?'; Account Manager: 'The risk is low, buy with confidence']"

[0129] 3. Results Output: The evaluation model quickly outputs a structured report containing dimensional scores, problem analysis, and improvement suggestions to help account managers clarify the direction of improvement.

[0130] In one embodiment of this application, the pre-configured coaching mode, which includes at least open coaching and restricted coaching, wherein the open coaching and restricted coaching can operate independently or be switched on demand, includes: pre-configuring the coaching mode as restricted coaching, setting key parameters of the process tree, wherein the key parameters of the process tree include at least the following parameter types: customer intent, used to define the core objectives of the node; node jump, used to control the direction of the training process; coach prompts, used to assist the account manager in answering accurately; intent keywords, used to identify the customer intent and the key points of the account manager's answer; standard answer, used as the benchmark for coaching scoring; and reusing the node attribute placeholders of the key parameters of the process tree to obtain the restricted coaching mode.

[0131] Specifically, the core parameter configuration provides configurable parameters, allowing business users to set key parameters for the process tree. Parameter types and examples are as follows:

[0132] Parameter type Configuration content example effect Customer Intent "Inquire about credit card annual fees" "Apply to increase loan limit" Define the core objectives of the nodes Node jump "Customer confirms annual fee → Redirect to 'Promotional Activities' section" Control the direction of the training process Coach's advice "Please specify the 'annual fee and fee reduction / waiver policy'." Assist account managers in answering questions accurately Intent Keywords Annual fee, charges, and fee reductions / exemptions Identifying customer intent and key points in the account manager's response Standard Answer "Our bank's Platinum Credit Card annual fee and waiver rules are clearly stated; overdue payments will incur fees as stipulated." As a scoring benchmark

[0133] In addition, it also includes placeholder reuse: node attributes support placeholders (such as {{product_name}} representing the product name, {{rate}} representing the interest rate), for example, "{{product_name}} has an annual interest rate of {{rate}}, and flexible repayment methods", which can be reused in all scripts after configuration, reducing the cost of repeated configuration.

[0134] The implementation of the eight core functions primarily addresses the issue of low training quality.

[0135] 1. Customer role-playing: Multiple customer roles (such as "rational consulting customer" and "picky complaining customer") can be configured under the same process tree. The system dynamically replaces the node dialogue and jump conditions according to the role tags to simulate the interaction scenarios of different customer characteristics.

[0136] 2. Real-time scoring per sentence: After each round of customer manager's speech, the small-parameter big language model is invoked to score in real time based on three dimensions: "intent keyword matching degree, standard answer coverage, and compliance", and the scoring results are displayed to help customer managers make immediate adjustments;

[0137] 3. Coaching script prompts: Account managers can choose whether to view the prompts. The system generates candidate scripts based on the current "standard answer" to help novice account managers quickly master professional expression;

[0138] 4. Small talk handling: Based on the keyword hit results of real-time scoring of single sentences, non-business small talk is identified, and the "small talk subtree" is triggered. After the response, the system automatically pulls back to the current process node to prevent training from deviating from the core.

[0139] 5. Script polishing: Based on the communication style of the customer roles, the standard script of the process tree is "style transferred" to add conversational and emotional expressions to make the interaction closer to real scenarios;

[0140] 6. Forced Redirect: When the system detects that "the customer's intentions have not been met multiple times" or "the account manager has repeatedly engaged in small talk and deviated from the intended purpose," it will automatically trigger a forced redirect, using prompts and virtual customer guidance to bring the training back to the subsequent core points.

[0141] 7. Comprehensive scoring, evaluation and suggestions: After the training session, the system summarizes the "real-time scoring results of individual sentences, completion rate of process nodes, and customer intent satisfaction", generating an overall score and multi-dimensional report to identify the account manager's strengths and areas for improvement;

[0142] 8. Batch expansion of process tree templates: The core structure of the process tree (such as "Consultation - Introduction - Objection Handling - Closing") is solidified into a template. When expanding to new scenarios, business personnel only need to select the template, replace fields, and preview the test to quickly complete the configuration without technical coding.

[0143] The process tree-based training program integrates eight core functions: Through a combination of features such as real-time scoring, script prompts, handling small talk, and forced redirection, it achieves real-time control, dynamic adaptation, and precise guidance throughout the training process, solving the "process out of control" problem of traditional solutions. It employs template-based expansion and variable replacement technology: the process tree is broken down into a "fixed framework + variable fields," with fallback scripts supporting dynamic variables. Business personnel can complete new scenario configurations without technical coding, adapting to the rapid iteration needs of banking operations.

[0144] This application also provides a training device 200 based on a large language model, such as... Figure 3The diagram shows a schematic representation of a language model-based tutoring device according to an embodiment of this application. The language model-based tutoring device 300 includes at least: a configuration module 310 and a selection / execution module 320, wherein:

[0145] In one embodiment of this application, the configuration module 310 is specifically used to: pre-configure a coaching mode, wherein the coaching mode includes at least open coaching and restricted coaching, the open coaching and the restricted coaching can run independently or be switched as needed, and both the open coaching and the restricted coaching are built based on a large language model.

[0146] The open-ended coaching service on the server side is based on a "hybrid retrieval + context optimization" approach. It uses a combination of vector retrieval and inverted index to recall knowledge base information and optimizes the model context by "retaining recent dialogue text" to balance interaction coherence and response speed.

[0147] The "parameterized configuration + dynamic role adaptation" method of the process tree-restricted coaching on the server side: supports configurable parameters such as customer intent and node jump, dynamically replaces the script and jump conditions through role tags, and realizes real-time scoring of single sentences and forced jump correction.

[0148] In one embodiment of this application, the selection execution module 320 is specifically used to: respond to the input of the user to be trained, select the training mode, and at the same time carry out training based on the fallback process and output the final training evaluation result, wherein the fallback process is used to ensure the continuity of the training process of the large language model.

[0149] On the server side, a fallback process is adopted based on the "indicator monitoring - script matching - dual-strategy recovery" method: monitoring multi-dimensional indicators such as model resources, response time, and content matching degree; matching variable fallback scripts according to scenarios and nodes; and providing two strategies, continuous fallback and dynamic recovery, to ensure the continuity of training.

[0150] In one embodiment of this application, the selection execution module 320 is further configured to:

[0151] The target coaching mode is determined based on the input content of the user to be coached;

[0152] After the target training mode is activated, the fallback process detects one or more scenarios, including abnormal responses during the training process, training gaps after training interruption, and abnormal process connection after interruption, and ensures the continuity of the training process in the target training mode.

[0153] In one embodiment of this application, the selection execution module 320 is further configured to:

[0154] Monitor the response time of each round of large language model and / or knowledge base retrieval small model;

[0155] When the response time exceeds the preset threshold, the agent internally suspends the current large language model or the retrieval of small models from the knowledge base, and records the training status of the user to be trained at the time of the interruption.

[0156] In one embodiment of this application, the selection execution module 320 is further configured to:

[0157] A backup script library is constructed, and backup scripts are pre-set according to the functional categories of actual business scenarios. The backup scripts cover the entire training process and support variable replacement to ensure the continuity of the scripts with the current training process.

[0158] Establish a dialogue matching logic. After the intelligent agent is interrupted, it reads the business function scenario, process tree node or historical dialogue keywords at the time of the interruption, and matches the optimal dialogue content from the fallback dialogue library.

[0159] In one embodiment of this application, the selection execution module 320 is further configured to:

[0160] The system implements a continuous backup mode, responding to the user's instruction to prioritize completing the coaching session, maintaining the backup dialogue interaction until the coaching session ends, and generating supplementary evaluations after the coaching session ends.

[0161] The system executes dynamic recovery mode in response to the user's instruction to wait for recovery. After a preset number of interaction rounds, it detects the model's running status. When the model is back to normal and usable, it switches back to intelligent coaching mode and continues the original dialogue from the interruption point, while supplementing key information in the fallback phase.

[0162] When executing the continuous fallback mode or the dynamic recovery mode, the dialogue data of the fallback phase is synchronized to the evaluation unit, which is used to generate the training evaluation results so that the final comprehensive evaluation generated when the large language model is available is based on the complete training process data.

[0163] In one embodiment of this application, the configuration module 310 is further configured to:

[0164] The coaching mode is pre-configured as an open coaching mode, and product information from business lines is collected, organized and constructed into a knowledge base according to product name, product function, and applicable scenarios.

[0165] Using a retrieval optimization strategy, when the large language model needs to generate customer scripts or verify customer manager answers, relevant fragments from the knowledge base are retrieved as prompt words and input into the large language model.

[0166] Virtual customer roles are set up based on a large language model. The virtual customer roles include role attribute definitions and role prompt word generation. The role attribute definitions are determined based on the bank's real customer profiles. The role prompt word generation is achieved by converting the role attributes into natural language descriptions and concatenating them into role prompt words.

[0167] Manage dialogue interaction and dialogue memory by designing dialogue logic, controlling casual conversation content, and optimizing contextual memory;

[0168] The training results in an open-ended training mode are obtained by evaluating and scoring the training sessions using a large language model according to preset evaluation dimensions.

[0169] In one embodiment of this application, the configuration module 310 is further configured to:

[0170] The coaching mode is pre-configured as restricted coaching, and key parameters of the process tree are set. These key parameters include at least the following parameter types:

[0171] Customer intent is used to define the core objectives of the node.

[0172] Node navigation is used to control the direction of the training process;

[0173] The coach's tips are designed to help account managers answer questions accurately.

[0174] Intent keywords are used to identify customer intent and the key points of the account manager's response;

[0175] The standard answer is used as the benchmark for scoring during practice sessions.

[0176] By reusing the node attribute placeholders of the key parameters of the process tree, a restricted coaching mode is obtained.

[0177] It is understood that the aforementioned tutoring device based on a large language model can implement all the steps of the tutoring method based on a large language model provided in the foregoing embodiments. The relevant explanations of the tutoring method based on a large language model are applicable to the tutoring device based on a large language model, and will not be repeated here.

[0178] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0179] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0180] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0181] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a transaction reconciliation mechanism at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0182] The training mode is pre-configured, which includes at least open training and restricted training. The open training and restricted training can run independently or be switched as needed. Both the open training and the restricted training are built based on a large language model.

[0183] In response to the input of the user seeking coaching, the coaching mode is selected, and coaching is conducted based on a fallback process, which outputs the final coaching evaluation result. The fallback process is used to ensure the continuity of the coaching process for large language models.

[0184] The above is as stated in this application. Figure 2The method for executing the training device based on a large language model disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0185] The electronic device can also perform Figure 2 This paper describes a method for implementing a language-model-based tutoring device, and demonstrates its application in this device. Figure 2 The functions of the embodiments shown are not described in detail here.

[0186] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 2 The method executed by the tutoring device based on a large language model in the illustrated embodiment is specifically used to perform:

[0187] The training mode is pre-configured, which includes at least open training and restricted training. The open training and restricted training can run independently or be switched as needed. Both the open training and the restricted training are built based on a large language model.

[0188] In response to the input of the user seeking coaching, the coaching mode is selected, and coaching is conducted based on a fallback process, which outputs the final coaching evaluation result. The fallback process is used to ensure the continuity of the coaching process for large language models.

[0189] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0190] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0191] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0192] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0193] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0194] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0195] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0196] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0197] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0198] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A tutoring method based on a large language model, characterized in that, Applied to the server side, the method includes: The training mode is pre-configured, which includes at least open training and restricted training. The open training and restricted training can run independently or be switched as needed. Both the open training and the restricted training are built based on a large language model. In response to the input of the user seeking coaching, the coaching mode is selected, and coaching is conducted based on a fallback process, which outputs the final coaching evaluation result. The fallback process is used to ensure the continuity of the coaching process for large language models.

2. The method according to claim 1, characterized in that, The process of responding to the input of the user seeking coaching, selecting the coaching mode, conducting coaching based on a fallback process, and outputting the final coaching evaluation result includes: The target coaching mode is determined based on the input content of the user to be coached; After the target training mode is activated, the fallback process detects one or more scenarios, including abnormal responses during the training process, training gaps after training interruption, and abnormal process connection after interruption, and ensures the continuity of the training process in the target training mode.

3. The method according to claim 2, characterized in that, The fallback process includes an agent self-interruption mechanism, which is used for: Monitor the response time of each round of large language model and / or knowledge base retrieval small model; When the response time exceeds the preset threshold, the agent internally suspends the current large language model or the retrieval of small models from the knowledge base, and records the training status of the user to be trained at the time of the interruption.

4. The method according to claim 2, characterized in that, The fallback process includes a fallback script library retrieval mechanism, which is used for: A backup script library is constructed, and backup scripts are pre-set according to the functional categories of actual business scenarios. The backup scripts cover the entire training process and support variable replacement to ensure the continuity of the scripts with the current training process. Establish a dialogue matching logic. After the intelligent agent is interrupted, it reads the business function scenario, process tree node or historical dialogue keywords at the time of the interruption, and matches the optimal dialogue content from the fallback dialogue library.

5. The method according to claim 2, characterized in that, The fallback process includes a recovery strategy, which is used for: The system implements a continuous backup mode, responding to the user's instruction to prioritize completing the coaching session, maintaining the backup dialogue interaction until the coaching session ends, and generating supplementary evaluations after the coaching session ends. The system executes dynamic recovery mode in response to the user's instruction to wait for recovery. After a preset number of interaction rounds, it detects the model's running status. When the model is back to normal and usable, it switches back to intelligent coaching mode and continues the original dialogue from the interruption point, while supplementing key information in the fallback phase. When executing the continuous fallback mode or the dynamic recovery mode, the dialogue data of the fallback phase is synchronized to the evaluation unit, which is used to generate the training evaluation results so that the final comprehensive evaluation generated when the large language model is available is based on the complete training process data.

6. The method according to claim 1, characterized in that, The pre-configured coaching modes include at least open coaching and restricted coaching modes, wherein the open coaching and restricted coaching modes can operate independently or be switched on demand, including: The coaching mode is pre-configured as an open coaching mode, and product information from business lines is collected, organized and constructed into a knowledge base according to product name, product function, and applicable scenarios. Using a retrieval optimization strategy, when the large language model needs to generate customer scripts or verify customer manager answers, relevant fragments from the knowledge base are retrieved as prompt words and input into the large language model. Virtual customer roles are set up based on a large language model. The virtual customer roles include role attribute definitions and role prompt word generation. The role attribute definitions are determined based on the bank's real customer profiles. The role prompt word generation is achieved by converting the role attributes into natural language descriptions and concatenating them into role prompt words. Manage dialogue interaction and dialogue memory by designing dialogue logic, controlling casual conversation content, and optimizing contextual memory; The training results in an open-ended training mode are obtained by evaluating and scoring the training sessions using a large language model according to preset evaluation dimensions.

7. The method according to claim 1, characterized in that, The pre-configured coaching modes include at least open coaching and restricted coaching modes, wherein the open coaching and restricted coaching modes can operate independently or be switched on demand, including: The coaching mode is pre-configured as restricted coaching, and key parameters of the process tree are set. These key parameters include at least the following parameter types: Customer intent is used to define the core objectives of the node. Node navigation is used to control the direction of the training process; The coach's tips are designed to help account managers answer questions accurately. Intent keywords are used to identify customer intent and the key points of the account manager's response; The standard answer is used as the benchmark for scoring during practice sessions. By reusing the node attribute placeholders of the key parameters of the process tree, a restricted coaching mode is obtained.

8. A tutoring device based on a large language model, characterized in that, Applied to the server side, the device includes: A configuration module is used to pre-configure coaching modes, which include at least open coaching and restricted coaching modes. The open coaching and restricted coaching modes can run independently or be switched as needed. The selection execution module is used to respond to the input of the user to be trained, select the training mode, and conduct training based on the fallback process and output the final training evaluation result. The fallback process is used to ensure the continuity of the training process.

9. An electronic device, characterized in that, include: processor; as well as A memory configured to store computer-executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that, when executed by an electronic device containing multiple applications, cause the electronic device to perform the method of any one of claims 1 to 7.

Images