Method and system for enhancing generated intelligent coaching for career training based on behavioral retrieval

Abstract

The present application relates to the field of artificial intelligence assisted vocational skill training, and relates to a vocational training intelligent tutoring method and system based on behavior retrieval enhanced generation. The method comprises the following steps: collecting real-time multi-modal behavior data of a user, actively identifying the implicit confusion state of the user through a five-dimensional confusion degree scoring model; taking the behavior feature vector and the natural language prompt as joint query conditions, performing three-modal dynamic weighted similarity matching retrieval in a knowledge base, generating heuristic prompts after multi-path semantic consistency verification; quantifying the intervention effect by Mahalanobis distance, upgrading to real-time operation video and voice guidance when invalid; constructing a Feynman reverse explanation mechanism, three-dimensionally scoring the concept accuracy, logical coherence and term standardization rate of the user's voice explanation, driving a "explanation-evaluation-interrogation" deep learning closed loop; writing successful tutoring cases into the knowledge base and optimizing the retrieval and interrogation strategies through reinforcement learning, realizing continuous self-evolution of the system.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence-assisted vocational skills training, specifically involving a vocational training intelligent tutoring method and system based on behavior-augmented retrieval-augmented generation (B-RAG). It is applicable to learning and tutoring scenarios of complex software with strong procedural interactive features, such as engineering drawing software, integrated development environment (IDE), financial ERP system and 3D modeling software. Background Technology

[0002] Against the backdrop of the continuous development of the digital economy, the ability to deeply apply software tools has become a core element of professional competitiveness. Currently, vocational training systems for software operation skills mainly rely on video courses, text-based tutorials, and one-on-one human tutoring, but face the following core technological bottlenecks in terms of large-scale intelligent tutoring.

[0003] First, AI tutoring systems suffer from a serious multimodal perception blind spot. Existing Retrieval Augmentation (RAG) technology primarily handles text reading scenarios and cannot perceive key elements in software learning—UI interface state changes and precise spatial coordinate interactions. When an AI system cannot identify the user's software interface state, operation path, and specific error location, the tutoring content it provides will inevitably be out of touch with the actual operating environment and lose its tutoring value.

[0004] Second, existing systems lack the ability to proactively perceive users' cognitive difficulties. When learners encounter cognitive obstacles in complex software interfaces, they often cannot accurately express the difficulties they face, manifesting as a state of implicit confusion: the mouse wanders erratically on the screen (high jitter index), meaningless repeated clicks, a sudden drop in operation speed, and frequent undoing. Existing intelligent systems all adopt a passive response mode, lacking the ability to perceive and proactively intervene in this state of "procedural silence," resulting in the loss of many valuable tutoring opportunities.

[0005] Third, there is a lack of a systematic learning closed-loop assessment mechanism. Current AI tutoring tools tend to directly output operation answers after detecting confusion, which solidifies learners' dependence on AI, blocks the active reconstruction of mental models, and leads to the formation of a "competency illusion." At the same time, traditional systems cannot provide real-time operation demonstrations based on the user's current screen state, nor can they accurately present the correct operation path in the form of video or audio, thus limiting the improvement of learning efficiency.

[0006] Fourth, existing RAG systems lack effective mechanisms to ensure the reliability of retrieval results. Multi-path retrieval results may exhibit semantic inconsistencies, leading to contradictions in tutoring content; tutoring activities and knowledge tracing lack tamper-proof evidence storage mechanisms, making it difficult to establish a foundation of trust in multi-node collaborative environments.

[0007] Therefore, there is an urgent need for a new intelligent tutoring solution that can overcome the above-mentioned technical bottlenecks. This solution should simultaneously possess: multimodal operation perception capability, proactive identification capability of implicit confusion, graded tutoring strategy, deep mastery assessment capability based on Feynman learning method, and reliable knowledge retrieval and evidence storage mechanism. Summary of the Invention

[0008] The purpose of this invention is to provide a vocational training intelligent tutoring method and system based on behavior retrieval and enhanced generation, so as to solve the above-mentioned problems in the background technology, realize the comprehensive perception of multimodal behavior during user software operation, the proactive identification and graded intervention of confusion state, the deep mastery assessment based on Feynman learning method, and the generation of real-time dynamic operation audio and video demonstrations.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A method for intelligent vocational training coaching based on behavior retrieval enhancement includes the following steps: Step S1, Multimodal Behavior Acquisition and Confusion Recognition: Real-time acquisition of multimodal interactive behavior data of users during the operation of the target learning software, including keyboard input sequences, mouse movement trajectories, and software operation interface status; extraction of multi-dimensional cognitive load features based on the acquired behavioral data, calculation of the user's confusion score, and determination of the user's confusion stage when the confusion score exceeds the preset confusion threshold, and active triggering of the tutoring mechanism, while responding to the user's active questions.

[0010] Step S2: Heuristic prompts generated based on behavior retrieval enhancement: The multimodal interaction behavior data is mapped into behavior feature vectors, which, together with the user's natural language prompts, serve as joint query conditions. A weighted similarity matching retrieval based on multimodal semantics and action topology is performed in a preset vocational training knowledge base. The response results of multiple retrieval paths are aggregated for consistency evaluation, generating reliable heuristic text or voice prompts. The weighted similarity matching is controlled by a retrieval strategy network. The retrieval strategy network is a lightweight attention mechanism network that uses perplexity as a comprehensive scoring factor. and text query embedding User multimodal action sequence embedding Embedding the current screen UI topology map As input, attention weights are calculated and normalized for the text channel, behavior channel, and UI topology channel respectively, and the dynamic output satisfies... Constrained text channel weights Behavioral channel weights UI topology channel weight The prompts are constructed based on the guidance strategy of Feynman learning and do not directly provide specific software operation instructions. Step S3, Intervention Effect Verification and Coaching Strategy Upgrade: Monitor the user's operational response after receiving heuristic prompts, and quantify the intervention effect by calculating the Mahalanobis distance between the user's operational feature vectors before and after the intervention; if several pre-effects are lower than the effective threshold, the coaching strategy upgrade is triggered, and specific operation video guidance or voice step guidance for the corresponding software operation position is directly output.

[0011] Step S4, Feynman Reverse Explanation and Real-time Scoring: When tutoring is triggered or the user actively chooses to enter the Feynman stage, the system acts as the questioner and evaluator, requiring the user to provide a voice explanation of the operating logic of the current software function as a "teacher," and to score the explanation in real time in three dimensions: conceptual accuracy, logical coherence, and standardization of terminology.

[0012] Step S5: In-depth mastery assessment and follow-up questioning loop: Simultaneously collect the user's software operation event flow and voice explanation text, identify key, difficult, and confusing operation nodes, and dynamically generate targeted follow-up questioning tasks; calculate a comprehensive mastery score based on the user's answers and operation accuracy. When the score is lower than a preset threshold, continue to generate in-depth follow-up questions, forming a closed loop of "explanation-assessment-follow-up questioning" until the user's mastery reaches the target. The follow-up questioning tasks are output by the follow-up questioning generation strategy network; the follow-up questioning generation strategy network uses the Feynman explanation three-dimensional quality score. The lowest-scoring quality dimension identifier ( , or (the lowest score), and overall mastery score The key, difficult, or confusing operation node identifier corresponding to the lowest-scoring follow-up question is used as input. Based on the lowest-scoring dimension identifier, targeted guiding follow-up question text is generated to address the reasons for the loss in that dimension, or targeted guiding follow-up question text is generated to address the operational deviation of that node based on the lowest-scoring node identifier and then output.

[0013] Step S6, Knowledge Base Self-Learning Closed Loop: Successful tutoring cases are written back into the system knowledge base, the cognitive load level metadata of the corresponding operation node in the software operation knowledge graph is updated, and the retrieval strategy network and the follow-up question generation strategy network are jointly optimized through reinforcement learning algorithm to complete the continuous dynamic iteration of the system.

[0014] A vocational training intelligent tutoring system based on behavior retrieval-enhanced generation and multimodal interaction is used to implement the above methods. It includes a multimodal monitoring and cognitive diagnosis module, a B-RAG retrieval and prompt generation module, an intervention effect verification and tutoring upgrade module, a Feynman reverse teaching and assessment module, and a self-learning closed-loop module. It is deployed using a cloud-edge collaborative computing architecture.

[0015] Multimodal monitoring and cognitive diagnosis module: It is used to monitor the status of the user software operation interface and the keyboard and mouse input behavior flow in real time with a millisecond sampling frequency. By extracting five-dimensional cognitive load features, it determines whether the user is in a confused stage according to the five-dimensional confusion degree formula, and outputs a three-level label of confusion degree. B-RAG Knowledge Retrieval and Hint Generation Module: Includes a local structured knowledge base, a multimodal vector database, and a blockchain collaborative verification engine. It performs trimodal dynamic weighted similarity retrieval and multi-path aggregation consistency evaluation, and generates heuristic text or voice prompts that are notarized on the blockchain. Intervention effect verification and coaching upgrade module: The heuristic intervention effect is evaluated by Mahalanobis distance calculation. The coaching strategy is automatically upgraded after the intervention is determined to be ineffective. AR operation guidance video containing Bézier curve mouse trajectory animation and TTS voice narration is retrieved and rendered in real time. Feynman Reverse Teaching and Assessment Module: As the decision-making center of the intelligent agent, it receives multimodal input from users while they are operating and explaining. It evaluates the user's understanding based on the three-dimensional explanation quality scoring formula and the comprehensive mastery scoring model, identifies key points, difficulties and points of confusion, dynamically generates follow-up questions, and drives the "explanation-assessment-follow-up questioning" closed loop until the user meets the requirements.

[0016] The self-learning closed-loop module writes successful tutoring cases back into the knowledge base, updates the knowledge graph cognitive load metadata, and uses the PPO reinforcement learning algorithm to jointly optimize the network parameters of the retrieval device and the agent based on the multi-dimensional reward value composed of comprehensive mastery score, operational rhythm recovery time and system response latency, so as to achieve continuous self-evolution of system performance.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) Breaking through the blind spots of multimodal perception. This invention introduces behavior retrieval augmented generation (B-RAG) into the software vocational training scenario for the first time. By capturing screen visual features, mouse trajectory, keyboard input and UI component status, it achieves full-dimensional perception of complex software operation environment, breaking through the perception limitation of traditional RAG systems that can only handle text reading scenarios.

[0018] (2) Proactively identify latent confusion. This invention proposes a five-dimensional confusion comprehensive scoring model, which accurately quantifies the user's cognitive load through five dimensions: operation speed ratio, replay frequency, gaze duration, jitter index, and invalid click rate, so as to achieve proactive identification and three-level intervention of the "procedural silence" state without requiring the user to ask questions.

[0019] (3) Engineering the Feynman Learning Technique. This invention innovatively constructs a dynamic role reversal mechanism in which "users act as teachers to explain and the system scores in real time". Through the Feynman explanation quality score and comprehensive mastery score model, it drives the deep learning closed loop of "explanation-evaluation-probing", effectively preventing the formation of "competency illusion".

[0020] (4) Real-time dynamic audio and video demonstration. This invention uses a cubic Bézier curve interpolation algorithm to overlay the correct mouse trajectory animation onto a real-time screenshot, combined with synchronous TTS voice narration, to achieve multimodal "hands-on" operation demonstration with zero recording cost, directly solving the core pain point that AI cannot provide specific operation guidance.

[0021] (5) Credibility assurance of tutoring content. This invention uses multi-path semantic consistency verification (Sim_mean) combined with blockchain evidence storage mechanism to ensure the accuracy, consistency and immutability of tutoring content, and establishes a complete and traceable record of tutoring behavior. Attached Figure Description

[0022] Figure 1 This is a flowchart of the overall workflow of the method of the present invention, showing the execution order of each step, the condition triggering, and the closed-loop relationship of follow-up questions.

[0023] Figure 2 The diagram shows the five-module architecture of the system of this invention, illustrating the complete data flow and feedback path from multimodal monitoring to self-learning closed loop.

[0024] Figure 3 This is a schematic diagram of the B-RAG three-modal dynamic weighted retrieval mechanism, showing the user's natural language text ( Action sequences UI topology () The complete process of matching the three-way joint query vector with the three-dimensional embedding of the knowledge base after dynamic weighting by the attention network.

[0025] Figure 4 This diagram illustrates the closed-loop evaluation of Feynman's reverse explanation and in-depth questioning. Three-dimensional scoring (conceptual accuracy / logical coherence / terminology standardization) and A two-tiered assessment cycle structure for comprehensive mastery evaluation.

[0026] Figure 5 The system cloud-edge collaborative deployment architecture diagram shows the division of labor between the local end perception agent (front-end listener + UI parser) and the cloud computing node (B-RAG retrieval layer + Feynman evaluation engine + knowledge base), as well as the complete deployment structure for bidirectional communication with the consortium blockchain protocol via an encrypted WebSocket channel. Detailed Implementation

[0027] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0028] Before proceeding with the detailed explanation, the relevant parameters and core terms involved in this invention will be defined as follows: Definition 1: Mouse trajectory jitter index (JitterIndex) ) When a user operates the mouse in the target learning software interface and is in a state of cognitive confusion, the mouse pointer will wander randomly within a small range on the screen, changing direction frequently, instead of moving along a fixed path. This invention quantifies this phenomenon as a jitter index. Its calculation formula is defined as: In the formula: The angle (in radians) of the deflection of the mouse movement direction vector between adjacent sampling points; The displacement distance (in pixels) between adjacent points; The effective displacement threshold (default 25 pixels); For indicator functions; This represents the total number of sampling points within the window. The time window length (in seconds).

[0029] A higher value indicates that the user's mouse moves more frequently within a small range, which is a typical behavioral characteristic of cognitive confusion and one of the basic input dimensions of the five-dimensional confusion score.

[0030] Definition 2: Procedural Silence and the Five-Dimensional Perplexity Score ) Procedural silence refers to the implicit state of confusion experienced by users when encountering cognitive obstacles during software operation, where they are unable to accurately express the difficulties they encounter. This manifests as behaviors such as random mouse hovering, meaningless repeated clicking, sudden drops in operation speed, and frequent undoing. This invention employs a five-dimensional cognitive load feature vector. The formula for quantifying procedural silence is as follows: In the formula: The operating speed ratio (current speed to historical baseline speed) reflects changes in operating rhythm. The frequency of review and undo (normalized after logarithmic transformation) reflects the user's degree of uncertainty about the content they have already manipulated; The duration of mouse hover over UI areas (mapped to 0-10 minutes) reflects the user's level of confusion regarding interface elements; The jitter index reflects the lack of targeted operations; Invalid click rate reflects a decrease in operational accuracy; weight The default value is [0.25, 0.25, 0.20, 0.15, 0.15], with a maximum score of 10 points.

[0031] Cognitive load levels are classified into three levels: ∈[7.5,8.5) represents mild confusion, [8.5,9.2) represents moderate confusion, and [9.2,10] represents severe confusion, corresponding to different intensities of counseling intervention strategies.

[0032] Definition 3: Behavior-Augmented RAG (B-RAG) three-modal similarity Traditional Retrieval Augmentation (RAG) only performs embedding matching on natural language text. This invention innovatively introduces a Behavioral Retrieval Augmentation (B-RAG) mechanism, incorporating user action sequences and UI state as independent modalities into the joint retrieval query, forming a three-way joint query vector. The similarity calculation formula for B-RAG is: In the formula: This is a client-side joint query vector, embedded from text queries. User multimodal action sequence embedding Embedding the current screen UI topology map It consists of three sub-vectors; This is the joint embedding vector of candidate knowledge entries in the knowledge base, formed by embedding the document text of the knowledge base. Knowledge base experts demonstrate action embedding Knowledge base UI state embedding It consists of three sub-vectors; Embedding natural language query text for users. Embedded in the text of knowledge base documents; Embedding user multimodal action sequences, Demonstrating action embedding to knowledge base experts; Embed the current screen UI topology map. Embedded in the knowledge base UI state; , , The weight coefficients are dynamically output by the lightweight attention mechanism network, satisfying... ; This is a function for calculating the similarity of UI topology based on graph neural networks.

[0033] When a user is in a state of procedural silence (unable to accurately express confusion), the attention network adaptively shrinks the text weights. And significantly increase behavioral weight UI topology weights This allows the system to perform accurate knowledge retrieval based solely on physical behavior trajectories and interface states, breaking through the traditional RAG system's prerequisite that "users must be able to accurately express the problem."

[0034] Definition 4: Mean of semantic consistency across multiple paths ( ) and blockchain trusted evidence storage To ensure the accuracy and reliability of the tutoring content, this invention employs a multi-path aggregation verification strategy. The system generates [the content] based on a semantic expansion strategy. Given semantically equivalent but different query paths, after generating path responses, the semantic consistency of each path response is evaluated using the following formula: In the formula: The mean of semantic consistency; Path response and Semantic embedding cosine similarity; This represents the total number of query paths.

[0035] like Greater than or equal to the consistency threshold If the heuristic prompt is deemed credible, it is output to the user; simultaneously, all semantic behavior chain transaction records are calculated. root hash value The root hash value is written into the blockchain master ledger for trusted collaborative verification and notarization, ensuring that the guidance content is immutable and traceable; if Below If so, the search will be re-executed until the verification is successful.

[0036] Definition 5: Mahalanobis Distance as a Quantitative Indicator for Intervention Effectiveness To avoid measurement bias caused by differences in feature dimensions and correlations in ordinary Euclidean distance, this invention uses Mahalanobis distance to quantify the degree of change in user operation status before and after heuristic intervention: In the formula: The Mahalanobis distance between the feature vectors of user actions before and after heuristic prompts is used to quantify the degree of change in user action state caused by heuristic intervention. The larger the value, the more significant the intervention effect. The feature vector of the user's operation before receiving the heuristic prompt (including mouse speed, pause duration, jitter index, and replay frequency). The feature vector of the user's actions after receiving heuristic prompts; This is the covariance matrix of user historical interaction data, used to eliminate the influence of correlation between features.

[0037] The larger the Mahalanobis distance, the more significant the difference in user behavior before and after the intervention, indicating a better intervention effect. When the Mahalanobis distance is below the effective intervention threshold... If the operational rhythm index fails to recover to more than 85% of the baseline standard, the current heuristic intervention is deemed ineffective, triggering an upgrade of the coaching strategy.

[0038] Definition 6: Feynman's Explanation Quality Rating ( ) The core principle of the Feynman Learning Technique is to expose knowledge gaps by explaining from the perspective of a "teacher." This invention quantifies the quality of user explanations using the following three-dimensional scoring model: In the formula: Score the conceptual accuracy (0-10); Logical coherence score (0-10); The term misuse rate (0-1); The default value is [0.45, 0.35, 0.20]. threshold The default score is 8 (out of 10).

[0039] Definition 7: Overall Mastery Score ( ) In the Feynman probing loop, the system monitors the user's... A comprehensive evaluation of the performance on the issue of road pursuit was conducted: In the formula: To track the total number of problems; For the first Feynman Quality Score Explained in the Problem Solving; Score the correctness of this step (10 for correct, 0 for incorrect). The weight is explained (default 0.6).

[0040] when Reaching the preset mastery threshold When the score is 85 (default), it is determined that the user has a deep understanding of the operation node.

[0041] Definition 8: Cubic Bézier Curve Mouse Trajectory In generating AR operation guidance videos, it is necessary to position the mouse at the current location. With the target UI node This generates a demonstration trajectory that features a natural transition of the starting tangent, precise alignment of the ending tangent, and continuous acceleration throughout. For this purpose, the invention uses a cubic Bézier curve with two intermediate control points. , The reasons are as follows: (1) The tangents at both ends are independently controllable: the direction of the starting tangent of the cubic Bézier curve is controlled by... → The direction of the tangent at the endpoint is determined by... → The decision is that the two are independent of each other, allowing the demonstration mouse to smoothly start from the user's current position (avoiding a harsh, linear feel) and vertically hit the target UI node (avoiding edge-grabbing misjudgments). However, a quadratic Bézier curve has only one control point, and the tangents at the start and end points are coupled, making it impossible to satisfy both constraints simultaneously. (2) Acceleration is continuous (C 2 (Continuous): The cubic curve is second-differentiable, demonstrating that the acceleration of the cursor does not change abruptly throughout the process, resulting in the most natural visual performance and conforming to the kinematic characteristics of human hand movement. (3) Optimal balance between computational efficiency and stability: While higher-order Bézier curves (fourth order and above) can provide more control points, they introduce curve oscillations and cause mouse visual jitter, which is inconsistent with the jitter index used to identify the confused state in step S1 of this invention. This is contradictory; at the same time, the increased computational complexity affects the response latency of real-time rendering at 30 frames per second. .

[0042] Considering the above factors, using a cubic Bézier curve (i.e., two control points) is the optimal technical choice for this invention in AR trajectory generation scenarios. Based on this choice, a cubic Bézier curve interpolation algorithm is used to calculate the trajectory from the current mouse position. via control point , To the target UI node Smooth transition mouse trajectory: In the formula: The parameters of the cubic Bézier curve are taken as follows: The corresponding two-dimensional screen coordinates at time t, i.e., the demonstration mouse position at time t. The instantaneous position; The current mouse position; , Control points (automatically calculated by a cubic Bézier curve interpolation algorithm to produce a natural curve shape); The target UI node position; For curve parameters, When =0, = , When =1, = .

[0043] Definition 9: Reinforcement Learning Multidimensional Reward Value ( ) This invention achieves continuous self-optimization of the coaching strategy by constructing a Markov Decision Process (MDP). The overall return value is defined as: In the formula: To enable users to gain a deeper understanding of their overall score; The time (in seconds) required for the user's operating rhythm to recover; the smaller the value, the more effective the intervention. Penalty for system response delay; , , Let be the weighting coefficient, satisfying The default value is [0.6, 0.3, 0.1].

[0044] The present invention specifically adopts the following technical solution: like Figure 1 As shown, the method of this invention includes six core steps, forming a complete closed-loop guidance link of "monitoring-identification-prompting-verification-evaluation-iteration". Figure 2 As shown, the system consists of five functional modules and is deployed on a cloud-edge collaborative computing architecture.

[0045] I. Methods for Collecting Multimodal Behavioral Data A non-intrusive front-end listener is mounted in the background of the operating system at a sampling frequency of no less than 50 milliseconds (i.e., sampling frequency ≥ 20Hz) to capture the following three types of raw data in real time: (1) Physical input stream: absolute mouse coordinates marked with millisecond timestamps The trajectory of change, the amount of scroll wheel displacement, and the keystroke sequence; (2) Interface state flow: Extract the UI component tree of the currently active window through the operating system accessibility APIs or lightweight computer vision algorithms, and identify the spatial bounding boxes of interactive controls such as buttons, input boxes, and canvases, as well as their current enabled / disabled / focused states; (3) Screen visual flow: The current software work area screen is captured in key frame mode and used as the base image overlay for AR operation video.

[0046] The collected raw data is denoised by differential coding and Bessel path compression algorithm, and then serialized into a structured multimodal interaction event stream (MIES) by adding millisecond-level timestamps through a high-precision timer. This serves as a unified data source for all subsequent analysis modules.

[0047] II. Pre-construction methods for knowledge bases Before the tutoring service was launched, a multimodal visual-language model (VLM) was used to deeply deconstruct multi-source data, including the official software operation manual, expert demonstration videos, and historical successful tutoring cases, expanding each knowledge block into a vector entity containing three modal embedding dimensions: (1) The language principle description embedding is derived from official documentation and explanatory text through an embedding model encoding. (2) : Continuous physical operation sequence mapping embedding, which is derived from the mouse coordinate sequence and UI interaction sequence extracted from the expert demonstration video and encoded by the behavior encoder; (3) The UI component topology graph before and after the operation is embedded, which is generated by encoding the before and after screenshots of the interface for each operation step through a graph neural network.

[0048] Three-dimensional vector entities are uniformly stored in a multimodal vector database, supporting Approximate Nearest Neighbor (ANN) retrieval based on dense semantic embedding. Simultaneously, the root hash value of each mentor output record that passes consistency verification is written into the blockchain master ledger, forming an immutable mentor content notarization chain.

[0049] III. The Tiered Trigger Mechanism of Coaching Strategies This invention employs a graded mentoring strategy mechanism that combines dual trigger sources and a three-level upgrade system. (1) Passive trigger source: continuous monitoring When the score exceeds the confusion threshold And the duration exceeds The system will proactively intervene at that time. (2) Active trigger source: The user submits a natural language question to the system at any time; (3) Level 1 intervention: based on Feynman learning method heuristic prompts, without directly giving the operation answer; (4) Secondary intervention: When Mahalanobis distance assessment determines that primary intervention is ineffective, it is upgraded to precise operation video + TTS voice explanation; (5) In-depth assessment: After the user completes the learning phase, enter the Feynman reverse explanation-probing loop to verify the degree of in-depth mastery.

[0050] IV. System Deployment Architecture like Figure 5As shown, this invention adopts a cloud-edge collaborative computing architecture: the front-end perception agent (the perception side of the multimodal monitoring and cognitive diagnosis module, the intervention effect verification and tutoring upgrade module) is deployed on the user's local computing device (Windows / macOS client or mobile terminal) to collect operation data with millisecond-level latency; the B-RAG knowledge retrieval and prompt generation module, the Feynman reverse teaching and evaluation module, and the self-learning closed-loop module are deployed on cloud servers or high-performance edge computing nodes to support the high-concurrency computing needs of large language model inference, trimodal vector retrieval, and reinforcement learning training; the front-end and the cloud conduct real-time bidirectional data transmission through encrypted WebSocket long connections, and synchronize blockchain storage records among multiple tutoring nodes through consortium blockchain protocols to ensure the consistency and credibility of tutoring content in multi-institution deployment scenarios.

[0051] Furthermore, the specific steps are as follows: Step S1: Multimodal behavior acquisition and confusion identification: The system captures user behavior data in real time through a multimodal monitoring and cognitive diagnostic module. For confusion quantification, the system slices the MIES event stream into fixed time windows (default 5 seconds) and extracts the following five-dimensional cognitive load features sequentially: (1) Operating speed ratio Defined as the ratio of the average combined speed of mouse movement and keyboard input within the current time window to the user's historical baseline speed, calculated as follows: In the formula: : Number of mouse events within the current time window Keyboard event count Sum divided by window duration The unit is times per second, that is ; This user's usage scenarios with similar software over the past 30 days The average value, personalized baseline.

[0052] when A value less than 1 indicates that the current operating speed is lower than the historical baseline, which is a quantitative manifestation of the slowdown in operation.

[0053] (2) Frequency of review and undo Normalize the number of back (Ctrl+Z) and mouse back operations within the window after logarithmic transformation; (3) Mouse hover duration : Cumulative time the user hovers their mouse over the same UI area (mapped to 0-10 minutes); (4) Jitter Index : Captures the irregular wandering behavior of the mouse within a small area; (5) Invalid click rate : The ratio of invalid clicks (clicks on grayed-out controls or repeated clicks on the same unresponsive area) to the total number of clicks.

[0054] Substitute the five-dimensional features into the following formula to calculate the comprehensive score of the five-dimensional perplexity: when Exceeding the preset confusion threshold (Default 7.5 points) and the duration exceeds (Default 3 seconds) Output a confusion status label and trigger corresponding intensity of counseling intervention according to the three levels of cognitive load. Mild confusion ( ∈[7.5,8.5)) triggers voice prompts; moderate confusion ( ∈[8.5,9.2)) triggers a dual-channel heuristic prompt involving text and audio; severe confusion ( (∈[9.2,10]) triggers direct demonstration video output.

[0055] Step S2: Heuristic hints generated based on behavior retrieval enhancement: like Figure 3 As shown, after the user triggers the tutoring mechanism, the B-RAG knowledge retrieval and prompt generation module performs a three-modal joint retrieval. The system extracts the user's current natural language question (corresponding to...). Figure 3 The input channel for "Natural Language Query" (or the query description automatically generated from the state of confusion), and the mouse trajectory operation sequence within the last 10 seconds (corresponding to...) Figure 3 The input channel for "operation action sequence" and the topological relationship of UI components in the current screenshot (corresponding to) Figure 3 The input channels of the "UI topology" are mapped into high-dimensional vectors by their respective dedicated embedded encoders and then fed into the input channel. Figure 3 The "B-RAG weighted similarity engine" shown performs trimodal dynamic weighted similarity matching according to the following formula: Figure 3 The attention weight network shown at the top is based on the current user's... The weights of the three paths are dynamically adjusted based on the user's level. This is especially important when the user is experiencing severe confusion. >9.2) When unable to express the problem, Compressed to below 0.1 and With a combined score exceeding 0.9, the system regressed to a "pure behavior + UI status retrieval mode," completely resolving the user's dilemma of "not knowing how to ask questions."

[0056] To ensure the reliability of search results, through Figure 3 The "Multi-path Semantic Consistency Validation" module shown at the bottom expands the original query to Semantic equivalent query paths (default) =5), after retrieving and generating path responses respectively, calculate the average semantic consistency of the multipath using the following formula: Only when ≥ The system only considers heuristic prompts reliable and passes them when the value is set to 0.85 (default). Figure 3 The "Search Results Output" node at the bottom will also display all semantic behavior chain transaction records. root hash value Write Figure 3 The blockchain notarization module in the knowledge base on the right performs notarization. If the consistency verification fails, then... Figure 3 The dashed feedback path shown returns to the B-RAG weighted similarity engine to re-execute semantic expansion and retrieval, with a maximum of 3 retries. The generated heuristic hints are framed using the Socratic questioning framework of Feynman Learning, guiding users to think actively without directly providing the operational answer.

[0057] Step S3: Validation of Intervention Effectiveness and Upgrading of Counseling Strategies After the heuristic prompt is output, the system collects the user's response within the following 5 seconds. Operation feature vector within the window The data collected before the prompt Perform Mahalanobis distance comparison using the covariance matrix of the user's historical 30-day operation data. Eliminate the influence of correlation between feature dimensions: when < (default =2.5 (based on historical sample calibration) and when the operational rhythm recovery does not reach 85% of the baseline, the system determines that the heuristic intervention is ineffective, immediately triggers the tutoring strategy upgrade, and enters the AR operational video rendering process: (1) Retrieve from the knowledge base a structured instruction set (including the coordinates of the starting UI node, the coordinates of the ending UI node, and the sequence of intermediate buttons) that precisely matches the current operation node. (2) Capture the current user's actual software workspace as the base image, and use the cubic Bézier curve in the following formula to calculate the curve from the base image. (Current mouse coordinates) Automatically calculated control points , To the center coordinates of the target UI node Smooth trajectory: (3) The trajectory is sampled at a frame rate of 30 frames / second, and a visually highlighted ripple animation is superimposed on the key frame (when the click is pressed). The TTS speech synthesis model is called simultaneously to generate a voice explanation of the operation steps. The TTS speech synthesis model takes the text of the operation steps extracted from the structured instruction set as input and outputs a speech waveform signal that corresponds to the text content of the operation steps and is synchronously aligned with the 30 frames / second trajectory sampling frame on the time axis, which serves as the audio track synthesized with the video frame sequence in the HLS streaming media encapsulation. The TTS speech synthesis model can be implemented based on well-known end-to-end text-to-speech neural network architectures such as Tacotron, FastSpeech, or VITS. Tacotron employs an encoder-attention-decoder structure, converting the text sequence into a Mel spectrum, which is then restored to a speech waveform via a vocoder. FastSpeech uses a non-autoregressive feedforward transform structure, explicitly controlling the pronunciation duration of each phoneme through a duration predictor, making it suitable for scenarios with high frame rate synchronization accuracy requirements. VITS uses an architecture combining variational inference and adversarial training, directly generating speech waveforms end-to-end from text. Those skilled in the art can choose one of these three well-known architectures based on the computational constraints and speech quality requirements of the deployment environment, or they can use other equivalent TTS architectures that meet the characteristics of "text input—waveform output—frame-level timing controllability."

[0058] (4) The video frame sequence and audio track are encapsulated into HLS streaming media format and overlaid and rendered above the user software workspace through the operating system's top-level window (Always-on-Top) mechanism to form an AR operation guidance layer, allowing users to watch the demonstration while operating synchronously.

[0059] Step S4, Feynman Reverse Engineering Explanation and Real-Time Scoring: This step, together with step S5, constitutes the following: Figure 4 The two-layer evaluation closed-loop structure shown corresponds to this step. Figure 4 The left side displays "Level 1: Explanation Quality Assessment Cycle." This is the system's response after the Feynman coaching session is triggered or the user actively selects to enter "Explanation Reproduction Mode." Figure 4 The user input shown (voice explanation + operation demonstration) acts as both questioner and evaluator, outputting Socratic questions and requiring the user to provide voice explanations as a "teacher." The system integrates external memory components (user's historical interaction records: historical points of confusion, historical explanation logs) and internal memory components (current multi-turn dialogue context) to form a comprehensive decision input state, strictly prohibiting the model from directly outputting complete operation step instructions.

[0060] After a user completes a presentation, the system uses natural language understanding algorithms (which can be implemented based on well-known pre-trained language models such as BERT and RoBERTa) to extract concepts, perform dependency parsing, and perform term density statistics on the presentation text. Figure 4 The Feynman Explanation Overall Score is calculated by substituting the three quality indicators—"Conceptual Accuracy," "Logical Coherence," and "Terminology Standardization"—under the "Explanation Quality Score" in the following formula: : Compare the user's explanations with the standard explanations in the knowledge base to assess the coverage and accuracy of the core concepts; : Evaluate the coherence of causal logic through dependency parsing and causal chain extraction; By analyzing term density, we can detect the proportion of sentences that "pile up terms without providing any substantive explanations".

[0061] like < (Default score 8), system enters Figure 4 The "No" branch of the "Explanation met the standard?" judgment shown uses the "Guidance" module to generate precise guiding follow-up questions for the lowest-scoring dimension (e.g., if the conceptual accuracy is insufficient, ask "Please explain the underlying logic of this operation?"; if the logical coherence is insufficient, ask "What is the causal relationship between this step and the previous step?"). This loop of "listening-scoring-follow-up questioning" is executed continuously until the user... When the threshold is reached, proceed. Figure 4 The second layer is shown as the "Comprehensive Mastery Assessment Cycle".

[0062] Step S5: Deepen understanding of the assessment and follow-up questioning loop: This step corresponds to Figure 4 The right side shows the "Second Layer: Comprehensive Mastery Assessment Loop." After the user enters the deep reproduction stage, the system uses a keyframe extraction algorithm to automatically identify keyframes from the user's operation sequence. Figure 4 The three types of high-value follow-up questions shown are: Key operational nodes: Steps where the accuracy rate is lower than the user's historical average indicate a persistent operational bias. Difficult operation points: The steps with the longest pause times recorded by most users in the system's history indicate that this part is generally difficult to understand; Points of confusion: The steps where the user triggers a replay action (mouse back or scroll to reread) during the current learning session, indicating that there is a real-time question here.

[0063] For the follow-up questions generated by the three types of nodes, the system uses the following formula to process the user's questions. The overall performance on the "Taoist Pursuit" question will be scored: The quality of explanation is also assessed for each follow-up question. ) and the correctness of synchronous operation ( The dual-dimensional verification avoids the pseudo-mastery state of "being able to talk but not to do" or "being able to do but not to understand".

[0064] when < When the score is 85 (default), the system enters... Figure 4 The "No" branch of the "Mastering the Standard?" judgment, as shown, uses the "Deep Dive" module to identify the operation node corresponding to the follow-up question with the lowest overall score, and generates more in-depth follow-up questions accordingly, following... Figure 4 The feedback loop returns to the "comprehensive mastery assessment," forming a recursive closed loop of "explanation-assessment-follow-up questioning," until... ≥ After meeting the standards, as follows: Figure 4 As shown in the diagram, successful tutoring cases (description of user confusion points, corresponding explanation strategies, and operation paths) are written into the knowledge base, triggering the self-learning update process in step S6.

[0065] Step S6, Knowledge Base Self-Learning Loop: After receiving the successful coaching case output in step S5, the self-learning closed-loop module performs the following three update operations: (1) Knowledge base writing: The confusion point-explanation pair (the user's original confusion description and the explanation strategy for successfully resolving the confusion) is written in a three-modal vector format. Write the data into a multimodal vector database to directly expand the coverage of tutoring cases in the knowledge base; (2) Knowledge graph update: update the knowledge graph corresponding to the operation node in this tutorial. The historical data of the level is updated to the software operation knowledge graph so that tutoring resources can be deployed in advance for nodes with high cognitive load in subsequent tutoring. (3) Enhanced learning optimization: Collect key indicators (before and after intervention) for the entire tutoring process. Change Final value and system Substitute these values ​​into the following calculation to determine the multidimensional return value. : Employing the Proximal Policy Optimization (PPO) algorithm, based on Attention weights of the retrieval strategy network (controlling network parameters) , , The system performs joint gradient updates of the network parameters (dynamic allocation) and the follow-up question generation strategy. After accumulating a sufficient number of tutoring cases, the system can: adaptively adjust the trimodal weight allocation to improve retrieval accuracy; optimize the switching timing of the first / secondary intervention strategies to reduce the number of ineffective interventions; and improve the targeting of the follow-up question generation strategy to accelerate user mastery.

[0066] Thus, the method of this invention forms a complete end-to-end intelligent tutoring closed loop of "collection → perception → retrieval → prompting → verification → evaluation → writing → optimization" (e.g., Figure 1 As shown in the figure, each successful coaching update the system's knowledge base and policy network, enabling continuous self-evolution of system performance.

[0067] It should be noted that the term "procedural silence" used in this invention specifically refers to a state of implicit confusion in which a user is unable to accurately express their confusion due to cognitive impairment during software operation. This is specifically manifested as: mouse jitter index. Exceeding the threshold, operation speed ratio Significantly lower than baseline, frequency of review / cancel The five-dimensional comprehensive score of the above behavioral characteristics is often elevated. Exceeding the preset threshold Only then can it be determined that it does not cover operational pauses caused by non-cognitive reasons such as network latency or device lag. "Deep mastery" specifically refers to... ≥ (Default score 85), must also meet the Feynman Explanation Quality Score. Score for correct operation Both dimensions must be met; passing only one dimension is not sufficient.

[0068] In this invention, the term "behavior retrieval enhancement generation" (B-RAG) specifically refers to embedding user action behavior sequences (… Embedding UI topology state () This technology, which involves independent modalities in joint retrieval queries, differs from the standard RAG architecture that only processes natural language text. The core retrieval mechanism described by the three-modal dynamic weighted similarity formula in the claims can be implemented through equivalent multimodal fusion methods, such as replacing the attention weight network with other dynamic weight allocation algorithms or replacing the graph neural network with other UI topology similarity calculation methods. As long as the core architecture of the three query channels—text, behavior sequence, and UI state—remains unchanged, it still falls within the scope of protection of this invention.

[0069] In this invention, the terms "comprising" and "including" are open-ended expressions, covering non-exclusive content. The technical features in the claims can be implemented through equivalent substitutions, such as replacing Mahalanobis distance intervention verification with other multivariate statistical distance metrics, replacing near-end policy optimization (PPO) with other reinforcement learning algorithms (such as SAC, TD3, etc.), replacing the blockchain evidence storage mechanism with other distributed trusted recording schemes, and replacing the TTS speech synthesis module with other speech generation technologies. As long as the core tutoring architecture (confusion recognition → retrieval prompts → intervention verification → Feynman evaluation → self-learning closed loop) remains unchanged, it still falls within the scope of protection of this invention.

[0070] In summary, this invention provides an end-to-end intelligent tutoring solution for complex software vocational training scenarios through collaborative innovation of multimodal behavior perception, behavior retrieval enhanced generation (B-RAG), tiered tutoring strategies, Feynman learning engineering, real-time AR operation demonstration, and reinforcement learning self-optimization. The specific implementation discloses algorithm parameters (such as default weights [0.25, 0.25, 0.20, 0.15, 0.15], confusion threshold, etc. =7.5, weight Mastering the threshold =85 points), system architecture design (such as cloud-edge collaborative deployment method, WebSocket encrypted communication protocol, consortium blockchain evidence storage mechanism), retrieval strategy (such as default number of multi-path semantic expansion entries). =5. Consistency Threshold =0.85) all support implementation by those skilled in the art without inventive effort, satisfying the "full disclosure" requirement of patent law. Any improvements based on this solution, including but not limited to replacement of the embedded model type, adjustment of the vector database product, changes in the form of the tutoring interaction interface (such as migrating from desktop to mobile or AR glasses), and expansion of the knowledge base data source (such as adding internal enterprise operation specification documents), should be considered equivalent embodiments and fall within the protection scope of this invention.

Claims

1. A method for intelligent vocational training coaching based on behavior retrieval enhancement, characterized in that, The steps are as follows: Step S1, Multimodal Behavior Acquisition and Confusion Recognition: Real-time acquisition of multimodal interactive behavior data of users during the operation of the target learning software, including keyboard input sequence, mouse movement trajectory and software operation interface status; extraction of multi-dimensional cognitive load features based on the acquired behavior data, calculation of the user's confusion score, and determination of the user's confusion stage when the confusion score exceeds the preset confusion threshold, and active triggering of the tutoring mechanism, while responding to the user's active questions. Step S2: Heuristic prompts generated based on behavior retrieval enhancement: The multimodal interaction behavior data is mapped into behavior feature vectors, which, together with the user's natural language prompts, serve as joint query conditions. A weighted similarity matching retrieval based on multimodal semantics and action topology is performed in a preset vocational training knowledge base. The response results of multiple retrieval paths are aggregated and evaluated for consistency, generating heuristic text or voice prompts with verified credibility. The weighted similarity matching is controlled by a retrieval strategy network. The prompts are constructed based on a guidance strategy using Feynman learning, without directly providing specific software operation instructions. Step S3, Intervention Effect Verification and Coaching Strategy Upgrade: Monitor the user's operational response after receiving heuristic prompts, and quantify the intervention effect by calculating the Mahalanobis distance between the user's operational feature vectors before and after the intervention; if several pre-effects are lower than the effective threshold, the coaching strategy upgrade is triggered, and specific operation video guidance or voice step guidance for the corresponding software operation position is directly output; Step S4, Feynman Reverse Explanation and Real-time Scoring: When tutoring is triggered or the user actively chooses to enter the Feynman stage, the system acts as the questioner and evaluator, requiring the user to give a voice explanation of the operating logic of the current software function as a "teacher", and to score the explanation in real time in three dimensions: conceptual accuracy, logical coherence and standardization of terminology. Step S5: In-depth mastery assessment and follow-up questioning loop: Simultaneously collect the user's software operation event flow and voice explanation text, identify key, difficult, and confusing operation nodes, and dynamically generate targeted follow-up questioning tasks; calculate the comprehensive mastery score based on the user's answers and operation accuracy; when the score is lower than the preset threshold, continue to generate in-depth follow-up questions, forming a closed loop of "explanation-assessment-follow-up questioning" until the user's mastery reaches the target; the follow-up questioning generation strategy network uses the Feynman Explanation 3D quality score. The lowest-scoring quality dimension identifier ( , or (the lowest score), and overall mastery score The key, difficult, or confusing operation node identifier corresponding to the lowest-scoring follow-up question is used as input. Based on the lowest-scoring dimension identifier, targeted guided follow-up questions are generated to address the reasons for the loss in that dimension, or targeted guided follow-up questions are generated to address operational deviations at that node, and then output. The parameters of the follow-up question generation strategy network are determined by the PPO algorithm described in step S6 based on multi-dimensional reward values. Update; Step S6, Knowledge Base Self-Learning Closed Loop: Successful tutoring cases are written back into the system knowledge base, the cognitive load level metadata of the corresponding operation node in the software operation knowledge graph is updated, and the retrieval strategy network and the follow-up question generation strategy network are jointly optimized through reinforcement learning algorithm to complete the continuous dynamic iteration of the system.

2. The intelligent vocational training tutoring method based on behavior retrieval enhancement generation according to claim 1, characterized in that, Step S1 is as follows: Employing a five-dimensional cognitive load feature vector The formula for quantifying procedural silence is as follows: In the formula: This refers to the operating speed ratio; Frequency of review and undo; The duration for mouse hover over the UI area; The jitter index; Invalid click-through rate; weight ; 10 points maximum; when Exceeding the preset confusion threshold =7.5 points and the duration exceeds When confusion occurs, a confusion status label is output, and corresponding intensity of counseling intervention is triggered according to the three-level cognitive load level; mild confusion ∈[7.5,8.5), triggers voice prompt; moderate confusion. ∈[8.5,9.2), triggering a dual-channel heuristic prompt combining text and audio; severe confusion. ∈[9.2,10], triggers direct demonstration video output.

3. The intelligent vocational training tutoring method based on behavior retrieval enhancement generation according to claim 2, characterized in that, The formula for calculating mouse trajectory jitter index is: In the formula: This represents the deflection angle of the mouse movement direction vector between adjacent sampling points; The displacement distance between adjacent points; The effective displacement threshold; For indicator functions; This represents the total number of sampling points within the window. This represents the length of the time window.

4. The intelligent vocational training tutoring method based on behavior retrieval enhancement generation according to claim 1, characterized in that, In step S2, three-modal dynamic weighted similarity matching is performed according to the following formula: In the formula: This is a client-side joint query vector, embedded from text queries. User multimodal action sequence embedding Embedding the current screen UI topology map It consists of three sub-vectors; This is the joint embedding vector of candidate knowledge entries in the knowledge base, formed by embedding the document text of the knowledge base. Knowledge base experts demonstrate action embedding Knowledge base UI state embedding It consists of three sub-vectors; Embedding natural language query text for users. Embedded in the text of knowledge base documents; Embedding user multimodal action sequences, Demonstrating action embedding to knowledge base experts; Embed the current screen UI topology map. Embedded in the knowledge base UI state; , , The weight coefficients are dynamically output by the lightweight attention mechanism network, satisfying... ; This is a function for calculating the topological similarity of UI structures based on graph neural networks; When the user is in a state of procedural silence, the attention network adaptively shrinks the text weights. And increase the weight of behavior. UI topology weights This enables the system to perform accurate knowledge retrieval based solely on physical behavior trajectories and interface states; To ensure the reliability of the search results, the original query is expanded to [a new query] based on a semantic expansion strategy. For each semantically equivalent query path, after retrieving and generating path responses, the average semantic consistency of the multiple paths is calculated using the following formula: In the formula: The mean of semantic consistency; Path response and Semantic embedding cosine similarity; This represents the total number of query paths. Only when ≥Consistency threshold Only then does the system recognize the heuristic prompt as credible and output it, while simultaneously recording all semantic behavior chain transactions. root hash value Write it into the blockchain master ledger for evidence storage; If the consistency verification fails, semantic expansion and retrieval will be re-executed, with a maximum of 3 retries.

5. The intelligent vocational training tutoring method based on behavior retrieval enhancement generation according to claim 1, characterized in that, Step S3 is as follows: After the heuristic prompt is output, collect user feedback in subsequent steps. Operation feature vector within the window The data collected before the prompt Perform Mahalanobis distance comparison using the covariance matrix of the user's historical 30-day operation data. Eliminate the influence of correlation between feature dimensions: In the formula: The Mahalanobis distance between the feature vectors of user actions before and after the heuristic prompt; The feature vector of the user's actions before receiving heuristic prompts; The feature vector of the user's actions after receiving heuristic prompts; The covariance matrix of user historical interaction data is used to eliminate the influence of correlation between features; when Effective intervention threshold Furthermore, if the recovery of the operational rhythm fails to reach 85% of the baseline, the heuristic intervention is deemed ineffective, and the coaching strategy is immediately upgraded, entering the AR operational video rendering process. (1) Retrieve a structured instruction set from the knowledge base that precisely matches the current operation node; (2) Capture the current user's actual software workspace as the base image, and use the cubic Bézier curve in the following formula to calculate the coordinates from the current mouse position. Automatically calculated control points , To the center coordinates of the target UI node Smooth trajectory: In the formula: The parameters of the cubic Bézier curve are taken as follows: The corresponding two-dimensional screen coordinates at time t, i.e., the demonstration mouse position at time t. The instantaneous position; (3) The trajectory is sampled at a frame rate of 30 frames / second, and visually highlighted ripple animation is superimposed on the key frames. The TTS speech synthesis model is called synchronously to generate speech explanation of the operation steps. The speech synthesis model is implemented based on Tacotron, FastSpeech or VITS network architecture: the operation step text extracted from the structured instruction set is used as input, and after processing by the network architecture, the output is a speech waveform signal that corresponds to the operation step text content and is synchronously aligned with the 30 frames / second trajectory sampling frame on the time axis, which is used as the audio track synthesized with the video frame sequence in the HLS streaming media encapsulation. (4) The video frame sequence and audio track are encapsulated into HLS streaming media format and overlaid and rendered on the top window mechanism of the operating system above the user software workspace to form an AR operation guidance layer, which allows users to watch the demonstration and operate synchronously.

6. The intelligent vocational training tutoring method based on behavior retrieval enhancement generation according to claim 1, characterized in that, Step S4 is as follows: After a user completes a presentation, a natural language understanding algorithm is used to extract concepts, perform dependency parsing, and conduct term density statistics on the presentation text. The following three-dimensional quality indicators are extracted and substituted into the following formula to calculate the Feynman Presentation Score, thereby quantifying the quality of the user's presentation: In the formula: Score the accuracy of the concept; Scoring is given for logical coherence; The rate of terminology misuse; ; like <threshold The system generates precise, guiding follow-up questions for the lowest-scoring dimension, and continuously executes a closed loop of "listening-scoring-follow-up questioning" until the user... The threshold has been reached.

7. The intelligent vocational training tutoring method based on behavior retrieval enhancement generation according to claim 1, characterized in that, Step S5 is as follows: For the follow-up questions generated by the three types of nodes, the following formula is used to address the user's questions. The overall performance on the "Taoist Pursuit" question will be scored: In the formula: To track the total number of problems; For the first Feynman Quality Score Explained in the Problem Solving; Score the correctness of this step; To explain weights; when Reaching the preset mastery threshold At that time, it is determined that the user has a deep understanding of the operation node; < At that time, the operation node corresponding to the question with the lowest overall score is identified, and more in-depth follow-up questions are generated accordingly, forming a recursive closed loop of "explanation-evaluation-follow-up questioning" until... ≥ Once the target is met, successful coaching cases will be added to the knowledge base.

8. The intelligent vocational training tutoring method based on behavior retrieval enhancement generation according to claim 1, characterized in that, Step S6 performs the following three update operations: (1) Knowledge base writing: The confusion point-explanation pair is written in a three-modal vector format. Write the data into a multimodal vector database to directly expand the coverage of tutoring cases in the knowledge base; (2) Knowledge graph update: update the knowledge graph corresponding to the operation node in this tutorial. The historical data of the level is updated to the software operation knowledge graph so that tutoring resources can be deployed in advance for nodes with high cognitive load in subsequent tutoring. (3) Reinforcement learning optimization: Calculate multidimensional reward value : In the formula: To enable users to gain a deeper understanding of their overall score; The time required for the user's operating rhythm to recover; Penalty for system response delay; , , Let be the weighting coefficient, satisfying ; The PPO algorithm is optimized using a near-end strategy, based on The attention weight network parameters of the retrieval strategy network and the follow-up question generation strategy network parameters are jointly updated using gradients.

9. The intelligent vocational training tutoring method based on behavior retrieval enhancement generation according to claim 1, characterized in that, The retrieval strategy network is a lightweight attention mechanism network that uses perplexity as a comprehensive score. and text query embedding User multimodal action sequence embedding Embedding the current screen UI topology map As input, attention weights are calculated and normalized for the text channel, behavior channel, and UI topology channel respectively, and the dynamic output satisfies... Constrained text channel weights Behavioral channel weights UI topology channel weight ; The questioning generation strategy network uses Feynman's explanation of the three-dimensional quality score. The lowest-scoring quality dimension indicator and the overall mastery score. The key, difficult, or confusing operation node identifier corresponding to the lowest-scoring follow-up question is used as input. Based on the lowest-scoring dimension identifier, targeted guiding follow-up question text is generated to address the reasons for the loss in that dimension, or targeted guiding follow-up question text is generated to address the operational deviation of that node based on the lowest-scoring node identifier and then output.

10. A vocational training intelligent tutoring system based on behavior retrieval enhancement, used to implement the method described in any one of claims 1-9, characterized in that, It includes a multimodal monitoring and cognitive diagnosis module, a B-RAG retrieval and prompt generation module, an intervention effect verification and tutoring upgrade module, a Feynman reverse teaching and assessment module, and a self-learning closed-loop module, deployed using a cloud-edge collaborative computing architecture; the specifics of each module are as follows: Multimodal monitoring and cognitive diagnosis module: It is used to monitor the status of the user software operation interface and the keyboard and mouse input behavior flow in real time with a millisecond sampling frequency. By extracting five-dimensional cognitive load features, it determines whether the user is in a confused stage according to the five-dimensional confusion degree formula, and outputs a three-level label of confusion degree. B-RAG Knowledge Retrieval and Hint Generation Module: Includes a local structured knowledge base, a multimodal vector database, and a blockchain collaborative verification engine. It performs trimodal dynamic weighted similarity retrieval and multi-path aggregation consistency evaluation, and generates heuristic text or voice prompts that are notarized on the blockchain. Intervention effect verification and coaching upgrade module: The heuristic intervention effect is evaluated by Mahalanobis distance calculation. The coaching strategy is automatically upgraded after the intervention is determined to be ineffective. AR operation guidance video containing Bézier curve mouse trajectory animation and TTS voice narration is retrieved and rendered in real time. Feynman Reverse Teaching and Assessment Module: As the decision-making center of the intelligent agent, it receives multimodal input from the user while operating and explaining, evaluates according to the three-dimensional explanation quality scoring formula and comprehensive mastery scoring model, identifies key points, difficulties and doubts, dynamically generates follow-up questions, and drives the "explanation-assessment-follow-up questioning" closed loop until the user meets the standard; The self-learning closed-loop module writes successful tutoring cases back into the knowledge base, updates the knowledge graph cognitive load metadata, and uses the PPO reinforcement learning algorithm to jointly optimize the network parameters of the retrieval device and the agent based on the multi-dimensional reward value composed of comprehensive mastery score, operational rhythm recovery time and system response latency, so as to achieve continuous self-evolution of system performance.

Images