Training method and device based on dynamic knowledge graph and multi-modal interaction

Abstract

The application provides a training method and device based on dynamic knowledge graph and multi-modal interaction. By extracting text data such as electricity price policy, process and operation specification from a business system, and after cleaning and normalization, the data is injected into a dynamically updated knowledge graph to realize structured storage and real-time evolution of knowledge. By using a large language model combined with domain adaptation training, an intelligent model with job understanding ability is constructed, and through voice recognition, augmented reality devices and mechanical arms, natural language query response, visual guidance and physical action demonstration are realized. Combined with image acquisition and Bayesian inference framework, the learner's operation behavior is perceived and the ability state is evaluated in real time to generate an individualized adaptive learning path. The scheme solves the problems of knowledge lag, single interaction and insufficient individualization in traditional training, and improves the intelligent level, learning efficiency and system adaptability of training.

Description

Technical Field

[0001] This invention relates to the technical field of integrating intelligent training systems with artificial intelligence technology, specifically to a training method and apparatus based on dynamic knowledge graphs and multimodal interaction. Background Technology

[0002] In existing technologies, corporate training systems generally rely on static knowledge bases and pre-set courseware for content delivery. Knowledge updates are often time-consuming and heavily reliant on manual maintenance, making it difficult to respond promptly to policy adjustments (such as electricity pricing policies) or business process changes, resulting in training content lagging behind actual operational requirements. Traditional training methods often employ one-way information delivery models such as video playback, document reading, or centralized lectures, lacking real-time interaction and personalized feedback mechanisms, leading to low learner participation and limited knowledge absorption efficiency. Although some systems have introduced question-and-answer robots or online testing functions, the language models they rely on are usually not domain-specifically trained, exhibiting limited understanding of complex semantics and professional terminology related to job roles, failing to accurately interpret deeper intentions, and resulting in low-quality interaction. Furthermore, existing training devices primarily rely on screen displays and keyboard input, lacking the integration of multimodal interaction methods such as voice, gestures, and physical operations. This fails to provide learners with intuitive operational guidance and an immersive experience, especially limiting their effectiveness in skills training involving equipment operation or high-risk tasks. While attempts have been made to combine augmented reality (AR) technology for visualization assistance, most only display content along fixed paths, lacking the ability to perceive learners' behavioral states in real time and dynamically assess their cognitive levels, and thus unable to generate adaptive learning strategies based on individual differences. Due to the lack of data collection and intelligent analysis mechanisms for the operational process, the system struggles to identify learning bottlenecks and proactively optimize training paths, remaining largely at a one-size-fits-all teaching stage. Therefore, existing technologies have significant shortcomings in terms of knowledge timeliness, natural interaction, context awareness, and personalized support, failing to meet the needs of efficient, accurate, and closed-loop intelligent training in complex and dynamic environments. Summary of the Invention

[0003] Based on this, in order to address the technical problems of low learning efficiency and poor adaptability caused by lagging knowledge updates, insufficient cross-modal information fusion, and weak personalized interaction capabilities in existing training methods, a training method and device based on dynamic knowledge graphs and multimodal interaction is proposed.

[0004] This invention protects a training method based on dynamic knowledge graphs and multimodal interaction, comprising the following steps:

[0005] S100. Obtain text data of electricity pricing policies, procedures, and operating specifications through the business system;

[0006] S200. Inject the text data into the knowledge graph to form a structured knowledge storage;

[0007] S300. Based on a large language model architecture, perform domain-adaptive training on pre-trained language models;

[0008] S400. Receives natural language query commands through the voice recognition module, outputs visual operation instructions in conjunction with augmented reality devices, and simultaneously demonstrates physical actions using a robotic arm;

[0009] S500. Capture learner's operational behavior through an image acquisition device, model and evaluate learner's ability status, and generate an adaptive learning path sequence based on the evaluation results.

[0010] Furthermore, in S100, heterogeneous data is extracted from the three-tier business systems at the city, district / county, and power supply station levels using ETL tools. The heterogeneous data includes work order records from the marketing system, metering data from the electricity consumption system, and unstructured electricity price policy documents and operating procedures.

[0011] The extracted data is cleaned, key entities in the text are identified, and the key entities are normalized by combining the rule engine and the predefined ontology library to form standardized knowledge units.

[0012] The standardized knowledge units are injected into the database in the form of nodes.

[0013] Furthermore, step S200, which injects text data into the dynamically updated knowledge graph to form structured knowledge storage, also includes: S210. Deploying an incremental update module to connect with the electricity price policy release API and the work order system message queue, triggering the knowledge parsing pipeline when an electricity price policy change event is detected; S220. Calling a text similarity algorithm to compare the old and new documents and locate the changed paragraphs; S230. Identifying the newly added or adjusted entity relationships in the changed paragraphs and generating a graph patch package; S240. Performing node addition, deletion, and edge reconstruction operations to achieve dynamic updates of the knowledge graph.

[0014] Furthermore, a timestamp and version identifier are added to each node in the knowledge graph; time difference logic is introduced to model the impact path of electricity price policy changes on downstream processes.

[0015] Further, step S300, based on the large language model architecture, performs domain-adaptive training on the pre-trained language model to obtain an artificial intelligence model with job-specific domain understanding capabilities. This further includes: S310. Using a dynamically updated knowledge graph as the data source for a job-specific customized large language model, extracting triple sequences related to specific jobs through SPARQL queries to construct contextualized question-and-answer pairs and inference training samples; S320. Extracting instances for fine-tuning for power supply station jobs; S330. Employing LoRA low-rank adaptation technology to update only the attention head parameters related to the power domain in the large language model; S340. Establishing a bidirectional coupling interface between the large language model and the knowledge graph, obtaining the latest electricity price policy basis through real-time queries of the knowledge graph during inference, and returning answers with traceable paths.

[0016] Furthermore, during the student interaction process, the new and reasonable expression after review is written back into the knowledge graph, triggering the incremental update module to perform node addition and relationship reconstruction operations.

[0017] Furthermore, the S400's reception of natural language query commands via the speech recognition module includes: constructing a corpus containing dialect annotations, covering accent variants from multiple geographical regions, and performing end-to-end training using a loss function; aligning dialect pronunciation variants to the Mandarin representation space using a phoneme mapping table, and achieving terminology consistency recognition by combining a dynamically expanded pronunciation dictionary; deploying the acoustic model on edge computing units, and performing low-latency online inference after accelerated quantization; constructing a response template by combining the triplet data returned by the graph during answer generation, and forcibly embedding the electricity price policy document number and process number as traceability fields through constraint decoding; inserting speech synthesis markers into the response text after post-processing, generating speech by the localization engine, pre-setting three role timbres, and dynamically adjusting speech rate and intonation parameters based on the interaction history.

[0018] Furthermore, step S500, which uses a Bayesian inference framework to model and evaluate learners' ability status, specifically includes: constructing a knowledge space topology graph using a dynamic Bayesian network structure, with knowledge concepts as nodes and prior-requirement / successor dependency relationships as edges; inputting the observation data at each time step into the Bayesian update engine after feature engineering processing, and iteratively calculating the mastery probability using a recursive Bayesian filtering mechanism; the transition probability is jointly modeled by the learning decay function and the skill consolidation model, and the observation likelihood is generated based on the weighted fusion result of the action deviation threshold and semantic matching score.

[0019] Furthermore, step S500, which generates an adaptive learning path sequence based on the evaluation results, includes: decomposing operational skills into atomic action units and configuring an independent mastery state tracker for each atomic action unit; mapping the motion trajectory data output by the visual perception submodule to a preset action template library after spatiotemporal alignment, and calculating the DTW distance as the observation input; synchronously recording joint angle sequences and force feedback data when the robotic arm performs standard actions, and constructing an action prototype library for deviation detection; transmitting the Bayesian evaluation results to the personalized path generator in real time through the API interface, and triggering reinforcement training tasks for knowledge nodes with a mastery probability lower than a set threshold; the state space of the path generator is composed of the Bayesian evaluation results, the action space corresponds to the optional teaching content modules, and the policy network uses the PPO algorithm to output the optimal intervention sequence.

[0020] This invention also provides a training device based on dynamic knowledge graphs and multimodal interaction, comprising: a data acquisition module for acquiring text data of electricity pricing policies, processes, and operational specifications through a business system; a knowledge injection module for injecting the text data into a dynamically updated knowledge graph to form structured knowledge storage; a model training module for performing domain-adaptive training on a pre-trained language model based on a large language model architecture to obtain an artificial intelligence model with job domain understanding capabilities; a multimodal interaction module for receiving natural language query commands through a speech recognition module and outputting visual operation guidance in conjunction with augmented reality devices, while simultaneously demonstrating physical actions using a robotic arm; and a behavior evaluation module for capturing learner operational behaviors through an image acquisition device, modeling and evaluating learner ability status, and generating an adaptive learning path sequence based on the evaluation results.

[0021] This invention protects a training method and apparatus based on dynamic knowledge graphs and multimodal interaction. By automatically collecting and structuring textual data of electricity pricing policies, processes, and operational procedures from business systems, and injecting it into a dynamically updatable knowledge graph, it achieves real-time evolution and unified management of the knowledge system, improving the accuracy and timeliness of training content. Employing a large language model architecture combined with domain-adaptive training strategies, it enables the artificial intelligence model to possess a deep understanding of job-specific professional knowledge, significantly enhancing the accuracy of semantic parsing and intelligent reasoning. Through a multimodal interaction system integrating speech recognition, augmented reality devices, and a robotic arm, it supports learners initiating queries using natural language. The system allows learners to inquire and simultaneously receive visual operation guidance and physical action demonstrations, breaking through the limitations of the traditional one-way teaching model and significantly improving the intuitiveness and immersion of the training process. Furthermore, the introduction of image acquisition devices and a Bayesian inference framework enables real-time perception of learners' operational behavior and cognitive state modeling, dynamically assessing their ability levels and generating personalized adaptive learning path sequences, thus improving learning efficiency and training effectiveness. The overall solution, through the deep integration of dynamic knowledge updates, intelligent semantic understanding, multi-channel interactive feedback, and personalized learning control, constructs a closed-loop, intelligent training system that demonstrates high adaptability, strong interactivity, and continuous optimization capabilities in complex skill transfer scenarios. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0023] Figure 1 The flowchart of the training method based on dynamic knowledge graph and multimodal interaction provided in the embodiments of this application is shown.

[0024] Figure 2 This application presents a schematic diagram of the structure of a training device based on dynamic knowledge graphs and multimodal interaction. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0026] Traditional power marketing training systems primarily rely on centralized lectures, paper-based manuals, and online video courses, supplemented by simple theoretical assessment modules. Trainees passively receive knowledge, lacking practical training. While some systems offer basic question-and-answer functionality, they only support keyword-based electricity pricing policy queries and lack intelligent reasoning capabilities. Simple training robot solutions, although integrating voice interaction and display terminals and capable of responding to standardized electricity pricing policy questions, lack tangible action execution mechanisms and cannot simulate key practical scenarios such as meter installation and work order processing. They also lack a professional knowledge graph, relying on a fixed rule base for electricity pricing policy interpretation, making it difficult to adapt to dynamically updated electricity pricing policies and business processes. Furthermore, their interaction methods are limited to voice-text feedback, lacking visual perception and motion coordination capabilities. None of these solutions integrate large language models and dynamic knowledge graph technologies, nor do they achieve deep adaptation of embodied intelligence to all power marketing business scenarios, resulting in a disconnect between theory and practice and low training efficiency. In addition, existing technologies generally lack multimodal interaction capabilities, failing to recognize the trainee's operational compliance or demonstrate standard actions via robotic arms, leading to a poor interactive experience. Meanwhile, the training pathways were not personalized based on trainees' knowledge gaps and skill differences, resulting in wasted resources and inconsistent training outcomes. At the deployment level, existing systems largely rely on cloud computing, lack localized reasoning capabilities, exhibit poor robustness in offline environments, and have insufficient dialect support, making it difficult to meet the diverse language needs of grassroots areas.

[0027] Based on this, please refer to Figure 1 This application provides a training method based on dynamic knowledge graphs and multimodal interaction. This method integrates business system data, constructs a dynamic knowledge graph, trains customized large language models for specific positions, and combines multimodal interaction methods such as speech recognition, augmented reality, and robotic arm execution to achieve a closed-loop training system that integrates theoretical learning and practical training in a personalized and intelligent manner. This training method specifically includes the following steps:

[0028] S100. Obtain text data of electricity pricing policies, procedures, and operating specifications through the business system;

[0029] S200. Inject the text data into a dynamically updated knowledge graph to form a structured knowledge storage;

[0030] S300. Based on the large language model architecture, the pre-trained language model is subjected to domain adaptation training to obtain an artificial intelligence model with job domain understanding ability;

[0031] The S400 receives natural language query commands through a voice recognition module, outputs visual operation instructions in conjunction with augmented reality devices, and simultaneously demonstrates physical actions using a robotic arm.

[0032] S500. The learner's operational behavior is captured by an image acquisition device, the learner's ability status is modeled and evaluated using a Bayesian inference framework, and an adaptive learning path sequence is generated based on the evaluation results.

[0033] The technical effects of this application's embodiments are reflected in the following aspects: First, by obtaining text data of electricity pricing policies, processes, and operational specifications from the business system and injecting them into a dynamically updated knowledge graph, structured storage and real-time synchronization of knowledge are achieved, solving the problems of fragmented knowledge and delayed updates in traditional training, and ensuring that training content remains consistent with the latest business. Second, based on a large language model architecture, domain-adaptive training is performed on the pre-trained model, enabling the artificial intelligence model to have a deep understanding of the job domain, improving the accuracy of electricity pricing policy interpretation, process deduction, and compliance verification, and meeting the differentiated needs of different levels of positions. Third, by receiving natural language instructions through a speech recognition module and outputting visual operation guidance in conjunction with augmented reality devices, while simultaneously using a robotic arm to perform physical action demonstrations, a multimodal interactive system integrating speech, vision, and action is constructed, enhancing the immersiveness and practicality of training. Finally, by capturing learners' operational behaviors through an image acquisition device, dynamically modeling and evaluating their ability status using a Bayesian inference framework, and generating an adaptive learning path sequence accordingly, targeted instruction is achieved, improving the accuracy and resource utilization of training. Overall, this method achieves a closed loop throughout the entire process of knowledge acquisition, intelligent understanding, embodied interaction, and personalized optimization, significantly improving training efficiency and skill mastery.

[0034] Preferably, in S100, when acquiring textual data of electricity pricing policies, processes, and operating procedures through the business system, raw data is first collected from the business systems at the city, district / county, and power supply station levels. This includes official documents from the electricity pricing policy release platform, work order transfer records from the marketing management system, metering operation logs from the electricity consumption information collection system, and various unstructured operating procedures and safety specification documents. These data come from a wide range of sources and are diverse in format, constituting a typical heterogeneous data set.

[0035] Preferably, an ETL (Extract-Transform-Load) tool is used to extract and stream the aforementioned data in batches, ensuring the completeness and timeliness of data acquisition. During the data extraction phase, structured data such as work order records and metering information are periodically synchronized to the local processing environment via a JDBC interface; for unstructured text files, a distributed crawler framework combined with API calls is used to automate downloading and archiving.

[0036] In S100, heterogeneous data is extracted from the three-tier business systems at the city, district / county, and power supply station levels using ETL tools. The heterogeneous data includes work order records from the marketing system, metering data from the electricity consumption system, and unstructured electricity price policy documents and operating procedures. Furthermore, a message queue mechanism is introduced during data transmission to improve data stability in concurrent scenarios, and checksums and timestamps are set to prevent data loss or duplicate loading.

[0037] The extracted data is cleaned, key entities in the text are identified, and the key entities are normalized using a rule engine and a predefined ontology library to form standardized knowledge units. Specifically, this includes preprocessing unstructured text such as denoising, segmentation, and unified encoding, followed by extracting core concepts in the power sector, such as "time-of-use pricing," "transformer line loss threshold," "meter replacement cycle," and "safety distance requirements." Subsequently, the identified key entities are input into the rule engine module, where semantic mapping and synonym merging are performed using a predefined power business ontology library. For example, "peak-valley pricing" and "time-of-use pricing" are uniformly categorized as "time-of-use pricing," resolving the issue of inconsistent expressions and ensuring that the same concept has a unique identifier within the system.

[0038] Preferably, the standardized knowledge units are injected into the database as nodes. After cleaning and normalization, the knowledge units are transformed into basic nodes in a graph structure. Each node carries a type label (such as electricity price policy terms and operational actions), attribute fields (such as release time, applicable positions, and responsible departments), and a unique ID, ready to enter the subsequent knowledge graph construction process. This process is executed periodically through batch processing tasks, supporting incremental updates and version tracking, providing a high-quality input source for dynamic knowledge graph modeling in the S200 stage.

[0039] In step S200, the text data is injected into a dynamically updated knowledge graph to form structured knowledge storage. After completing the data collection and standardization process in stage S100, the knowledge modeling stage of this step begins. This process uses a graph database as the core carrier to construct a dynamic knowledge network covering the business logic of the city, district / county, and power supply station levels, realizing multi-dimensional semantic associations between electricity price policy terms, business processes, practical actions, and job roles.

[0040] In practice, the generated standardized knowledge units are imported into the graph database as basic nodes.

[0041] Preferably, node types are categorized into four core entities: electricity pricing policy clauses, business processes, practical actions, and job roles; edge relationships are defined by semantic types such as "trigger," "dependency," "constraint," "applicable," and "impact," supporting cross-level and multi-hop reasoning. For example, the "new time-of-use electricity pricing policy" node can be connected to the "electricity bill calculation process" node through the "impact" relationship, and then linked to the "meter time period setting" operation action node through the "requirement" relationship, forming a complete link from electricity pricing policy to process to action, enabling trainees to trace the electricity pricing policy basis and business objectives behind a certain operation.

[0042] Preferably, an incremental update module is deployed to connect to the electricity price policy release API and the work order system message queue. When an electricity price policy change event is detected, the knowledge parsing pipeline is triggered. To ensure the timeliness of the knowledge graph, the system integrates an incremental update mechanism to monitor the API interface of the electricity price policy release platform and the message queue of the intranet marketing system in real time. Once a signal such as an electricity price policy revision notification or process adjustment announcement is received, the knowledge parsing pipeline is immediately started, and the automated update process begins.

[0043] A text similarity algorithm is used to compare the old and new documents to locate the changed paragraphs. After obtaining the new version of the electricity pricing policy document, the algorithm is first used to perform a full text similarity analysis between the old and new versions to identify the differences. Paragraphs with high similarity are considered as unchanged content, while low similarity parts are marked as potential change areas, and the corresponding chapters are extracted for subsequent refined processing.

[0044] Preferably, the text similarity algorithm can be a cosine similarity algorithm based on BERT pre-trained word vectors, with the word vector dimension set to 768, and word segmentation using the jieba power industry customized dictionary; when comparing new and old files, the similarity threshold is set to 0.7, and paragraphs below the threshold are judged as changed paragraphs.

[0045] Identify newly added or adjusted entity relationships in the modified paragraphs and generate a graph patch package. This patch package can be in JSON format, containing fields such as 'operation type (add / delete / modify), node ID, node attributes (including timestamp / version number), associated edge ID, and relationship type', for example: {'operation': 'add', 'node_id':'P202405', 'node_attr': {'name': '2024 Time-of-Use Electricity Pricing Policy', 'timestamp': '2024-05-01', 'version': 'V1'}, 'edge_id': 'E202405', 'relation': 'impact'}. For the identified modified paragraphs, input them into a relationship extraction model to automatically identify the newly added or modified relationships between entities. For example, if a clause adds "abnormal work orders must be closed within 7 working days," the system extracts the "constraint" relationship between the "abnormal work order handling process" and the "time limit requirement." All identification results are packaged into a structured graph patch package, containing information on nodes and edges to be added and / or deleted.

[0046] Preferably, node addition, deletion, and edge reconstruction operations are performed to achieve dynamic updates of the knowledge graph. Using automatically generated Cypher scripts, atomic operations are executed in the Neo4j database to complete node creation, attribute updates or deletions, and edge reconstruction. The entire update process supports transaction rollback mechanisms to ensure data consistency, and the average synchronization latency is controlled within 24 hours, meeting the power industry's requirements for knowledge freshness.

[0047] Each node in the knowledge graph is assigned a timestamp and version identifier. To further support historical tracing and state replay, the system adds a timestamp field and version number to each node and edge, recording its first creation time, most recent modification time, and the corresponding electricity price policy version cycle. Users can query a snapshot of the graph at a specific time node by timeline, such as viewing all operating procedures corresponding to the "electricity price policy implemented in Q2 2025".

[0048] Preferably, time-difference logic is introduced to model the impact path of electricity price policy changes on downstream processes. Based on this, time-difference logic is introduced to perform causal modeling of the change propagation path. The system reverse-engineers along the electricity price policy, processes, work orders, and operation links to automatically identify the scope of historical business affected by this change.

[0049] For example, after an electricity price adjustment, all work orders involving the pricing rule within the past month can be marked, and relevant personnel can be prompted to retrain or review the operating standards, thereby achieving a closed-loop linkage from knowledge update to training response.

[0050] Through the above mechanism, S200 not only realizes the structured organization of static knowledge, but also builds a dynamic knowledge system with self-evolution capabilities, providing continuous, accurate and traceable knowledge support for subsequent large language model training and personalized training.

[0051] Step S300 includes performing domain-adaptive training on the pre-trained language model based on a large language model architecture to obtain an artificial intelligence model with job-specific domain understanding capabilities. After completing the dynamic knowledge graph construction in stage S200, the system uses the structured triplet data stored in the graph as high-quality corpus to train a dedicated artificial intelligence model for the electricity marketing field. This process is based on a general large language model and enhances its understanding and reasoning capabilities in interpreting electricity pricing policies, judging process compliance, and generating practical instructions by introducing domain knowledge.

[0052] In practice, a pre-trained language model is used as the basic architecture. To achieve differentiated adaptation for different positions, subgraph paths related to specific positions are extracted from the knowledge graph output by S200 through SPARQL queries, and contextualized training sample sets are constructed. For example, for management positions in the municipal company, the links of "electricity price policy terms, indicator deduction, and decision impact" are extracted to generate question-and-answer pairs for electricity price policy analysis; for operations positions in districts and counties, process review training data is collected from the path of "from work order process to compliance verification rules, and then to anomaly judgment conditions"; for front-line positions in power supply stations, the focus is on the sequence of "safety specifications, operating actions, and risk warnings" to generate on-site operation guidance instruction samples.

[0053] During training, LoRA (Low-Rank Adaptation) fine-tuning technique is used. While freezing most of the parameters of the original model, only the key attention head and the low-rank matrix injected into the feedforward network layer are updated to reduce the consumption of computational resources.

[0054] Preferably, job identifiers are embedded on the input side, enabling the model to identify the job attributes of the current service recipient and activate the corresponding knowledge reasoning path. After multiple rounds of iterative training, the resulting artificial intelligence model can not only accurately answer "What is the scope of time-of-use electricity pricing?", but also further explain its relationship with line loss accounting, the corresponding operational adjustment steps, and potential customer complaint scenarios, achieving a deep understanding across levels and dimensions.

[0055] Preferably, the model establishes a bidirectional coupling interface with the dynamic knowledge graph constructed by S200. During the inference phase, the model can obtain the latest electricity pricing policy basis by querying the graph in real time and return answers with traceable paths. During the interaction process, reasonable new expressions or operational logic proposed by trainees can be written back into the knowledge graph after manual review, driving the continuous expansion of the knowledge system. This forms a collaborative evolution mechanism between the model and the knowledge graph, ensuring that the artificial intelligence model always keeps pace with actual business and has job-level semantic understanding and intelligent service capabilities.

[0056] In the S400, natural language query commands are received through a voice recognition module, and visual operation instructions are output in conjunction with augmented reality devices. Simultaneously, a robotic arm performs physical action demonstrations. Building upon the training of the customized AI model for the S300 job, the system enters the multimodal embodied interaction phase. This step transforms the training process from one-way knowledge transfer to an immersive, perceptible, hands-on guided experience.

[0057] In practice, the system first collects natural language query commands from trainees via a microphone array, such as "How to handle abnormal time-of-use electricity pricing work orders?" or "What are the safety precautions when wiring meters?". After the voice signal is input to the voice recognition module, the end-to-end recognition process is initiated. To improve robustness in grassroots scenarios, a specially constructed corpus containing several annotated Hebei dialects (e.g., 1000 hours) covering accent variants from Baoding, Chengde, Zhangjiakou, and other regions has been built to ensure that frontline employees using local accents can still be accurately understood.

[0058] Preferably, a dialect-annotated corpus is constructed, covering accent variants in multiple geographical regions, and end-to-end training is performed using a loss function. The corpus is manually transcribed and aligned, and the acoustic model is driven by the CTC (Connectionist Temporal Classification) loss function for end-to-end training to capture long-term speech dependencies and enhance the modeling ability of connected speech and weak pronunciation in dialects.

[0059] Dialectal pronunciation variations are aligned to the Standard Mandarin representation space using a phoneme mapping table, and terminology consistency recognition is achieved by combining this with a dynamically expanding pronunciation dictionary. For typical sound change issues in dialects, a phoneme mapping rule table is designed to map non-standard pronunciations to the standard Chinese Pinyin system. Simultaneously, an integrated pronunciation dictionary of power industry terminology is provided, supporting an online expansion mechanism. When new electricity pricing policy terms are released, dictionary entries are automatically updated, ensuring consistency and accuracy in key terminology recognition.

[0060] Preferably, the acoustic model is deployed on the edge computing unit and executed for low-latency online inference after accelerated quantization. To adapt to remote areas without network access, the acoustic model is quantized and compressed and deployed on the local edge computing unit. While maintaining high recognition accuracy, it achieves an end-to-end response latency of less than 300ms, ensuring smooth voice interaction.

[0061] After the speech recognition results are converted into text, they are fed into a job-specific large language model trained on the S300 for semantic analysis. The model, combined with a real-time knowledge graph query interface, retrieves triplet data of relevant electricity pricing policy clauses, process nodes, and operational specifications, and generates accurate answers accordingly. For example, when explaining the electricity billing process, the model not only outputs the calculation formula but also displays the relevant electricity pricing policy regulations and associated operational action IDs.

[0062] During answer generation, a response template is constructed using the triplet data returned by the graph. Constraint decoding forces the embedding of electricity price policy document numbers and process numbers as traceable fields. The generation process incorporates a structured constraint decoding mechanism to ensure that the output text must include a source identifier field, forming a standardized response format of "conclusion + basis + path," enhancing the credibility of the answer and the standardization of teaching. The response text is post-processed with inserted speech synthesis markers, and the localization engine generates speech, pre-configured with three voice roles and dynamically adjusting speech rate and intonation parameters based on interaction history. The generated answer text enters the speech synthesis module, employing a FastSpeech2+HiFiGAN architecture localization engine, supporting offline speech generation. The system pre-configures three voice roles: electricity price policy explanation (e.g., a steady male voice), process guidance (e.g., a clear female voice), and operation prompts (short and forceful), automatically switching according to the current interaction scenario. Speech rate and intonation parameters are dynamically adjusted based on the student's age, learning progress, and historical interaction rhythm, improving information delivery efficiency.

[0063] Preferably, the system initiates a multimodal output collaboration mechanism. For operational issues, augmented reality devices (such as AR glasses or helmet displays) simultaneously load a 3D sandbox scene, overlaying virtual guide arrows, highlighted areas, and step prompts to achieve a synchronized demonstration of electricity pricing policies, actions, and the environment. For example, when explaining the meter replacement process, the AR interface displays the device disassembly sequence, screw tightening direction, and key points for insulation protection.

[0064] During this process, the six-degree-of-freedom robotic arm receives motion commands from the control system and calls upon more than 20 preset standard operation templates (such as "meter installation," "work order scanning," and "terminal crimping") to reproduce standard operation actions in real time. The robotic arm simultaneously records joint angle trajectories and force feedback data during execution, building a motion prototype library for subsequent comparison with trainees' behavior. The response latency is controlled within 200ms, ensuring a high degree of synchronization between the demonstration, voice narration, and AR visuals.

[0065] Through the above mechanism, the S400 achieves a complete closed loop of voice input, intelligent understanding, multi-channel output, and physical demonstration, enabling students to learn in an environment where they can hear clearly, see clearly, observe clearly, and imitate clearly, thus bridging the gap between theoretical knowledge and practical skills.

[0066] In step S500, learner actions are captured using an image acquisition device, and a Bayesian inference framework is used to model and evaluate the learner's ability state. Based on the evaluation results, an adaptive learning path sequence is generated. After completing the multimodal interactive demonstration in stage S400, the system enters the key evaluation and optimization phase of the training loop. This step achieves intelligent evolution from "teaching" to "evaluation" to "adjustment," constructing a personalized and evolvable learning path generation mechanism through continuous perception of the learner's operational process and dynamic inference of their cognitive state.

[0067] In practice, the system uses binocular vision sensors and depth cameras deployed in the training environment to collect real-time video streams of trainees' operations. This captures behavioral data such as hand movement trajectories, tool usage sequence, and equipment interaction logic as trainees perform tasks like meter installation and work order processing on AR sandboxes or physical simulation platforms. After preprocessing, the image data is input to the visual perception submodule, which uses a Transformer-based spatiotemporal motion detection model to extract key motion segments and combines this with a pose estimation algorithm to identify operational deviations, such as not wearing insulated gloves, improper crimping of wiring terminals, and meter installation angle misalignment.

[0068] Preferably, a dynamic Bayesian network structure is used to construct the knowledge space topology graph, with knowledge concepts as nodes and prerequisite / successor dependencies as edges. To achieve structured modeling of trainees' abilities, the system constructs a capability graph covering core knowledge points of electricity marketing, using abstract concepts such as "understanding time-of-use pricing policies," "mastering electricity billing processes," and "complying with meter installation standards" as latent variable nodes, and setting prerequisite and successor dependencies based on business logic. For example, mastering the skill of "handling abnormal electricity bills" depends on the state output of two prerequisite knowledge nodes: "understanding electricity pricing policies" and "standard work order completion," forming a knowledge space topology graph with a causal structure.

[0069] The observation data at each time step, after feature engineering, is input into the Bayesian update engine, which iteratively calculates the mastery probability using a recursive Bayesian filtering mechanism. During training, the system continuously collects multi-dimensional observation signals, including the accuracy rate of answering questions about electricity pricing policies, operation completion time, action standardization scores, and voice command response accuracy. These raw data are standardized, weighted, fused, and feature-encoded, transforming them into observation vectors that can be used for state updates, and then input into the Bayesian update engine.

[0070] Operational skills are broken down into atomic action units, and each atomic action unit is equipped with an independent mastery status tracker. To improve the granularity of evaluation, complex operational tasks are decomposed into indivisible atomic action units, such as "removing the meter's protective cover," "disconnecting the incoming power supply," and "calibrating the voltage sampling line." Each atomic action corresponds to an independent binary latent variable (mastered / not mastered) and has a prior distribution and update path to ensure fine-grained capability tracking.

[0071] The motion trajectory data output by the visual perception submodule is spatiotemporally aligned and mapped to a pre-defined motion template library. The DTW distance is calculated as the observation input. The system pre-establishes a standard motion template library, which contains high-precision motion prototypes executed and recorded by a robotic arm. The trainee's actual operation trajectory is matched with its corresponding template using a dynamic time warping algorithm. The calculated time warping distance serves as the core indicator of motion deviation, used to quantify the degree of operational inaccuracy, and is converted into an observation likelihood value to participate in Bayesian updates.

[0072] Preferably, when the robotic arm performs standard movements, it simultaneously records joint angle sequences and force feedback data to build a motion prototype library for deviation detection. During the demonstration phase, the six-DOF robotic arm collects the angle change curves of each joint, end effector pose information, and contact force feedback data in real time when performing each standard operation, forming a multimodal motion prototype. This prototype library is not only used for AR and physical demonstrations but also serves as a comparison benchmark for visual perception systems, supporting cross-modal motion consistency analysis.

[0073] Preferably, the transition probability is jointly modeled by the learning decay function and the skill consolidation model, that is, the transition probability is modeled as a combination function of the learning decay function and the skill consolidation model; the observation likelihood is generated based on the weighted fusion result of the action deviation threshold and the semantic matching score, that is, the observation likelihood is defined as the weighted sum of the action deviation threshold and the semantic matching score.

[0074] The Bayesian evaluation results are transmitted in real time to the personalized path generator via API. Knowledge nodes whose mastery probability is below a set threshold trigger reinforcement training tasks. When the posterior mastery probability of a knowledge node is below a preset threshold (e.g., 60%), the system automatically marks it as an "item to be reinforced" and pushes it to the path generation module. For example, if a trainee repeatedly misses the data comparison step in the "transformer line loss verification" process, the system determines that their understanding of the process is flawed and additional case training is required.

[0075] The path generator's state space is composed of Bayesian evaluation results, and the action space corresponds to selectable teaching content modules. The policy network uses an algorithm to output the optimal intervention sequence. The path generator is modeled using a reinforcement learning framework. Its state space consists of the mastery probability vectors of all current knowledge nodes, and its action space covers various teaching intervention methods such as theoretical explanation, AR simulation, robotic arm demonstration, and in-class quizzes. The reward function is designed as a weighted difference between the increase in mastery probability and resource consumption (time, computing power). After being trained using the PPO (Proximal Policy Optimization) algorithm, the policy network can output the optimal teaching sequence. For example, for new employees with weak foundations, the recommended path is "interpretation of electricity price policy, robotic arm demonstration, AR imitation, deviation correction"; while for on-the-job training personnel, it directly jumps to "complex work order drills, collaborative operation simulations".

[0076] All evaluations and path decisions are completed in the local edge computing unit. The Bayesian model, after being converted and quantized in ONNX format, is executed in the ONNX Runtime environment, with a single inference latency controlled within 50ms, meeting the requirements for real-time interaction. At the same time, the trainee's ability evolution data is synchronously written to the "Trainee Ability Evolution" subgraph of the background knowledge graph according to the timestamp, for use in subsequent model iterations and training strategy optimization.

[0077] Through the above steps, the S500 achieves a closed-loop process from behavior perception to capability modeling and then to personalized path generation, thereby achieving intelligent training goals and significantly improving training efficiency and skill attainment rates.

[0078] Building upon the completion of the S200 stage dynamic knowledge graph construction and the S300 initial model training process, we further deepen the integration mechanism between the large language model and domain knowledge to ensure that the artificial intelligence model not only has general semantic understanding capabilities, but also can accurately adapt to the actual business needs of different levels of positions such as municipal companies, district and county companies and power supply stations.

[0079] In another embodiment, preferably, step S300, based on a large language model architecture, further includes performing domain-adaptive training on the pre-trained language model by: using a dynamically updated knowledge graph as the data source for a job-specific customized large language model; extracting triple sequences related to specific jobs through SPARQL queries to construct contextualized question-and-answer pairs and inference training samples; in specific implementation, the system uses the structured knowledge graph output in S200 as the core data support and sets differentiated data extraction strategies for different job roles. For example, for the municipal company's electricity price policy analysis position, the system queries the triple links on the path of "electricity price policy terms, indicator impact, and decision deduction" to generate high-order inference samples such as "After the implementation of the new time-of-use electricity price, what is the expected range of change in the line loss rate of the transformer area?"; for district and county work order managers, the system extracts the subgraphs of "process nodes, compliance verification rules, and anomaly handling mechanisms" to construct a training dataset for process review.

[0080] For power supply station positions, examples were extracted for fine-tuning. Knowledge paths strongly related to on-site operations were selected from the knowledge graph, such as "safe operation procedures, meter installation actions, and risk warning conditions." At least 5,000 sets of operational logic triplets from real-world scenarios were extracted and transformed into natural language question-and-answer pairs by combining historical work order records and expert experience. For example, the answer to the question "When replacing an old meter, what power outage confirmation steps must be performed?" embeds the standard operating sequence, forming a fine-tuned corpus with educational value, improving the model's accuracy in understanding frontline operational scenarios.

[0081] This method employs LoRA (Local Low-Rank Adaptation) to update only the attention head parameters relevant to the power sector within the large language model. Based on a pre-trained language model built upon AI architectures such as DeepSeek, most parameters of the backbone network are frozen, and a low-rank factorization matrix is ​​injected into the key attention layer specifically to capture long-distance dependencies between power-related terms. This approach reduces training memory usage by over 60% while maintaining fundamental language capabilities, enabling the model to quickly focus on core tasks such as interpreting electricity pricing policies, process reasoning, and operational guidance, achieving efficient and lightweight domain transfer.

[0082] A bidirectional coupling interface is established between the large language model and the knowledge graph. During inference, the model retrieves the latest electricity pricing policy information by querying the knowledge graph in real time and returns an answer with a traceable path. When students ask questions, the model does not rely solely on static training memory but activates the knowledge graph query channel, accessing the Neo4j database via the SPARQL protocol to obtain currently valid electricity pricing policy clauses and process definitions. For example, when asked "What are the current peak-valley electricity pricing periods for residents?", the model retrieves the "time-of-use pricing policy" node with the latest timestamp and its associated information from the knowledge graph in real time, generating an answer such as "According to the document, the daily peak hours are 8:00–11:00 and 18:00–23:00," and automatically attaches a source link for verification.

[0083] Preferably, the bidirectional interface supports reverse writing: during training interactions, if trainees propose reasonable operational logic or expressions that have not been included, after manual review and confirmation by the backend, these can be injected into the knowledge graph as new knowledge nodes, driving the continuous expansion of the graph. This forms a collaborative evolutionary closed loop of "training the model through the graph and supplementing the graph with the reasoning model," ensuring that the knowledge system of the artificial intelligence system always evolves in sync with business development.

[0084] It is evident that the above steps enhance the professionalism, timeliness, and interpretability of the model trained in the S300 stage, making it the intelligent teaching hub that connects electricity pricing policies, processes, and practical operations, and providing a foundation for subsequent S400 multimodal interaction and S500 personalized assessment.

[0085] Figure 2 This is a schematic diagram of a training device based on dynamic knowledge graphs and multimodal interaction, provided as an embodiment of this application. As shown in the figure, the training device includes:

[0086] The data acquisition module 110 is used to acquire textual data of electricity pricing policies, processes, and operating procedures through the business system. Specifically, the data acquisition module 110 extracts heterogeneous data from the three-tier business system at the city, district / county, and power supply station levels using ETL tools. The heterogeneous data includes work order records from the marketing system, metering data from the user acquisition system, and unstructured electricity pricing policy documents and operating procedures. The extracted data is cleaned, key entities in the text are identified, and the key entities are normalized using a rule engine and a predefined ontology library to form standardized knowledge units. The standardized knowledge units are then injected into the database in the form of nodes.

[0087] The knowledge injection module 120 is used to inject the text data into a dynamically updated knowledge graph to form structured knowledge storage. The knowledge injection module 120 further includes an incremental update submodule, which is used to deploy the incremental update module to connect to the electricity price policy release API and the work order system message queue. When an electricity price policy change event is detected, the knowledge parsing pipeline is triggered; the text similarity algorithm is called to compare the old and new documents and locate the changed paragraphs; the newly added or adjusted entity relationships in the changed paragraphs are identified and a graph patch package is generated; node addition, deletion and edge reconstruction operations are performed to realize the dynamic update of the knowledge graph; wherein, each node in the knowledge graph is appended with a timestamp and version identifier, and time difference logic is introduced to model the impact path of the electricity price policy change on the downstream process before and after.

[0088] The model training module 130 is used to perform domain adaptation training on a pre-trained language model based on a large language model architecture, resulting in an artificial intelligence model with job-specific domain understanding capabilities. Specifically, the model training module 130 uses a dynamically updated knowledge graph as the data source for a job-specific customized large language model. It extracts triple sequences related to specific jobs through queries such as SPARQL, constructing contextualized question-and-answer pairs and inference training samples. For power supply station positions, instances are extracted for fine-tuning. LoRA low-rank adaptation technology is used to update only the attention head parameters related to the power industry in the large language model. A bidirectional coupling interface is established between the large language model and the knowledge graph. During inference, the latest electricity price policy basis is obtained by querying the knowledge graph in real time, and an answer with a traceable path is returned. During student interaction, the newly approved reasonable expression is written back into the knowledge graph, triggering the incremental update module to perform node addition and relationship reconstruction operations.

[0089] The multimodal interaction module 140 is used to receive natural language query commands through the speech recognition module and output visual operation guidance in conjunction with augmented reality devices, while simultaneously using a robotic arm to perform physical action demonstrations. The speech recognition module constructs a corpus containing dialect annotations, covering accent variants from multiple geographical regions, and uses a loss function for end-to-end training. It aligns dialect pronunciation variants to the Mandarin representation space through a phoneme mapping table and achieves terminology consistency recognition in conjunction with a dynamically expanded pronunciation dictionary. The acoustic model is deployed on an edge computing unit and performs low-latency online inference after accelerated quantization. When generating the answer, it constructs a response template based on the triplet data returned by the graph, and forcibly embeds the electricity price policy document number and process number as traceability fields through constraint decoding. The response text is post-processed to insert speech synthesis markers, and the speech is generated by the localization engine, with three preset role timbres and dynamically adjusted speech rate and intonation parameters based on the interaction history.

[0090] The behavior assessment module 150 is used to capture learner operational behaviors through an image acquisition device, model and assess the learner's ability status using a Bayesian inference framework, and generate an adaptive learning path sequence based on the assessment results. Specifically, the behavior assessment module 150 constructs a knowledge space topology graph using a dynamic Bayesian network structure, with knowledge concepts as nodes and prerequisite / successor dependencies as edges. The observation data at each time step is processed by feature engineering and then input into the Bayesian update engine, which iteratively calculates the mastery probability using a recursive Bayesian filtering mechanism. The transition probability is jointly modeled by the learning decay function and the skill consolidation model, and the observation likelihood is generated based on a weighted fusion of the action deviation threshold and semantic matching score. The skill is broken down into atomic action units, and each atomic action unit is equipped with an independent mastery state tracker. The motion trajectory data output by the visual perception submodule is mapped to a preset action template library after spatiotemporal alignment, and the DTW distance is calculated as the observation input. When the robotic arm performs standard actions, it simultaneously records the joint angle sequence and force feedback data to build an action prototype library for deviation detection. The Bayesian evaluation results are transmitted to the personalized path generator in real time through the API interface. Knowledge nodes with a mastery probability lower than a set threshold trigger reinforcement training tasks. The state space of the path generator is composed of the Bayesian evaluation results, the action space corresponds to the optional teaching content modules, and the policy network uses the PPO algorithm to output the optimal intervention sequence.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0093] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0095] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method based on dynamic knowledge graphs and multimodal interaction, characterized in that, S100. Obtain text data of electricity pricing policies, procedures, and operating specifications through the business system; S200. Inject the text data into the knowledge graph to form a structured knowledge storage; S300. Based on the large language model architecture, perform domain-adaptive training on the pre-trained language model; The S400 receives natural language query commands through a voice recognition module, outputs visual operation instructions in conjunction with augmented reality devices, and simultaneously demonstrates physical actions using a robotic arm. S500. Capture learner's operational behavior through an image acquisition device, model and evaluate learner's ability status, and generate an adaptive learning path sequence based on the evaluation results; Step S300, based on the large language model architecture, further includes performing domain-adaptive training on the pre-trained language model, including: S310. Use the dynamically updated knowledge graph as the data source for the job-customized large language model, query and extract the triple sequence related to specific jobs, and construct contextualized question-answering pairs and reasoning training samples; S320. For power supply station positions, extract examples for fine-tuning; S330. Update the attention head parameters related to the power domain in the large language model; S340. Establish a bidirectional coupling interface between the large language model and the knowledge graph. During inference, obtain the latest electricity price policy basis by querying the knowledge graph in real time and return the answer with the source path. Step S500, which generates an adaptive learning path sequence based on the evaluation results, includes: Operational skills are broken down into atomic action units, and each atomic action unit is equipped with an independent mastery status tracker; The motion trajectory data output by the visual perception submodule is mapped to a preset motion template library after spatiotemporal alignment, and the DTW distance is calculated as the observation input. When the robotic arm performs standard movements, it simultaneously records joint angle sequences and force feedback data, and builds a movement prototype library for deviation detection. Bayesian evaluation results are transmitted to the personalized path generator in real time via API interface, and knowledge nodes with a probability below a set threshold trigger reinforcement training tasks. The state space of the path generator is composed of Bayesian evaluation results, the action space corresponds to the optional teaching content modules, and the policy network uses the PPO algorithm to output the adaptive learning path sequence.

2. The training method according to claim 1, characterized in that, In S100, heterogeneous data is extracted from the three-level business systems at the city, district / county, and power supply station levels using ETL tools. The heterogeneous data includes work order records from the marketing system, metering data from the electricity consumption system, and unstructured electricity price policy documents and operating procedures. The extracted data is cleaned, key entities in the text are identified, and the key entities are normalized by combining the rule engine and the predefined ontology library to form standardized knowledge units. The standardized knowledge units are injected into the database in the form of nodes.

3. The training method according to claim 1, characterized in that, Step S200, injecting the text data into the knowledge graph to form structured knowledge storage, further includes: S210. Deploy the incremental update module to connect with the electricity price policy release API and the work order system message queue. When an electricity price policy change event is detected, the knowledge parsing pipeline is triggered. S220. Use a text similarity algorithm to compare the old and new files and locate the changed paragraphs; S230. Identify newly added or adjusted entity relationships in the modified paragraphs and generate a map patch package; S240. Perform node addition, deletion, and edge reconstruction operations.

4. The training method according to claim 3, characterized in that, S200 also includes the following steps: Add a timestamp and version identifier to each node in the knowledge graph; We introduce time difference logic modeling to model the impact path of electricity price policy changes on downstream processes.

5. The training method according to claim 4, characterized in that, During student interaction, the new and reasonable expression after review is written back into the knowledge graph, triggering the incremental update module to perform node addition and relationship reconstruction operations.

6. The training method according to claim 1, characterized in that, The S400's reception of natural language query commands via the speech recognition module further includes: We constructed a dialect-annotated corpus covering accent variants from multiple geographical regions and used a loss function for end-to-end training. Dialectal pronunciation variations are aligned to the Mandarin representation space using a phoneme mapping table, and terminology consistency recognition is achieved by combining a dynamically expanded pronunciation dictionary. The acoustic model is deployed on edge computing units and then executed for low-latency online inference after accelerated quantization. When generating the answer, the response template is constructed by combining the triple data returned by the graph, and the electricity price policy document number and process number are forcibly embedded as traceable fields through constraint decoding; The response text is post-processed and has speech synthesis markers inserted. The speech is generated by the localization engine, with three preset voice timbres and dynamic adjustment of speech rate and intonation parameters based on the interaction history.

7. The training method according to claim 1, characterized in that, In step S500, a Bayesian inference framework is used to model and assess the learner's ability status, including: A dynamic Bayesian network structure is used to construct a knowledge space topology graph, with knowledge concepts as nodes and successor-precedence dependencies as edges. The observation data at each time step is processed by feature engineering and then input into the Bayesian update engine, which uses a recursive Bayesian filtering mechanism to iteratively calculate the mastery probability. The transition probability is modeled by both the learning decay function and the skill consolidation model.

8. A training device based on dynamic knowledge graph and multimodal interaction, comprising a storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method of any one of claims 1 to 7.

Images