A personalized practical training guidance and evaluation method and system based on a private knowledge base

Abstract

The application relates to the field of artificial intelligence and industrial internet technology, in particular to a personalized practical training guidance and evaluation method and system based on a private knowledge base; the method comprises the following steps: acquiring user feature data, mapping a practical training task configuration file; generating guidance instructions and feeding back to an operation terminal; collecting and recording time sequence operation data; after the practical training task is completed, multi-dimensional comparison and analysis are carried out, and an evaluation report containing task completion quantitative data, expert operation model deviation degree analysis and parameter correction suggestions is generated; the application provides a personalized practical training guidance and evaluation method and system based on a private knowledge base, and solves the technical problems of lack of adaptability in practical training scheme configuration, lack of safe and controllable real-time guidance, and lack of objective quantitative means in practical skill evaluation in the prior art.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and industrial internet technology, and in particular to an intelligent guidance and automated quantitative assessment method and system based on a private knowledge base for industrial skills training scenarios. Background Technology

[0002] In modern industrial fields such as industrial automation, intelligent manufacturing, and robotics, the demand for highly skilled personnel is becoming increasingly urgent. However, existing industrial skills training models still face problems such as low digitalization, severe data silos, unsatisfactory teaching results, and strong subjectivity in evaluation. First, the training program configuration is rigid; the traditional model usually adopts unified teaching materials and fixed training tasks, and lacks a dynamic configuration generation mechanism based on the data characteristics of trainees; the data characteristics of trainees include at least professional background, historical operation level, learning ability and current training goals, resulting in low utilization of equipment resources and training efficiency, and making it difficult to achieve individualized teaching.

[0003] Secondly, on-site guidance lacks real-time capability and security. During equipment operation, trainees often encounter unexpected and specific problems. While existing general-purpose large model (LLM) possesses some question-and-answer capabilities, LLMs directly connected to the public internet pose a serious risk of knowledge illusion and are highly susceptible to leakage of sensitive data such as core processes, key parameters, and operating procedures. Industrial scenarios urgently need a companion guidance solution that is isolated from the public internet and based on private domain knowledge.

[0004] Finally, skills assessment lacks objective and fair unified standards. The evaluation of practical skills currently relies mainly on manual observation by teachers or trainers, which is highly subjective and lacks comprehensive data collection and automated analysis of the operational process. This makes it impossible to accurately quantify minor deviations in the operation process, nor to provide a comprehensive evaluation result reflecting proficiency, accuracy, and complexity based on objective data. Summary of the Invention

[0005] This invention aims to solve the technical problems existing in current technical education, such as the lack of adaptability in the configuration of practical training programs, the lack of safe and controllable real-time guidance, and the lack of objective quantitative means for the assessment of practical skills.

[0006] To address the aforementioned technical problems, this invention provides a personalized training guidance and evaluation method based on a private knowledge base, comprising: S101: Obtain user feature data submitted by the operation terminal. The user feature data includes at least professional background identifier, historical operation level rating and current training target. Based on the preset feature mapping model, map the user feature data into a training task configuration file containing specific task sequences, equipment control parameter constraints and assessment indicator weights to generate a training scheme based on user features. S102: In response to detecting that the training equipment has started running according to the training task configuration file, receive the operation query request submitted by the operation terminal, and retrieve the context information matching the operation query request in a private knowledge base that is physically or logically isolated from the public network and has pre-set professional domain knowledge data, generate guidance instructions and feed them back to the operation terminal. S103: Through the data interface connected to the training equipment, collect and record multi-dimensional time-series operation data in real time during the execution of the training task. The multi-dimensional time-series operation data includes at least: timestamp sequence of operation steps, sequence of execution actions, and equipment operation process parameters. S104: After the training task is completed, the multi-dimensional time-series operation data is compared and analyzed with the pre-stored expert operation model for the same training task in a multi-dimensional manner to perform automatic quantitative evaluation. The expert operation model is constructed based on a directed acyclic graph and includes a standard action node sequence, state transition conditions between nodes, reference time thresholds for each step, and qualified ranges for key parameters. S105: Based on the comparative analysis results, generate an evaluation report that includes quantified data on task completion, deviation analysis from the expert operation model, and parameter correction suggestions.

[0007] Furthermore, in step S102, before retrieving relevant information from the private knowledge base, the following steps are also included: The operation query request is semantically parsed and intent is identified using a natural language processing model to extract core keywords and intent vectors. The retrieval of context information matching the operation query request includes: using the intent vector to perform similarity matching in the vector database of the private knowledge base, obtaining the Top-N relevant knowledge fragments, and re-ranking them in combination with the core keywords.

[0008] Furthermore, the professional domain knowledge data stored in the private knowledge base includes structured and unstructured data, specifically including: design documents, design drawings, equipment operation manuals, standard operating procedures (SOP) documents, historical fault feature databases, and teaching video data with timestamps; the generation of guidance instructions includes: using retrieval-enhanced generation technology, inputting relevant knowledge fragments retrieved into a pre-trained generative model to generate guidance instructions in natural language form.

[0009] Furthermore, in step S103, the multi-dimensional time-series operation data also includes: equipment status change code or abnormal alarm code triggered by the training equipment; the acquisition is carried out through industry standard protocols such as OPC-UA or Modbus-TCP between the device side and the edge side, and the edge side communicates with the platform side through the MQTT protocol, and the acquisition frequency is not less than 10Hz.

[0010] Furthermore, the expert operation model also includes a standard error handling subprocess associated with a specific node of the directed acyclic graph; The multi-dimensional comparative analysis also includes: when the multi-dimensional time-series operation data is detected to deviate from the standard action node sequence, determining whether it matches the standard error handling sub-process; if it matches, evaluating the standardization of its abnormal handling operation; if it does not match, determining it as an operation error.

[0011] Furthermore, the evaluation report generated in step S105 also includes quantitative scores for each sub-dimension, which includes at least: operation accuracy evaluation based on action node matching degree, operation proficiency evaluation based on timestamp sequence, and complexity evaluation based on preset weight calculation.

[0012] Furthermore, a personalized training guidance and evaluation system based on a private knowledge base is also provided, which includes: The user feature acquisition module is used to acquire user feature data submitted by the operation terminal. The task configuration generation module is used to map the user feature data into a training task configuration file containing a specific task sequence, equipment control parameter constraints, and assessment indicator weights based on a preset feature mapping model. The storage module is used to store a private knowledge base and expert operation model database that are isolated from the public network; The interactive guidance module is used to receive operation query requests during the operation of the training equipment, retrieve matching information from the private knowledge base of the storage module, generate guidance instructions, and provide feedback. The data acquisition module is used to collect multi-dimensional time-series operation data in real time during the task execution process through the data interface; An automated evaluation module is used to compare and analyze the multi-dimensional time-series operation data with the expert operation model pre-stored in the storage module, which is constructed based on a directed acyclic graph, and generate an evaluation report that includes both quantitative scoring and qualitative evaluation.

[0013] Furthermore, the interactive guidance module includes: The intent recognition unit is used to identify the core intent of the query request using natural language processing algorithms. The retrieval generation unit is used to perform vector retrieval in the private knowledge base based on the intent recognition results, and to integrate and generate guidance instructions using retrieval enhancement generation technology.

[0014] Furthermore, the automated evaluation module includes: The data comparison unit is used to perform node matching, parameter comparison, and proficiency analysis based on timestamp sequences between the multi-dimensional time-series operation data and the expert operation model. The report generation unit is used to calculate the scores for each dimension based on the comparison results and generate a structured evaluation report that includes both quantitative scoring and qualitative evaluation content.

[0015] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the aforementioned personalized training guidance and evaluation methods based on a private knowledge base.

[0016] Compared with the prior art, the beneficial effects of this invention are significant and multifaceted: 1. Improved adaptability of training configuration: By introducing user characteristic data including professional background, historical operation level, learning ability and current training objectives, differentiated task configuration files are generated, and personalized training solutions are further formed to achieve individualized teaching.

[0017] 2. Enhanced accompanying guidance and security controllability: Based on a privately deployed knowledge base and RAG technology, it provides real-time guidance in physically or logically isolated environments, effectively reducing the risk of knowledge illusion and sensitive data leakage.

[0018] 3. Improved objectivity and fairness of training evaluation: By automatically analyzing multi-dimensional time-series operational data, an evaluation report is generated that includes both quantitative scoring and qualitative evaluation, providing clear data support for the evaluation results.

[0019] 4. Multi-dimensional evaluation is achieved: The practical process of trainees is comprehensively evaluated based on dimensions such as accuracy, proficiency, and complexity, thereby improving the interpretability of evaluation results and their suitability for teaching. Attached Figure Description

[0020] To enable those skilled in the art to more clearly and comprehensively understand the technical solutions of the present invention, preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the accompanying drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the accompanying drawings: Figure 1 This is a schematic diagram of a system application scenario according to an embodiment of the present invention.

[0021] Figure 2 This is a system functional module block diagram of an embodiment of the present invention.

[0022] Figure 3 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, several preferred embodiments of the invention will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely for explaining the invention and do not constitute any limitation on its scope of protection. Any modifications, equivalent substitutions, or improvements made based on the spirit and principles of this invention should be included within the scope of protection of this invention.

[0024] Example 1: This example provides a complete process for a personalized training guidance and evaluation method based on a private knowledge base, such as... Figure 3 As shown. This method can be executed by a system deployed on a local server or edge computing device, ensuring that data does not leave the internal network.

[0025] Step S101: Obtain user feature data and generate training task configuration file.

[0026] The system first receives user feature data through the operating terminal. For example, user A's feature data is {Major ID: "Automation", Grade: 1, Historical Rating: null, Learning Ability: "Medium", Current Training Goal: "Basic Operation of Industrial Robots"}, and user B's feature data is {Major ID: "Computer Science", Grade: 4, Historical Rating: "A", Learning Ability: "Relatively Strong", Current Training Goal: "Visual Collaboration"}.

[0027] The task configuration generation module inside the system uses preset mapping rules or classification models to map the above feature data into machine-readable configuration files; the configuration files are further used to generate personalized training schemes that match the current training objectives.

[0028] Example configuration files are as follows: { "task_id": "T-101", "safety_constraints": {"max_speed": 0.5, "force_limit": 10N}, "target_points": [[100, 200, 300], [150, 250, 350]], "evaluation_weights": {"accuracy": 0.6, "efficiency": 0.4} } For user A, the system generates a basic task configuration, limiting the maximum axis speed to 50%; for user B, the system generates an advanced task configuration, opening TCP / IP communication interface permissions and setting a higher weight for code execution efficiency assessment.

[0029] Step S102: Provide interactive guidance based on a private knowledge base.

[0030] During the operation of the training equipment, the system parses the user's query through the natural language processing unit.

[0031] For example, a user asks, "How do I switch coordinate systems?" The system first identifies the intent as [Query: Coordinate_System_Switch].

[0032] The system then performs a search in a private knowledge base. This knowledge base pre-stores data such as design documents, design drawings, equipment manuals, SOP documents, a fault case library, and instructional videos, and has undergone vectorization processing.

[0033] The system retrieves relevant SOP sections, design document snippets, or instructional video clips, and uses Retrieval Enhanced Generation (RAG) technology to generate a message similar to: "Please click the 'Coordinates' button on the main interface of the teach pendant... For details, please refer to the video [link]" and reply to the user. The entire process ensures data security by avoiding public network access.

[0034] Step S103: Collect multi-dimensional time-series operation data in real time during the practical operation process.

[0035] The data acquisition module connects to the training equipment via industry-standard protocols (such as OPC-UA, Modbus-TCP) or vendor APIs between the device side and the edge side, and communicates between the edge side and the platform side via the MQTT protocol.

[0036] The collected data is multi-dimensional time-series data, including: 1. Equipment operating parameters: equipment operating status, alarm or pre-alarm status, standby status, shutdown status, current, temperature, or other parameters reflecting the health status of the equipment; 2. Process parameters: Process execution parameters related to the object being machined or trained, such as cutting speed, feed rate, and magnification. 3. Event data: button press, command sent, alarm triggered, etc.; 4. Timestamp data: Each operation step, each action node, and the entire action sequence is timestamped.

[0037] The sampling frequency is not less than 10Hz, and preferably can reach the 100ms level.

[0038] Step S104: Perform automatic quantitative and qualitative evaluation analysis based on expert models.

[0039] This is the core step of the invention. The system compares the collected time-series data with the pre-stored "expert operation model".

[0040] The expert operation model is constructed based on a directed acyclic graph (DAG).

[0041] The comparison process includes: 1. Sequence Alignment: The Dynamic Time Warping (DTW) algorithm is used to map the user's actual operation sequence to the nodes of the expert DAG model.

[0042] 2. Deviation Calculation: Calculate the Euclidean distance between the actual trajectory and the standard trajectory (accuracy score), and calculate the ratio of actual time consumption to reference time consumption (efficiency score).

[0043] 3. Logical verification: Check for undefined node jumps (i.e., incorrect operation order) or redundant loops.

[0044] 4. Anomaly Handling Evaluation: When an operation deviates from the standard path, the system will further search the DAG to see if there is a corresponding "error handling sub-process". For example, if the user triggers an emergency stop while moving, the system will check whether the user subsequently performed "reset" and "return to origin" operations. If they were performed, it is determined that the "anomaly handling is correct"; otherwise, it is determined that the "operation is incorrect".

[0045] Step S105: Generate a multi-dimensional evaluation report.

[0046] The system generates an evaluation report based on the calculation results of S104. The evaluation report includes at least a quantitative scoring section and a qualitative evaluation section.

[0047] For example, the evaluation report may include: Task completion rate: 100%; Operational accuracy score: 98 points; Operational proficiency evaluation: The loading and moving processes are smooth and the operation is relatively proficient, but the reaction time for some steps is a bit long; Overall complexity evaluation: Under the current training task complexity weights, the overall performance is good; Improvement suggestion: It was detected that the axis speed setting was only 50% during the PTP movement process. It is recommended to increase it to 80% to optimize the cycle time, while ensuring safety.

[0048] Example 2: This example provides a personalized training guidance and evaluation system based on a private knowledge base, such as... Figure 2 As shown. The system includes: 1. User Feature Acquisition Module: Used to receive user login information and background data.

[0049] 2. Task Configuration Generation Module: Internally integrates a rule engine to generate machine-readable task configuration files based on user feature data, and forms personalized training schemes based on the task configuration files.

[0050] 3. Storage module: includes a vectorized private knowledge base and an expert model database; the private knowledge base stores resources such as SOPs, equipment manuals, design documents, design drawings, fault case libraries, and teaching videos.

[0051] 4. Interactive guidance module: includes BERT intent recognition unit and RAG retrieval generation unit.

[0052] 5. Data Acquisition Module: Used to acquire device-side data via protocols such as OPC-UA or Modbus-TCP, and transmit data from the edge to the platform side via the MQTT protocol.

[0053] 6. Automated Evaluation Module: This module includes a sequence matching algorithm, scoring logic, and qualitative evaluation analysis logic, used to output a final evaluation report that simultaneously includes quantitative scoring and qualitative evaluation content. Finally, it should be emphasized that the embodiments described above are merely illustrative of the technical concept and preferred implementation of the present invention, and are not intended to exhaustively describe or limit the scope of protection of the present invention. Any person skilled in the art, after understanding the spirit and core technical solutions of the present invention, may make various modifications, equivalent substitutions, or improvements based on the content disclosed in the present invention without departing from the basic principles of the present invention. These obvious variations or substitutions should all be considered to be included within the scope of protection claimed by the present invention.

Claims

1. A personalized training guidance and evaluation method based on a private knowledge base, characterized in that, include: S101: Obtain user feature data submitted by the operation terminal. The user feature data includes at least a professional background identifier, historical operation level rating, and current training target. Based on a preset feature mapping model, map the user feature data into a training task configuration file containing a specific task sequence, equipment control parameter constraints, and assessment indicator weights. S102: In response to detecting that the training equipment has started running according to the training task configuration file, receive the operation query request submitted by the operation terminal, and retrieve the context information matching the operation query request in a private knowledge base that is physically or logically isolated from the public network and has pre-set professional domain knowledge data, generate guidance instructions and feed them back to the operation terminal. S103: Through the data interface connected to the training equipment, collect and record multi-dimensional time-series operation data in real time during the execution of the training task. The multi-dimensional time-series operation data includes at least: timestamp sequence of operation steps, sequence of execution actions, and equipment operation process parameters. S104: After the training task is completed, the multi-dimensional time-series operation data is compared and analyzed with the pre-stored expert operation model for the same training task in a multi-dimensional manner to perform automatic quantitative evaluation. The expert operation model is constructed based on a directed acyclic graph and includes a standard action node sequence, state transition conditions between nodes, reference time thresholds for each step, and qualified ranges for key parameters. S105: Based on the comparative analysis results, generate an evaluation report that includes quantified data on task completion, deviation analysis from the expert operation model, and parameter correction suggestions.

2. The personalized training guidance and evaluation method based on a private knowledge base according to claim 1, characterized in that, In step S102, before retrieving relevant information from the private knowledge base, the method further includes: The operation query request is semantically parsed and intent is identified using a natural language processing model to extract core keywords and intent vectors. The retrieval of context information matching the operation query request includes: using the intent vector to perform similarity matching in the vector database of the private knowledge base, obtaining the Top-N relevant knowledge fragments, and re-ranking them in combination with the core keywords.

3. The personalized training guidance and evaluation method based on a private knowledge base according to claim 1, characterized in that, The professional domain knowledge data stored in the private knowledge base includes structured and unstructured data, specifically including: design documents, design drawings, equipment operation manuals, standard operating procedures (SOP) documents, historical fault feature databases, and teaching video data with timestamps; the generation of guidance instructions includes: using retrieval-enhanced generation technology, inputting relevant knowledge fragments retrieved into a pre-trained generative model to generate guidance instructions in natural language form.

4. The personalized training guidance and evaluation method based on a private knowledge base according to claim 1, characterized in that, In step S103, the multi-dimensional time-series operation data further includes: equipment status change code or abnormal alarm code triggered by the training equipment; the acquisition is carried out through industry standard protocols such as OPC-UA or Modbus-TCP between the equipment side and the edge side, and the edge side communicates with the platform side through the MQTT protocol, and the acquisition frequency is not less than 10Hz.

5. A personalized training guidance and evaluation method based on a private knowledge base according to claim 1, characterized in that, The expert operation model also includes a standard error handling sub-process associated with a specific node of the directed acyclic graph; The multi-dimensional comparative analysis also includes: when the multi-dimensional time-series operation data is detected to deviate from the standard action node sequence, determining whether it matches the standard error handling sub-process; If a match is found, the standardization of the exception handling operation is evaluated; if a match is not found, it is determined to be an operational error.

6. The personalized training guidance and evaluation method based on a private knowledge base according to claim 1, characterized in that, The evaluation report generated in step S105 also includes quantitative scores for each sub-dimension, which includes at least: operation accuracy evaluation based on action node matching degree, operation proficiency evaluation based on timestamp sequence, and complexity evaluation based on preset weight calculation.

7. A personalized practical training guidance and evaluation system based on a private knowledge base, characterized in that, include: The user feature acquisition module is used to acquire user feature data submitted by the operation terminal. The task configuration generation module is used to map the user feature data into a training task configuration file containing a specific task sequence, equipment control parameter constraints, and assessment indicator weights based on a preset feature mapping model. The storage module is used to store a private knowledge base and expert operation model database that are isolated from the public network; The interactive guidance module is used to receive operation query requests during the operation of the training equipment, retrieve matching information from the private knowledge base of the storage module, generate guidance instructions, and provide feedback. The data acquisition module is used to collect multi-dimensional time-series operation data in real time during the task execution process through the data interface; An automated evaluation module is used to compare and analyze the multi-dimensional time-series operation data with the expert operation model pre-stored in the storage module, which is constructed based on a directed acyclic graph, and generate an evaluation report that includes both quantitative scoring and qualitative evaluation.

8. A personalized training guidance and evaluation system based on a private knowledge base according to claim 7, characterized in that, The interactive guidance module includes: The intent recognition unit is used to identify the core intent of the query request using natural language processing algorithms. The retrieval generation unit is used to perform vector retrieval in the private knowledge base based on the intent recognition results, and to integrate and generate guidance instructions using retrieval enhancement generation technology.

9. A personalized practical training guidance and evaluation system based on a private knowledge base according to claim 7, characterized in that, The automated evaluation module includes: The data comparison unit is used to perform node matching, parameter comparison, and proficiency analysis based on timestamp sequences between the multi-dimensional time-series operation data and the expert operation model. The report generation unit is used to calculate the scores for each dimension based on the comparison results and generate a structured evaluation report that includes both quantitative scoring and qualitative evaluation content.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a personalized training guidance and evaluation method based on a private knowledge base as described in any one of claims 1 to 6.

Images