Electromechanical operation and maintenance teaching knowledge question-answering and evaluation method based on domain knowledge graph and large model
By combining domain knowledge graphs and large language models, a domain expert model is constructed, which solves the problems of lagging behind and insufficient professional depth in traditional operation and maintenance training, realizes intelligent operation and maintenance teaching and assessment, and improves the accuracy of question and answer and the professionalism of exercise generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHWEST NORMAL UNIVERSITY
- Filing Date
- 2026-01-20
- Publication Date
- 2026-04-24
AI Technical Summary
Traditional operation and maintenance training suffers from outdated knowledge, high costs, and insufficient personalization. Furthermore, the application of large language models in the field of electromechanical operation and maintenance exhibits insufficient professional depth and errors, making it difficult to meet the needs of precise teaching and assessment.
A two-stage strategy based on domain knowledge graphs and large models is adopted. By constructing domain expert models and combining knowledge graphs and large language models, an intelligent question answering and evaluation system is realized. LoRA fine-tuning technology is used to reduce training costs and generate high-quality operation and maintenance knowledge question answer pairs.
It has enabled intelligent teaching in the field of electromechanical operation and maintenance, improved the accuracy of question and answer and the professionalism of exercise generation, provided personalized skills assessment tools, and reduced training costs.
Smart Images
Figure CN121920471A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of highway electromechanical operation and maintenance technology, and in particular to a teaching knowledge question-and-answer and assessment method for electromechanical operation and maintenance based on domain knowledge graphs and large models. Background Technology
[0002] As highway electromechanical equipment becomes increasingly intelligent and complex, higher demands are placed on the professional skills and knowledge of maintenance personnel. Traditional maintenance training relies heavily on paper manuals and centralized lectures, which suffers from drawbacks such as outdated knowledge, high training costs, and insufficient personalization. In recent years, large language models have demonstrated powerful general capabilities in natural language processing tasks; however, when directly applied to highly specialized fields such as electromechanical maintenance, they often exhibit problems such as "illusion" phenomena, factual errors, and insufficient professional depth, making it difficult to meet the needs of precise teaching and assessment.
[0003] Knowledge graphs can encapsulate entities, concepts, and their relationships within a domain in a structured form, providing an accurate and reliable semantic foundation for operating knowledge systems. Combining the accuracy of knowledge graphs with the powerful generation and reasoning capabilities of large language models is considered an effective approach to building domain expert systems. Currently, related research still faces challenges: firstly, how to efficiently and cost-effectively inject structured knowledge into parameterized models; and secondly, how to construct a closed-loop teaching system integrating knowledge question answering and ability assessment, rather than a single-function implementation. Summary of the Invention
[0004] The purpose of this invention is to address the practical needs of electromechanical operation and maintenance (EMG) training by proposing a Q&A and assessment method for EMG operation and maintenance knowledge based on domain knowledge graphs and large-scale models. The main contributions of this invention include: 1. A method for automatically constructing domain data based on knowledge graphs and prompting engineering is proposed, which transforms triple knowledge into high-quality question-answer pairs, effectively solving the problem of scarce domain fine-tuning data.
[0005] 2. Construct a domain expert model for electromechanical operation and maintenance, and adopt a two-stage strategy of "domain pre-training + LoRA fine-tuning" to significantly reduce training costs while ensuring the professional performance of the model.
[0006] 3. Design and implement a teaching system that integrates intelligent question answering and automatic evaluation. This system can not only provide accurate Q&A on fault knowledge, but also automatically generate multiple types of exercises based on specified knowledge points, providing intelligent tools for the skills assessment and personalized learning of maintenance personnel.
[0007] To achieve the above objectives, the present invention provides the following solution: A question-and-answer and assessment method for electromechanical operation and maintenance teaching based on domain knowledge graphs and large models includes: Obtain the operation and maintenance guide text data, preprocess the operation and maintenance guide text data, and obtain the token sequence data required for pre-training; The knowledge graph data of highway electromechanical equipment faults is transformed into operation and maintenance knowledge question and answer pairs data. The token sequence data is input into a preset large language model for model pre-training; The pre-trained model is fine-tuned using the aforementioned operation and maintenance knowledge Q&A data to obtain a domain expert model. Question answering evaluation is applied based on the aforementioned domain expert model.
[0008] Optionally, preprocessing the operation and maintenance guide text data includes: The operation and maintenance guide text data is subjected to information filtering, redundant data removal, and word segmentation operations to retain key operation and maintenance information text. In the information filtering stage, the established keyword recognition technology is used to identify and remove noise and irrelevant data in the text. In the redundancy removal stage, duplicate information is removed using screening techniques; In the word segmentation stage, the organized operation and maintenance text is segmented to generate the token sequence required for pre-training.
[0009] Optionally, the knowledge graph data on highway electromechanical equipment faults can be converted into operation and maintenance knowledge question-and-answer pairs, including: The structural triples in the knowledge graph of highway electromechanical equipment faults are transferred to the general large model, and questions and answers are generated by combining them with preset prompt engineering statements; Based on the generated questions and answers, and with the guidance of experts in the field of transportation operation and maintenance, a semi-automatic filtering and proofreading process is carried out using scripts.
[0010] Optionally, the general large model adopts the GPT-4o model.
[0011] Optionally, the preset large language model adopts the LLaMa2 model.
[0012] Optionally, before inputting the token sequence data into a preset large language model, the following steps are included: An embedding method is used to map the token sequence data into a real number vector, and root mean square layer normalization is used to remove the mean in the calculation.
[0013] Optionally, inputting the token sequence data into a preset large language model includes: Convert the input vector into a query vector matrix, a key matrix, and a value matrix, respectively. The query vector and key vector are subjected to rotational positional encoding, which uses absolute positional encoding to achieve relative positional encoding. Perform matrix dot product on the encoded query vector and key vector to obtain the attention score; The Softmax operation is used to assign attention weights in the range [0, 1] to different attention scores; The calculated attention weights are multiplied by the Value matrix to form the output of the self-attention layer; The output vector is obtained through a feedforward neural network, thus completing the vector pre-training operation.
[0014] Optionally, fine-tuning the pre-trained model using the operation and maintenance knowledge Q&A data includes: Based on the aforementioned operation and maintenance knowledge question-and-answer pair, a low-rank adaptation technique based on a large language model is used to fine-tune the pre-trained model.
[0015] Optionally, question-answering evaluation applications based on the domain expert model include: When a user inputs a natural language question, the domain expert model directly generates a well-structured and professionally designed answer. At the same time, the question-and-answer record can be exported as a standardized document with one click. When a user inputs any knowledge point, the domain expert model automatically generates various question types covering that knowledge point. The generated questions are used to build test papers to assess maintenance personnel. The domain expert model automatically scores the answers or provides reference answers, thus achieving a closed-loop evaluation of teaching effectiveness.
[0016] The beneficial effects of this invention are as follows: This invention addresses the intelligent needs of highway electromechanical operation and maintenance (EMG) teaching and training by proposing a Q&A and assessment method based on domain knowledge graphs and large-scale models. Through automated construction of domain-specific fine-tuning data and the adoption of efficient LoRA fine-tuning technology, specialized EMG knowledge is successfully injected into a general large-scale model, constructing a domain expert model. The system developed based on this model demonstrates excellent performance in both intelligent Q&A and exercise generation: accuracy is significantly improved in specialized Q&A tasks; and the generated exercises are recognized by experts for their high accuracy and practicality. This invention provides an effective intelligent solution for personnel training, skills assessment, and knowledge management in the EMG field, and has promising application prospects. Future work will focus on optimizing the difficulty control mechanism of exercises and exploring the application of multimodal knowledge in teaching systems. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a schematic diagram of a teaching knowledge question-and-answer and assessment method for electromechanical operation and maintenance based on domain knowledge graphs and large models, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the Neo4j visualization results of the electromechanical operation and maintenance knowledge graph in an embodiment of the present invention; Figure 3 This is a logical structure diagram of the LLaMa2 model according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the LoRA fine-tuning principle of an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the generative operation and maintenance guidance of an embodiment of the present invention; Figure 6 This is a log illustration of the generative guidance process implementation according to an embodiment of the present invention; Figure 7 This is a schematic diagram illustrating the generation of intelligent exercises according to an embodiment of the present invention; Figure 8 This is a schematic diagram showing the log generated by the intelligent exercises in an embodiment of the present invention. Detailed Implementation
[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0021] like Figure 1 As shown, this embodiment proposes a question-and-answer and assessment method for electromechanical operation and maintenance teaching knowledge based on domain knowledge graphs and large models, including: Obtain the operation and maintenance guide text data, preprocess the operation and maintenance guide text data, and obtain the token sequence data required for pre-training; The knowledge graph data of highway electromechanical equipment faults is transformed into operation and maintenance knowledge question and answer pairs data. The token sequence data is input into a preset large language model for model pre-training; The pre-trained model is fine-tuned using the aforementioned operation and maintenance knowledge Q&A data to obtain a domain expert model. Question answering evaluation is applied based on the aforementioned domain expert model.
[0022] Furthermore, the preprocessing of the operation and maintenance guide text data includes: The operation and maintenance guide text data is subjected to information filtering, redundant data removal, and word segmentation operations to retain key operation and maintenance information text. In the information filtering stage, the established keyword recognition technology is used to identify and remove noise and irrelevant data in the text. In the redundancy removal stage, duplicate information is removed using screening techniques; In the word segmentation stage, the organized operation and maintenance text is segmented to generate the token sequence required for pre-training.
[0023] Furthermore, the knowledge graph data on highway electromechanical equipment faults will be transformed into operation and maintenance knowledge question-and-answer pairs data, including: The structural triples in the knowledge graph of highway electromechanical equipment faults are transferred to the general large model, and questions and answers are generated by combining them with preset prompt engineering statements; Based on the generated questions and answers, and with the guidance of experts in the field of transportation operation and maintenance, a semi-automatic filtering and proofreading process is carried out using scripts.
[0024] Specifically, in this embodiment, the construction of domain data includes: High-quality domain data is the foundation for fine-tuning large models. This study first organizes electromechanical operation and maintenance technical guidelines and related policy documents, performing information filtering, redundant data removal, and token segmentation on the raw data to retain key operation and maintenance information. In the information filtering stage, a pre-defined keyword recognition technology is used to identify and remove noise and irrelevant data from the text. In the redundancy removal stage, it was found that the operation and maintenance documents contained duplicate records of the same factual information. Therefore, this embodiment uses filtering techniques to remove duplicate information to improve token information processing efficiency. Finally, in the token segmentation stage, the organized operation and maintenance text is segmented to generate the token sequence required for pre-training.
[0025] Furthermore, utilizing the existing knowledge graph of highway electromechanical equipment faults, the statistical results of the number of relation types and the visualization results are shown in Table 1 and Table 2, respectively. Figure 2As shown, based on a general large model, triples in the form of (head entity, relation, tail entity) are transformed into natural language question-answer pairs. The implementation process is as follows: First, the structural triples in the knowledge graph of highway electromechanical equipment faults are uploaded to GPT-4o, and questions and answers are generated by combining them with self-made prompt engineering statements; then, the generated content is stored in the format of "Question" and "Answer"; finally, under the guidance of experts in the field of transportation operation and maintenance, semi-automatic filtering and proofreading are performed using scripts, including supplementing incomplete content and deleting erroneous information. The self-made prompt engineering statements are shown in Table 2.
[0026] Table 1. Statistics on the number of different relationship types Table 2. Custom-made prompt statements Furthermore, before inputting the token sequence data into the preset large language model, the following steps are included: An embedding method is used to map the token sequence data into a real number vector, and root mean square layer normalization is used to remove the mean in the calculation.
[0027] Furthermore, inputting the token sequence data into a preset large language model includes: Convert the input vector into a query vector matrix, a key matrix, and a value matrix, respectively. The query vector and key vector are subjected to rotational positional encoding, which uses absolute positional encoding to achieve relative positional encoding. Perform matrix dot product on the encoded query vector and key vector to obtain the attention score; The Softmax operation is used to assign attention weights in the range [0, 1] to different attention scores; The calculated attention weights are multiplied by the Value matrix to form the output of the self-attention layer; The output vector is obtained through a feedforward neural network, thus completing the vector pre-training operation.
[0028] Specifically, in this embodiment, model fine-tuning training includes: This embodiment uses traffic operation and maintenance technical guidelines and Q&A data to pre-train and fine-tune the general-purpose LLaMa2 model, enabling it to better understand and complete tasks related to intelligent Q&A in traffic operation and maintenance. The LLaMa2 model employs an autoregressive language method and includes three versions: 7B, 13B, and 70B. Its logical structure is as follows: Figure 3 As shown.
[0029] The implementation process of the LLaMa2 model is as follows: First, the input operation and maintenance information is segmented and converted into a token sequence; then, an embedding method is used to map the tokens into real number vectors; finally, root mean square layer normalization and mean removal calculation are used to enhance the stability of training. This process can be expressed as follows: (1); (2); in, These are learnable parameters.
[0030] Similar to the Transformer architecture, LLaMa2 transforms the input vector into a query vector matrix, a key matrix, and a value matrix, respectively. Then, it performs Rotary Position Embedding (RoPE) on the query and key vectors. RoPE achieves relative position encoding using absolute position encoding, thus improving model expressiveness while maintaining computational efficiency. Furthermore, to represent the importance of different positions to the current position, it performs matrix multiplication on the query and key vectors to obtain an attention score. Its expression is as follows: (3); in, The Softmax operation assigns attention weights in the range [0, 1] to different attention scores. The calculation process can be represented as follows: (4); (5); (6); The attention weights calculated above are multiplied by the Value matrix to form the output of the self-attention layer. Finally, the output vector is obtained through a feed-forward network (FFN). This completes the vector pre-training operation.
[0031] Furthermore, fine-tuning the pre-trained model using the aforementioned operation and maintenance knowledge question-and-answer method includes: Based on the aforementioned operation and maintenance knowledge question-and-answer pair, a low-rank adaptation technique based on a large language model is used to fine-tune the pre-trained model.
[0032] Considering that the pre-trained LLaMa2 model may still suffer from the illusion problem when handling domain-specific question-answering tasks, this embodiment uses an electromechanical maintenance question-answering dataset to fine-tune the model. Furthermore, to reduce the computational cost of full parameter fine-tuning, a low-rank adaptation of large language (LoRA) method based on a large language model is employed. The LoRA method constructs an updated model of model parameters through low-rank decomposition techniques. It uses a pair of linear projection matrices to perform a low-rank approximation of the increment matrix to capture the specific messages required for this task, thereby significantly reducing the number of parameters. Its principle is as follows: Figure 4 As shown.
[0033] exist Figure 4 In the image, the blue section on the left represents the pre-trained model parameters, with input and output dimensions of [missing information]. , Let be the rank of the model. and These are two smaller matrices. In the early stages of training, and When the parameter is set to 0, it is expressed as follows: (7); It can also be expressed as: (8); in, This represents the adjusted weight value. The updated weight values, These are the pre-trained weight values.
[0034] get After the low-rank update, the scaling factor can be adjusted. The value of is used to enhance the model's adaptability to new training iterations while maintaining its performance. Its expression is as follows: (9); As can be seen from the above process, the LoRA model does not directly fine-tune the pre-trained parameters, but achieves an approximate fine-tuning effect by optimizing the low-rank decomposition matrix, thereby reducing the computational complexity and storage cost of the model.
[0035] Furthermore, the application of question-answering evaluation based on the aforementioned domain expert model includes: When a user inputs a natural language question, the domain expert model directly generates a well-structured and professionally designed answer. At the same time, the question-and-answer record can be exported as a standardized document with one click. When a user inputs any knowledge point, the domain expert model automatically generates various question types covering that knowledge point. The generated questions are used to build test papers to assess maintenance personnel. The domain expert model automatically scores the answers or provides reference answers, thus achieving a closed-loop evaluation of teaching effectiveness.
[0036] Specifically, in this embodiment, intelligent question answering and exercise generation are applied: Based on the fine-tuned domain expert model, the following two core applications are built: Intelligent Question Answering System: Users input natural language questions, and the system leverages the powerful understanding capabilities of its finely tuned model to directly generate clear, well-structured, and professionally relevant answers. Simultaneously, the system supports one-click export of question-and-answer records into standardized documents, facilitating knowledge preservation and sharing.
[0037] Intelligent Exercise Generation and Assessment System: Users input any knowledge point (such as "transformer input voltage"), and the system can automatically generate various question types (multiple choice, fill-in-the-blank, short answer) covering that knowledge point. The generated exercises can be used to create test papers to assess maintenance personnel. The system can automatically score the answers or provide reference answers, achieving a closed-loop evaluation of teaching effectiveness.
[0038] Specifically, in this embodiment, the experiments and analyses are as follows: To verify the effectiveness of combining application domain knowledge graphs and large models in intelligent question answering tasks, this embodiment sets up question answering statement tests and operation and maintenance question bank tests to provide corresponding proof of effectiveness.
[0039] Experimental setup: Baseline models: ChatGPT-4, LLaMa3.1-8b, and unadjusted LLaMa2 were selected as the comparison baselines.
[0040] Evaluation dataset: Professional Question Bank: Three sets of multiple-choice test papers on electromechanical operation and maintenance (TF_1, TF_2, TF_3) compiled by experts in the field, each with a full score of 100 points.
[0041] Exercise Quality Assessment Set: 100 exercises generated by the system for 10 core knowledge points, which are then scored on a 5-point scale by 3 domain experts from three dimensions: “content accuracy”, “scenario relevance”, and “difficulty appropriateness”.
[0042] Results and Analysis: (a) Question and answer statement test experiment: For the same traffic operation and maintenance problem, this embodiment compares and analyzes the proposed model with ChatGPT-4, LLaMa3.1-8b and the untuned LLaMa2 model. The results of the question and answer are shown in Tables 3 and 4.
[0043] Table 3. Comparison of Question-Answering Results for Different Large Models Table 4. Comparison of Question-Answering Results for Different Large Models By comparing the above models, it can be found that for electromechanical maintenance professional questions, the ChatGPT-4 and LLaMa3.1-8b models, which have not undergone pre-training and fine-tuning, provide 6 candidate answers, and each answer is relatively lengthy. The LLaMa model, which has been pre-trained but not fine-tuned, provides 4 short answers. In contrast, the model in this embodiment, which is pre-trained using electromechanical maintenance guidelines and fine-tuned using knowledge-based question answering, provides only 3 answers, and the content is more concise and closer to the domain knowledge provided by experts. These results indicate that combining domain knowledge graphs with general large language models can improve the professionalism of the output answers and significantly enhance the comprehension ability of question-answering applications.
[0044] Furthermore, based on the implemented application domain knowledge graph and large model, this embodiment develops an operation and maintenance fault Q&A system and displays the front-end page, such as... Figure 5 and Figure 6 As shown, when maintenance personnel input actual maintenance problems into the interactive interface, such as "Why can't the air conditioner be controlled by the remote control?", the system analyzes and reasons about the problem using an integrated large language model, generating a structured list of fault causes and handling suggestions. The entire process is completed within seconds. The system also automatically converts the Markdown format output of the model into a standard Chinese Word document for easy download and archiving by users. This process not only enables rapid diagnosis and knowledge response to electromechanical equipment faults but also completes the one-click generation from interactive Q&A to formatted documents, effectively improving the efficiency of on-site maintenance work and the automation level of knowledge management.
[0045] Operations and maintenance question bank test experiment: In addition, to objectively evaluate the ability of different large-scale models to handle electromechanical operation and maintenance issues, this embodiment designed three sets of multiple-choice questions covering knowledge in the field of electromechanical operation and maintenance, and tested their answers in different large-scale models. These multiple-choice questions were reviewed by experts and included questions of varying difficulty levels, with a total score of 100 points. TF_(1-3) represents different test papers, and the model performance is shown in Table 5.
[0046] Table 5. Score statistics for different models Based on the scores of TF_1, TF_2, and TF_3, the model in this embodiment outperformed other models in all three sets of professional domain test papers. Specifically, the model in this embodiment, which integrates the electromechanical operation and maintenance guide and knowledge question-and-answer pairs, can more accurately answer professional questions related to operation and maintenance based on a full understanding of user intent. Furthermore, as shown in Table 5, by combining the domain knowledge graph, the accuracy of the model in the professional domain was significantly improved. Fine-tuning significantly enhanced the performance of the LLaMa2 model in the electromechanical operation and maintenance knowledge question-and-answer task, thereby improving its effectiveness in professional applications.
[0047] In addition, based on the implemented electromechanical operation and maintenance knowledge point self-assessment system, this embodiment develops and displays a front-end page for an intelligent exercise generation and assessment system based on a generative large model, such as... Figure 7 and Figure 8 As shown, when maintenance personnel input a specific knowledge point, such as "transformer input terminal (AC220V) voltage," the system, relying on an integrated large language model, performs semantic understanding and expansion of the knowledge point, automatically generating various types of exercises covering that knowledge point and its related content, including multiple-choice, fill-in-the-blank, and short-answer questions. During the generation process, the system can organize the question structure and answers in real time based on the input content, ensuring a reasonable distribution of question types and content that fits actual maintenance scenarios. The entire process achieves one-click output from knowledge point input to formatted exercise sets, effectively supporting maintenance personnel in self-testing and consolidating professional knowledge, and effectively improving the automation level of training and assessment.
[0048] In addition, the experts' scoring results for the generated exercises are shown in Table 6.
[0049] Table 6. Expert Scoring Results for Exercise Generation Quality (Average Score) The results show that the generated exercises received extremely high scores (both exceeding 4.5 points) in "content accuracy" and "scenario relevance," indicating that the exercises are professional and closely related to actual operation and maintenance scenarios. The "difficulty suitability" score was slightly lower, but still at a good level, reflecting that the system has room for optimization in adaptive difficulty control. The overall score of 4.6 points demonstrates the practical value of the system. In conclusion, the above experiments fully demonstrate that combining domain graphs and large language models can improve the accuracy of intelligent question answering in the field of electromechanical operation and maintenance.
[0050] This embodiment addresses the intelligent needs of highway electromechanical operation and maintenance teaching and training, proposing a question-answering and assessment method that integrates domain knowledge graphs and large language models. By automatically constructing domain-specific fine-tuning data and employing efficient LoRA fine-tuning technology, specialized operation and maintenance knowledge is successfully injected into a general large model, constructing a domain expert model. The system developed based on this model demonstrates excellent performance in both intelligent question answering and exercise generation: accuracy is significantly improved in specialized question-answering tasks; the generated exercises are recognized by experts for their high accuracy and practicality. This embodiment provides an effective intelligent solution for personnel training, skills assessment, and knowledge management in the electromechanical operation and maintenance field, and has promising application prospects. Future work will focus on optimizing the difficulty control mechanism of exercises and exploring the application of multimodal knowledge in teaching systems.
[0051] To address the shortcomings of general-purpose large language models in the field of highway electromechanical operation and maintenance (EMG / EM) due to a lack of professional knowledge, low answer credibility, and insufficient targeted assessment capabilities, this embodiment proposes a Q&A and assessment method for EMG / EM teaching that integrates domain knowledge graphs and large language models. First, based on an existing EMG / EM knowledge graph, high-quality domain-specific question-and-answer pairs are automatically generated using prompting engineering techniques to construct a fine-tuning dataset. Next, the open-source LLaMa2 model is pre-trained using domain text, and EMG knowledge is injected using LoRA technology to achieve lightweight and efficient fine-tuning of the model, constructing a domain expert model. Finally, based on this model, an EMG / EM teaching system with intelligent question answering and automatic exercise generation functions is developed, realizing a closed loop from user question understanding and knowledge retrieval to answer generation and teaching assessment. Experimental results show that the model fine-tuned with domain knowledge achieves an average accuracy of 88.3% in professional question bank tests, significantly outperforming general-purpose large language models; the generated exercises scored over 90% in both content accuracy and scenario relevance in expert reviews, verifying the effectiveness and practicality of this method in EMG / EM training and skills assessment.
[0052] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A method for teaching and evaluating electromechanical operation and maintenance knowledge based on domain knowledge graphs and large models, characterized in that, include: Obtain the operation and maintenance guide text data, preprocess the operation and maintenance guide text data, and obtain the token sequence data required for pre-training; The knowledge graph data of highway electromechanical equipment faults is transformed into operation and maintenance knowledge question and answer pairs data. The token sequence data is input into a preset large language model for model pre-training; The pre-trained model is fine-tuned using the aforementioned operation and maintenance knowledge Q&A data to obtain a domain expert model. Question answering evaluation is applied based on the aforementioned domain expert model.
2. The electromechanical operation and maintenance teaching knowledge question-and-answer and assessment method based on domain knowledge graph and large model as described in claim 1, characterized in that, Preprocessing the operation and maintenance guide text data includes: The operation and maintenance guide text data is subjected to information filtering, redundant data removal, and word segmentation operations to retain key operation and maintenance information text. In the information filtering stage, the established keyword recognition technology is used to identify and remove noise and irrelevant data in the text. In the redundancy removal stage, duplicate information is removed using screening techniques; In the word segmentation stage, the organized operation and maintenance text is segmented to generate the token sequence required for pre-training.
3. The electromechanical operation and maintenance teaching knowledge question-and-answer and assessment method based on domain knowledge graphs and large models as described in claim 1, characterized in that, The knowledge graph data on highway electromechanical equipment faults was transformed into operation and maintenance knowledge question-and-answer pairs data, including: The structural triples in the knowledge graph of highway electromechanical equipment faults are transferred to the general large model, and questions and answers are generated by combining them with preset prompt engineering statements; Based on the generated questions and answers, and with the guidance of experts in the field of transportation operation and maintenance, a semi-automatic filtering and proofreading process is carried out using scripts.
4. The electromechanical operation and maintenance teaching knowledge question-and-answer and assessment method based on domain knowledge graphs and large models as described in claim 3, is characterized in that, The general large model adopts the GPT-4o model.
5. The electromechanical operation and maintenance teaching knowledge question-and-answer and assessment method based on domain knowledge graph and large model as described in claim 1, characterized in that, The preset large language model adopts the LLaMa2 model.
6. The electromechanical operation and maintenance teaching knowledge question-and-answer and assessment method based on domain knowledge graphs and large models as described in claim 1, characterized in that, Before inputting the token sequence data into the preset large language model, the following steps are included: An embedding method is used to map the token sequence data into a real number vector, and root mean square layer normalization is used to remove the mean in the calculation.
7. The electromechanical operation and maintenance teaching knowledge question-and-answer and assessment method based on domain knowledge graphs and large models as described in claim 1, characterized in that, Inputting the token sequence data into a preset large language model includes: Convert the input vector into a query vector matrix, a key matrix, and a value matrix, respectively. Perform rotational positional encoding on the query vector and key vector; Perform matrix dot product on the encoded query vector and key vector to obtain the attention score; The Softmax operation is used to assign attention weights in the range [0, 1] to different attention scores; The calculated attention weights are multiplied by the Value matrix to form the output of the self-attention layer; The output vector is obtained through a feedforward neural network, thus completing the vector pre-training operation.
8. The electromechanical operation and maintenance teaching knowledge question-and-answer and assessment method based on domain knowledge graph and large model as described in claim 1, characterized in that, Fine-tuning the pre-trained model using the aforementioned operation and maintenance knowledge Q&A includes: Based on the aforementioned operation and maintenance knowledge question-and-answer pair, a low-rank adaptation technique based on a large language model is used to fine-tune the pre-trained model.
9. The electromechanical operation and maintenance teaching knowledge question-and-answer and assessment method based on domain knowledge graph and large model as described in claim 1, characterized in that, Applications of question-answering evaluation based on the aforementioned domain expert model include: When a user inputs a natural language question, the domain expert model directly generates a well-structured and professionally designed answer. At the same time, the question-and-answer record can be exported as a standardized document with one click. When a user inputs any knowledge point, the domain expert model automatically generates various question types covering that knowledge point. The generated questions are used to build test papers to assess maintenance personnel. The domain expert model automatically scores the answers or provides reference answers, thus achieving a closed-loop evaluation of teaching effectiveness.