Self-adaptive safety training system and method based on virtual reality and large language model

Abstract

The invention discloses a self-adaptive safety training system and method based on virtual reality and a large language model. The system comprises a VR module, an AI training module and a self-adaptive evaluation module. The VR module constructs a virtual training environment, provides a virtual training environment scene for a user, provides a user interaction interface and collects user interaction data; the AI training module is used for pushing training content to the user and performing question and answer interaction with the user; and the self-adaptive evaluation module is used for evaluating all user interaction data collected by the VR module and the AI training module, and dynamically adjusting subsequent task difficulty or recommending supplementary training content. According to the invention, a fusion architecture of three modules of VR + LLM under an adaptive learning framework is proposed for the first time, and a complete closed-loop learning system is formed. The AI tutoring and real-time evaluation module is embedded in the VR scene, so that the cooperative ability among the modules is enhanced, the illusion problem is solved, the follow-up training scene and difficulty are dynamically adjusted, and the training effect is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an adaptive safety training system and method based on virtual reality and a large language model. Background Technology

[0002] Nuclear power, as a clean and efficient new energy source, is receiving increasing attention in the current climate change and sustainable development context. However, as a high-risk industry, nuclear power plant operators are crucial to ensuring the safe and efficient operation of the plant and its production activities. Therefore, ensuring continuous and high-quality operator training is essential. However, current traditional safety training (such as classroom lectures) suffers from drawbacks such as low interactivity, low participation, and low learning efficiency. Furthermore, due to the unique characteristics of nuclear power plants—the dose limit for occupational exposure of nuclear power plant workers is an average effective dose of no more than 20 mSv / year over five consecutive years, and no more than 50 mSv / year in any single year—the frequency and duration of on-site practical training for operators are further limited.

[0003] CN119626055A discloses a simulation scenario training method and apparatus based on a large language model. The method includes: determining the demonstration text information of the target interaction task, the individual ability information of the trainee, and the first prefix information based on the simulation scenario training request; outputting the simulation scenario dialogue script through the simulation training large language model using the individual ability information, demonstration text information, and first prefix information; simulating a real-time dialogue scenario of life insurance services based on the task feature information of the target interaction task and the simulation scenario dialogue script; obtaining the voice feedback information of the trainee; adding the voice feedback information to the simulation training large language model, and outputting the simulation training score of the trainee, which is of great benefit to improving the trainee's communication ability in life insurance service-related work.

[0004] CN118918744A discloses a virtual training method, apparatus, device, and medium based on a large language model, relating to the field of virtual reality technology. The method includes: dividing the virtual training process of a virtual training file into multiple training steps according to a knowledge system; reading the corresponding knowledge graph based on the virtual training file, with different virtual training files corresponding to their own knowledge graphs, which are obtained by inputting all the knowledge systems of the virtual training file into a large language model according to the knowledge information corresponding to the training steps; for each training step, monitoring and acquiring the virtual training information for that step; matching the virtual training information with the knowledge graph using an AI teaching assistant virtual role model based on the knowledge system corresponding to the training step, determining whether the virtual training information matches the knowledge information of that training step; if not, outputting virtual training guidance information for that training step through the AI ​​teaching assistant virtual role model.

[0005] Existing safety training methods (traditional lectures and single VR simulations) lack interactivity, have low learning motivation, and poor knowledge retention. Traditional VR training lacks personalized real-time feedback and cannot dynamically adjust training content based on trainees' performance. The integration of VR and AI technologies is not deep enough to achieve voice interaction and real-time guidance based on large language models. In particular, the problem of how to improve training efficiency and safety through a closed-loop feedback mechanism in high-risk scenarios remains unresolved.

[0006] Virtual reality technology can provide trainees with an immersive experience while avoiding radiation risks, effectively improving training efficiency and effectiveness. Furthermore, the nuclear power industry, being a high-risk sector with high staff turnover, struggles to preserve accumulated experience and lessons learned. Utilizing RAG (Rapid Language Analysis) technology based on large language models to build an expert system adapted to the actual conditions of the factory can significantly enhance the natural language processing and understanding capabilities of the VR system, thereby enabling efficient communication and interaction between the user and the system. This patent provides a safety training system that integrates virtual reality, large language models, and adaptive learning methods, addressing the shortcomings of existing technologies in training interactivity, personalization, and knowledge retention. Summary of the Invention

[0007] This invention addresses the shortcomings of existing training systems based on virtual reality technology, such as insufficient interactivity and failure to implement a closed-loop feedback mechanism. It provides an adaptive safety training method based on virtual reality and a large language model (RAG). This method utilizes RAG technology to build an expert system adapted to the actual factory conditions, significantly improving the natural language processing and understanding capabilities of the VR system. This enables efficient communication and interaction between the user and the system, resolving the deficiencies of existing technologies in training interactivity, personalization, and knowledge retention.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] An adaptive security training system based on virtual reality and a large language model is disclosed. The adaptive security training system includes a virtual reality (VR) module, an AI training module, and an adaptive evaluation module; the modules communicate with each other to achieve closed-loop feedback.

[0010] The VR module is used to scan and acquire the physical layout data of the training environment, construct a virtual training environment, provide users with a virtual training environment scene and a user interaction interface, and collect user interaction data.

[0011] The AI ​​training module is used to push training content to users and interact with them through questions and answers.

[0012] The adaptive evaluation module is used to evaluate all user interaction data collected by the VR module and AI training module, and dynamically adjust the difficulty of subsequent tasks or recommend supplementary training content.

[0013] This invention presents a safety training system that organically combines virtual reality, large language models, and adaptive learning theory. Through the collaborative work of its modules, it achieves an immersive, highly interactive, and personalized training experience. Retrieval-enhanced generation technology reduces the risk of hallucinations, enabling voice and text interaction and providing real-time, targeted, and personalized training feedback and guidance to ensure the accurate transfer of professional knowledge in the nuclear power industry. An adaptive learning module, centered on data collection, intelligent assessment, and dynamic feedback, is established. By collecting trainee operation data and assessment scores, and combining this with standard answers or expert rules for quantitative analysis, the VR and AI training content is dynamically adjusted based on the assessment results, forming a continuously optimized closed-loop learning system. A unified data management architecture is constructed to achieve efficient data interaction and real-time communication between the VR module, AI tutoring module, and adaptive learning module, ensuring accurate recording and seamless integration of data at each stage of the training process, thereby improving the overall training effectiveness.

[0014] When applying general-purpose LLM models to industrial scenarios, the main challenge lies in the limited understanding of specialized industry knowledge and the inability to directly access local domain information. Furthermore, directly importing LLM models without adaptation may generate inaccurate or misleading content. Therefore, this system employs retrieval-enhanced generation technology to establish an AI training module, combining a large language model with virtual reality to improve training effectiveness in specific industry domains.

[0015] Preferably, the AI ​​training module includes a knowledge base module, a retrieval-augmented generation (RAG) module, and a large language model module;

[0016] The knowledge base module includes training scenario descriptions and detailed security training knowledge documents related to the training scenarios.

[0017] The RAG retrieval module is used to receive users' natural language questions. After the question content is vectorized, similar content is retrieved in the knowledge base, and the retrieval results are organized in natural language form.

[0018] The large language model module is used to receive the search content and user questions from the RAG module, call the pre-trained language model (such as the ChatGLM model) via API to generate answers, and pass the answer results to the VR module.

[0019] The AI ​​training module was constructed by adding the ChatGLM model and the Tongyi Qianwen model to the Langchain-Chatchat application framework.

[0020] The semantic vector database includes any one of the following: FAISS vector database, Milvus vector database, Pinecone vector database, and Weaviate vector database.

[0021] The content of the knowledge documents in the knowledge base module can be dynamically updated;

[0022] The construction of the knowledge base module includes: writing detailed professional security documents as knowledge documents, manually segmenting and optimizing the knowledge documents, and embedding them into a semantic vector database after vectorization to obtain the knowledge base module. The knowledge documents include descriptions of training scenarios, a knowledge base based on the training environment, workflow instructions, equipment descriptions in the work environment, potential security issues, and solutions.

[0023] The VR module has a built-in speech recognition module, such as Windows Speech Recognition (WSR), which converts the user's voice input into text in real time and transmits it to the AI ​​training module. When the user asks a question, the large language model module in the AI ​​training module vectorizes the query, and the RAG retrieval module retrieves the most relevant content from the knowledge base. The question and the retrieval results are then passed to the large language model module through a prompting process, and finally the VR module responds to the user, thus generating a coherent and accurate answer.

[0024] To enable real-time interaction with the LLM in a virtual environment, the AI ​​module incorporates a question-and-answer database to store user interaction data; preferably, this database includes any one of MySQL, SQLite, or PostgreSQL. All interaction data between the client and server, including user requests, server responses, and timestamps, is recorded in real-time in the data intermediary. The VR module can execute data queries through custom database connection scripts and bind the retrieved information to relevant elements of the user interface for seamless data display.

[0025] The adaptive evaluation module includes a data acquisition module, an intelligent evaluation module, and a dynamic feedback module. During the training process, the AI ​​training module and the adaptive evaluation module share data through a unified interactive record database interface. User speech problems in the VR scene and their responses generated by the large language model are recorded by the AI ​​training module and shared with the adaptive evaluation module through the database for subsequent querying.

[0026] The data acquisition module is used to collect all data from user interactions with the VR module and AI module. The question-and-answer database is also embedded in the data acquisition module of the adaptive evaluation module, which collects data including user behavior and user questions.

[0027] The intelligent assessment module has a built-in question bank, question bank answers, and assessment criteria, which are used to ask questions to users and conduct assessments based on the users' answers;

[0028] The dynamic feedback module assesses the user's training status based on user interaction data and answer performance, and dynamically adjusts the task difficulty of the AI ​​training module or recommends supplementary training content.

[0029] All data collected from user interactions in the VR module, including user operation data, evaluation scores, and interactive behaviors, are stored in a database to provide data support for subsequent analysis.

[0030] More specifically, collecting all data from user interactions within the VR module includes:

[0031] User operation records, which record the interactive behavior of students in the VR environment, such as device operation steps, error rate, execution time, etc.

[0032] The student's answer performance, including the content, accuracy, and completion time of the questions posed by the AI ​​during the training process;

[0033] Learning path data records the content accessed and questions asked by trainees during the training process, tracks their learning trajectory, and analyzes their interests and weaknesses.

[0034] This invention also provides an adaptive security training method based on virtual reality and a large language model, the adaptive security training method comprising:

[0035] A security training knowledge base is constructed based on RAG technology. The security training knowledge base includes descriptions of training scenarios and detailed security training knowledge documents related to the training scenarios.

[0036] An AI training module based on RAG technology and a large language model is built, enabling users to enter a virtual training environment through VR headsets and interact with the AI ​​training module for training exercises;

[0037] Based on the user's interaction data with the AI ​​training module, questions are asked to the user and the user's answers are recorded. The user's training status is evaluated by combining the interaction data and the answers, and the task difficulty of the AI ​​training module is dynamically adjusted or supplementary training content is recommended.

[0038] The dynamic adjustment of the task difficulty or recommendation of supplementary training content for the AI ​​training module specifically includes: an adaptive algorithm based on Bayesian knowledge tracing, which calculates the probability of knowledge mastery based on the user's historical answer records, predicts future performance, and dynamically adjusts subsequent training content to optimize the training path of the AI ​​training module.

[0039] The dynamic adjustment includes strengthening weak knowledge points, providing additional explanations or supplementary exercises for low-scoring items, personalized training content recommendations based on user interests and learning progress, and recommending more suitable training modules.

[0040] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the adaptive security training method.

[0041] The present invention also provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory, wherein the memory stores instructions executable by the at least one processor, the instructions being configured to cause the processor to perform the adaptive security training method.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) Existing safety training products often adopt a single technical approach, such as using only VR for scenario simulation or only using AI question-and-answer systems for knowledge transfer, lacking systematic integration. This invention proposes for the first time a three-module integrated architecture of "VR+LLM" under an adaptive learning framework, forming a complete closed-loop learning system. By embedding AI tutoring and real-time assessment modules into VR scenarios, the isolation of modules is broken, and the collaborative ability between modules is enhanced.

[0044] (2) Compared with traditional LLM products, this invention introduces a Retrieval-Augmented Generation (RAG) mechanism, which vectorizes local professional security documents and embeds them into the knowledge base, enabling LLM to have local contextual understanding capabilities and solving the illusion problem.

[0045] (3) Traditional VR training paths are fixed and cannot be adjusted in real time according to the trainees' performance. This invention uses an AI assessment module to score and analyze the trainees' answers and behaviors in the VR environment, and dynamically adjusts the subsequent training scenarios and difficulty based on the scores. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the overall structure of the adaptive safety training system based on virtual reality and large language model of the present invention.

[0047] Figure 2 This is a schematic diagram of the training process of the adaptive safety training system of the present invention.

[0048] Figure 3 This is a virtual training scenario diagram of the adaptive safety training system of the present invention - pipeline rust removal.

[0049] Figure 4A virtual training scenario diagram of the adaptive safety training system of the present invention - adjusting pipeline flow.

[0050] Figure 5 This is a virtual training scenario diagram of the adaptive safety training system of the present invention - user interaction with a large language model.

[0051] Figure 6 This is a virtual training scenario diagram of the adaptive safety training system of the present invention - the question-and-answer session.

[0052] Figure 7 This is a virtual training scenario diagram of the adaptive safety training system of the present invention - the training tasks are adjusted according to dynamic feedback and highlighted in red. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Modifications or equivalent substitutions made by those skilled in the art based on their understanding of the technical solutions of this invention, without departing from the spirit and scope of the invention, should be covered within the protection scope of this invention.

[0054] All raw materials used in the following specific implementation methods were purchased from the market.

[0055] 1. Hardware components:

[0056] VR headset: HTC VIVE Pro Eye, resolution: 1080×1200 per eye, refresh rate: 90Hz, field of view: 108°×97°;

[0057] Computer host: Intel i7-12700F processor, NVIDIA RTX4070 graphics card, 32GB RAM;

[0058] Voice input device: Headset integrated microphone, WSR voice recognition engine;

[0059] Server-side: Used to deploy the LangChain-ChatChat framework and integrate the RAG module and MySQL database.

[0060] 2. Software architecture and connection methods:

[0061] like Figure 1 As shown, the adaptive security training system based on virtual reality and a large language model described in this invention adopts a modular software architecture, consisting of a virtual reality module, an AI training module, and an adaptive evaluation module. The modules communicate decoupledly through RESTful APIs and message queues.

[0062] The virtual reality module runs on VR terminal devices, enabling users to interact naturally in virtual scenes through voice recognition and controllers. During training, users can proactively ask questions or issue commands; the system captures the user's voice in real time, converts it into text, and sends it to the AI ​​training module.

[0063] After receiving user input, the AI ​​training module first performs embedding vector transformation on the input and then searches a structured private knowledge base based on semantic matching algorithms (such as FAISS or Milvus) to organize the top-k similar text segments into supplementary contextual material. The AI ​​training module calls the large language model API (such as ChatGLM or Qwen) to generate a response, and the generated result is sent back to the VR terminal for presentation through the middle layer interface.

[0064] The adaptive evaluation module assesses user performance during training, dynamically determining whether to proceed to retraining based on the score, and providing visual feedback in VR through hotspot highlighting and guided prompts. The data acquisition module within this module manages all user data, evaluation results, and interaction records, storing them in a backend database (such as MySQL). It supports queries based on multiple criteria, including timestamp, user ID, and evaluation stage. The system can also generate learning curves and training reports based on historical records for subsequent analysis or report generation.

[0065] Asynchronous message passing (such as RabbitMQ) is used between modules to improve concurrent processing capabilities. The model service, database service, and front-end VR interaction service communicate via HTTP / HTTPS and WebSocket interfaces to achieve module decoupling and system fault tolerance. Key modules are deployed on a local server.

[0066] Example 1

[0067] An adaptive security training system based on virtual reality and a large language model is provided. The adaptive security training system includes a VR module, an AI training module, and an adaptive evaluation module; the modules communicate with each other to achieve closed-loop feedback.

[0068] The VR module is used to scan and acquire the physical layout data of the training environment, construct a virtual training environment, provide users with a virtual training environment scene and a user interaction interface, and collect user interaction data.

[0069] The AI ​​training module is used to push training content to users and interact with them through questions and answers.

[0070] The adaptive evaluation module is used to evaluate the user interaction data collected by the VR module and dynamically adjust the task difficulty of the AI ​​training module or recommend supplementary training content.

[0071] Specifically, the VR module uses 3D scanning to acquire the physical layout data of the nuclear power safety training environment, uses Blender and Unity3D to build the virtual training environment required for the experiment, provides users with a virtual training environment scene and a user interface, which is convenient for scene simulation and real-time operation, and collects user interaction data.

[0072] In this embodiment, the HTC VIVE Pro Eye head-mounted display is used to ensure a clear and smooth visual experience for users during VR training. Furthermore, the system runs on a custom computer platform, guaranteeing efficient rendering and real-time interactive response during training.

[0073] The VR module has a built-in speech recognition module, such as Windows Speech Recognition (WSR), which converts the user's voice input into text in real time and transmits it to the AI ​​training module. When the user asks a question, the large language model module in the AI ​​training module vectorizes the query, and the RAG retrieval module retrieves the most relevant content from the knowledge base. The question and the retrieval results are then passed to the large language model module through a prompting process, and finally the VR module responds to the user, thus generating a coherent and accurate answer.

[0074] To enable real-time interaction with LLM in a virtual environment, the AI ​​training module incorporates a MySQL database to store user interaction data. All interaction data between the client and server, including user requests, server responses, and timestamps, is recorded in real-time in the data intermediary. The VR module can execute data queries through custom database connection scripts and bind the retrieved information to relevant elements in the user interface for seamless data display.

[0075] The AI ​​training module is constructed by adding the ChatGLM model and the Tongyi Qianwen model to the Langchain-Chatchat application framework, and includes a knowledge base module, a RAG retrieval module and a large language model module.

[0076] The knowledge base module includes training scenario descriptions and detailed safety training knowledge documents related to the training scenarios. To enable the system to understand virtual safety training scenarios, detailed professional safety documents were written to describe the training situations and to establish a knowledge base based on the spent fuel cooling chamber, including work process descriptions, equipment descriptions in the factory, and possible safety issues and solutions, which were then uploaded to the knowledge base.

[0077] The imported professional documents are manually segmented and optimized to improve retrieval efficiency and accuracy. The processed text is vectorized and embedded into the FAISS semantic vector database to obtain the knowledge base module.

[0078] The RAG retrieval module retrieves the closest content from the knowledge module based on user queries and then transmits the search results to the large language model module. Utilizing RAG technology during interactive sessions, the AI ​​training module can prioritize relevant content, improving response accuracy. Furthermore, the system's knowledge base module is highly flexible, with domain documents dynamically updated to adapt to different security training needs.

[0079] The large language model module is used to vectorize user queries and transmit them to the RAG retrieval module, and then send the retrieval results obtained by the RAG retrieval module back to the user via the VR module.

[0080] The process of user interaction with the AI ​​training module includes:

[0081] Step 1: The VR module collects user interaction data, which is then displayed on the Unity workstation and stored in a MySQL database.

[0082] Step 2: User interaction data is transmitted from the MySQL database into the RAG model. After being vectorized by the Tongyi Qianwen model, the user interaction data is processed by the ChatGLM model to output the consultation results.

[0083] Step 3: The ChatGLM model transmits the consultation results to the MySQL database and displays them to the user through the UI in Unity.

[0084] The adaptive evaluation module includes a data acquisition module, an intelligent evaluation module, and a dynamic feedback module. During the training process, the AI ​​training module and the adaptive evaluation module share data through a unified interactive record database interface. Specifically, the user's voice questions in the VR scene and their responses generated by the large language model are recorded by the AI ​​training module and shared with the adaptive evaluation module through the database for subsequent querying.

[0085] The adaptive evaluation module assesses the user's knowledge mastery based on their answers and an adaptive algorithm, and dynamically adjusts the knowledge points that require further training.

[0086] like Figure 2 As shown, the data acquisition module is used to collect all data from the user's interaction with the VR module and the AI ​​training module; the intelligent evaluation module has a built-in question bank, question bank answers and evaluation criteria, which are used to ask questions to the user and evaluate based on the user's answers;

[0087] Specifically, all data collected from user interactions in the VR module, including user operation data, evaluation scores, and interactive behaviors, are stored in a database to provide data support for subsequent analysis.

[0088] More specifically, collecting all data from user interactions within the VR module includes:

[0089] User operation records, which record the interactive behavior of students in the VR environment, such as device operation steps, error rate, execution time, etc.

[0090] The student's answer performance, including the content, accuracy, and completion time of the questions posed by the AI ​​during the training process;

[0091] Learning path data records the content accessed and questions asked by trainees during the training process, tracks their learning trajectory, and analyzes their interests and weaknesses.

[0092] During training, the system will pose security-related questions to trainees, and their answers will be transmitted to the intelligent evaluation module for assessment via a prompting process. The main dimensions of the assessment include:

[0093] Operational understanding: Have the trainees correctly mastered the safe operating procedures?

[0094] Risk identification: Can trainees accurately identify safety hazards in the training environment?

[0095] Risk avoidance: Whether trainees make reasonable and effective decisions in emergency situations.

[0096] In addition, the system combines standard answers from the database with semantic similarity calculations to quantitatively evaluate students' answers and record their progress.

[0097] The dynamic feedback module assesses the user's training status based on user interaction data and answer status. Based on the adaptive algorithm of Bayesian knowledge tracking, it calculates the probability of knowledge mastery according to the user's historical answer records, predicts future performance, and dynamically adjusts subsequent training content. Based on the student's historical answer records, it calculates the probability of knowledge mastery, predicts future performance, decides whether certain content needs to be retrained, and dynamically adjusts the task difficulty of the AI ​​training module or recommends supplementary training content.

[0098] The adjustments include: strengthening weak knowledge points, providing additional explanations or supplementary exercises for low-scoring areas; and personalized content recommendations and more suitable training modules based on students' interests and learning progress to improve the learning experience.

[0099] In the dynamic feedback module, after dynamically adjusting the task difficulty of the AI ​​training module or recommending supplementary training content, the system automatically highlights information that trainees have not mastered or fully understood, making subsequent training more targeted. Through a continuous cycle of evaluation-feedback-adjustment, a closed-loop learning mechanism is formed, and the system gradually improves trainees' safety awareness and practical skills.

[0100] VR Environment Adjustment: Based on the student's performance in the assessment, the system highlights the learning content that the student has not mastered in the VR environment;

[0101] AI feedback optimization: Based on the learner's learning performance, the intelligent agent outputs the learner's learning performance in terms of operational understanding, risk identification, and risk avoidance.

[0102] Data storage and tracking: The system records trainees' training data in the long term, forming learning archives to support subsequent training optimization.

[0103] Through a cyclical mechanism of data collection, intelligent evaluation, and dynamic feedback, the adaptive learning module can continuously optimize training content, making safety training more personalized and efficient, and ensuring that trainees continuously improve their safe operating skills.

[0104] A user-specific adaptive security training method based on virtual reality and a large language model, the adaptive security training method including:

[0105] A security training knowledge base is constructed based on RAG technology. The security training knowledge base includes descriptions of training scenarios and detailed security training knowledge documents related to the training scenarios.

[0106] Build an AI training module based on RAG technology and a large language model, enabling users to enter a virtual training environment through AR and interact with the AI ​​training module to conduct training exercises;

[0107] Based on the user's interaction data with the AI ​​training module, questions are asked to the user and the user's answers are recorded. The user's training status is evaluated by combining the interaction data and the answers, and the task difficulty of the AI ​​training module is dynamically adjusted or supplementary training content is recommended.

[0108] The dynamic adjustment of the task difficulty or recommendation of supplementary training content for the AI ​​training module specifically includes: an adaptive algorithm based on Bayesian knowledge tracing, which calculates the probability of knowledge mastery based on the user's historical answer records, predicts future performance, and dynamically adjusts subsequent training content to optimize the training path of the AI ​​training module.

[0109] The dynamic adjustment includes strengthening weak knowledge points, providing additional explanations or supplementary exercises for low-scoring items, personalized training content recommendations based on user interests and learning progress, and recommending more suitable training modules.

[0110] The operation process includes: users wearing VR devices enter a virtual training environment. After triggering task nodes in the scene, users complete the tasks and freely explore the VR scene. Trainees can ask any questions about the scene via voice, which WSR converts into text and enters the AI ​​training module. The virtual training scene is shown below. Figures 3-7As shown, the system calls the RAG retrieval module to retrieve relevant information from the local knowledge base of the knowledge base module, and generates feedback through the large language model module, which then sends the response to the user. The adaptive evaluation module analyzes and scores the user's understanding based on the answer data recorded by the VR module. The scoring results are fed back to the Unity engine through the database, dynamically adjusting the difficulty of subsequent tasks or recommending supplementary training modules.

[0111] Once users enter the scenario, they need to complete a series of learning tasks, such as Figure 3 As shown, the user is completing a learning task on pipe rust removal. Figure 4 The user is completing a task to learn how to regulate pipeline flow. Using the controller, the user can not only interact with the virtual environment in various ways, but also directly ask questions to the large language model within the scene. For example... Figure 5 As shown, the user is asking the large language model how to avoid noise risks. After each learning task, the system automatically enters evaluation mode, where it asks the user questions based on their performance in the previous learning task, such as... Figure 6 As shown, after the rust removal learning task is completed, the system asks the user questions about related operations, which the user can answer directly by voice input. After the user answers, the system will provide feedback based on the user's answer, highlighting the content that needs to be retrained in the scene. For example... Figure 7 As shown.

Claims

1. An adaptive safety training system based on virtual reality and a large language model, characterized in that, The adaptive safety training system includes a VR module, an AI training module, and an adaptive assessment module; The VR module is used to scan and acquire the physical layout data of the training environment, construct a virtual training environment, provide users with a virtual training environment scene and a user interaction interface, and collect user interaction data. The AI ​​training module is used to push training content to users and interact with them through questions and answers. The adaptive evaluation module is used to evaluate all user interaction data collected by the VR module and AI training module, and dynamically adjust the difficulty of subsequent tasks or recommend supplementary training content.

2. The adaptive safety training system based on virtual reality and large language model according to claim 1, characterized in that, The AI ​​training module includes a knowledge base module, a RAG retrieval module, and a large language block and large-scale language model module. The knowledge base module is used to store training scenario descriptions and detailed security training knowledge documents related to the training scenarios. The RAG retrieval module is used to receive users' natural language questions. After the question content is vectorized, similar content is retrieved in the knowledge base, and the retrieval results are organized in natural language form. The large language model module is used to receive the search content and user questions from the RAG module, call the pre-trained language model via API to generate answers, and pass the answer results to the VR module.

3. The adaptive safety training system based on virtual reality and large language model according to claim 2, characterized in that, The AI ​​training module was constructed by adding the ChatGLM model and the Tongyi Qianwen model to the Langchain-Chatchat application framework. The semantic vector database includes any one of the following: FAISS vector database, Milvus vector database, Pinecone vector database, and Weaviate vector database.

4. The adaptive safety training system based on virtual reality and large language model according to claim 2, characterized in that, The content of the knowledge documents in the knowledge base module can be dynamically updated; And / or, the construction of the knowledge base module includes: writing detailed professional security documents as knowledge documents, inputting the knowledge documents into the AI ​​training module, and embedding them into a semantic vector database after manual segmentation, optimization, and vectorization to obtain the knowledge base module.

5. The adaptive safety training system based on virtual reality and large language model according to claim 1, characterized in that, The VR module has a built-in speech recognition module, which is used to convert the user's voice input into text in real time and transmit it to the AI ​​training module. The AI ​​training module has a built-in question-and-answer database for storing user interaction data; preferably, the question-and-answer database includes any one of MySQL, SQLite, or PostgreSQL databases.

6. The adaptive safety training system based on virtual reality and large language model according to claim 1, characterized in that, The adaptive evaluation module includes a data acquisition module, an intelligent evaluation module, and a dynamic feedback module; The data acquisition module is used to collect all data from user interactions in the VR module and AI module. The question-and-answer database is also embedded in the data acquisition module of the adaptive evaluation module, which collects data including user behavior and user questions. The intelligent assessment module has a built-in question bank, question bank answers, and assessment criteria, which are used to ask questions to users and conduct assessments based on the users' answers; The dynamic feedback module assesses the user's training status based on user interaction data and answer performance, and dynamically adjusts the task difficulty of the AI ​​training module or recommends supplementary training content.

7. An adaptive security training method based on virtual reality and a large language model, characterized in that, The adaptive security training method includes: A security training knowledge base is constructed based on RAG technology. The security training knowledge base includes descriptions of training scenarios and detailed security training knowledge documents related to the training scenarios. An AI training module based on RAG technology and a large language model is built, enabling users to enter a virtual training environment through VR headsets and interact with the AI ​​training module for training exercises; Based on the user's interaction data with the AI ​​training module, questions are asked to the user and the user's answers are recorded. The user's training status is evaluated by combining the interaction data and the answers, and the task difficulty of the AI ​​training module is dynamically adjusted or supplementary training content is recommended.

8. The adaptive security training method based on virtual reality and large language model according to claim 7, characterized in that, The dynamic adjustment of the task difficulty or recommendation of supplementary training content for the AI ​​training module specifically includes: an adaptive algorithm based on Bayesian knowledge tracing, which calculates the probability of knowledge mastery based on the user's historical answer records, predicts future performance, and dynamically adjusts subsequent training content, thereby dynamically adjusting the task difficulty or recommendation of supplementary training content for the AI ​​training module.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the adaptive security training method of claim 7 or 8.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, in, The memory stores instructions executable by the at least one processor, the instructions being configured to cause the processor to perform the adaptive security training method as described in claim 7 or 8.

Images