Medium-voltage micro-grid generator car training system based on mixed reality technology and use method thereof

Abstract

The invention belongs to the technical field of medium-voltage micro-grid power generation car training, and particularly relates to a medium-voltage micro-grid power generation car training system based on a mixed reality technology and a use method thereof, and the system comprises a client and a medium-voltage micro-grid power generation private cloud platform. The client is used for data acquisition, processing and transmission, has an interaction function, can realize a virtual-real fusion technology, and comprises an attachment identification module, a principle teaching module, an operation process virtual training module, a troubleshooting training module, an assessment mode module and a data uploading module; the medium-voltage micro-grid power generation private cloud platform is used for carrying out information interaction with the client, carrying out management analysis on data and realizing multi-user cooperative training, and the medium-voltage micro-grid power generation cloud platform comprises a management module, a statistics module, a cloud service module and a medium-voltage micro-grid emergency power generation vehicle knowledge graph module; and the training requirements of skilled and refined daily operation and general troubleshooting of medium-voltage micro-grid power generation operators can be met.

Description

Technical Field

[0001] The invention relates to a medium-voltage microgrid generator vehicle training system based on mixed reality technology, belonging to the field of medium-voltage microgrid generator vehicle training. Background Art

[0002] In recent years, international experts have come to a conclusion that in terms of strengthening power grid security, developing distributed power generation systems is easier and faster than transforming the power grid. An effective way to solve the impact of distributed power sources on the power grid is to connect them to the distribution system as microgrids. When the large power grid collapses and unexpected disasters (such as earthquakes, blizzards, man-made damage, and war) occur, since microgrids can operate independently in isolated grids, they can ensure uninterrupted power supply to important users and provide power support for rapid recovery after the collapse of the large power grid. It is precisely because of some unique advantages of microgrids that their research and application training work has been highly valued. Microgrids usually supply power to local loads to improve reliability and power quality, so the power supply range is relatively small. For medium-voltage microgrids, the power supply range is usually several thousand meters, while for low-voltage microgrids, the power supply range is only a hundred meters, or even shorter, but the flexible operation mode of microgrids can provide power supply guarantee for power outage areas as soon as possible, effectively restore the power supply to some areas where power outages are caused by disasters, and provide strong emergency protection for people's livelihood. However, since medium-voltage emergency generators are expensive and cannot meet the full staffing needs of municipal and county companies, the independent operation of medium-voltage generators is relatively complex and requires a high level of operating skills from operators. Therefore, refined operation training for medium-voltage microgrid emergency power generation is very necessary.

[0003] However, traditional training methods have many shortcomings: (1) The cost of medium-voltage microgrid emergency power generation equipment is relatively high, and practical teaching greatly increases training time and training costs; (2) It is difficult to reproduce fault characteristics during practical training, and troubleshooting training cannot be conducted; (3) Field training is limited by time and space, and regular operational training cannot be conducted. In order to solve the problems of long traditional training cycles, low efficiency, high costs, and difficulty in reproducing faults, a set of medium-voltage microgrid emergency power generation teaching tools is urgently needed. For example, the comparative document entitled "Grid Substation Integrated Simulation Training System and Its Implementation Method" with publication number CN103136976A only simulates signal faults and does not involve practical training.

[0004] In view of the above-mentioned deficiencies, the present invention has developed a medium-voltage microgrid generator vehicle training system based on mixed reality technology and its use method, which is used to realize medium-voltage microgrid power generation equipment, equipment identification, medium-voltage microgrid emergency power generation working principle demonstration, medium-voltage microgrid power generation operation and troubleshooting virtual training and assessment, so as to meet the training needs of medium-voltage microgrid power generation operators for proficiency, refinement of daily operation and general troubleshooting. Summary of the invention

[0005] In view of the above deficiencies in the prior art, a medium-voltage microgrid generator vehicle training system based on mixed reality technology is provided.

[0006] The medium-voltage microgrid power generation vehicle training system based on mixed reality technology of the present invention includes a client and a medium-voltage microgrid power generation private cloud platform;

[0007] The client is used for data collection, processing and transmission. The client has an interactive function and can realize virtual-real fusion technology. The client includes an attachment recognition module, a principle teaching module, an operation process virtual training module, a troubleshooting training module, an assessment mode module and a data upload module;

[0008] The medium-voltage microgrid power generation private cloud platform is used to interact with the client, manage and analyze data, and realize multi-user collaborative training. The medium-voltage microgrid power generation cloud platform includes a management module, a statistics module, a cloud service module, and a medium-voltage microgrid emergency power generation vehicle knowledge graph module.

[0009] The accessory recognition module is used to identify the various components of the medium-voltage microgrid power generation vehicle operated by the trainees based on the feature point detection and matching algorithm of three-dimensional modeling, and use the binocular depth algorithm to locate the position of the operating components and identify the characteristics of the components, and provide component-related information.

[0010] The principle teaching module is used for holographic images based on mixed reality glasses to provide functions such as operation principle demonstration, disassembly and assembly demonstration, and maintenance process demonstration of medium-voltage microgrid power generation equipment.

[0011] The operation process virtual training module is used to load the simulation model corresponding to each process in the medium-voltage power generation vehicle parts model library based on the virtual-reality fusion technology, and to integrate the simulation device model with the real medium-voltage power generation vehicle device.

[0012] The troubleshooting training module is used to load the fault library of medium-voltage generator vehicle parts based on virtual-reality fusion technology, simulate the fault characteristics of real medium-voltage generator vehicles, and provide fault handling methods and principles to train trainees to troubleshoot.

[0013] The assessment mode module is used to conduct relevant job assessments, evaluate assessment results, and provide fault settings and AI coordination assessment functions;

[0014] The data upload module is used to upload operation training data and assessment data to the medium voltage microgrid generator vehicle operation training for subsequent statistical analysis.

[0015] The management module includes a database management unit, a data dictionary maintenance unit, a training subject setting and issuing unit, a fault setting and issuing unit, and a record receiving unit;

[0016] The database management unit is used to create and maintain the relevant database;

[0017] The data dictionary maintenance unit is used to store database related information and define and describe data objects in the medium voltage generator car database;

[0018] The training subject setting and issuing unit is used to set the corresponding medium voltage generator vehicle operation process guidance and issue it to the client for simulation training;

[0019] The fault setting and sending unit is used to set component faults and send component fault characteristics and maintenance plans to the client, so as to conduct simulated troubleshooting training;

[0020] The record receiving unit is used to receive the data uploaded by the client and save it to the cloud platform database for subsequent statistical analysis.

[0021] The statistical module includes a training situation statistical analysis unit and an assessment situation statistical analysis unit, and the statistical module is used to perform corresponding statistical analysis on the received data.

[0022] The cloud service module includes a real-time communication service unit, a multi-user coordination service unit, and a knowledge graph real-time push service unit;

[0023] The real-time communication service unit is used to realize multi-user real-time communication between clients and real-time communication between clients and cloud platform;

[0024] The multi-user coordination service unit is used to implement multi-user collaborative training between users;

[0025] The knowledge graph real-time push service unit is used to push the medium-voltage microgrid power generation knowledge graph on the cloud platform to the client in real time.

[0026] The medium-voltage microgrid emergency power generation vehicle knowledge graph module is used to model and record the association and knowledge between data knowledge in the medium-voltage microgrid power generation process. The medium-voltage microgrid emergency power generation vehicle knowledge graph module includes a multi-category data import unit, a cross-platform call unit, an AI natural language interaction unit, an AI information push unit, and a troubleshooting guidance unit;

[0027] The multi-type data importing unit is used to import the data collected by the client.

[0028] The cross-platform calling unit is used to synchronously call knowledge graph data between multiple terminals;

[0029] The AI ​​natural language interaction unit is used to perform speech processing, communicate, exchange and provide feedback with trainees;

[0030] The AI ​​information push unit is used to collect and identify information about the components of the medium voltage generator vehicle during the training process, and to match and provide relevant operation process and equipment knowledge information;

[0031] The troubleshooting guidance unit is used to intelligently provide relevant information and operations required for troubleshooting according to the matching of fault characteristics with the fault library during troubleshooting training.

[0032] The method for using the medium voltage microgrid generator vehicle training system based on mixed reality technology includes the following steps:

[0033] Step 1: Get the current medium voltage generator equipment information, enter the UI interface, and enter the training;

[0034] Step 2: Select the training mode you want in the UI interface;

[0035] Step 3: Load the virtual model of the medium voltage generator car and conduct video, 3D animation theory, and holographic model teaching;

[0036] Step 4: Virtual process training: The virtual model is loaded onto the physical medium-voltage generator truck through the virtual-reality fusion module to conduct virtual process training, followed by virtual practice, and evaluation and scoring based on the students’ practice performance;

[0037] Step 5: Troubleshooting training: Use virtual-reality fusion technology to load the virtual model onto the physical medium-voltage generator vehicle and load the medium-voltage generator vehicle virtual fault library;

[0038] Step 6: Assessment: Receive the assessment model and conduct the assessment of the corresponding assessment model. After the assessment is completed, evaluate and score it and upload it to the cloud.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. The system of the present invention introduces mixed reality technology into the virtual training of medium-voltage microgrid generator vehicles for the first time. Compared with the traditional medium-voltage microgrid generator vehicle training method and the VR-based operation training method, the present invention does not need to actually start the medium-voltage microgrid generator vehicle, which greatly reduces the waste of resources and the loss of the medium-voltage generator vehicle, and overcomes the problem that the virtual model in the VR system cannot be combined with the physical medium-voltage generator vehicle.

[0041] 2. The system of the present invention uses multi-modal human-computer interaction such as eye tracking, gestures, and voice of the MR head display to truly free the hands of trainees and improve the comfort of operation training.

[0042] 3. The system of the present invention is provided with a learning module for the mechanical equipment structure of a medium-voltage generator truck, which can visually display the component structure that cannot be seen inside a real medium-voltage generator truck. By operating the simulation model in an interactive manner, it helps trainees understand the mechanical structure of the medium-voltage generator truck and quickly master the working principle of the medium-voltage generator truck.

[0043] 4. The method of the present invention can integrate the real medium-voltage power generation vehicle with the virtual medium-voltage power generation vehicle model, track and register the relative positions of the holographic models of each medium-voltage power generation vehicle device, and the real medium-voltage power generation vehicle equipment can be intuitively seen during training. The sensory effect is just like operating a real medium-voltage power generation vehicle. At the same time, it is accompanied by an operation analysis and evaluation section to evaluate the operator in real time, which is more efficient and convenient than traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a system framework diagram of the present invention;

[0045] Figure 2 It is a flow chart of the implementation method of the system of the present invention.

[0046] Figure 3 It is a flow chart of the method for using the system of the present invention. DETAILED DESCRIPTION

[0047] The present invention is further described below in conjunction with specific embodiments.

[0048] However, the description of the present invention is merely an embodiment of structural and even functional description, and the scope of rights of the present invention is not limited by the embodiments described in the text.

[0049] For example, multiple embodiments may have various modifications and various forms, and it should be understood that the scope of rights of the present invention includes equivalents that can implement the technical concept.

[0050] like Figure 1 to Figure 3 As shown, the medium-voltage microgrid power generation vehicle training system based on mixed reality technology includes a client and a medium-voltage microgrid power generation private cloud platform;

[0051] The client is used for data collection, processing and transmission. The client has interactive functions and can realize virtual-reality fusion technology. The client includes an attachment recognition module, a principle teaching module, an operation process virtual training module, a troubleshooting training module, an assessment mode module and a data upload module; in this application, the client adopts an MR headset, which is a head-mounted portable holographic computer device that does not require cables. It can realize camera posture estimation and posture tracking, spatial mapping of virtual information, real-time capture of gestures, and voice commands to realize interaction and other functions.

[0052] The accessory recognition module is used to identify the various components of the medium-voltage microgrid power generation vehicle operated by the trainees based on the feature point detection and matching algorithm of three-dimensional modeling, and use the binocular depth algorithm to locate the position of the operating components and identify the characteristics of the components, and provide component-related information.

[0053] The principle teaching module is used to provide functions such as the operation principle demonstration, disassembly and assembly demonstration, and maintenance process demonstration of medium-voltage microgrid power generation equipment based on the holographic image of mixed reality glasses. During the demonstration process, the principle teaching module uses text, pictures, voice and other forms to explain the two-dimensional and three-dimensional animation demonstration process. The animation demonstration provides operations such as play, pause, stop, and jump to a specified step. Through a vivid display form, students can have an intuitive understanding and deep cognition of the system composition and operation of medium-voltage microgrid emergency power generation equipment.

[0054] The operation process virtual training module is used to load the simulation models corresponding to each process in the medium-voltage power generation vehicle parts model library based on the virtual-reality fusion technology, and integrate the simulation device model with the real medium-voltage power generation vehicle device. The trainees operate the virtual medium-voltage power generation vehicle parts through gesture recognition and eye tracking technology of multi-modal triggered human-computer natural interaction technology to complete the operation training task of the medium-voltage power generation vehicle.

[0055] The troubleshooting training module is used to load the fault library of medium-voltage generator vehicle parts based on virtual-reality fusion technology, simulate the fault characteristics of real medium-voltage generator vehicles, and provide fault handling methods and principles to train trainees to troubleshoot.

[0056] The assessment mode module is used to conduct relevant operation assessments. Specifically, the assessment mode module assesses the user's emergency power generation operation and troubleshooting operation in an augmented reality environment based on virtual-reality fusion technology. In the assessment mode, the trainees will not have any prompts and will complete the specified tasks within a limited time. The teaching tool will evaluate the assessment results based on the completion status, time and accuracy, and provide fault settings and AI cooperation assessment functions;

[0057] The data upload module is used to upload each step of the training data and assessment data of the trainees to the medium-voltage microgrid generator vehicle operation training based on wireless network communication technology for subsequent statistical analysis.

[0058] The medium-voltage microgrid power generation private cloud platform is used to exchange information with the client, manage and analyze data, and realize multi-user collaborative training. The medium-voltage microgrid power generation cloud platform includes a management module, a statistical module, a cloud service module, and a medium-voltage microgrid emergency power generation vehicle knowledge graph module. Specifically, it is used to accept data uploaded by the client, manage and analyze the model library, fault library, training process, training assessment records, etc. in the cloud, provide real-time communication to provide the cloud knowledge graph to the client, and realize multi-user collaborative training.

[0059] The management module includes a database management unit, a data dictionary maintenance unit, a training subject setting and issuing unit, a fault setting and issuing unit, and a record receiving unit;

[0060] The database management unit is used to create and maintain relevant databases. Specifically, the database management unit performs activities such as creation, loading, daily maintenance, and security control of data such as medium-voltage power generation vehicle component models, component failure databases, operation process virtual training databases, assessment process databases, and trainee information databases;

[0061] The data dictionary maintenance unit is used to store relevant information used by the database and define and illustrate data objects in the medium voltage generator car database;

[0062] The training subject setting and issuing unit is used to set the corresponding medium-voltage generator vehicle operation process guidance according to different medium-voltage generator vehicle application scenarios, and issue it to the client, so that the trainees can conduct simulation training;

[0063] The fault setting and sending unit is used to set component faults. The fault setting and sending unit can set component faults through the medium-voltage power car component fault library, and send the component fault characteristics and maintenance plans to the client-MR head display, so that trainees can conduct simulated troubleshooting training;

[0064] The recording receiving unit is used to receive the training and assessment data of each step of the trainees' operation uploaded by the client-MR headset and save it to the cloud platform database for subsequent statistical analysis.

[0065] The statistical module includes a training situation statistical analysis unit and an assessment situation statistical analysis unit. The statistical module is used to perform corresponding statistical analysis on each step of the training or assessment data of the trainees received by the management module so that the administrator can view it.

[0066] The cloud service module includes a real-time communication service unit, a multi-user coordination service unit, and a knowledge graph real-time push service unit;

[0067] The real-time communication service unit adopts wireless network communication technology including WiFi and 5G communication modes to realize multi-user real-time communication between clients and real-time communication between clients and cloud platform terminals;

[0068] The multi-user coordination service unit realizes multi-user collaborative training between users through the real-time communication service unit;

[0069] The knowledge graph real-time push service unit is used to push the cloud platform end medium-voltage microgrid power generation knowledge graph to the client in real time. Specifically, the knowledge graph real-time push service unit pushes the cloud platform end medium-voltage microgrid power generation knowledge graph to the client-MR head display in real time through the real-time communication service when the trainee needs it, and the holographic display is displayed in the trainee's field of view.

[0070] The medium-voltage microgrid emergency power generation vehicle knowledge graph module adopts a graph structure to model and record the association and knowledge between data knowledge in the medium-voltage microgrid power generation process, so as to effectively realize more accurate object-level search. The medium-voltage microgrid emergency power generation vehicle knowledge graph module includes a multi-class data import unit, a cross-platform call unit, an AI natural language interaction unit, an AI information push unit, and a troubleshooting guidance unit;

[0071] The multi-type data import unit is used to integrate and import the medium-voltage power generation vehicle image data, multimedia data, binocular depth data, various sensor data, and various text data provided by the medium-voltage power generation vehicle instructor collected by the client-MR headset.

[0072] The cross-platform calling unit can synchronously call the knowledge graph data between multiple terminals such as the MR terminal and the PC terminal;

[0073] The AI ​​natural language interaction unit is used for voice processing, mainly processing the trainee's voice data through artificial intelligence, converting human language into a form that can be read, understood and processed by computers, and communicating, exchanging and providing feedback with the trainees;

[0074] The AI ​​information push unit is used to collect and identify information about the parts of the medium-voltage power generation vehicle through the client-MR head-mounted binocular depth camera during the training process, and match and provide relevant operation process and equipment knowledge information;

[0075] The troubleshooting guidance unit is used to intelligently provide relevant information and operations required for troubleshooting according to the matching of fault characteristics with the fault library during troubleshooting training.

[0076] The medium-voltage microgrid generator vehicle training system based on mixed reality technology in this application is equipped with the following technologies:

[0077] 1. Human-computer natural interaction technology

[0078] Specifically, through the client-MR head display, eye tracking, gesture interaction, voice interaction, etc., the training and control of the operation of the virtual-reality fusion medium-voltage generator vehicle can be realized.

[0079] Eye tracking: The physics.Raycast method is used to detect whether the ray emitted from the MR headset worn by the trainee collides with an object in the scene, thereby triggering eye tracking;

[0080] Gesture interaction: Use the binocular depth camera of the MR headset to collect gesture information, collision between the hand and the recognized object, etc., design basic gestures, and use basic gestures to select events, drag, scale, rotate, and disassemble virtual models. The method is to first define the gesture recognition object, set the gesture type to be monitored, set the processing to be done after the gesture is recognized, and start and end the gesture recognition function;

[0081] Voice interaction: includes the control of voice devices and the acquisition of voice data. By defining voice keywords and corresponding events, registering recognition events, starting recognition, and responding to design voice command recognition modules, the recognition of multiple command sets can be realized. The commands in the command set are composed of words and phrases.

[0082] 2. Environmental Understanding Technology

[0083] The MR headset collects binocular depth information, IMU sensor information, visual information and other information through corresponding sensors, and calculates the surrounding environment of the power plant and the position of trainees through VSLAM algorithm, binocular depth generation algorithm, IMU calibration algorithm, multi-sensor fusion algorithm, plane detection algorithm, and environmental point cloud patch generation algorithm.

[0084] 3. Virtual and real fusion technology

[0085] The simulation model of the power generation vehicle in the model library of the medium-voltage power generation vehicle parts is registered on the power generation vehicle, and the virtual-real fusion is completed through the MR head display. The specific method is: first, the SLAM algorithm is used to study the initial calibration of positioning, algorithm principles, feasibility analysis, and the implementation and error testing of actual positioning, so that it can finally meet the tracking requirements of the head position in MR. Then, using the MR three-dimensional registration algorithm, a registration algorithm based on the natural features of the video sequence is developed, so that the virtual information is superimposed on the real scene and displayed on the optical perspective helmet display, superimposed with the real scene seen by the user, to achieve the fusion display of virtual and real information.

[0086] 4. Multi-person collaboration technology

[0087] Using the Large Space Real-time Location Service (LSRTLS), with the support of computer vision algorithms, the camera is used to scan the user's surrounding environment through a sparse spatial map (Sparse Spatial Map), generate a three-dimensional visual map of the environment, and provide real-time visual positioning and tracking functions. The established visual map can be saved or shared in real time between multiple devices. When other devices load the corresponding map and determine the position and posture of the device relative to the map through positioning in the loaded map, it is suitable for persistent holographic applications or multi-person interactive holographic applications. LSRTLS supports loading multiple maps, locating in multiple maps and returning the ID of the corresponding map and the position and posture of the device relative to the map.

[0088] 5. Knowledge graph generation technology

[0089] Using AI+KG technology, we conduct deep learning and mining of various multimodal stock data generated during medium-voltage microgrid emergency power generation and troubleshooting operations, build knowledge nodes, and generate operation knowledge graphs through knowledge fusion and knowledge calculation, providing operators with intelligent search engines and greatly improving the ability to retrieve relevant knowledge. Using mixed reality technology, operators can obtain intelligent recommendations for medium-voltage microgrid emergency power generation knowledge anytime and anywhere through natural human-computer interaction, and solve problems encountered during the operation quickly and conveniently, thereby greatly improving the efficiency of the operation.

[0090] The implementation method of the medium voltage microgrid generator vehicle training system based on mixed reality technology includes the following steps:

[0091] Step 1: Perform a three-dimensional scan of the contour of the medium-voltage generator truck operation site by taking photos and recording videos, and collect data on maintenance and operation process records;

[0092] Step 2: Modeling is done based on the scanned contours, surface texture is restored through photos, and the entire repair or operation process is sorted out based on the data;

[0093] Step 3: Integrate the model with maintenance and operation processes through the Unity engine, and complete the basic environment construction in the Unity project scene.

[0094] Step 4: Build a holographic video section: Use the Unity 3D video component to play the internal structure video of the medium-voltage microgrid generator model; Build a text section: Use the Unity 3D image component to display the relevant structural text knowledge of the medium-voltage microgrid generator; Build an interactive section: Load the medium-voltage generator model, mount the Voice_control.cs script to monitor the triggering voice interaction event, and mount the AllButton.cs script to monitor the triggering gesture interaction event. Set the tool panel through the image component and add the onclick event to each button.

[0095] Build the virtual-reality fusion module: When the unity system is running, the MR-headset position is used as the origin of the real coordinate system, the virtual environment coordinate system is set to coincide with the world coordinate system, and the MR-headset updates the 6DOF pose of the virtual camera relative to the real coordinate system in real time. Load the virtual camera, image recognition prefab, feature point database, and medium-voltage power generation vehicle model into the project, adjust the relative position of the identification map and the model, use the console model in the instantiated model library, set its parent as the identification map object, and realize dynamic loading of the model. Write a spatial anchor control script and mount it on the model to achieve space retention and complete the construction of the virtual-reality fusion module.

[0096] Build a virtual-reality fusion operation training module: Principle teaching mode: Create a video player in the system, load the holographic video and step-by-step instructions in the UI, load the medium-voltage generator model principle animation, and display the medium-voltage generator operation process. Training mode: Create a gesture recognition function, write a control operation logic program, realize the user's gesture interaction with the virtual model, write an operation score judgment calculation program, and output the current score of the user training.

[0097] The method for using the medium voltage microgrid generator vehicle training system based on mixed reality technology includes the following steps:

[0098] Step 1: The user wears the MR headset, starts the system, obtains the current medium-voltage generator vehicle equipment information, enters the UI interface, and clicks to enter the training;

[0099] Step 2: Select a training mode in the UI interface, including principle teaching mode, process virtual training, troubleshooting training, and assessment mode, and start training;

[0100] Step 3: Select the principle teaching module, load the virtual model of the medium-voltage generator car, and after completing the video, 3D animation theory, and holographic model teaching, the trainees interact with the virtual model through eye tracking, voice, and gestures to learn the principle and structure of the medium-voltage generator car;

[0101] Step 4: Select process virtual training, call the camera of the MR headset, and load the virtual model onto the physical medium-voltage generator truck through the virtual-reality fusion module. After the trainees complete the process virtual training through eye tracking, voice, and gestures, they enter the virtual practice. At the same time, they are evaluated and scored based on their practice situation to complete the training.

[0102] Step 5: Select troubleshooting training, call the camera of the MR headset, load the virtual model onto the physical medium-voltage generator truck through the virtual-reality fusion module, load the virtual fault library of the medium-voltage generator truck, and the trainees interact with the virtual model through eye tracking, voice, and gestures to learn the fault principles and troubleshooting knowledge of the medium-voltage generator truck.

[0103] Step 6: Select the assessment mode. The instructor selects the process assessment mode or troubleshooting assessment mode and distributes it to the student client. The student enters the assessment mode for assessment. In the assessment mode, there is no prompt. After the assessment, the system evaluates and scores, and uploads it to the cloud for the instructor to view.

[0104] This application meets the training needs and practical teaching exercises for the skilled, refined, standardized and customized virtual operation training and assessment of medium-voltage microgrid emergency power generation and troubleshooting operations, and realizes the following functions:

[0105] 1. Training on identification of medium voltage microgrid power generation equipment and devices

[0106] It provides functions such as displaying and hiding the 3D models of medium-voltage microgrid power generation equipment and equipment, displaying and hiding the description information of equipment and equipment, grabbing, viewing and resetting equipment and equipment, and zooming equipment and equipment. The description information includes the name, function, and structural composition of the equipment and equipment, and the types of information include text, graphics, audio, and video. Grab and view means grabbing, moving, rotating, and viewing by gestures, eye control, and gaze.

[0107] In the MR environment, a variety of means such as text, pictures, 2D video, panoramic video, holographic stereo video, 3D holographic display, holographic animation display, etc. are used to display the teaching tool overview, function, composition, installation position relationship, performance indicators, etc. of medium-voltage microgrid emergency power generation equipment and equipment. Let the students learn the relevant knowledge of medium-voltage microgrid emergency power generation system equipment and equipment in a 3D holographic environment.

[0108] 2. Principle teaching and training function

[0109] It provides functions such as demonstration of the operating principle, disassembly and assembly, and maintenance process of medium-voltage microgrid emergency power generation equipment. During the demonstration, the two-dimensional and three-dimensional animation demonstration process is explained in the form of text, pictures, and voice. The animation demonstration provides operations such as play, pause, stop, and jump to a specified step. Through vivid presentation, students can have an intuitive understanding and deep cognition of the system composition and operation of medium-voltage microgrid emergency power generation equipment.

[0110] 3. Virtual training function

[0111] There are two training modes: virtual teaching mode and free practice mode:

[0112] a. VR / MR teaching mode, in which the VR instructor guides the trainees to carry out virtual training in a virtual environment, and instructs the trainees on the relevant operations of power work through VR or MR holographic guidance. This mode is generally used for beginners.

[0113] b. Free practice mode. In this mode, the virtual training teaching tool will generate a virtual practice environment and training entities to provide training support for users. Compared with the first method, there is no guidance from a VR instructor. This method is generally suitable for users with a certain foundation.

[0114] During emergency power generation operations and troubleshooting, it is sometimes necessary to replace, disassemble, and debug certain equipment. The teaching tool builds a virtual simulation environment that allows users to perform disassembly and assembly operations in a purely virtual or a combination of virtual and real environments.

[0115] 4. Assessment function

[0116] The system can assess the emergency power generation and troubleshooting operations of users in a mixed reality (MR) environment. In the assessment mode, users will not receive any prompts and will complete the specified tasks within a limited time. The teaching tool will evaluate the assessment results based on the completion status, time, and accuracy. It also provides fault settings and AI-assisted assessment functions.

[0117] In the assessment mode, users will be required to complete specific subjects within a limited time, and each step of the operation will be recorded and automatically judged. After the operation is completed or the time is up, the teaching tool will automatically score and record the information in the database.

[0118] 5. Accessibility

[0119] a. Auxiliary functions include training monitoring, background management, subjects, fault settings, data query, training data upload, assessment data upload, etc.

[0120] Training monitoring means that the instructor can see the student's first-person perspective or third-party teaching and training operation screen through Hololens or a large touch screen, so as to grasp the student's learning dynamics in a timely manner.

[0121] b. The purpose of the backend management function is to implement operations such as backup, import, and export of user permissions, training subjects, and databases.

[0122] c. The purpose of subject and fault settings is to set which component of which subject has failed, so as to facilitate targeted assessment.

[0123] d. The purpose of data query is to be able to check pictures, documents, videos and other materials in a virtual environment at any time when performing power operations.

[0124] e. The training data upload function refers to the result data of the user's training while wearing the MR headset, which is uploaded to the power operation private cloud in an offline manner to provide data support for statistical analysis of the training situation.

[0125] f. The assessment data upload function refers to the result data of the assessment performed by the user wearing the MR helmet display, which is uploaded to the medium-voltage microgrid power generation private cloud platform in an offline manner to provide data support for statistical analysis of the assessment situation.

[0126] Of course, the above contents are only preferred embodiments of the present invention and cannot be considered to limit the scope of the embodiments of the present invention. The present invention is not limited to the above examples, and equal changes and improvements made by ordinary technicians in the technical field within the essential scope of the present invention should all fall within the scope of the patent coverage of the present invention.

Claims

1. A medium voltage microgrid generator vehicle training system based on mixed reality technology, characterized in that: Including client and medium voltage microgrid power generation private cloud platform; The client is used for data collection, processing and transmission. The client has an interactive function and can realize virtual-real fusion technology. The client includes an attachment recognition module, a principle teaching module, an operation process virtual training module, a troubleshooting training module, an assessment mode module and a data upload module; The medium-voltage microgrid power generation private cloud platform is used to interact with the client, manage and analyze data, and realize multi-user collaborative training. The medium-voltage microgrid power generation cloud platform includes a management module, a statistics module, a cloud service module, and a medium-voltage microgrid emergency power generation vehicle knowledge graph module.

2. According to claim 1, the medium voltage microgrid generator vehicle training system based on mixed reality technology is characterized in that: The accessory recognition module is used to identify the various components of the medium-voltage microgrid power generation vehicle operated by the trainees based on feature point detection and matching algorithms, and use a binocular depth algorithm to locate the position of the operating components and identify the characteristics of the components, and provide component-related information; The principle teaching module is used to provide the operation principle demonstration, disassembly and assembly demonstration and maintenance process demonstration functions of the medium-voltage microgrid power generation equipment based on the holographic image of the mixed reality glasses.

3. According to claim 1, the medium voltage microgrid generator vehicle training system based on mixed reality technology is characterized in that: The operation process virtual training module is used to load the simulation model corresponding to each process in the medium-voltage power generation vehicle parts model library based on the virtual-reality fusion technology, and to integrate the simulation device model with the real medium-voltage power generation vehicle device.

4. According to claim 1, the medium voltage microgrid generator vehicle training system based on mixed reality technology is characterized in that: The troubleshooting training module is used to load the fault library of medium-voltage generator vehicle parts based on virtual-reality fusion technology, simulate the fault characteristics of real medium-voltage generator vehicles, and provide fault handling methods and principles to train trainees to troubleshoot.

5. According to claim 1, the medium voltage microgrid generator vehicle training system based on mixed reality technology is characterized in that: The assessment mode module is used to conduct relevant job assessments, evaluate assessment results, and provide fault settings and AI coordination assessment functions; The data upload module is used to upload operation training data and assessment data to the medium voltage microgrid generator vehicle operation training for subsequent statistical analysis.

6. According to claim 1, the medium voltage microgrid generator vehicle training system based on mixed reality technology is characterized in that: The management module includes a database management unit, a data dictionary maintenance unit, a training subject setting and issuing unit, a fault setting and issuing unit, and a record receiving unit; The database management unit is used to create and maintain the relevant database; The data dictionary maintenance unit is used to store database related information and define and describe data objects in the medium voltage generator car database; The training subject setting and issuing unit is used to set the corresponding medium voltage generator vehicle operation process guidance and issue it to the client for simulation training; The fault setting and sending unit is used to set component faults and send component fault characteristics and maintenance plans to the client, so as to conduct simulated troubleshooting training; The record receiving unit is used to receive the data uploaded by the client and save it to the cloud platform database for subsequent statistical analysis.

7. According to claim 1, the medium voltage microgrid generator vehicle training system based on mixed reality technology is characterized in that: The statistical module includes a training situation statistical analysis unit and an assessment situation statistical analysis unit, and the statistical module is used to perform corresponding statistical analysis on the received data.

8. According to claim 1, the medium voltage microgrid generator vehicle training system based on mixed reality technology is characterized in that: The cloud service module includes a real-time communication service unit, a multi-user coordination service unit, and a knowledge graph real-time push service unit; The real-time communication service unit is used to realize multi-user real-time communication between clients and real-time communication between clients and cloud platform; The multi-user coordination service unit is used to implement multi-user collaborative training between users; The knowledge graph real-time push service unit is used to push the medium-voltage microgrid power generation knowledge graph on the cloud platform to the client in real time.

9. According to claim 1, the medium voltage microgrid generator vehicle training system based on mixed reality technology is characterized in that: The medium-voltage microgrid emergency power generation vehicle knowledge graph module is used to model and record the association and knowledge between data knowledge in the medium-voltage microgrid power generation process. The medium-voltage microgrid emergency power generation vehicle knowledge graph module includes a multi-class data import unit, a cross-platform call unit, an AI natural language interaction unit, an AI information push unit, and a troubleshooting guidance unit; The multi-type data importing unit is used to import the data collected by the client; The cross-platform calling unit is used to synchronously call knowledge graph data between multiple terminals; The AI ​​natural language interaction unit is used to perform speech processing, communicate, exchange and provide feedback with trainees; The AI ​​information push unit is used to collect and identify information about the components of the medium voltage generator vehicle during the training process, and to match and provide relevant operation process and equipment knowledge information; The troubleshooting guidance unit is used to intelligently provide relevant information and operations required for troubleshooting according to the matching of fault characteristics with the fault library during troubleshooting training.

10. The method for using the medium voltage microgrid generator vehicle training system based on mixed reality technology as described in any one of claims 1 to 9 is characterized in that: The following steps are involved: Step 1: Get the current medium voltage generator equipment information, enter the UI interface, and enter the training; Step 2: Select the training mode you want in the UI interface; Step 3: Load the virtual model of the medium voltage generator car and conduct video, 3D animation theory, and holographic model teaching; Step 4: Virtual process training: The virtual model is loaded onto the physical medium-voltage generator truck through the virtual-reality fusion module to conduct virtual process training, followed by virtual practice, and evaluation and scoring based on the students’ practice performance; Step 5: Troubleshooting training: Use virtual-reality fusion technology to load the virtual model onto the physical medium-voltage generator vehicle and load the medium-voltage generator vehicle virtual fault library; Step 6: Assessment: Receive the assessment model and conduct the assessment of the corresponding assessment model. After the assessment is completed, evaluate and score it and upload it to the cloud.

Images