Virtual assembly training system based on UE

Abstract

The invention discloses a virtual assembly training system based on UE. The virtual assembly training system comprises a training course making module, an interactive training module, a data model library module, a user management module and the like. The training course making module imports a three-dimensional model from multiple sources, can import external animation resources and provides a visual management interface; the interactive training module carries out training, simulation operation assessment, theoretical knowledge assessment and training effect assessment in a three-dimensional virtual system constructed based on the UE by importing the made training course; the data model library is used for completing storage of training course data and user use data; and the user management module is used for realizing storage and management of training process data, training examination data and training effect evaluation data of the user, and is also used for controlling classification and authority of the user. The method has the beneficial effects of improving the learning efficiency, reducing the training cost, improving the safety, enhancing the practical ability, improving the student participation degree, accurately evaluating the training result and the like.

Description

Technical Field

[0001] The invention belongs to the field of virtual reality training / exercise, and relates to a UE-based virtual assembly training system for virtual training and exercise of equipment. Background Art

[0002] Virtual training is a training method that uses virtual reality, augmented reality, network platforms and other technical means. First of all, virtual training can break the limitations of time and space. Trainees can participate in training at any time anywhere with an Internet connection, which greatly improves the flexibility and convenience of training. Secondly, by simulating real work scenes and situations, trainees can have a more immersive learning experience, which helps to better understand and master knowledge and skills. Thirdly, virtual training can be repeated, and trainees can practice repeatedly according to their own needs to consolidate what they have learned. At the same time, in some fields where actual operations are difficult and the operational risks are high, such as medical, industrial, aviation, etc., virtual training can provide trainees with close-to-real practical opportunities under the premise of ensuring safety.

[0003] Unreal Engine (UE for short) is a powerful game development engine that is widely used to create high-quality games, virtual reality, augmented reality and other applications. It has excellent graphics rendering capabilities, physical simulation effects and a rich toolset that can help developers achieve amazing visual effects and immersive experiences.

[0004] At present, in the field of equipment assembly and use training, there is still a lack of virtual training application systems that meet the actual needs of equipment virtual assembly and debugging. In view of this, it is necessary to provide a UE-based virtual training system to address the current equipment training needs, such as the need to use physical objects, low organizational efficiency, high training costs, and narrow training coverage. The virtual training system can provide an infinitely repeatable virtual immersive learning environment, which can greatly reduce learning costs and practical risks and improve the efficiency of equipment use training. Summary of the invention

[0005] The purpose of the present invention is to provide a UE-based virtual assembly training system, which has the functions of three-dimensional lightweight model import, disassembly and assembly animation production, step-by-step training process production, training / training result evaluation, etc., and can support equipment users to carry out training and assessment in a virtual environment.

[0006] The technical solution of the present invention is:

[0007] A UE-based virtual assembly training system consists of five modules: training course production, interactive training, data model library, user management, and hardware support, among which:

[0008] 1) Training tutorial production module

[0009] The training tutorial production module is the foundation of the training system, providing tutorial producers with an efficient and customized content creation environment. This module consists of three sub-modules: lightweight model import, interactive animation production, and training process design. The lightweight model import sub-module supports importing models from multiple sources and uses automatic optimization algorithms to ensure the smooth operation of the model in the virtual environment. The interactive animation production sub-module is based on the UE engine and imports external animation resources. The training process design sub-module uses graphical programming technology and provides a visual management interface. Users can easily build training flowcharts and add corresponding training content through operations such as dragging and dropping, connecting, etc. Through this module, training tutorials that meet the requirements of the learning curve can be efficiently generated to improve the effect of virtual training.

[0010] Lightweight model import supports the use of OpenGL and USD (Universal Scene Description) technology to import product models, tool and fixture models, and camera equipment models from a variety of sources (such as CAD files, 3D scan data, etc.). During the import process, algorithms are used to optimize the imported models to improve the details and quality of the models. When importing models from external sources (such as CAD files, 3D scan data, etc.), the system will automatically apply algorithms to optimize these models, such as mesh simplification, texture compression, LOD (Level of Detail) generation, etc., to ensure smooth operation of the models in the virtual environment and better visual effects, and store the imported models in the data model library.

[0011] The interactive animation production submodule turns lightweight 3D models into interactive training animations. This module is based on the UE engine and provides animation production capabilities. Tutorial production users can extract the required models from the data model library by themselves, connect multiple models together, and implement preliminary basic actions such as model assembly or disassembly. Then, by externally importing relevant model action animations, combine them with the captured actions to achieve the specific details of device docking and more complex interactive animation effects, and finally realize the interactive animation production of the overall process of the equipment. The module has built-in animation production tools that support keyframe animation and timeline editing functions. This allows more flexible creation and editing of animations, more complex interactive animation effects, and better display and explanation of related knowledge or skills.

[0012] The training process design submodule connects various interactive training animations in series, adopts graphical programming technology, and provides a visual management interface. Users can easily build training flow charts and add corresponding training content through operations such as dragging and connecting. It can efficiently generate training tutorials that meet the requirements of the learning curve and improve the effect of virtual training. Finally, the completed tutorials are saved in the data model library.

[0013] 2) Interactive training modules

[0014] The interactive training module includes four sub-modules: UE virtual training, UE virtual training assessment, electronic answering system, and training effect evaluation. This module is used to implement training, simulation operation assessment, theoretical knowledge assessment, and training effect evaluation in a three-dimensional virtual system built based on UE. Among them: the UE virtual training sub-module is the core part of the interactive training module, which is used to import the completed training tutorials and carry out training, simulation operation assessment, theoretical knowledge assessment, and training effect evaluation in the three-dimensional interactive training module built based on UE. Import training tutorials and test question banks from the data model library, carry out virtual training, simulation operation and operation assessment in UE virtual training, and record trainee learning and training data. The electronic answering system randomly extracts test questions from the test question bank to form a theoretical knowledge test. The training effect evaluation forms the trainee training effect evaluation results through the trainee training data and theoretical test scores.

[0015] The data model library is the data storage center of the training system, including interactive assembly animation library, product model library, debugging equipment model library, tool and fixture library, assembly process library and test electronic question library. It completes the storage of training course data, user usage data, etc.

[0016] 3) User management module

[0017] User management includes user data management and user authority management. User data management includes the storage and management of user training process data, training assessment data, training effect evaluation data and other data. User authority management completes functions such as user classification and authority control.

[0018] 4) System hardware support

[0019] System hardware is an important means of supporting software operation, immersive 3D display, data model storage, and human-computer interaction. It consists of a high-performance graphics workstation, a common display, a 3D head display device, and an interactive handle. The high-performance graphics workstation supports the operation of the 3D scene and is responsible for the storage of the data model; the common display completes the 3D scene display, training / training test display, and evaluation result display; the 3D head display device completes the 3D immersive display; and the interactive handle completes the human-computer interaction operation.

[0020] The beneficial effects of the present invention are:

[0021] After the present invention is put into use in equipment use training, it can make up for the shortcomings of current equipment training that is carried out in kind, with low organizational efficiency, high training cost, and narrow training coverage, and can effectively improve the learning efficiency of equipment users, reduce training costs, increase safety, enhance practical ability, improve student participation, accurately evaluate training results, and other beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 : Virtual training system composition diagram.

[0023] Figure 2 : Flowchart of system operation. DETAILED DESCRIPTION

[0024] Example 1

[0025] A UE-based virtual assembly training system consists of five modules: training course production, interactive training, data model library, user management and hardware support.

[0026] Training tutorial production, interactive training, and data model library are the core parts of the system.

[0027] The training tutorial production module is completed by the tutorial production user, who imports simplified (lightweight) product models, tool and fixture models, and debugging equipment models into the training tutorial production module. It supports the import of models in FBX, STL, and OBJ formats.

[0028] The interactive animation production sub-module is built based on UE. It can make the imported model into assembly, wiring, and debugging animations, and perform collision detection and motion constraints on the model movement. It also supports the import of wrl format animations produced by other animation software.

[0029] The training process design sub-module is built based on UE and is designed by classification according to learning or operation types. It is combined into various learning and operation training tutorials such as product composition, product assembly, and product debugging according to different training types.

[0030] Electronic test questions are imported into the system through the electronic question bank, and the electronic test question bank is formed and provided to the electronic answering system. The imported models, test questions, animations and training tutorials are all stored and managed uniformly through the data model library.

[0031] Interactive training constructs a 3D scene based on UE, imports training tutorials and test electronic question banks from the data model library, and conducts virtual training, exercises and assessments in the 3D scene. After logging into the training system, trainees can enter the 3D training scene through a normal display or 3D head display device, and perform simulated operations through an interactive handle. The device simulation operation function uses VR devices such as Oculus Rift and HTC Vive as the basis, combined with degree of freedom tracking technology and dynamic simulation, to achieve accurate tracking of hand movements and physical feedback of device operations, thereby improving the realism of actual operations.

[0032] After students have completed their studies and training, they will be assessed theoretically through an electronic question-answering system. The electronic question-answering system uses advanced online examination system technology and has a built-in diverse question bank, covering multiple-choice questions, fill-in-the-blank questions, true-or-false questions, and other question types, ensuring the comprehensiveness and flexibility of the assessment.

[0033] The system records the students' learning, training and assessment data and feedback, including training data such as learning time and simulation operation results. The training effect is comprehensively evaluated and analyzed by analyzing the students' training data. The evaluation results will serve as an important basis for subsequent teaching improvement and optimization and provide reference and guidance for the students' personal development.

[0034] The user management module is responsible for the comprehensive management of system user information and permission control. This module includes user data management and user permission management. User data management is responsible for storing and managing user personal information, learning records and other data to ensure the security and integrity of user data. User permission management assigns different operation permissions and access rights according to the user's role and responsibilities. Students can view virtual tutorials and take subject tests, teachers can make tutorials, administrators can manage permissions and user data, etc. The system's strict permission system can ensure the security and stability of the system.

[0035] The 3D head-mounted display uses a standard resolution display screen to provide a clear visual experience, reduce pixel graininess, and enhance immersion. It has a larger field of view design, so you can feel a wider field of view when wearing the head-mounted display, which improves the realism and immersion of the training. The head-mounted display device focuses on ergonomic design, and ensures that it remains comfortable even when worn for a long time by adjusting the tightness of the headband and balancing the weight distribution. At the same time, it supports a variety of head sizes to meet the needs of different students.

[0036] The interactive handle has a built-in high-precision sensor to accurately capture the trainees' hand movements and position information, and achieve precise virtual interactive operations. The handle is equipped with multiple function buttons and a touchpad, supporting multiple operation modes, such as clicking, sliding, and joystick control, to meet the needs of different training scenarios. Made of durable materials, it ensures that the handle maintains good performance and feel during long-term use. At the same time, the ergonomically designed handle shape and grip make it easier to operate.

[0037] Example 2

[0038] The workflow of the system of the present invention includes:

[0039] ① User login. Log in to the system according to user permissions and carry out training tutorial production or training / exercise;

[0040] ②Users with the permission to create training courses enter the training course creation module;

[0041] ③ Import lightweight models of products, tooling tools, and debugging equipment into the system and save them in the system database;

[0042] ④ Use the imported lightweight model to create interactive animations. You can also import the created interactive animations in wrl format into the system and store them in the database;

[0043] ⑤ Decompose the training process into sub-processes such as component assembly, large-scale assembly, debugging wiring, and debugging. Prepare training content for each sub-process, combine each training process into assembly process and training tutorials according to the assembly and debugging sequence, and store them in the warehouse. The training tutorial production module is completed;

[0044] ⑥ The electronic question bank for theoretical tests is imported into the system and stored through the electronic question bank import function;

[0045] ⑦ Trainee users with interactive training permissions enter the interactive training module and conduct virtual training;

[0046] ⑧ Enter the UE virtual training / training system to carry out virtual assembly and debugging training or training. After the training, a virtual training assessment can be conducted, and the learning and assessment results are stored in the trainee user data management library;

[0047] ⑨Students enter the electronic answering system to study and test the assembly and debugging theory, and the learning and test results are stored in the student user data management database;

[0048] ⑩ The system automatically evaluates students’ learning progress and effectiveness in real time based on their training and assessment results, and provides reminders to students whose learning time and assessment do not meet the standards.

Claims

1. A UE-based virtual assembly training system, characterized by: The training system includes a graphics workstation and a 3D head display device and an interactive handle connected to the graphics workstation; the graphics workstation is used to support the operation of three-dimensional scenes, realize the storage of data models, and display three-dimensional scenes, display of training / training tests, and display of evaluation results; the 3D head display device is used to realize three-dimensional immersive display; the interactive handle is used to realize human-computer interactive operation; The training system also includes a training course production module, an interactive training module, a data model library module and a user management module, among which: The training course production module is used to provide an efficient and customized content creation environment for course production personnel. The module consists of three sub-modules: lightweight model import, interactive animation production, and training process design. The lightweight model import sub-module supports the import of 3D models from a variety of sources; the interactive animation production sub-module implements the import of external animation resources based on the UE engine; the training process design sub-module provides a visual management interface, and users can build a training flowchart and add corresponding training content through drag and drop and connection operations; The interactive training module consists of four submodules: UE virtual training, UE virtual training assessment, electronic answering system, and training effect evaluation. The module is used to implement training, simulation operation assessment, theoretical knowledge assessment, and training effect evaluation in a three-dimensional virtual system built based on UE; wherein: the UE virtual training submodule is the core part of the interactive training module, which is used to carry out three-dimensional virtual training in a three-dimensional virtual training scene built based on UE by importing the completed training tutorial, including equipment composition, working principle, assembly operation, debugging operation, and assembly and debugging simulation operation; the electronic answering system is used to form test questions by randomly extracting electronic question banks to complete regular tests and assessments of theoretical knowledge; the UE virtual training assessment submodule is used to carry out theoretical and virtual operation assessments for trainees who have completed theoretical learning and virtual operation and debugging training in a three-dimensional virtual training assessment system based on UE. The theoretical assessment is completed by the electronic answering system, and the virtual operation and debugging assessments are completed by UE virtual training. At the end of the assessment, the assessment results of the trainees are given; the training effect evaluation submodule evaluates the training effect of the trainees by collecting the trainees' learning process data and combining the assessment results; The data model library is used to complete the storage of training course data and user usage data, and is composed of an interactive assembly animation library, a product model library, a debugging equipment model library, a tool and fixture library, an assembly process library, and an electronic question library; The user management module includes user data management and user authority management. The user data management includes the storage and management of the user's training process data, training assessment data, and training effect evaluation data. The user authority management completes the classification and authority control of users.

2. The virtual training system according to claim 1, characterized in that: The lightweight model import supports the use of OpenGL and USD to import product models, tool and fixture models, and camera equipment models from multiple sources. During the import process, algorithms are used to optimize the imported models to improve the details and quality of the models. When models are imported from the outside, algorithms are automatically applied to optimize these models and store the imported models in a data model library.

3. The virtual training system according to claim 1, characterized in that: The interactive animation production submodule turns the lightweight three-dimensional model into an interactive training animation; the tutorial production user first extracts the required model from the data model library by himself, connects multiple models together, and realizes the basic actions leading to the assembly or disassembly of the model; then, by externally importing the relevant model action animation, it is combined with the captured action to realize the specific details of the device docking and its more complex interactive animation effects; finally, the interactive animation production of the overall process of the equipment is realized; the module has a built-in animation production tool and supports keyframe animation and timeline editing functions.

4. The virtual training system according to claim 1, characterized in that: The training process design submodule connects various interactive training animations in series, adopts graphical programming technology, and provides a visual management interface. Users can easily build training flowcharts through dragging and connecting operations, add corresponding training content, and save the completed tutorials to the data model library.

5. The virtual training system according to claim 4, characterized in that: The training process is decomposed into component assembly, large-section assembly, debugging and wiring, and sub-processes. The sub-processes are combined into assembly processes and training tutorials in the order of assembly and debugging and stored in the warehouse.

6. The virtual training system according to claim 1, characterized in that: The electronic test questions are imported into the system through the electronic question bank, and a test electronic question bank is formed, which is provided to the electronic answering system for use; the imported models, test questions, produced animations and produced training tutorials are all uniformly stored and managed through the data model library.

7. The virtual training system according to claim 1, characterized in that: The interactive training enters the three-dimensional training scene through an ordinary display or a 3D head display device, and performs equipment simulation operation through an interactive handle; the equipment simulation operation uses VR equipment and combines degree of freedom tracking and dynamic simulation to achieve accurate tracking of hand movements and physical feedback of equipment operation, thereby improving the realism of actual operation.

8. The virtual training system according to any one of claims 1 to 7, characterized in that: The electronic answering system adopts an online examination method, and the system has built-in diverse question bank resources.

9. The virtual training system according to any one of claims 1 to 7, characterized in that: The collected student learning process data includes learning time, regular test scores, operation time and operation practice accuracy.

10. The virtual training system according to any one of claims 1 to 7, characterized in that: The model imported from outside includes CAD files and 3D scanning data; The various sources include CAD files and 3D scan data.

Images