VR multi-person collaborative mechanical cloud classroom interaction method based on Unity3D

Abstract

The invention discloses a VR multi-person collaborative mechanical cloud classroom interaction method based on Unity3D, and belongs to the crossing field of a virtual reality technology and an education technology. According to the method, a high-precision virtual machine scene is constructed through three-dimensional modeling, and translation (linear driving) and rotation (rotation driving) operations of a user on virtual equipment are realized in combination with a physical engine and interactive logic design; a real-time communication framework is used for supporting multi-user collaborative operation and data synchronization, and a teacher can dynamically guide students to operate and monitor the learning progress; operation deviations (such as cutter angle errors and cutting depth overrun) are detected through a real-time feedback mechanism, warning is generated, and meanwhile operation data are recorded for teaching evaluation. The VR teaching system solves the problems that an existing VR teaching system is insufficient in interactivity, difficult in multi-person cooperation and lack of real-time feedback, is especially suitable for subjects needing complex operation training such as mechanical engineering, and can effectively improve the teaching efficiency and the practical operation restoration degree.

Description

Technical Field

[0001] The present invention relates to the fields of virtual reality (VR) technology, three-dimensional modeling technology, network collaboration technology, and educational technology, and particularly relates to an interaction algorithm for a VR multi-person collaborative mechanical cloud classroom based on Unity3D. Background Art

[0002] Traditional mechanical course teaching relies on teachers' explanations and students' actual operations. However, due to factors such as high equipment costs, large operation risks, time and space limitations, etc., students are difficult to obtain sufficient practical opportunities. With the development of virtual reality technology, VR technology has gradually been applied to the education field, especially in disciplines that require a large amount of practical operations such as mechanical engineering. However, most existing VR teaching systems lack efficient interaction algorithms and cannot achieve multi-person collaborative operations and real-time feedback, resulting in poor teaching effects. Therefore, it is of great significance to develop an interaction method for a VR multi-person collaborative mechanical cloud classroom.

[0003] Prior Art 1 (CN117648036A) discloses a VR teaching system that unidirectionally shows mechanical structures and working principles to students through pre-recorded three-dimensional models and teaching videos. Students passively watch the teaching content through VR devices. The system only supports scene switching in a fixed process and has no real-time interaction function. The defect of Prior Art 1 is the lack of interactivity, only unidirectional information transmission can be achieved, and there is a lack of real-time interaction between students and the virtual environment (such as unable to perform operation feedback such as grasping and rotating mechanical parts); it cannot support complex mechanical operation simulations (such as multi-step assembly, dynamic fault diagnosis, or physical engine-driven mechanical behavior simulations), resulting in the disconnection between the teaching scenario and actual operations.

[0004] Prior Art 2 (CN222255931U) discloses a VR teaching system that is built based on a single-user operation environment and only supports a single user to complete mechanical structure observation and basic theory learning through an independent VR device. The operation permissions, perspective switching, and data feedback in the virtual scenario in the system are all designed around a single user, and the function of multi-terminal synchronous access and collaboration is not opened. The defect of Prior Art 2 is the difficulty in multi-person collaboration. The system architecture cannot support multi-user synchronous access and real-time collaboration (such as teacher-student perspective sharing, multi-person collaborative disassembly and assembly of mechanical parts, or parallel operation verification); the teacher-student interaction is missing. There is a lack of a teacher-end instruction transmission, real-time monitoring of students' operations, and multi-role task collaboration mechanism in the virtual environment, resulting in the teaching scenario being unable to restore the grouping collaboration and dynamic guidance requirements in a real classroom.

[0005] The prior art 3 (CN108109451A) discloses a method for implementing a VR teaching system, which conducts skill training by constructing a three-dimensional virtual scene, but only provides one-way operation guidance during the teaching process. The defects of the prior art 3 are as follows: (1) The system cannot capture the deviation of the action trajectory in real time during the user's operation (such as the angle of the surgical instrument deviating from the standard value by more than 5°), resulting in key operation errors not being immediately warned; (2) There is a lack of a multi-dimensional feedback mechanism, and cross-verification cannot be carried out for multi-channel behaviors that require collaborative operations (such as simultaneously controlling the displacement and clamping force of the robotic arm), making it difficult to effectively identify compound operation errors.

[0006] The present invention aims to solve the defects in the prior art such as the inability of virtual reality interaction to support complex operation simulation, difficulty in multi-person collaborative operation, and insufficient real-time feedback. Summary of the Invention

[0007] The present invention provides a VR multi-person collaborative mechanical cloud classroom interaction method based on Unity3D, which specifically includes the following steps:

[0008] S1. Virtual scene construction:

[0009] Use the Unity3D engine to construct a virtual mechanical teaching scene, including a three-dimensional model of mechanical equipment, an operation interface, and interaction logic. By creating a three-dimensional model of mechanical equipment and importing it into Unity3D for scene construction. The specific steps are as follows:

[0010] Three-dimensional modeling: Use modeling software to create a three-dimensional model of mechanical equipment. The size and structure of the model refer to the CAD drawings of the actual equipment to ensure the accuracy of the model.

[0011] Scene construction: Import the created three-dimensional model into Unity3D to construct a virtual teaching scene. The scene includes elements such as mechanical equipment, an operation interface, and virtual tools to ensure the realism and operability of the scene.

[0012] Physical engine setting: Through the physical engine and collision detection function of Unity3D, ensure that the operation of the equipment in the virtual scene conforms to the real physical laws.

[0013] S2. Interaction logic design:

[0014] Use programming languages to write interaction logic, and combine virtual reality development frameworks and interaction toolkits to implement the interaction functions of VR devices. Realize the translation and rotation interaction of the device through linear drive and rotation drive components, specifically including:

[0015] Linear drive:

[0016] Used to control the translational motion of the device, such as the X - Y - Z axis movement of the tool slide. Through the linear drive script, the user can control the movement of the tool slide in three - dimensional space via the VR handle to achieve precise cutting operations.

[0017] Rotary drive:

[0018] Used to control the rotational motion of the device, such as the rotation of the handwheel, the forward and reverse rotation of the chuck, etc. Through the rotary drive script, the user can control the rotation of the handwheel via the VR handle to achieve operations such as the forward and reverse rotation of the chuck and the feed of the tool.

[0019] Interaction logic implementation: Write the interaction logic through programming language to ensure that the operations of the user in the virtual scene can be reflected on the device in real - time.

[0020] S3. Multi - person collaborative network development:

[0021] Use the real - time communication framework for network development of multi - user collaboration to achieve collaborative operations of teachers and students in the same virtual scene. Through the network synchronization function of the real - time communication framework, ensure that the operations of multiple users in the virtual scene can be synchronized in real - time and real - time feedback can be carried out. The specific steps are as follows:

[0022] Network synchronization: Through the network synchronization function of the real - time communication framework, ensure that the operations of multiple users in the virtual scene can be synchronized in real - time.

[0023] Role assignment: In the virtual scene, teachers and students can be assigned different roles.

[0024] Real - time communication: Through the real - time communication function of the real - time communication framework, teachers and students can conduct real - time voice or text communication in the virtual scene to ensure the interactivity of the teaching process.

[0025] S4. Real - time feedback mechanism:

[0026] Set up a real - time feedback mechanism in the virtual scene. When the student performs an operation, the system will give real - time feedback according to the correctness of the operation. The specific steps are as follows:

[0027] Operation detection: Through the physical engine and collision detection function of Unity3D, detect in real - time whether the student's operation is correct.

[0028] Feedback generation: According to the operation detection result, the system will generate corresponding feedback.

[0029] Data recording: The system will record the operation data of the student, including operation time, operation steps, operation results, etc. Through data recording, teachers can understand the learning progress and operation situation of students and provide targeted guidance.

[0030] S5. Teaching module design:

[0031] The mechanical course is divided into multiple teaching modules, and each module corresponds to different knowledge points and operation steps. For example, the descriptive geometry module, the lathe practical operation module, etc. Each module contains detailed operation steps and interaction logics. Students can perform virtual operations through VR devices, and the system will generate corresponding feedback according to the operation results. The specific steps are as follows:

[0032] Module division: The course is divided into multiple modules according to the teaching content of the mechanical course.

[0033] Operation step design: Each module contains detailed operation steps and interaction logics.

[0034] Feedback generation: In each module, the system will generate corresponding feedback according to the operation results of the students.

[0035] Preferably, the models imported into Unity3D can be created by the modeling software Blender.

[0036] Preferably, the programming language for writing the interaction logic uses the C# scripting language.

[0037] Preferably, the real-time communication framework uses Photon.

[0038] Preferably, the virtual reality development framework uses SteamVR.

[0039] Preferably, the interaction toolkit uses XR Interaction Toolkit.

[0040] Preferably, the components for realizing the translation and rotation interactions of the device respectively adopt Linear Drive and Circular Drive. Description of the Drawings

[0041] Figure 1 is the interaction logic flow chart of the present invention; Detailed Embodiments

[0042] In order to be able to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention will be described in detail below with reference to the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In the following description, many specific details are set forth in order to fully understand the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this invention belongs. The terms used in the description of this invention are for the purpose of describing specific embodiments only and are not intended to limit the invention.

[0044] Specific Embodiment 1 (refer to the appendix Figure 1 ): Lathe machining

[0045] S1. Virtual scene construction:

[0046] Use Blender to create a 3D model of the lathe tool machining mechanical equipment, including a lathe, a lathe tool, and a workpiece model, and import it into Unity3D. Through the physical engine and collision detection function of Unity3D, ensure that the operation of these devices in the virtual scene conforms to the real physical laws.

[0047] S2. Interaction logic design:

[0048] Use C# scripts to write the interaction logic, and combine SteamVR and XR Interaction ToolKit to implement the interaction function of VR devices. Control the X-Y-Z axis movement of the lathe tool slide through the Linear Drive script, and control the rotation of the machined workpiece through the Circular Drive script.

[0049] S3. Multi-person collaborative network development:

[0050] Use Photon for multi-user collaborative network development. Through the network synchronization function of Photon, teachers and students can perform collaborative operations in the same virtual scene and can see the operation results of each other in real time.

[0051] S4. Real-time feedback mechanism:

[0052] Set up a real-time feedback mechanism in the virtual scene. When students perform lathe operations, the system will display parameters such as cutting depth and tool position, and give real-time feedback according to the correctness of the operations.

[0053] S5. Teaching module design:

[0054] Divide the lathe into a descriptive geometry module and a lathe practical operation module. Each module corresponds to different knowledge points and operation steps in the lathe machining process. At the same time, each module contains detailed operation steps and interaction logic. Students can perform virtual operations of lathe machining through VR devices, and the system generates corresponding feedback according to the operation results.

[0055] In the above embodiments, a VR multi-person collaborative mechanical cloud classroom interaction method based on Unity3D is provided. By combining VR technology, 3D modeling technology, and network collaboration technology, virtual reality interaction, multi-person collaborative operation, and real-time feedback in mechanical course teaching are realized. This algorithm can effectively improve the teaching effect of mechanical courses, reduce teaching costs, and has broad application prospects.

[0056] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle described in the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A VR multi-person collaborative mechanical cloud classroom interaction method based on Unity3D, characterized in that: The following steps are involved: Step 1: Virtual scene construction: Create an accurate 3D model of the mechanical equipment through 3D modeling software, import it into the Unity3D engine to build a virtual teaching scene, and configure the physics engine and collision detection function to make the operation of the virtual equipment conform to the laws of real physics; Step 2: Interaction logic design: Write the interaction logic based on the programming language, combine the virtual reality development framework and the interaction toolkit, control the translation movement of the device through the linear drive component, and control the rotation movement of the device through the rotation drive component, so that users can accurately operate the virtual device through the VR handle; Step 3: Multi-person collaborative network development: Use a real-time communication framework to achieve multi-user network synchronization, support teachers and students to collaborate in the same virtual scene, synchronize operation data and role allocation in real time, and provide voice and text communication functions; Step 4: Real-time feedback mechanism: Detect user operation behavior through the physical engine, generate real-time feedback on operation correctness, and record operation data to support teaching evaluation; Step 5: Teaching module design: Divide the mechanical course into multiple modules. Each module contains interactive operation steps and dynamic feedback mechanism to support students to complete phased learning tasks through VR devices.

2. The method according to claim 1, characterized in that: In step 1, the 3D modeling software is Blender, and the model size is built based on the actual device CAD drawings.

3. The method according to claim 1, characterized in that In step 2, the programming language is C#, and the linear drive component and the rotation drive component are respectively implemented by translation control of axis alignment constraints and rotation control of quaternion interpolation.

4. The method according to claim 1, characterized in that: In step 3, the real-time communication framework is Photon, which is used for network synchronization and data distribution.

5. The method according to claim 1, characterized in that In step 2, the virtual reality development framework is SteamVR, and the interaction toolkit is XR Interaction Toolkit.

6. The method according to claim 1, characterized in that In step 2, the linear drive component and the rotary drive component are implemented using Linear Drive and Circular Drive scripts respectively.

7. The method according to claim 1, characterized in that In step 4, the real-time feedback mechanism includes a visual display of operating parameters and an immediate warning function for erroneous operations.

8. The method according to claim 7, characterized in that In step 4, the operating parameters include cutting depth and tool position.

9. The method according to claim 1, characterized in that: In step 5, the teaching module is divided into multiple modules according to the course; each module includes operation steps and interactive logic support.

Images