Teaching practical training implementation method and system based on element cosmic force feedback technology

Abstract

The invention discloses a teaching practical training implementation method and system based on a meta-universe force feedback technology, and the method comprises the steps: obtaining the current practical training user information, the current practical training type selection information and a meta-universe practical training scene through a host, and transmitting the current practical training data to VR head-mounted display equipment; the VR head-mounted display device displays the current virtual reality display data; the force feedback touch glove obtains current practical training operation data aiming at the current virtual reality display data and sends the current practical training operation data to the host; and if the host detects a practical training ending instruction, the host obtains the current practical training operation data and the stored current practical training operation data and forms a current practical training data set. According to the embodiment of the invention, in combination with the host, the VR head display device and the force feedback touch glove, the current practical training operation data for the current virtual reality display data in the virtual reality scene displayed by the VR head display device by the user can be timely acquired, and the corresponding touch data is received in the operation process, so that the practical training simulation effect is greatly enhanced.

Description

Technical Field

[0001] The present invention relates to the field of metaverse technology, and in particular to a teaching and training implementation method and system based on metaverse force feedback technology. Background Art

[0002] In the traditional field of power professional training, the training methods mainly rely on video tutorials, written teaching materials and limited field operations. Although the above model has its basic role, it inevitably faces several key challenges: 1) The limitations of actual operation simulation, that is, due to the limitations of physical space, professional equipment configuration and safety regulations, traditional training is difficult to fully and realistically simulate the complex scenarios and emergency situations in power work, which makes it difficult for trainees to accumulate sufficient practical operation experience and affects their ability to deal with practical problems; 2) Delayed monitoring and feedback. In other words, in the traditional training process, due to the lack of an efficient real-time monitoring mechanism, it is difficult for instructors to obtain the trainees' operation details in a timely manner, and thus it is impossible to quickly provide targeted guidance and instant feedback, which limits the improvement of the training effect. 3) Insufficient practical experience, that is, the content of traditional training is relatively fixed and single, and it is difficult to fully cover the diverse work scenarios and challenges in the power profession. Therefore, it is difficult for trainees to obtain rich and real operation experience, which affects the overall improvement of their professional skills and comprehensive quality.

[0003] In response to the above problems in the practical training of electric power professionals, a virtual reality electric power professional training system has been introduced. This system uses virtual reality (VR, the full name is Virtual Reality) technology to build a highly simulated electric power working environment, allowing trainees to conduct various electric power operation training in virtual scenes, significantly improving their practical ability and teamwork level. However, when using the above virtual reality electric power professional training system, when the trainees perform specific operations in the virtual reality environment, there is no real physical feedback from the operating object, that is, the immersive touch is poor; and it is impossible to obtain the trainees' operation details in time, so it is impossible to quickly provide targeted guidance and instant feedback. Summary of the invention

[0004] The embodiments of the present invention provide a teaching and training implementation method, device, computer equipment and medium based on metaverse force feedback technology, aiming to solve the problem in the prior art that when a virtual reality power professional training system and trainees perform specific operations in a virtual reality environment, there is no real physical feedback of the operation object and the operation details of the trainees cannot be obtained in time.

[0005] In a first aspect, an embodiment of the present invention provides a teaching and training implementation method based on Metaverse force feedback technology, which is applied to a teaching and training implementation system based on Metaverse force feedback technology. The teaching and training implementation system based on Metaverse force feedback technology includes a host, a display, a user input device, a VR head display device, and a force feedback tactile glove; the display, the user input device, and the VR head display device are all connected to the host in communication, the VR head display device is connected to the force feedback tactile glove in communication, and the host is also connected to a cloud server through a network; the teaching and training implementation method based on Metaverse force feedback technology includes: The host responds to the user login instruction and obtains the current training user information and current training type selection information corresponding to the user login instruction; If the host determines that the current training type selection information corresponds to the metaverse training scene, the host sends the current training data corresponding to the current training type selection information to the VR head display device; The VR head display device receives the current training data and displays the current virtual reality display data corresponding to the current training data; The force feedback tactile glove obtains the user's current training operation data for the current virtual reality display data, and sends the current training operation data to the host in real time for storage until a training end instruction is detected; wherein, the glove model mapping corresponding to the force feedback tactile glove is displayed in the virtual reality scene displayed by the VR head display device, and the user's hand operation data collected by the force feedback tactile glove is used to synchronize with the virtual reality scene displayed by the VR head display device to perform corresponding operations on the current virtual reality display data, and receive corresponding tactile data; the tactile data is generated and sent to the force feedback tactile glove when the VR head display device performs corresponding operations on the current virtual reality display data based on the user's hand operation data; If the host detects a training end instruction, it obtains the saved current training operation data and the current training operation data corresponding to the current training operation data and the current virtual reality display data, and forms a current training data set corresponding to the current training data.

[0006] In a second aspect, an embodiment of the present invention further provides a teaching and training implementation system based on the metaverse force feedback technology, which includes: A host, a display, a user input device, a VR head display device and a force feedback haptic glove; the display, the user input device and the VR head display device are all connected to the host in communication, the VR head display device is connected to the force feedback haptic glove in communication, and the host is also connected to a cloud server via a network; The host is used to respond to the user login instruction and obtain the current training user information and current training type selection information corresponding to the user login instruction; The host is further configured to send current training data corresponding to the current training type selection information to the VR head display device if it is determined that the current training type selection information corresponds to a metaverse training scene; The VR head display device is used to receive the current training data and display the current virtual reality display data corresponding to the current training data; The force feedback tactile glove is used to obtain the user's current training operation data for the current virtual reality display data, and send the current training operation data to the host in real time for storage until a training end instruction is detected; wherein the glove model corresponding to the force feedback tactile glove is mapped and displayed in the virtual reality scene displayed by the VR head display device, and the user's hand operation data collected by the force feedback tactile glove is used to synchronize with the virtual reality scene displayed by the VR head display device to perform corresponding operations on the current virtual reality display data, and receive corresponding tactile data; the tactile data is generated and sent to the force feedback tactile glove when the VR head display device performs corresponding operations on the current virtual reality display data based on the user's hand operation data; The host is also used to obtain the saved current training operation data and the current training operation data corresponding to the current training operation data and the current virtual reality display data if a training end instruction is detected, and to form a current training data set corresponding to the current training data.

[0007] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0008] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the method described in the first aspect above can be implemented.

[0009] The embodiment of the present invention provides a teaching and training implementation method and system based on the metaverse force feedback technology, the method comprising: the host responds to the user login instruction, obtains the current training user information and the current training type selection information corresponding to the user login instruction; if the host determines that the current training type selection information corresponds to the metaverse training scene, the current training data corresponding to the current training type selection information is sent to the VR head display device; the VR head display device receives the current training data, and displays the current virtual reality display data corresponding to the current training data; the force feedback tactile gloves obtain the user's current training operation data for the current virtual reality display data, and displays the current training operation data When the training is finished, the glove model corresponding to the force feedback tactile glove is mapped and displayed in the virtual reality scene displayed by the VR head display device, and the user hand operation data collected by the force feedback tactile glove is used to synchronize to the virtual reality scene displayed by the VR head display device to perform corresponding operations on the current virtual reality display data, and receive corresponding tactile data; if the host detects the training end instruction, the saved current training operation data and current training operation data corresponding to the current training operation data and the current virtual reality display data are obtained, and a current training data set corresponding to the current training data is formed. The embodiment of the present invention combines the host, the VR head display device and the force feedback tactile glove to realize that the current training operation data of the user for the current virtual reality display data in the virtual reality scene displayed by the VR head display device can be collected in time, and the corresponding tactile data is received during the operation, which greatly enhances the simulation effect of the training. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0011] Figure 1 A schematic diagram of an application scenario of a teaching and training implementation method based on the metaverse force feedback technology provided by an embodiment of the present invention; Figure 2 A flowchart of a teaching and training implementation method based on the metaverse force feedback technology provided by an embodiment of the present invention; Figure 3 A schematic diagram of a sub-process of a teaching and training implementation method based on the metaverse force feedback technology provided in an embodiment of the present invention; Figure 4 A schematic block diagram of a teaching and training implementation device based on the metaverse force feedback technology provided by an embodiment of the present invention; Figure 5 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0014] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0015] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0016] Please also refer to Figure 1 and Figure 2 ,in Figure 1 This is a schematic diagram of an application scenario of a teaching and training implementation method based on the metaverse force feedback technology according to an embodiment of the present invention. Figure 2 1 is a flow chart of a teaching and training method based on the metaverse force feedback technology provided by an embodiment of the present invention. Figure 1 As shown, the teaching and training implementation method based on Metaverse force feedback technology provided by an embodiment of the present invention is applied to a teaching and training implementation system based on Metaverse force feedback technology, wherein the teaching and training implementation system based on Metaverse force feedback technology includes a host 1, a display 2, a user input device 3, a VR head display device 4 and a force feedback tactile glove 5; the display 2, the user input device 3 and the VR head display device 4 are all communicatively connected to the host 1, the VR head display device 4 is communicatively connected to the force feedback tactile glove 5, and the host 1 is also connected to a cloud server via a network.

[0017] Among them, still refer to Figure 1, the host 1 is also connected to a router 6 by wired or wireless connection, and the router 6 serves as a medium for connecting the host 1 to the cloud server. Moreover, the VR head display device 4 can be connected to the router 6 to be located in the same local area network as the host 1. More specifically, if the host 1 is placed on the main frame of the training room, the display 2, the user input device 3, the VR head display device 4 and the force feedback haptic gloves 5 can all be placed on the main frame when not in use. When training is required, the participants can take off the VR head display device 4 and the force feedback haptic gloves 5 and put them on to start the training operation.

[0018] like Figure 2 As shown, the method includes the following steps S110-S150.

[0019] S110. The host responds to a user login instruction and obtains current training user information and current training type selection information corresponding to the user login instruction.

[0020] In this embodiment, a Metaverse comprehensive training platform is deployed on the host. After the user operates the user input device to open the Metaverse comprehensive training platform, the user interaction interface corresponding to the Metaverse comprehensive training platform is displayed on the display. At this time, the user can enter the user login information (such as user account and user password) in the user interaction interface corresponding to the Metaverse comprehensive training platform, thereby triggering the generation of a user login instruction. After the user login information passes the user identity authentication of the Metaverse comprehensive training platform, the current training user information corresponding to the user login instruction is obtained. At this time, the user interaction interface corresponding to the Metaverse comprehensive training platform is no longer the user login interface, but switches to the main operation interface of the Metaverse comprehensive training platform. At this time, the user can select one of the four virtual buttons of "speed evaluation", "training module", "data observation" and "active user statistics" provided on the main operation interface to click, thereby starting the corresponding training mode. More specifically, the Metaverse comprehensive training platform is a WEB application platform developed based on the JAVA language, adopts a WEB application microservice architecture based on the general open source framework Springcloud, provides HTTP web page services and API interfaces, and supports multiple people to enter the Metaverse virtual world for real-time training at the same time. The Metaverse comprehensive training platform and VR head-mounted display device communicate data through the general HTTP protocol, which can schedule and manage the scenes in the VR head-mounted display device in real time. In addition, the force feedback glove data received by the VR head-mounted display device and collected by the force feedback tactile gloves can be uploaded to the Metaverse comprehensive training platform for corresponding storage and analysis. The Metaverse comprehensive training platform has a relevant user management system, which integrates the four core functions of "speed evaluation", "training module", "data observation" and "active user statistics". Among them, "speed evaluation" is to start the Metaverse training scene in the VR head-mounted display device through the Metaverse comprehensive training platform, and view the real-time streaming screen of the VR head-mounted display device. The trainees can observe the real-time monitoring screen of the internal operations of the trainees in the VR head-mounted display device corresponding to the Metaverse training scene in the monitoring list, and follow up the details of the trainees during the training process. "Training module" is that the trainees can view the training status, equipment list, equipment name, equipment status and other information, and can also control the start and end operations of the Metaverse training scene in the VR head-mounted display device. "Data observation" means that trainees can view the number of system experiences today, cumulative number of experiences, today's executed projects, total number of devices, currently running devices, currently idle devices, currently unconnected devices, hot project rankings, and project classifications through the Metaverse Comprehensive Training Platform. "Active user statistics" means that trainees can view training records and related training progress through the Metaverse Comprehensive Training Platform and conduct data analysis.Under the core function of "training module", the first-level sub-scenarios such as Metaverse Power Comprehensive Training Scenarios, and the second-level sub-scenarios such as Metaverse Power Fault Training Scenarios and Metaverse Power Equipment Assembly Training Scenarios are respectively assembled and defined in the format of JSON data, and the JSON data is sent to the VR headset device for scene initialization and scene switching scheduling. These sub-scenario services are developed in the form of microservices, and provide services to the outside world through an API interface based on the HTTP protocol, realizing the interaction of training scene data and scheduling instruction management with the VR headset device, and each microservice in the Metaverse Comprehensive Training Platform adopts a distributed deployment method.

[0021] For example, when the user operates the user input device to select the virtual button "training module" on the main operation interface, the corresponding current training type selection information is the Metaverse training scene; when the user operates the user input device to select the virtual button "speed evaluation" on the main operation interface, the corresponding current training type selection information is the Metaverse training viewing scene; when the user operates the user input device to select the virtual button "data observation" on the main operation interface, the corresponding current training type selection information is the data observation scene; when the user operates the user input device to select the virtual button "active user statistics" on the main operation interface, the corresponding current training type selection information is the data analysis scene. It can be seen that the above-mentioned Metaverse comprehensive training platform integrates the four core functions of "speed evaluation", "training module", "data observation" and "active user statistics". Moreover, the Metaverse comprehensive training platform can also support multiple users to participate in training, that is, multiple users respectively operate the corresponding teaching and training implementation system based on Metaverse force feedback technology and log in to the Metaverse comprehensive training platform.

[0022] S120. If the host determines that the current training type selection information corresponds to the metaverse training scene, the current training data corresponding to the current training type selection information is sent to the VR head display device.

[0023] In this embodiment, when the user operates the user input device to select the virtual button "Training Module" on the main operation interface, the corresponding current training type selection information is the metaverse training scene. At this time, the host first obtains one of the training data corresponding to the metaverse training scene, and sends it to the VR headset as the current training data, using the VR headset as the display medium for the metaverse training scene. Among them, since the Metaverse training scenario includes multiple sub-scenarios, the host can select a sub-scenario based on the user's training needs, such as the Metaverse comprehensive power training scenario, and the first-level sub-scenario of the Metaverse comprehensive power training scenario can specifically include the Metaverse power failure training scenario, the Metaverse power equipment assembly training scenario and other second-level sub-scenarios. When a sub-scenario such as the Metaverse power failure training scenario is arbitrarily selected from the second-level sub-scenario, the corresponding current training data can be determined. These current training data can be regarded as the initial virtual reality scene corresponding to the second-level sub-scenario selected based on the user's training needs, and in the initial virtual reality scene, there are several virtual reality model objects corresponding to the selected sub-scenario, such as electric meters, high-voltage cables, distribution transformers, central cabinets, vacuum ring network cabinets, SF6 cabinets (which are a type of ring network cabinet), etc.

[0024] In one embodiment, step S120 includes: If it is determined that the current training type selection information is the Metaverse Electric Power Comprehensive Training Scene in the Metaverse Training Scene, then the current training data corresponding to the Metaverse Electric Power Comprehensive Training Scene is obtained, and the current training data is sent to the VR head display device.

[0025] In this embodiment, the Metaverse training scene includes multiple first-level sub-scenes, for example, the Metaverse electric power comprehensive training scene is one of the sub-scenes, and the Metaverse electric power comprehensive training scene can also include second-level sub-scenes such as the Metaverse electric power failure training scene and the Metaverse electric power equipment assembly training scene. When the user selects the first-level sub-scene of the Metaverse electric power comprehensive training scene, if no specific second-level sub-scene is selected, the Metaverse comprehensive training platform in the host can determine a corresponding default training scene (such as meter assembly) based on the Metaverse electric power comprehensive training scene, and obtain the current training data corresponding to the default training scene, and send the current training data to the VR head display device. It can be seen that through the above method, the specific training scene corresponding to the information can be quickly selected in the host based on the current training type, and the corresponding current training data can be sent to the VR head display device and displayed in the virtual reality scene.

[0026] In one embodiment, the obtaining of the current training data corresponding to the Metaverse Electric Power Comprehensive Training Scenario includes: If it is determined that the Metaverse comprehensive power training scenario is a Metaverse power failure training scenario, then the current training data corresponding to the Metaverse power failure training scenario is generated based on a preset power failure simulation production strategy; If it is determined that the Metaverse Electric Power Comprehensive Training Scenario is a Metaverse Electric Power Equipment Assembly Training Scenario, the current training data corresponding to the Metaverse Electric Power Equipment Assembly Training Scenario is generated based on the acquired electric power equipment assembly project information.

[0027] In this embodiment, the Metaverse Electric Power Comprehensive Training Scenario is one of the first-level sub-scenarios in the Metaverse Training Scenario, which includes the Metaverse Electric Power Failure Training Scenario, the Metaverse Electric Power Equipment Assembly Training Scenario, and other second-level sub-scenarios. When the user selects the Metaverse Electric Power Failure Training Scenario as a second-level sub-scenario, it means that the user needs to perform training operations on power failure maintenance. At this time, the Metaverse Comprehensive Training Platform generates the current training data corresponding to the Metaverse Electric Power Failure Training Scenario based on the electric power failure simulation production strategy. For example, the electric power failure simulation production strategy is used to determine at least one target electric power failure type from the electric power failure simulation library based on the personnel labels of the trainees (such as junior electricians, intermediate electricians, senior electricians, etc.), or based on the training heat values ​​of various types of electric power failures in the electric power failure simulation library (such as the topK of the training heat value, K=3 or a positive integer set based on actual needs), and then determine the current training data based on the target electric power failure type. For example, the virtual reality model objects that need to be included in the target power fault type are electric meters, high-voltage cables, and distribution transformers. Then, based on the virtual reality model objects of the electric meters, high-voltage cables, and distribution transformers (each virtual reality model object has a corresponding initial three-dimensional space position and initial object state in the initial virtual reality scene), the current training data displayed in the initial virtual reality scene is generated accordingly.

[0028] When the user selects the Metaverse power equipment assembly training scene, it means that the user needs to perform training operations on power equipment assembly. At this time, the user can further obtain one of the power equipment assembly project information selected by the user in the power equipment assembly project list provided by the user interaction interface (the power equipment assembly project information can include specific power equipment such as electric meters, high-voltage cables, distribution transformers, central cabinets, vacuum ring network cabinets, etc.), and the virtual reality model objects corresponding to the power equipment specifically included in the power equipment assembly project information are placed and set in the initial spatial position specified in the initial virtual reality scene. The trainee needs to assemble the power equipment based on the power equipment assembly operation. After obtaining the power equipment assembly project information, the current training data displayed in the initial virtual reality scene is generated based on the virtual reality model objects of each power equipment specifically included in the power equipment assembly project information.

[0029] Of course, during the specific implementation, the first-level sub-scenario of the Metaverse training scenario includes not only the second-level sub-scenario such as the Metaverse power failure training scenario and the Metaverse power equipment assembly training scenario, but also the second-level sub-scenario such as the Metaverse power failure evacuation and escape training scenario and the Metaverse power equipment emergency handling operation training scenario. The corresponding training scenarios can be set according to the actual needs of users.

[0030] S130. The VR head display device receives the current training data and displays current virtual reality display data corresponding to the current training data.

[0031] In this embodiment, when the VR head display device receives the current training data sent by the host based on the local area network, it first obtains the virtual reality model objects of each power equipment included in the current training data, and then combines the virtual reality model objects into the current virtual reality display data, and displays them in the virtual reality scene displayed by the VR head display device.

[0032] S140: The force feedback tactile glove obtains the user's current training operation data for the current virtual reality display data, and sends the current training operation data to the host in real time for storage until a training end instruction is detected.

[0033] Among them, the glove model mapping corresponding to the force feedback tactile glove is displayed in the virtual reality scene displayed by the VR head display device, and the user hand operation data collected by the force feedback tactile glove is used to synchronize with the virtual reality scene displayed by the VR head display device to perform corresponding operations on the current virtual reality display data, and receive corresponding tactile data; the tactile data is generated and sent to the force feedback tactile glove when the VR head display device performs corresponding operations on the current virtual reality display data based on the user hand operation data.

[0034] In this embodiment, in order to simulate the users participating in the training to perform corresponding operations on various virtual reality model objects in the virtual reality scene displayed by the VR head display device through the feedback tactile gloves, it is first necessary to map the glove model corresponding to the feedback tactile gloves and display it in the virtual reality scene displayed by the VR head display device, and more specifically, to integrate it into the virtual reality scene corresponding to the current virtual reality display data. Moreover, the operations performed by the user's hand driving force feedback tactile gloves will be synchronized to the virtual reality scene in real time to perform corresponding operations on the current virtual reality display data and receive corresponding tactile data.

[0035] In order to synchronize the user's hand operation data to the virtual reality scene by the force feedback tactile gloves so as to perform corresponding operations on the current virtual reality display data and receive the corresponding tactile data, the force feedback tactile gloves can specifically adopt the VR gloves SenseGlove (which integrates active contact feedback, force feedback and vibration tactile feedback functions; active contact feedback is used to feel the impact and grip of the palm; force feedback is used to feel the size and hardness of virtual objects; vibration tactile feedback is used to feel signals and basic textures), and it has the use effects of gesture recognition, immersive tactile experience, etc. The user's hand operation data collected by the force feedback tactile gloves can be synchronized in real time to the glove model in the virtual reality scene displayed by the VR head display device. The operations performed by the user wearing the force feedback tactile gloves can be mapped to the glove model and the glove model can be used to perform corresponding operations on the current virtual reality display data.

[0036] In one embodiment, if Figure 3 As shown, step S140 includes: S141, the feedback tactile glove obtains the user's current operation model object in the current virtual reality display data, and generates tactile data corresponding to the current operation model object for simulating real tactile sensation; S142: The feedback tactile glove obtains the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, and sends it to the host for storage.

[0037] In this embodiment, when the user wears the feedback tactile gloves for practical training, when the user's hand drives the feedback tactile gloves to perform an operation (for example, moving the hand position to contact a faulty cable, etc.), the hand model in the virtual reality scene will also move to the location of the corresponding current operation model object in the current virtual reality display data to perform the same operation. Since the current operation model object is currently known (that is, the specific physical type of the current operation model object is known), the tactile data corresponding to the current operation model object can be obtained in the background model object database corresponding to the Metaverse Comprehensive Training Platform to simulate the user's real tactile feeling for the current operation model object. For example, if the current operation model object is a faulty cable, when the user operates the feedback tactile gloves to drive the glove model to move to the cable, the tightness of the cable can be felt by the tactile data generated by the VR head display device, so that the user can get a real sense of force when operating electrical equipment, thereby greatly enhancing the simulation effect of the practical training.

[0038] Afterwards, the feedback tactile gloves obtain the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data. This current training operation data represents the specific operation of the user on the current operation model object in the current virtual reality display data. The feedback tactile gloves can fully capture and identify the user's hand movements and subtle gestures, ensuring that the user's operations in the virtual reality environment can be accurately mapped to the actual power operation (that is, the user's operations in the virtual reality environment also obtain the same operation experience in the actual power operation). Whether it is a simple switch operation or a complex troubleshooting, it can be accurately simulated. The current training operation data obtained is sent to the host in real time for storage. The above process is to save the current training operation data of the user wearing the feedback tactile gloves to perform an operation action. Other operation actions are also processed in accordance with this process until the training end instruction is detected.

[0039] In one embodiment, step S142 includes: The feedback tactile glove obtains the current user hand operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, and synchronously maps the current user hand operation data to the glove model in the virtual reality scene of the VR head display device, and synchronously drives the glove model to perform corresponding operations on the current operation model object and generate the current training operation data.

[0040] In this embodiment, when the feedback tactile glove obtains the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, the feedback tactile glove, as a physical device, first captures and recognizes the user's hand movements and subtle gestures, and maps these hand movements and subtle gestures to the glove model in the virtual reality scene of the VR head display device, and uses the glove model as the digital twin model of the feedback tactile glove to perform corresponding operations on the current operation model object and generate the current training operation data. The above method enables the user's operation process to obtain an immersive tactile experience, simulating that the user is in a real operation environment and can intuitively obtain physical feedback from each operation.

[0041] S150: If the host detects a training end instruction, it obtains the saved current training operation data and the current training operation data corresponding to the current training operation data and the current virtual reality display data, and forms a current training data set corresponding to the current training data.

[0042] In this embodiment, if the host detects a training end instruction, it means that the user's current round of Metaverse comprehensive training is over. The saved current training operation data and the current virtual reality display data corresponding to the current training operation data and the current training operation data can be obtained first, and the current training data set corresponding to the current training data can be formed; then, the current training data set is saved in the background database of the host. Among them, the current training data set has a binding relationship with the user corresponding to the current training user information, that is, it means that this current training data set is the data generated by the user corresponding to the current training user information performing Metaverse training operations within the user operation time interval. The user operation time interval can be the time when the user login instruction is generated as the starting time point, and the time when the training end instruction is generated as the end time point.

[0043] In one embodiment, after step S110, the method further includes: If the host determines that the current training type selection information corresponds to the metaverse training viewing scene, it obtains the currently viewable training operation video stream corresponding to the metaverse training viewing scene, and sends the currently viewable training operation video stream to the VR head display device.

[0044] In this embodiment, when the user operates the user input device to select the virtual button "Speed ​​Evaluation" on the main operation interface, the corresponding current training type selection information is the Metaverse training viewing scene. At this time, if the host has been receiving the current training operation data from the current user participating in the training, then other users can obtain the current training operation data of the above-mentioned user participating in the training from the host in the Metaverse training viewing scene, and send it as the currently viewable training operation video stream to the VR head display devices of other viewing users (for example, teachers who comment on and operate the operations of the training users), and synchronize the current operation step information corresponding to the currently viewable training operation video stream on the interface of the currently viewable training operation video stream, so that the VR head display devices of other users can obtain the currently viewable training operation video stream in time, thereby realizing real-time monitoring and transmission of training operation data.

[0045] Still referring to the above example, if the teacher wears feedback tactile gloves in addition to the VR headset, the current training operation data of the users participating in the training can also be simulated with real touch on the teacher's feedback tactile gloves, so that the teacher can quickly obtain the students' operation data and give timely and accurate feedback.

[0046] In one embodiment, after step S110, the method further includes: If the host determines that the current training type selection information corresponds to a data observation scenario, the host obtains project equipment related data corresponding to the data observation scenario to display it through the display connected to the host.

[0047] In this embodiment, when the user operates the user input device to select the virtual button "Data Observation" on the main operation interface, the corresponding current training type selection information is the data observation scene, and the user can obtain project equipment related data such as the number of experiences today, the cumulative number of experiences, today's execution projects, executing projects, the total number of devices, the current running devices, the current idle devices, the current unconnected devices, the hot project rankings and project classifications, and display them through the display connected to the host. Of course, it can also be displayed by sending it to a VR head display device.

[0048] When the user operates the user input device to select the virtual button "Active User Statistics" on the main operation interface, the corresponding current training type selection information is a data analysis scene. The user can view the active user statistics data corresponding to the data analysis scene on the display or VR headset device, so that trainees can view the training records and related training progress, and perform data analysis.

[0049] It can be seen that the embodiment of the method combined with the host, VR head display device and force feedback tactile gloves realizes that the user's current training operation data for the current virtual reality display data in the virtual reality scene displayed by the VR head display device can be collected in a timely manner, and the corresponding tactile data is received during the operation, which greatly enhances the simulation effect of the training.

[0050] Figure 4 is a schematic block diagram of a teaching and training implementation system based on the metaverse force feedback technology provided by an embodiment of the present invention. Figure 4 As shown, corresponding to the above teaching and training implementation method based on the metaverse force feedback technology, the present invention also provides a teaching and training implementation system 100 based on the metaverse force feedback technology. Figure 1 and Figure 4 The teaching and training implementation system 100 based on the metaverse force feedback technology includes a host 1, a display 2, a user input device 3, a VR head display device 4 and a force feedback tactile glove 5; the display 2, the user input device 3 and the VR head display device 4 are all communicatively connected to the host 1, the VR head display device 4 is communicatively connected to the force feedback tactile glove 5, and the host 1 is also connected to the cloud server through a network.

[0051] Among them, still refer to Figure 1, the host 1 is also connected to a router 6 by wired or wireless connection, and the router 6 serves as a medium for connecting the host 1 to the cloud server. Moreover, the VR head display device 4 can be connected to the router 6 so as to be located in the same local area network as the host 1. More specifically, if the host 1 is placed on the main rack of the training room, the display 2, the user input device 3, the VR head display device 4 and the force feedback tactile gloves 5 can all be placed on the main rack when not in use. When training is required, the trainees can remove the VR head display device 4 and the force feedback tactile gloves 5 and put them on to start the training operation. The user input device 3 is implemented using a keyboard and a mouse.

[0052] The host 1 is used to respond to a user login instruction and obtain current training user information and current training type selection information corresponding to the user login instruction.

[0053] In this embodiment, a Metaverse comprehensive training platform is deployed on the host. After the user operates the user input device to open the Metaverse comprehensive training platform, the user interaction interface corresponding to the Metaverse comprehensive training platform is displayed on the display. At this time, the user can enter the user login information (such as user account and user password) in the user interaction interface corresponding to the Metaverse comprehensive training platform, thereby triggering the generation of a user login instruction. After the user login information passes the user identity authentication of the Metaverse comprehensive training platform, the current training user information corresponding to the user login instruction is obtained. At this time, the user interaction interface corresponding to the Metaverse comprehensive training platform is no longer a user login interface, but switches to the main operation interface of the Metaverse comprehensive training platform. At this time, the user can select one of the four virtual buttons of "Speed ​​Evaluation", "Training Module", "Data Observation" and "Active User Statistics" provided on the main operation interface. Click on it to start the corresponding training mode.

[0054] For example, when the user operates the user input device to select the virtual button "training module" on the main operation interface, the corresponding current training type selection information is the Metaverse training scene; when the user operates the user input device to select the virtual button "speed evaluation" on the main operation interface, the corresponding current training type selection information is the Metaverse training viewing scene; when the user operates the user input device to select the virtual button "data observation" on the main operation interface, the corresponding current training type selection information is the data observation scene; when the user operates the user input device to select the virtual button "active user statistics" on the main operation interface, the corresponding current training type selection information is the data analysis scene. It can be seen that the above-mentioned Metaverse comprehensive training platform integrates the four core functions of "speed evaluation", "training module", "data observation" and "active user statistics". Moreover, the Metaverse comprehensive training platform can also support multiple users to participate in training, that is, multiple users respectively operate the corresponding teaching and training implementation system based on Metaverse force feedback technology and log in to the Metaverse comprehensive training platform.

[0055] The host 1 is also used to send the current training data corresponding to the current training type selection information to the VR head display device if it is determined that the current training type selection information corresponds to the metaverse training scene.

[0056] In this embodiment, when the user operates the user input device to select the virtual button "Training Module" on the main operation interface, the corresponding current training type selection information is the metaverse training scene. At this time, the host first obtains one of the training data corresponding to the metaverse training scene, and sends it to the VR headset as the current training data, using the VR headset as the display medium for the metaverse training scene. Among them, since the Metaverse training scenario includes multiple sub-scenarios, the host can select a sub-scenario based on the user's training needs, such as the Metaverse comprehensive power training scenario, and the first-level sub-scenario of the Metaverse comprehensive power training scenario can specifically include the Metaverse power failure training scenario, the Metaverse power equipment assembly training scenario and other second-level sub-scenarios. When a sub-scenario such as the Metaverse power failure training scenario is arbitrarily selected from the second-level sub-scenario, the corresponding current training data can be determined. These current training data can be regarded as the initial virtual reality scene corresponding to the second-level sub-scenario selected based on the user's training needs, and in the initial virtual reality scene, there are several virtual reality model objects corresponding to the selected sub-scenario, such as electric meters, high-voltage cables, distribution transformers, central cabinets, vacuum ring network cabinets, SF6 cabinets (which are a type of ring network cabinet), etc.

[0057] In one embodiment, the host 1 is specifically used for: If it is determined that the current training type selection information is the Metaverse Electric Power Comprehensive Training Scene in the Metaverse Training Scene, then the current training data corresponding to the Metaverse Electric Power Comprehensive Training Scene is obtained, and the current training data is sent to the VR head display device.

[0058] In this embodiment, the Metaverse training scene includes multiple first-level sub-scenes, for example, the Metaverse electric power comprehensive training scene is one of the sub-scenes, and the Metaverse electric power comprehensive training scene can also include second-level sub-scenes such as the Metaverse electric power failure training scene and the Metaverse electric power equipment assembly training scene. When the user selects the first-level sub-scene of the Metaverse electric power comprehensive training scene, if no specific second-level sub-scene is selected, the Metaverse comprehensive training platform in the host can determine a corresponding default training scene (such as meter assembly) based on the Metaverse electric power comprehensive training scene, and obtain the current training data corresponding to the default training scene, and send the current training data to the VR head display device. It can be seen that through the above method, the specific training scene corresponding to the information can be quickly selected in the host based on the current training type, and the corresponding current training data can be sent to the VR head display device and displayed in the virtual reality scene.

[0059] In one embodiment, the obtaining of the current training data corresponding to the Metaverse Electric Power Comprehensive Training Scenario includes: If it is determined that the Metaverse comprehensive power training scenario is a Metaverse power failure training scenario, then the current training data corresponding to the Metaverse power failure training scenario is generated based on a preset power failure simulation production strategy; If it is determined that the Metaverse Electric Power Comprehensive Training Scenario is a Metaverse Electric Power Equipment Assembly Training Scenario, the current training data corresponding to the Metaverse Electric Power Equipment Assembly Training Scenario is generated based on the acquired electric power equipment assembly project information.

[0060] In this embodiment, the Metaverse Electric Power Comprehensive Training Scenario is one of the first-level sub-scenarios in the Metaverse Training Scenario, which includes the Metaverse Electric Power Failure Training Scenario, the Metaverse Electric Power Equipment Assembly Training Scenario, and other second-level sub-scenarios. When the user selects the Metaverse Electric Power Failure Training Scenario as a second-level sub-scenario, it means that the user needs to perform training operations on power failure maintenance. At this time, the Metaverse Comprehensive Training Platform generates the current training data corresponding to the Metaverse Electric Power Failure Training Scenario based on the electric power failure simulation production strategy. For example, the electric power failure simulation production strategy is used to determine at least one target electric power failure type from the electric power failure simulation library based on the personnel labels of the trainees (such as junior electricians, intermediate electricians, senior electricians, etc.), or based on the training heat values ​​of various types of electric power failures in the electric power failure simulation library (such as the topK of the training heat value, K=3 or a positive integer set based on actual needs), and then determine the current training data based on the target electric power failure type. For example, the virtual reality model objects that need to be included in the target power fault type are electric meters, high-voltage cables, and distribution transformers. Then, based on the virtual reality model objects of the electric meters, high-voltage cables, and distribution transformers (each virtual reality model object has a corresponding initial three-dimensional space position and initial object state in the initial virtual reality scene), the current training data displayed in the initial virtual reality scene is generated accordingly.

[0061] When the user selects the Metaverse power equipment assembly training scene, it means that the user needs to perform training operations on power equipment assembly. At this time, the user can further obtain one of the power equipment assembly project information selected by the user in the power equipment assembly project list provided by the user interaction interface (the power equipment assembly project information can include specific power equipment such as electric meters, high-voltage cables, distribution transformers, central cabinets, vacuum ring network cabinets, etc.), and the virtual reality model objects corresponding to the power equipment specifically included in the power equipment assembly project information are placed and set in the initial spatial position specified in the initial virtual reality scene. The trainee needs to assemble the power equipment based on the power equipment assembly operation. After obtaining the power equipment assembly project information, the current training data displayed in the initial virtual reality scene is generated based on the virtual reality model objects of each power equipment specifically included in the power equipment assembly project information.

[0062] Of course, during the specific implementation, the first-level sub-scenario of the Metaverse training scenario includes not only the second-level sub-scenario such as the Metaverse power failure training scenario and the Metaverse power equipment assembly training scenario, but also the second-level sub-scenario such as the Metaverse power failure evacuation and escape training scenario and the Metaverse power equipment emergency handling operation training scenario. The corresponding training scenarios can be set according to the actual needs of users.

[0063] The VR head display device 4 is used to receive the current training data and display the current virtual reality display data corresponding to the current training data.

[0064] In this embodiment, when the VR head display device receives the current training data sent by the host based on the local area network, it first obtains the virtual reality model objects of each power equipment included in the current training data, and then combines the virtual reality model objects into the current virtual reality display data, and displays them in the virtual reality scene displayed by the VR head display device.

[0065] The force feedback tactile gloves 5 are used to obtain the user's current training operation data for the current virtual reality display data, and send the current training operation data to the host in real time for storage until a training end instruction is detected.

[0066] Among them, the glove model mapping corresponding to the force feedback tactile glove is displayed in the virtual reality scene displayed by the VR head display device, and the user hand operation data collected by the force feedback tactile glove is used to synchronize with the virtual reality scene displayed by the VR head display device to perform corresponding operations on the current virtual reality display data, and receive corresponding tactile data; the tactile data is generated and sent to the force feedback tactile glove when the VR head display device performs corresponding operations on the current virtual reality display data based on the user hand operation data.

[0067] In this embodiment, in order to simulate the users participating in the training to perform corresponding operations on various virtual reality model objects in the virtual reality scene displayed by the VR head display device through the feedback tactile gloves, it is first necessary to map the glove model corresponding to the feedback tactile gloves and display it in the virtual reality scene displayed by the VR head display device, and more specifically, to integrate it into the virtual reality scene corresponding to the current virtual reality display data. Moreover, the operations performed by the user's hand driving force feedback tactile gloves will be synchronized to the virtual reality scene in real time to perform corresponding operations on the current virtual reality display data and receive corresponding tactile data.

[0068] In order to synchronize the user's hand operation data to the virtual reality scene by the force feedback tactile gloves so as to perform corresponding operations on the current virtual reality display data and receive the corresponding tactile data, the force feedback tactile gloves can specifically adopt the VR gloves SenseGlove (which integrates active contact feedback, force feedback and vibration tactile feedback functions; active contact feedback is used to feel the impact and grip of the palm; force feedback is used to feel the size and hardness of virtual objects; vibration tactile feedback is used to feel signals and basic textures), and it has the use effects of gesture recognition, immersive tactile experience, etc. The user's hand operation data collected by the force feedback tactile gloves can be synchronized in real time to the glove model in the virtual reality scene displayed by the VR head display device. The operations performed by the user wearing the force feedback tactile gloves can be mapped to the glove model and the glove model can be used to perform corresponding operations on the current virtual reality display data.

[0069] In one embodiment, the force feedback tactile glove 5 is specifically used for: Acquire a current operation model object in the current virtual reality display data by the user, and generate tactile data corresponding to the current operation model object for simulating a real tactile sensation; The current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data is obtained, and sent to the host for storage.

[0070] In this embodiment, when the user wears the feedback tactile gloves for training operations, when the user's hand drives the feedback tactile gloves to perform an operation (for example, moving the hand position to contact a faulty cable, etc.), the hand model in the virtual reality scene will also move to the location of the corresponding current operation model object in the current virtual reality display data to perform the same operation. Since the current operation model object is currently known (that is, the specific physical type of the current operation model object is known), the tactile data corresponding to the current operation model object can be obtained in the background model object database corresponding to the Metaverse Comprehensive Training Platform to simulate the user's real touch for the current operation model object. For example, if the current operation model object is a faulty cable, when the user operates the feedback tactile gloves to drive the glove model to move to the cable, the user can feel the tightness of the cable, etc., so that the user can get a real sense of force when operating electrical equipment, thereby greatly enhancing the simulation effect of the training.

[0071] Afterwards, the feedback tactile gloves obtain the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data. This current training operation data represents the specific operation of the user on the current operation model object in the current virtual reality display data. The feedback tactile gloves can fully capture and identify the user's hand movements and subtle gestures, ensuring that the user's operations in the virtual reality environment can be accurately mapped to the actual power operation (that is, the user's operations in the virtual reality environment also obtain the same operation experience in the actual power operation). Whether it is a simple switch operation or a complex troubleshooting, it can be accurately simulated. The current training operation data obtained is sent to the host in real time for storage. The above process is to save the current training operation data of the user wearing the feedback tactile gloves to perform an operation action. Other operation actions are also processed in accordance with this process until the training end instruction is detected.

[0072] In one embodiment, the step of obtaining the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data and sending the data to the host for storage includes: The feedback tactile glove obtains the current user hand operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, and synchronously maps the current user hand operation data to the glove model in the virtual reality scene of the VR head display device, and synchronously drives the glove model to perform corresponding operations on the current operation model object and generate the current training operation data.

[0073] In this embodiment, when the feedback tactile glove obtains the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, the feedback tactile glove, as a physical device, first captures and recognizes the user's hand movements and subtle gestures, and maps these hand movements and subtle gestures to the glove model in the virtual reality scene of the VR head display device, and uses the glove model as the digital twin model of the feedback tactile glove to perform corresponding operations on the current operation model object and generate the current training operation data. The above method enables the user's operation process to obtain an immersive tactile experience, simulating that the user is in a real operation environment and can intuitively obtain physical feedback from each operation.

[0074] The host 1 is also used to obtain the saved current training operation data and the current training operation data corresponding to the current training operation data and the current virtual reality display data if a training end instruction is detected, and to form a current training data set corresponding to the current training data.

[0075] In this embodiment, if the host detects a training end instruction, it means that the user's current round of Metaverse comprehensive training is over. The saved current training operation data and the current virtual reality display data corresponding to the current training operation data and the current training operation data can be obtained first, and the current training data set corresponding to the current training data can be formed; then, the current training data set is saved in the background database of the host. Among them, the current training data set has a binding relationship with the user corresponding to the current training user information, that is, it means that this current training data set is the data generated by the user corresponding to the current training user information performing Metaverse training operations within the user operation time interval. The user operation time interval can be the time when the user login instruction is generated as the starting time point, and the time when the training end instruction is generated as the end time point.

[0076] In one embodiment, the host 1 is further used for: Determine that the current training type selection information corresponds to the metaverse training viewing scene, then obtain the currently viewable training operation video stream corresponding to the metaverse training viewing scene, and send the currently viewable training operation video stream to the VR head display device.

[0077] In this embodiment, when the user operates the user input device to select the virtual button "Speed ​​Evaluation" on the main operation interface, the corresponding current training type selection information is the Metaverse training viewing scene. At this time, if the host has been receiving the current training operation data from the current user participating in the training, then other users can obtain the current training operation data of the above-mentioned user participating in the training from the host in the Metaverse training viewing scene, and send it as the currently viewable training operation video stream to the VR head display devices of other viewing users (for example, teachers who comment on and operate the operations of the training users), and synchronize the current operation step information corresponding to the currently viewable training operation video stream on the interface of the currently viewable training operation video stream, so that the VR head display devices of other users can obtain the currently viewable training operation video stream in time, thereby realizing real-time monitoring and transmission of training operation data.

[0078] Still referring to the above example, if the teacher wears feedback tactile gloves in addition to the VR headset, the current training operation data of the users participating in the training can also be simulated with real touch on the teacher's feedback tactile gloves, so that the teacher can quickly obtain the students' operation data and give timely and accurate feedback.

[0079] In one embodiment, the host 1 is further used for: If it is determined that the current training type selection information corresponds to a data observation scenario, project equipment related data corresponding to the data observation scenario is acquired to be displayed through the display connected to the host.

[0080] In this embodiment, when the user operates the user input device to select the virtual button "Data Observation" on the main operation interface, the corresponding current training type selection information is the data observation scene, and the user can obtain project equipment related data such as the number of experiences today, the cumulative number of experiences, today's execution projects, executing projects, the total number of devices, the current running devices, the current idle devices, the current unconnected devices, the hot project rankings and project classifications, and display them through the display connected to the host. Of course, it can also be displayed by sending it to a VR head display device.

[0081] When the user operates the user input device to select the virtual button "Active User Statistics" on the main operation interface, the corresponding current training type selection information is a data analysis scene. The user can view the active user statistics data corresponding to the data analysis scene on the display or VR headset device, so that trainees can view the training records and related training progress, and perform data analysis.

[0082] It can be seen that the implementation example of the system combined with the host, VR head display device and force feedback tactile gloves realizes that the user's current training operation data for the current virtual reality display data in the virtual reality scene displayed by the VR head display device can be collected in a timely manner, and the corresponding tactile data is received during the operation, which greatly enhances the simulation effect of the training.

[0083] The above-mentioned teaching and training implementation system based on the metaverse force feedback technology can be implemented in the form of a computer program. The computer program can be used in Figure 5 Runs on the computer device shown.

[0084] See also Figure 5 , Figure 5 1 is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device integrates any teaching and training implementation system based on the metaverse force feedback technology provided by an embodiment of the present invention.

[0085] See also Figure 5 The computer device 400 includes a processor 402 , a memory and a network interface 405 connected via a system bus 401 , wherein the memory may include a storage medium 403 and an internal memory 404 .

[0086] The storage medium 403 can store an operating system 4031 and a computer program 4032. The computer program 4032 includes program instructions, and when the program instructions are executed, the processor 402 can execute the above-mentioned teaching and training implementation method based on the metaverse force feedback technology.

[0087] The processor 402 is used to provide computing and control capabilities to support the operation of the entire computer device.

[0088] The internal memory 404 provides an environment for the operation of the computer program 4032 in the storage medium 403. When the computer program 4032 is executed by the processor 402, the processor 402 can execute the above-mentioned teaching and training implementation method based on the metaverse force feedback technology.

[0089] The network interface 405 is used to communicate with other devices over the network. Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0090] Among them, the processor 402 is used to run the computer program 4032 stored in the memory to implement the teaching and training method based on the metaverse force feedback technology as mentioned above.

[0091] It should be understood that in the embodiment of the present invention, the processor 402 may be a central processing unit (CPU), and the processor 402 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0092] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0093] Therefore, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the above-mentioned teaching and training implementation method based on the metaverse force feedback technology.

[0094] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0095] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0096] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0097] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0099] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A teaching and training implementation method based on metaverse force feedback technology, applied to a teaching and training implementation system based on metaverse force feedback technology, characterized in that: The teaching and training implementation system based on the metaverse force feedback technology includes a host, a display, a user input device, a VR head display device and a force feedback tactile glove; the display, the user input device and the VR head display device are all connected to the host in communication, the VR head display device is connected to the force feedback tactile glove in communication, and the host is also connected to a cloud server through a network; The teaching and training implementation method based on the metaverse force feedback technology includes: The host responds to the user login instruction and obtains the current training user information and current training type selection information corresponding to the user login instruction; If the host determines that the current training type selection information corresponds to the metaverse training scene, the host sends the current training data corresponding to the current training type selection information to the VR head display device; The VR head display device receives the current training data and displays the current virtual reality display data corresponding to the current training data; The force feedback tactile glove obtains the user's current training operation data for the current virtual reality display data, and sends the current training operation data to the host in real time for storage until a training end instruction is detected; wherein, the glove model mapping corresponding to the force feedback tactile glove is displayed in the virtual reality scene displayed by the VR head display device, and the user's hand operation data collected by the force feedback tactile glove is used to synchronize with the virtual reality scene displayed by the VR head display device to perform corresponding operations on the current virtual reality display data, and receive corresponding tactile data; the tactile data is generated and sent to the force feedback tactile glove when the VR head display device performs corresponding operations on the current virtual reality display data based on the user's hand operation data; If the host detects a training end instruction, it obtains the saved current training operation data and the current training operation data corresponding to the current training operation data and the current virtual reality display data, and forms a current training data set corresponding to the current training data.

2. The method according to claim 1, characterized in that The determining that the current training type selection information corresponds to the metaverse training scene, and sending the current training data corresponding to the current training type selection information to the VR head display device, includes: If it is determined that the current training type selection information is the Metaverse Electric Power Comprehensive Training Scene in the Metaverse Training Scene, then the current training data corresponding to the Metaverse Electric Power Comprehensive Training Scene is obtained, and the current training data is sent to the VR head display device.

3. The method according to claim 2, characterized in that The obtaining of the current training data corresponding to the Metaverse Electric Power Comprehensive Training Scenario includes: If it is determined that the Metaverse comprehensive power training scenario is a Metaverse power failure training scenario, then the current training data corresponding to the Metaverse power failure training scenario is generated based on a preset power failure simulation production strategy; If it is determined that the Metaverse Electric Power Comprehensive Training Scenario is a Metaverse Electric Power Equipment Assembly Training Scenario, the current training data corresponding to the Metaverse Electric Power Equipment Assembly Training Scenario is generated based on the acquired electric power equipment assembly project information.

4. The method according to claim 1, characterized in that: The force feedback tactile glove obtains the user's current training operation data for the current virtual reality display data, and sends the current training operation data to the host in real time for storage, including: The feedback tactile glove obtains the user's current operation model object in the current virtual reality display data, and generates tactile data corresponding to the current operation model object for simulating real tactile sensation; The feedback tactile glove obtains the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, and sends it to the host for storage.

5. The method according to claim 4, characterized in that The feedback tactile glove obtains the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, including: The feedback tactile glove obtains the current user hand operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, and synchronously maps the current user hand operation data to the glove model in the virtual reality scene of the VR head display device, and synchronously drives the glove model to perform corresponding operations on the current operation model object and generate the current training operation data.

6. The method according to claim 1, characterized in that After the host responds to the user login instruction and obtains the current training user information and current training type selection information corresponding to the user login instruction, the method further includes: If the host determines that the current training type selection information corresponds to the metaverse training viewing scene, it obtains the currently viewable training operation video stream corresponding to the metaverse training viewing scene, and sends the currently viewable training operation video stream to the VR head display device.

7. The method according to claim 1, characterized in that After the host responds to the user login instruction and obtains the current training user information and current training type selection information corresponding to the user login instruction, the method further includes: If the host determines that the current training type selection information corresponds to a data observation scenario, the host obtains project equipment related data corresponding to the data observation scenario to display it through the display connected to the host.

8. A teaching and training implementation system based on metaverse force feedback technology, characterized in that: The system comprises a host, a display, a user input device, a VR head display device and a force feedback tactile glove; the display, the user input device and the VR head display device are all connected to the host in communication, the VR head display device is connected to the force feedback tactile glove in communication, and the host is also connected to a cloud server via a network; The host is used to respond to the user login instruction and obtain the current training user information and current training type selection information corresponding to the user login instruction; The host is further configured to send current training data corresponding to the current training type selection information to the VR head display device if it is determined that the current training type selection information corresponds to a metaverse training scene; The VR head display device is used to receive the current training data and display the current virtual reality display data corresponding to the current training data; The force feedback tactile glove is used to obtain the user's current training operation data for the current virtual reality display data, and send the current training operation data to the host in real time for storage until a training end instruction is detected; wherein the glove model corresponding to the force feedback tactile glove is mapped and displayed in the virtual reality scene displayed by the VR head display device, and the user's hand operation data collected by the force feedback tactile glove is used to synchronize with the virtual reality scene displayed by the VR head display device to perform corresponding operations on the current virtual reality display data, and receive corresponding tactile data; the tactile data is generated and sent to the force feedback tactile glove when the VR head display device performs corresponding operations on the current virtual reality display data based on the user's hand operation data; The host is also used to obtain the saved current training operation data and the current training operation data corresponding to the current training operation data and the current virtual reality display data if a training end instruction is detected, and to form a current training data set corresponding to the current training data.

9. The teaching and training implementation system based on the metaverse force feedback technology according to claim 8 is characterized in that: The host is specifically used for: If it is determined that the current training type selection information is the Metaverse Electric Power Comprehensive Training Scene in the Metaverse Training Scene, then the current training data corresponding to the Metaverse Electric Power Comprehensive Training Scene is obtained, and the current training data is sent to the VR head display device.

10. The teaching and training implementation system based on the metaverse force feedback technology according to claim 8 is characterized in that: The feedback tactile gloves are specifically used for: Acquire a current operation model object in the current virtual reality display data by the user, and generate tactile data corresponding to the current operation model object for simulating a real tactile sensation; The current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data is obtained, and sent to the host for storage.

Images