Teaching practical training implementation method and system based on element cosmic force feedback technology

Abstract

The invention discloses a teaching practical training implementation method and system based on a meta-universe force feedback technology, and the method comprises the steps: obtaining the current practical training user information, the current practical training type selection information and a meta-universe practical training scene through a host, and transmitting the current practical training data to VR head-mounted display equipment; the VR head-mounted display device displays the current virtual reality display data; the force feedback touch glove obtains current practical training operation data aiming at the current virtual reality display data and sends the current practical training operation data to the host; and if the host detects a practical training ending instruction, the host obtains the current practical training operation data and the stored current practical training operation data and forms a current practical training data set. According to the embodiment of the invention, in combination with the host, the VR head display device and the force feedback touch glove, the current practical training operation data for the current virtual reality display data in the virtual reality scene displayed by the VR head display device by the user can be timely acquired, and the corresponding touch data is received in the operation process, so that the practical training simulation effect is greatly enhanced.

Description

[0001] This application claims priority to the Chinese patent application No. CN202411797419.4, filed on December 9, 2024, with the Chinese Patent Office and entitled “Teaching and practical training implementation method and system based on meta-universe force feedback technology”, the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of meta-universe, in particular to a teaching and practical training implementation method and system based on meta-universe force feedback technology. BACKGROUND

[0003] In the traditional field of power professional training, the training method mainly relies on video tutorials, written materials and limited field operations. Although the above-mentioned mode has its basic role, it inevitably faces several key challenges: 1) Limitations of actual operation simulation, i.e. limited by physical space, professional equipment configuration and safety specifications, traditional training is difficult to fully and realistically simulate complex scenarios and emergency situations in power work, making it difficult for trainees to accumulate sufficient practical operation experience, affecting their ability to deal with actual problems; 2) Lag in monitoring and feedback, i.e. in the traditional training process, due to the lack of an efficient real-time monitoring mechanism, the instructor is difficult to obtain the operation details of the trainee in time, so as to provide targeted guidance and immediate feedback, limiting the improvement of training effect; 3) Lack of practical experience, i.e. the traditional training content is relatively fixed and single, which is difficult to fully cover the diversified work scenarios and challenges in the power profession, so that the trainee is difficult to obtain rich and realistic operation experience, affecting the comprehensive improvement of their professional skills and comprehensive quality.

[0004] In view of the above problems existing in the power professional training, a virtual reality power professional training system has appeared, which builds a highly simulated power work environment through virtual reality (VR) technology, enabling trainees to perform various power operation training in a virtual scenario, significantly improving their operation ability and team collaboration level. However, when using the above virtual reality power professional training system, the personnel participating in the training have no real physical feedback of the operation object when operating in the virtual reality environment, i.e. the immersive tactile sensation is poor; and the operation details of the trainee cannot be obtained in time, so as to provide targeted guidance and immediate feedback. SUMMARY

[0005] The embodiment of the present application provides a teaching practical implementation method and device based on meta-universe force feedback technology, computer equipment and medium, aiming at solving the problems that there is no real physical feedback of operation object when the personnel in the virtual reality environment specifically operate the virtual reality power professional practical training system and the practical training personnel in the prior art, and the operation details of the practical training personnel cannot be obtained in time.

[0006] In a first aspect, the embodiment of the present application provides a teaching practical implementation method based on meta-universe force feedback technology, applied to a teaching practical implementation system based on meta-universe force feedback technology, which includes a host, a display, a user input device, a VR head-mounted device and a force feedback haptic glove; the display, the user input device and the VR head-mounted device are all in communication connection with the host, the VR head-mounted device is in communication connection with the force feedback haptic glove, and the host is also connected with a cloud server through a network; the teaching practical implementation method based on meta-universe force feedback technology includes: The host acquires current practical user information and current practical type selection information corresponding to the user login instruction in response to a user login instruction; If the host determines that the current practical type selection information corresponds to a meta-universe practical scene, the host sends current practical data corresponding to the current practical type selection information to the VR head-mounted device; the host generates the current practical data based on a fault type in the meta-universe practical scene to generate a strategy or a power fault simulation production strategy; The VR head-mounted device receives the current practical data and displays current virtual reality display data corresponding to the current practical data; The force feedback haptic glove acquires current practical operation data of the user for the current virtual reality display data and sends the current practical operation data to the host in real time for saving until a practical end instruction is detected; wherein a glove model corresponding to the force feedback haptic glove is displayed in the virtual reality scene displayed by the VR head-mounted device, and the user hand operation data collected by the force feedback haptic glove is used to be synchronized to the virtual reality scene displayed by the VR head-mounted device to perform corresponding operations on the current virtual reality display data, and the force feedback haptic glove receives touch data determined according to a force feedback intensity determination strategy; the touch data is force feedback intensity data or a force feedback intensity data set corresponding to the force feedback intensity data generated and sent to the force feedback haptic glove when the VR head-mounted device performs corresponding operations on the current virtual reality display data based on the user hand operation data; If the host detects the end-of-training instruction, the saved current training operation data corresponding to the current training operation data and the current training operation data are obtained, and a current training data set corresponding to the current training data is formed.

[0007] In a second aspect, the embodiments of the present application also provide a teaching training implementation system based on meta-universe force feedback technology, which comprises: a host, a display, a user input device, a VR head-mounted device and a force feedback haptic glove; the display, the user input device and the VR head-mounted device are in communication connection with the host, the VR head-mounted device is in communication connection with the force feedback haptic glove, and the host is further connected with a cloud server through a network; The host is configured to, in response to a user login instruction, obtain current training user information and current training type selection information corresponding to the user login instruction. The host is further configured to, if it is determined that the current training type selection information corresponds to a meta-universe training scene, send current training data corresponding to the current training type selection information to the VR head-mounted device; and the host generates the current training data based on a fault type in the meta-universe training scene to generate a strategy or a power failure simulation production strategy. The VR head-mounted device is configured to receive the current training data and display current virtual reality display data corresponding to the current training data. The force feedback haptic glove is configured to obtain current training operation data of a user with respect to the current virtual reality display data, and send the current training operation data to the host in real time for saving until an end-of-training instruction is detected; wherein a glove model corresponding to the force feedback haptic glove is displayed in a virtual reality scene displayed by the VR head-mounted device, and user hand operation data collected by the force feedback haptic glove is used to be synchronized to the virtual reality scene displayed by the VR head-mounted device to perform corresponding operations on the current virtual reality display data, and touch data determined by the host according to a force feedback intensity determination strategy is received; the touch data is force feedback intensity data or a force feedback intensity data set corresponding to the force feedback intensity data generated by the VR head-mounted device based on the user hand operation data when performing corresponding operations on the current virtual reality display data and sent to the force feedback haptic glove. The host is further configured to, if an end-of-training instruction is detected, obtain saved current training operation data corresponding to the current training operation data and the current training operation data, and form a current training data set corresponding to the current training data.

[0008] In a third aspect, the embodiments of the present application further provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0009] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program, wherein the computer program comprises program instructions, and the program instructions can implement the method of the first aspect when executed by a processor.

[0010] The embodiments of the present application provide a teaching training implementation method and system based on meta-universe force feedback technology, which comprises the following steps: a host computer responds to a user login instruction to obtain current training user information and current training type selection information corresponding to the user login instruction; if the host computer determines that the current training type selection information corresponds to a meta-universe training scene, the host computer sends current training data corresponding to the current training type selection information to a VR head-mounted device; the VR head-mounted device receives the current training data and displays current virtual reality display data corresponding to the current training data; a force feedback haptic glove obtains current training operation data of a user for the current virtual reality display data and sends the current training operation data to the host computer in real time for storage until a training end instruction is detected; wherein a glove model corresponding to the force feedback haptic glove is displayed in a virtual reality scene displayed by the VR head-mounted device, and user hand operation data collected by the force feedback haptic glove is used to synchronize to the virtual reality scene displayed by the VR head-mounted device to perform corresponding operations on the current virtual reality display data, and tactile data determined by the host computer according to a force feedback intensity determination strategy is received; if the host computer detects the training end instruction, the host computer obtains saved current training operation data and current training operation data corresponding to the current training operation data and the current virtual reality display data, and forms a current training data set corresponding to the current training data. The embodiments of the present application combine the host computer, the VR head-mounted device and the force feedback haptic glove to realize that the current training operation data of the user in the virtual reality scene displayed by the VR head-mounted device for the current virtual reality display data can be collected in time, and tactile data determined by the host computer according to the force feedback intensity determination strategy is received during the operation process, which greatly enhances the simulation effect of the training. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0012] Figure 1 An application scenario schematic diagram of the teaching practical training implementation method based on the meta-universe force feedback technology is provided for an embodiment of the present application. Figure 2 A flowchart schematic diagram of the teaching practical training implementation method based on the meta-universe force feedback technology is provided for an embodiment of the present application. Figure 3 A sub-flowchart schematic diagram of the teaching practical training implementation method based on the meta-universe force feedback technology is provided for an embodiment of the present application. Figure 4 A schematic block diagram of the teaching practical training implementation device based on the meta-universe force feedback technology is provided for an embodiment of the present application. Figure 5 A schematic block diagram of the computer device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0013] The technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0014] It should be understood that when used in the present specification and the appended claims, the terms “comprise” and “include” indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0015] It should also be understood that the terms used in the present specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present specification and the appended claims, unless otherwise clearly indicated by the context, the singular forms “a”, “an” and “the” are intended to include the plural forms.

[0016] It should be further understood that the term “and / or” used in the present specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0017] Please refer to Figure 1 and Figure 2 , wherein Figure 1 An application scenario schematic diagram of the teaching practical training implementation method based on the meta-universe force feedback technology is provided for an embodiment of the present application, Figure 2 A flowchart schematic diagram of the teaching practical training implementation method based on the meta-universe force feedback technology is provided for an embodiment of the present application. As Figure 1As shown, the teaching practice implementation method based on the metaverse force feedback technology provided by the embodiment of the application is applied to a teaching practice implementation system based on the metaverse force feedback technology. The teaching practice implementation system based on the metaverse force feedback technology comprises a host computer 1, a display 2, a user input device 3, a VR head-mounted device 4, and a force feedback haptic glove 5. The display 2, the user input device 3, and the VR head-mounted device 4 are all in communication connection with the host computer 1. The VR head-mounted device 4 is in communication connection with the force feedback haptic glove 5. The host computer 1 is also connected to a cloud server through a network.

[0018] Wherein, still referring to Figure 1 , the host computer 1 is also connected to a router 6 through a wired connection or a wireless connection, so that the router 6 serves as a medium for the connection between the host computer 1 and the cloud server. Moreover, the VR head-mounted device 4 can be connected to the router 6 so as to be located in the same local area network as the host computer 1. More specifically, if the host computer 1 is placed on a host computer rack in a practice room, the display 2, the user input device 3, the VR head-mounted device 4, and the force feedback haptic glove 5 can all be placed on the host computer rack when not in use. When practice is needed, the personnel participating in the practice can take off the VR head-mounted device 4 and the force feedback haptic glove 5 and then wear them to start the practice operation.

[0019] As shown in Figure 2 , the method comprises the following steps S110-S150.

[0020] S110, the host computer acquires current practice user information and current practice type selection information corresponding to the user login instruction in response to the user login instruction.

[0021] In this embodiment, a metaverse comprehensive practice platform is deployed on the host computer. After a user opens the metaverse comprehensive practice platform by operating the user input device, a user interaction interface corresponding to the metaverse comprehensive practice platform is displayed on the display. At this time, the user can enter user login information (such as including a user account and a user password) in the user interaction interface corresponding to the metaverse comprehensive practice platform, so as to trigger a user login instruction. After the user login information passes the user identity verification of the metaverse comprehensive practice platform, current practice user information corresponding to the user login instruction is acquired. At this time, the user interaction interface corresponding to the metaverse comprehensive practice platform is no longer a user login interface, but is switched to a main operation interface of the metaverse comprehensive practice platform. At this time, the user can select one of four virtual buttons of “speed competition evaluation”, “practice module”, “data observation”, and “active user statistics” provided on the main operation interface and click the virtual button, so as to start a corresponding practice mode.

[0022] For example, when the user operates the user input device to select the "training module" virtual button on the main operation interface, the corresponding current training type selection information is the meta universe training scene; when the user operates the user input device to select the "race evaluation" virtual button on the main operation interface, the corresponding current training type selection information is the meta universe training viewing scene; when the user operates the user input device to select the "data observation" virtual button on the main operation interface, the corresponding current training type selection information is the data observation scene; when the user operates the user input device to select the "active user statistics" virtual button on the main operation interface, the corresponding current training type selection information is the data analysis scene. It can be seen that the above meta universe comprehensive training platform integrates the four core functions of "race evaluation", "training module", "data observation" and "active user statistics". Moreover, the meta universe comprehensive training platform can also support multiple users to participate in training, that is, multiple users respectively operate the corresponding teaching and training implementation system based on the meta universe force feedback technology and log in to the meta universe comprehensive training platform.

[0023] In step S120, the host sends the current training data corresponding to the current training type selection information to the VR head-mounted device if it is determined that the current training type selection information corresponds to the meta universe training scene.

[0024] In the embodiment, when the user operates the user input device to select the "training module" virtual button on the main operation interface, the corresponding current training type selection information is the meta universe training scene. At this time, the host first acquires one of the training data corresponding to the meta universe training scene and sends it as the current training data to the VR head-mounted device, taking the VR head-mounted device as the display medium of the meta universe training scene. Since the meta universe training scene corresponds to multiple sub-scenes, the host can select a sub-scene such as the meta universe power comprehensive training scene based on the user's training requirements, and the meta universe power comprehensive training scene can further include secondary sub-scenes such as the meta universe power fault training scene and the meta universe power equipment assembly training scene. When a sub-scene such as the meta universe power fault training scene is selected from the secondary sub-scenes, the corresponding current training data can be determined. These current training data can be regarded as the initial virtual reality scene corresponding to the secondary sub-scene selected based on the user's training requirements, and there are a plurality of virtual reality model objects such as electric meters, high-voltage cables, distribution transformers, middle-position cabinets, vacuum ring network cabinets, and SF6 cabinets (a kind of ring network cabinet) in the initial virtual reality scene.

[0025] In an embodiment, step S120 comprises: If it is determined that the current training type selection information is a meta-universe power comprehensive training scene in the meta-universe training scene, the current training data corresponding to the meta-universe power comprehensive training scene is obtained, and the current training data is sent to the VR head-mounted device.

[0026] In the embodiment, a plurality of first-level sub-scenes are included in the meta-universe training scene, for example, the meta-universe power comprehensive training scene is one of the sub-scenes, and the meta-universe power fault training scene and the meta-universe power equipment assembly training scene are second-level sub-scenes included in the meta-universe power comprehensive training scene. When the user selects the meta-universe power comprehensive training scene as the first-level sub-scene, and does not specifically select the second-level sub-scene, the meta-universe comprehensive training platform in the host computer determines a corresponding default training scene (such as a power meter assembly) based on the meta-universe power comprehensive training scene, obtains the current training data corresponding to the default training scene, and sends the current training data to the VR head-mounted device. It can be seen that by the above method, the specific training scene corresponding to the current training type selection information can be quickly determined in the host computer, and the corresponding current training data can be sent to the VR head-mounted device and displayed in the virtual reality scene.

[0027] In an embodiment, the current training data corresponding to the meta-universe power comprehensive training scene is obtained by: If it is determined that the meta-universe power comprehensive training scene is a meta-universe power fault training scene, the current training data corresponding to the meta-universe power fault training scene is generated based on a preset power fault simulation production strategy; If it is determined that the meta-universe power comprehensive training scene is a meta-universe power equipment assembly training scene, the current training data corresponding to the meta-universe power equipment assembly training scene is generated based on the obtained power equipment assembly project information.

[0028] In this embodiment, the meta-universe power comprehensive training scene is one of the first-level sub-scenes in the meta-universe training scene, which includes the meta-universe power failure training scene and the meta-universe power equipment assembly training scene. When the user selects the meta-universe power failure training scene, it indicates that the user needs to perform power failure maintenance training operation. At this time, the meta-universe comprehensive training platform generates the current training data corresponding to the meta-universe power failure training scene based on the power failure simulation production strategy. For example, the power failure simulation production strategy is used to determine at least one target power failure type from the power failure simulation library based on the personnel label of the training personnel (such as junior electrician, intermediate electrician, senior electrician, etc.) or based on the training heat value of each type of power failure in the power failure simulation library (such as topK of the training heat value, K=3 or a positive integer set based on actual demand), and then determine the current training data based on the target power failure type. For example, the virtual reality model objects included in the target power failure type are electric meter, high-voltage cable, and distribution transformer. The virtual reality model objects of the electric meter, high-voltage cable, and distribution transformer (each virtual reality model object has a corresponding initial three-dimensional space position and initial object state in the initial virtual reality scene) are used to generate the current training data displayed in the initial virtual reality scene.

[0029] When the user selects the meta-universe power equipment assembly training scene, it indicates that the user needs to perform power equipment assembly training operation. At this time, the user can further select one power equipment assembly project information from the power equipment assembly project list provided in the user interaction interface (the power equipment assembly project information can include specific power equipment such as electric meter, high-voltage cable, distribution transformer, intermediate cabinet, and vacuum ring network cabinet). The virtual reality model objects corresponding to the specific power equipment included in the power equipment assembly project information are placed at the initial space position specified in the initial virtual reality scene. The training personnel needs to assemble the power equipment based on the power equipment assembly operation. After obtaining the power equipment assembly project information, the virtual reality model objects of the specific power equipment included in the power equipment assembly project information are used to generate the current training data displayed in the initial virtual reality scene.

[0030] Of course, in the specific implementation, in addition to the meta-universe power failure training scene and the meta-universe power equipment assembly training scene, the meta-universe training scene can also include the meta-universe power failure evacuation training scene and the meta-universe power equipment emergency handling operation training scene. The corresponding training scene can be set according to the actual demand of the user.

[0031] Wherein, the personnel label construction and ability portrait are preformed in the host, that is, the meta-universe comprehensive training platform generates skill level label, previous training completion, skill level score, operation accuracy, core data, typical error record, generates exclusive skill level label (such as primary, intermediate and advanced), weak operation link and weak link label (such as high-voltage wiring, fault troubleshooting, etc.), training frequency and other multi-dimensional portraits (personnel label) for each participant. After the student user logs in the meta-universe comprehensive training platform, the personnel label can also be used to preferentially screen the scene difficulty coefficient matching the skill level of the personnel, and select the TopK fault types with high heat value in the fault library combined with the power fault simulation production strategy as the candidate set, taking into account the teaching value and popular training demand. Moreover, the candidate scene can also be introduced into the weight adjustment mechanism based on the historical weak link: if there are some high-risk or weak operation links in the personnel portrait, the system will increase the probability of generating fault scenes containing related links, so as to realize the personalized training guided by the teaching goal. In the fault library, the heat value of each type of fault / training scene can be calculated (the heat value is determined by weighting the sum of the frequency of being triggered, the degree of teaching attention, the importance of the accident, the accident statistics in recent years, etc.), and each type of fault / training scene can be dimensionally labeled (difficulty, danger level, required skill, etc.). In the fault library, the training heat value of each type of fault is calculated in real time (based on the dimensions of the platform training frequency, key training demand, etc.), the TopK high-frequency fault types are screened, and the fault scenes with corresponding difficulty coefficients are matched according to the skill level label of the training personnel (such as focusing on basic line fault for primary students and focusing on complex interlocking fault for advanced students); the generation probability of corresponding fault types can also be adjusted based on the weight proportion of the historical weak operation link of the student user (such as a student with a high-voltage operation failure rate, the corresponding fault scene generation probability is increased by 30%), so as to realize targeted intensive training. At the same time, the training list for the student user can be generated according to the teaching goal (class / course demand) and team cooperation training goal, and the personnel label and scene heat value can be updated after training through the score / error playback to form a closed-loop iterative optimization.

[0032] In an embodiment, step S120 comprises: If it is determined that the current training type selection information corresponds to a meta-universe training scene, the current student information logged in is obtained; wherein the current student information at least includes student unique user number, student training skill level and student training weak link information; The training heat value of each type of fault included in the preset fault library, the matching degree with the skill level of the current student, the weight of the corresponding weak link of the fault and the total number of matchable fault types are obtained, and the generation probability of each fault scene matched with the current user is determined based on the preset fault type generation strategy; determine a fault scenario with the maximum generation probability from the generation probability of each fault scenario matched with the current user, and generate the current training data correspondingly; send the current training data to the VR head-mounted device.

[0033] In this embodiment, as another embodiment of step S120, after the training trainee inputs the login account and login password and successfully logs in the meta-universe comprehensive training platform, the current trainee information including at least the trainee unique user number, the trainee training skill level and the trainee weak link information can also be obtained. Then, combined with the training heat value of each fault included in the preset fault library in the host, the matching degree with the skill level of the current trainee, the weight of the weak link corresponding to the fault and the total number of the matchable fault types, the following formula corresponding to the fault type generation strategy is input as a parameter: wherein, P i represents the generation probability of the i-th fault scenario matched with the current user, H i represents the training heat value of the i-th fault scenario in the fault library (wherein, the training heat value of each fault scenario in the fault library is normalized and the value range is 0 to 1), M i，d represents the matching degree of the i-th fault scenario with the skill level of the current trainee (specifically, if matched, M i，d =1, if not matched, M i，d =0.2), W i represents the weight of the weak link corresponding to the i-th fault scenario, n represents the total number of the matchable fault types in the fault library, and the value range of i is 1 to n.

[0034] H i associated with all the data of the fault library, the high-frequency fault type (such as transformer insulation fault, line short circuit fault) can be screened out, the training necessity of the fault is quantified by the heat value, and it is ensured that the trainee trains the high-frequency fault in the industry first, and the emergency handling ability is improved. M i，d Then, the skill level label (junior / middle / senior) is generated according to the historical completion degree of the trainee, the matching degree coefficient ensures that the junior trainee does not touch the super difficult fault and the senior trainee does not repeat the basic training, and at the same time, 0.2 of the low matching degree (such as the junior trainee touching the simplified middle-level fault with low probability) is reserved, and the training safety and expansibility are considered. W iThe system records the history operation failure rate of the trainee (such as high-voltage wiring failure rate 30%, fault troubleshooting failure rate 10%) in the fault library, and through the weight quantification of the weak link, the generation probability of high-voltage wiring related faults is significantly improved; the summation term corresponding to the denominator ensures the probability normalization, so that the generation probability of all selectable fault scenarios is 1, avoiding the logical error of probability exceeding 100%, and ensuring the scientificity of scenario generation.

[0035] Through the above implementation manner, it is different from the fixed scene selection mode, and individualized training scenes suitable for the skill level of different trainees, focusing on weak links and covering high-frequency needs are generated through data-driven probability calculation, avoiding the inefficient training of one-size-fits-all. In the traditional training platform, trainees often face the problem that the training content does not match their own ability (such as forcing junior trainees to practice complex faults, leading to frustration, and wasting time for senior trainees to repeatedly practice basic faults), and the training focus is out of line with industry needs (such as ignoring high-frequency faults and missing personal weak links). In this application, the probability is calculated through multi-dimensional data fusion, so that the training scene of each trainee is tailor-made, covering core training needs and accurately supplementing individual shortcomings, greatly improving training efficiency. Ultimately, through quantitative calculation of the formula, the Meta Universe Comprehensive Training Platform converts vague training needs (such as precise touch, individual training, and long-term stability) into executable, monitorable, and optimized technical logic, becoming the core mathematical support of the Meta Universe Comprehensive Training Platform. At the same time, it also avoids retaining skill expansion possibilities, balances heat value and difficulty matching, avoids resource waste, and provides data support for dynamic skill upgrading.

[0036] S130, the VR head-mounted device receives the current training data and displays current virtual reality display data corresponding to the current training data.

[0037] In this embodiment, when the VR head-mounted device receives the current training data sent by the host based on the local area network, it first acquires virtual reality model objects including each power equipment in the current training data, then groups each virtual reality model object into current virtual reality display data, and displays it in the virtual reality scene displayed by the VR head-mounted device.

[0038] S140, the force feedback haptic glove acquires current training operation data of the user for the current virtual reality display data, and sends the current training operation data to the host in real time for saving until a training end instruction is detected.

[0039] The glove model corresponding to the force feedback haptic glove is displayed in the virtual reality scene displayed by the VR head-mounted device, and the user hand operation data collected by the force feedback haptic glove is used to synchronize to the virtual reality scene displayed by the VR head-mounted device to perform corresponding operations on the current virtual reality display data, and the tactile data determined by the host according to the force feedback intensity determination strategy is received; the tactile data is the force feedback intensity data or force feedback intensity data set corresponding to the force feedback intensity data generated and sent to the force feedback haptic glove when the VR head-mounted device performs corresponding operations on the current virtual reality display data based on the user hand operation data.

[0040] In this embodiment, in order to simulate the user participating in the training to perform corresponding operations on each virtual reality model object in the virtual reality scene displayed by the VR head-mounted device through the force feedback haptic glove, first, the glove model corresponding to the force feedback haptic glove needs to be displayed in the virtual reality scene displayed by the VR head-mounted device, more specifically, fused in the virtual reality scene corresponding to the current virtual reality display data, and the operation made by the user's hand driving the force feedback haptic glove is synchronized to the virtual reality scene in real time to perform corresponding operations on the current virtual reality display data, and the tactile data determined by the host according to the force feedback intensity determination strategy is received.

[0041] In order to realize the synchronization of the user hand operation data to the virtual reality scene by the force feedback haptic glove to perform corresponding operations on the current virtual reality display data, and receive the tactile data determined by the host according to the force feedback intensity determination strategy, the force feedback haptic glove can specifically adopt VR glove SenseGlove (which integrates active contact feedback, force feedback and vibration haptic feedback functions; active contact feedback is used to feel the palm impact and grip feeling; force feedback is used to feel the size and hardness of virtual objects; vibration haptic feedback is used to feel signals and basic textures), and it has gesture recognition, immersive tactile experience and other use effects. The user hand operation data collected by the force feedback haptic glove can be synchronized to the glove model in the virtual reality scene displayed by the VR head-mounted device in real time, and the operation made by the user wearing the force feedback haptic glove can be mapped to the glove model and perform corresponding operations on the current virtual reality display data through the glove model.

[0042] If the data transmission link in step S140 (i.e. from the force feedback haptic glove to the host) is regarded as an uplink data transmission link, its specific data collection, processing and transmission process is as follows: A1) Force feedback haptic gloves (at least equipped with posture sensors, pressure sensors, motion trajectory sensors, etc.) collect multi-dimensional operation data of user's hands in real time and upload them to the host computer synchronously, providing accurate input for physical simulation. Specifically, the force feedback haptic gloves obtain multi-dimensional sensor data (such as joint angle, speed, contact position, local force / torque estimation, touch event data, etc.) in the current practical operation data of the user for the current virtual reality display data, and use a low-latency, reliable transmission protocol (such as a transmission protocol supporting timestamp and serialization) to ensure the real-time consistency of physical computing on the host side.

[0043] A2) After the host (which is considered as a physical computing layer) receives the current practical operation data received by the force feedback haptic gloves, the first aspect needs to obtain the material physical model of the equipment in the virtual reality scene (such as metal conductivity, cable flexibility, switch mechanical structure parameters, etc.), combine the hand operation data in the current practical operation data, and perform real-time physical simulation calculation to generate corresponding tactile feedback logic; the second aspect needs to obtain the material and geometric physical model of the current object being operated in the virtual reality scene (elasticity / viscosity / friction / damping, etc.), combine the hand operation data in the current practical operation data, and perform real-time physical simulation calculation to generate tactile description parameters (such as a tactile parameter matrix), and update the rendering state of the virtual reality scene synchronously.

[0044] More specifically, in the meta-universe electric power comprehensive practical training scene, the VR glove SenseGlove is mainly used for the following advantages: B1) High-precision force feedback mechanism, such as simulating multiple physical force sensations in electrical equipment operation (switch resistance, cable tightness, component weight gradient, etc.), achieving accurate restoration from macro operation to micro tactile sensation; supporting multi-dimensional force / torque feedback, accurately simulating object weight, contact impedance, friction and texture differences; in electric power training, it can realistically reproduce operation mechanics characteristics such as switch feedback force, cable stress, fastener torque, etc., improving the training transfer effect of operation actions.

[0045] B2) Full-dimensional gesture recognition, that is, capturing subtle data such as hand posture, motion trajectory, and force intensity through built-in multi-dimensional sensors to ensure that the action logic of virtual operation is highly consistent with actual electric power operation; integrating high-resolution position / angle and tactile sensors, supporting high-precision capture of fine finger joint angles, grip posture, and palm position; can accurately map complex hand movements to virtual electric power equipment operation (such as tool changing, wiring, power-off inspection, etc.), supporting simulation of complex processes such as fault diagnosis.

[0046] B3) Immersive tactile reproduction, which combines virtual reality environment construction to allow trainees to obtain tactile feedback consistent with real work scenarios in virtual scenes, enhancing operational memory and skill transfer ability; and it also works with VR rendering engine to provide a spatiotemporally consistent visual and tactile closed loop, allowing trainees to obtain near-realistic tactile feedback, enhancing learning motivation and memory retention.

[0047] In one embodiment, such as Figure 3 As shown, step S140 includes: S141. The force feedback haptic glove acquires the user's current operation model object in the current virtual reality display data, and generates haptic data corresponding to the current operation model object for simulating real haptic sensation. S142. The force feedback tactile glove acquires the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, and sends it to the host for storage.

[0048] In this embodiment, when a user wears a force feedback haptic glove for practical training, and the user's hand drives the glove to perform an action (such as moving the hand to touch a faulty cable), the hand model in the virtual reality scene will also move to the corresponding position of the current operation model object in the current virtual reality display data and perform the same action. Since the current operation model object (i.e., its specific physical type) is known, tactile data corresponding to the current operation model object can be obtained from the background model object database of the Metaverse Integrated Training Platform to simulate the user's real tactile sensation towards the current operation model object. For example, if the current operation model object is a faulty cable, when the user operates the force feedback haptic glove to drive the glove model to move to the cable, the tactile data generated by the VR headset can be used to feel the cable's tension, etc., allowing the user to experience the realistic force sensation when operating electrical equipment, thereby greatly enhancing the simulation effect of the training.

[0049] Subsequently, the force feedback haptic glove acquires the user's current training operation data on the current operation model object within the simulated haptic environment corresponding to the haptic data. This current training operation data represents the user's specific operation on the current operation model object within the current virtual reality display data. The force feedback haptic glove comprehensively captures and recognizes the user's hand movements and subtle gestures, ensuring that the user's operations in the virtual reality environment are accurately mapped to actual power operations (i.e., the user's operations in the virtual reality environment also provide the same operational experience as in actual power operations). Whether it's a simple switch operation or complex troubleshooting, accurate simulation is achieved. The obtained current training operation data is sent to the host computer in real time for storage. The above process saves the current training operation data of a user performing an operation while wearing the force feedback haptic glove. Subsequent operations are processed using the same process until a training end command is detected.

[0050] In one embodiment, step S142 includes: The force feedback haptic glove acquires the user's current hand operation data on the current operation model object in the simulated haptic environment corresponding to the haptic data, and synchronously maps the current user's hand operation data to the glove model in the virtual reality scene of the VR headset, synchronously driving the glove model to perform corresponding operations on the current operation model object and generate the current training operation data.

[0051] In this embodiment, when the force feedback haptic glove acquires the user's current training operation data on the current operation model object in the simulated haptic environment corresponding to the haptic data, specifically, the physical device of the force feedback haptic glove first captures and recognizes the user's hand movements and subtle gestures, and maps these hand movements and subtle gestures onto the glove model in the virtual reality scene of the VR headset. The glove model serves as a digital twin model of the force feedback haptic glove to perform corresponding operations on the current operation model object and generate the current training operation data. This method provides the user with an immersive haptic experience, simulating the user being in a real operating environment, allowing them to intuitively receive physical feedback from each operation.

[0052] In one embodiment, after step S142 and before step S150, the method further includes: The host obtains the student's hand operation force, the equipment material coefficient of the current operation model object, the scene difficulty coefficient and feedback calibration compensation factor corresponding to the current training operation data from the current training operation data, determines the strategy and the force feedback intensity dataset corresponding to the current training operation data according to the preset force feedback intensity, and sends it to the force feedback tactile glove.

[0053] In this embodiment, when the host obtains the current training operation data, if it is considered as a time sequence and includes multiple current training operation sub-data sequences, when combining the force feedback intensity determination strategy with the force feedback intensity data corresponding to the current training operation sub-data sequence for a current training operation sub-data sequence, it is first necessary to obtain the student's hand operation force, the equipment material coefficient of the current operation model object, the scene difficulty coefficient and the feedback calibration compensation factor corresponding to the current training data from the current training operation sub-data sequence. Then, the above parameters are input into the calculation formula F = Fop × Km × Kd × (1+ΔK) corresponding to the force feedback intensity determination strategy to calculate the force feedback intensity data corresponding to the current training operation sub-data sequence.

[0054] In the calculation formula F = Fop × Km × Kd × (1+ΔK), the specific parameter functions are as follows: F represents the force feedback intensity data (its unit is N); Fop represents the force exerted by the student's hand, which can be collected by a multi-dimensional pressure sensor in the force feedback tactile glove (ensuring the authenticity of the input data and providing a user operation benchmark for subsequent calculations), and is measured in N. Km represents the material coefficient of the current operation model object (for example, in the metaverse power comprehensive training scenario, based on the different power equipment materials of the current operation model object, such as switch metal material Km=0.8, cable rubber material Km=0.3, etc.); Km can be matched to the device material model in the physical calculation layer of the host, in which the material parameters of different power equipment are preset (such as metal switches have high hardness, cable rubber has strong flexibility). By quantifying the material differences through coefficients, students can feel greater resistance when operating metal switches and feel a soft touch when operating cables, thus restoring the physical characteristics of real equipment; Kd represents the scene difficulty coefficient corresponding to the current training data (e.g., in the metaverse power comprehensive training scene, if the basic scene difficulty is selected, Kd=0.6; if the intermediate scene difficulty is selected, Kd=0.9; and if the advanced scene difficulty is selected, Kd=1.2). Kd can be linked with the intelligent scene generation module in the host to set the scene difficulty coefficient of the basic scene difficulty to 0.6, reducing the force feedback intensity to lower the operation threshold; and to set the scene difficulty coefficient of the advanced scene difficulty to 1.2, increasing the force feedback complexity (e.g., simulating the jamming resistance of old equipment), so as to achieve simultaneous upgrade of difficulty and tactile sensation. ΔK represents the feedback calibration compensation factor (derived from the execution error calculated by the feedback calibration stream, ΔK∈[-0.1,0.1], positive execution error calculated by the feedback calibration stream results in negative compensation, and negative execution error results in positive compensation); ΔK can be linked with the feedback calibration stream module in the host. The force feedback haptic glove reports the actual execution error (e.g., theoretically calculated force feedback is 10N, but only 8N is actually output). The calculation parameters are dynamically adjusted through this feedback calibration compensation factor to compensate for hardware execution deviations and ensure the long-term accuracy of haptic feedback.

[0055] The force feedback intensity determination strategy for a current training operation sub-data sequence is combined with the force feedback intensity data corresponding to the current training operation sub-data sequence in the manner described above. The force feedback intensity data for other current training operation sub-data sequences are also determined in the same way. When the force feedback intensity data is determined according to the temporal order of the current training operation sub-data sequence, it is also arranged in temporal order to form a force feedback intensity dataset.

[0056] As can be seen, the above methods provide users with an immersive tactile experience during operation. The force feedback tactile gloves, combined with the force feedback intensity data calculated by the host in real time, simulate the user being in a real operating environment, allowing them to intuitively receive the physical feedback from each operation. This solves the problem of the disconnect between virtual operation tactile sensation and real power operation. Through multi-factor quantitative calculation, the force feedback output by the force feedback tactile gloves is highly consistent with the actual operating scenario, avoiding skill transfer failure caused by the inconsistency between the force applied in virtual operation and the real equipment.

[0057] In this process, after the host determines the force feedback intensity dataset based on the current training operation data uploaded by the force feedback haptic glove and the force feedback intensity determination strategy, the force feedback intensity dataset is sent from the host to the force feedback haptic glove as a downlink data transmission link. In practice, after the host obtains the current training operation data and determines the corresponding force feedback intensity data based on each current training operation sub-data sequence and the force feedback intensity determination strategy, it can also simultaneously determine the core parameters such as tactile frequency and resistance gradient corresponding to the force feedback intensity data. That is, each force feedback intensity data can also form a tactile parameter matrix with the corresponding tactile frequency, resistance gradient, and other core parameters, and send it to the force feedback haptic glove. The force feedback haptic glove then executes accordingly based on the force feedback intensity data or the tactile parameter matrix. It is important to note that the downlink data transmission link and the visual rendering data transmission channel corresponding to step S120 (i.e., the data transmission channel through which the host sends the current training data corresponding to the current training type selection information to the VR headset) are parallel but independent, ensuring that the timing of the downlink data transmission link is synchronized with the timing of the virtual rendering events.

[0058] ΔK is linked to the feedback calibration flow module in the host. The force feedback haptic glove reports the actual execution error, and the calculation parameters are dynamically adjusted through this feedback calibration compensation factor, providing a basis for the dynamic calibration of the hardware execution deviation of the force feedback haptic glove. Specifically, refer to the formula ΔF=∣Fcal-Fact∣ / Fcal and another formula ΔK=−α×ΔF; where ΔF represents the relative error of force feedback execution (dimensionless), Fcal represents the force feedback intensity data theoretically calculated by the host (in N), Fact represents the actual force feedback intensity executed by the force feedback haptic glove (collected by the actuator sensor in the force feedback haptic glove, and in N), and α represents the calibration coefficient (which is an empirical value, such as 0.8, which can be dynamically adjusted according to the training scenario corresponding to the current training data).

[0059] ΔK works in conjunction with the feedback calibration stream module in the host machine. Firstly, it addresses the issue of accumulated hardware execution deviations in force feedback haptic gloves over time. Through error calculation and compensation factor adjustment, it ensures the force feedback system maintains long-term accuracy, preventing initial tactile accuracy followed by later feedback distortion that could negatively impact training effectiveness. Secondly, Fcal is the output from the host's physical computing layer. The host determines the force feedback intensity dataset based on the current training operation data uploaded upstream and the force feedback intensity determination strategy. Each force feedback intensity data point serves as a benchmark value for error comparison. Fact is the upstream data from the feedback calibration stream of the force feedback haptic glove, representing the real-time acquisition of actual output force feedback by the built-in sensors in the glove's actuator, serving as the actual value for error comparison. α, a calibration coefficient verified through extensive training data (e.g., 0.8), ensures that the compensation factor adjustment is neither excessive nor insufficient, preventing reverse deviations after error compensation and achieving stable calibration. Thirdly, during long-term use, force feedback tactile gloves may experience discrepancies between actual output and theoretical calculations due to mechanical wear and sensor drift (e.g., theoretically outputting 5N, but actually only outputting 4N). Without calibration, trainees will gradually adapt to the distorted tactile sensation, leading to errors in force judgment during actual operation. The aforementioned calibration operation calculates the error in real time and dynamically adjusts the compensation factor to ensure that the force feedback tactile gloves always maintain the same feedback accuracy as the theory and reality, ensuring that the tactile memory formed during training is accurate and reliable.

[0060] S150. If the host detects a training end command, it acquires the saved current training operation data and the current training operation data corresponding to the current training operation data and the current virtual reality display data, and forms a current training dataset corresponding to the current training data.

[0061] In this embodiment, if the host detects a training end command, it indicates that the user's current round of metaverse comprehensive training has ended. First, it acquires the saved current training operation data and the current virtual reality display data corresponding to the current training operation data and the current training operation data, and forms a current training dataset corresponding to the current training data. Then, it saves the current training dataset in the host's background database. The current training dataset is bound to the user corresponding to the current training user information; that is, this current training dataset is generated by the user corresponding to the current training user information performing metaverse training operations within the user's operation time interval. The user operation time interval can start from the time the user logs in and end at the time the training end command is generated.

[0062] In one embodiment, the method further includes the following after step S110: If the host determines that the current training type selection information corresponds to the metaverse training viewing scenario, it obtains the currently viewable training operation video stream corresponding to the metaverse training viewing scenario and sends the currently viewable training operation video stream to the VR headset.

[0063] In this embodiment, when a user selects the "Speed ​​Competition" virtual button on the main operation interface using the user input device, the corresponding current training type selection information is the Metaverse Training Viewing Scene. If the host is continuously receiving current training operation data from the current participating user, other users in the Metaverse Training Viewing Scene can obtain the current training operation data of the aforementioned participating user from the host and send it as the currently viewable training operation video stream to the VR headsets of other viewing users (e.g., teachers who comment on and operate the training user's operations). The current operation step information corresponding to the currently viewable training operation video stream is also displayed synchronously on the interface of the currently viewable training operation video stream, so that the VR headsets of other users can obtain the currently viewable training operation video stream in a timely manner, realizing real-time monitoring and transmission of training operation data.

[0064] Referring back to the example above, if the teacher wears both a VR headset and force feedback haptic gloves, the current training operation data of the participants can also be realistically simulated through the teacher's force feedback haptic gloves, enabling the teacher to quickly obtain the students' operation data and provide timely and accurate feedback.

[0065] In one embodiment, the method further includes the following after step S110: If the host determines that the current training type selection information corresponds to a data observation scenario, it acquires the relevant data of the project equipment corresponding to the data observation scenario and displays it through the display connected to the host.

[0066] In this embodiment, when a user selects the "Data Observation" virtual button on the main operation interface using the user input device, the corresponding current training type selection information is "Data Observation Scene." The user can obtain project-related data such as today's number of experiences, cumulative number of experiences, projects executed today, projects in progress, total number of devices, currently running devices, currently idle devices, currently unconnected devices, popular project rankings, and project categories, which are then displayed on the monitor connected to the host. Alternatively, the data can be sent to a VR headset for display.

[0067] When a user selects the "Active User Statistics" virtual button on the main operation interface using the user input device, the corresponding current training type is selected as a data analysis scenario. The user can view the active user statistics data corresponding to the data analysis scenario on the monitor or VR headset, allowing trainees to view training records and related training progress, and perform data analysis.

[0068] As can be seen, the embodiment of the method, which combines the host, VR headset, and force feedback haptic gloves, enables the timely collection of the user's current training operation data in the virtual reality scene displayed on the VR headset. Furthermore, during the operation, the host receives haptic data determined by the force feedback intensity determination strategy, which greatly enhances the simulation effect of the training.

[0069] Figure 4 This is a schematic block diagram of a teaching and training system based on metacosmic force feedback technology provided in an embodiment of the present invention. Figure 4 As shown, corresponding to the above-mentioned teaching and training implementation method based on metacosmic force feedback technology, this invention also provides a teaching and training implementation system 100 based on metacosmic force feedback technology. Figure 1 and Figure 4 As shown, the teaching and training system 100 based on metaverse force feedback technology includes a host 1, a display 2, a user input device 3, a VR headset 4, and a force feedback tactile glove 5. The display 2, the user input device 3, and the VR headset 4 are all communicatively connected to the host 1. The VR headset 4 is communicatively connected to the force feedback tactile glove 5. The host 1 is also connected to a cloud server via a network.

[0070] Among them, still refer to Figure 1The host 1 is also connected to a router 6 via a wired or wireless connection, with the router 6 serving as the medium for the connection between the host 1 and the cloud server. Furthermore, the VR headset 4 can be connected to the router 6 to be on the same local area network as the host 1. More specifically, if the host 1 is placed on a rack in the training room, the monitor 2, user input device 3, VR headset 4, and force feedback haptic gloves 5 can all be placed on the rack when not in use. When training is required, the participants can remove the VR headset 4 and force feedback haptic gloves 5 and put them on to begin the training. The user input device 3 specifically uses a keyboard and mouse.

[0071] The host 1 is used to respond to a user login command by obtaining the current training user information and the current training type selection information corresponding to the user login command.

[0072] In this embodiment, a metaverse integrated training platform is deployed on the host computer. After the user opens the metaverse integrated training platform using a user input device, the corresponding user interface of the metaverse integrated training platform is displayed on the monitor. At this time, the user can enter user login information (such as user account and user password) on the user interface of the metaverse integrated training platform, thereby triggering a user login command. After the user login information passes the user authentication of the metaverse integrated training platform, the current training user information corresponding to the user login command is obtained. At this time, the user interface of the metaverse integrated training platform is no longer the user login interface, but switches to the main operation interface of the metaverse integrated training platform. At this time, the user can select one of the four virtual buttons provided on the main operation interface, namely "Speed ​​Competition", "Training Module", "Data Observation" and "Active User Statistics", to start the corresponding training mode.

[0073] For example, when a user selects the virtual button "Training Module" on the main interface using their input device, the corresponding training type is the Metaverse training scenario; when they select the virtual button "Speed ​​Competition," the training type is the Metaverse training viewing scenario; when they select the virtual button "Data Observation," the training type is the data observation scenario; and when they select the virtual button "Active User Statistics," the training type is the data analysis scenario. Thus, the Metaverse integrated training platform integrates four core functions: "Speed ​​Competition," "Training Module," "Data Observation," and "Active User Statistics." Furthermore, the Metaverse integrated training platform supports multi-user participation, meaning multiple users can operate their respective teaching and training systems based on Metaverse force feedback technology and log into the Metaverse integrated training platform.

[0074] The host 1 is further configured to send the current training data corresponding to the current training type selection information to the VR headset if it is determined that the current training type selection information corresponds to the metaverse training scenario.

[0075] In this embodiment, when the user selects the virtual button "Training Module" on the main operation interface using the user input device, the corresponding current training type selection information is the Metaverse Training Scene. At this time, the host first obtains one of the training data corresponding to the Metaverse Training Scene and sends it to the VR headset as the current training data, using the VR headset as the display medium for the Metaverse Training Scene. Since the metaverse training scenario includes multiple sub-scenarios, the host can select a sub-scenarios, such as the metaverse power comprehensive training scenario, based on the user's training needs. This first-level sub-scenarios can further include second-level sub-scenarios such as the metaverse power fault training scenario and the metaverse power equipment assembly training scenario. When any sub-scenarios, such as the metaverse power fault training scenario, is selected from the second-level sub-scenarios, the corresponding current training data can be determined. This current training data can be regarded as the initial virtual reality scenario corresponding to the second-level sub-scenarios selected based on the user's training needs. In this initial virtual reality scenario, there are several virtual reality model objects corresponding to the selected sub-scenarios, such as electricity meters, high-voltage cables, distribution transformers, medium-voltage switchgear, vacuum ring network cabinets, and SF6 cabinets (which are a type of ring network cabinet).

[0076] In one embodiment, the host 1 is specifically used for: If the current training type selection information is determined to be the Metaverse Power Comprehensive Training Scenario in the Metaverse Training Scenario, then the current training data corresponding to the Metaverse Power Comprehensive Training Scenario is obtained, and the current training data is sent to the VR headset device.

[0077] In this embodiment, the metaverse training scenario includes multiple primary sub-scenes, such as the metaverse comprehensive power training scenario, which is one of them. The metaverse comprehensive power training scenario may also include secondary sub-scenes such as the metaverse power fault training scenario and the metaverse power equipment assembly training scenario. When a user selects the metaverse comprehensive power training scenario as a primary sub-scene, without specifically selecting a secondary sub-scene, the metaverse comprehensive training platform in the host machine can determine a corresponding default training scenario (such as meter assembly) based on the metaverse comprehensive power training scenario, obtain the current training data corresponding to the default training scenario, and send the current training data to the VR headset. Therefore, through the above method, the specific training scenario corresponding to the current training type can be quickly selected in the host machine, and the corresponding current training data can be sent to the VR headset for display in the virtual reality scene.

[0078] In one embodiment, acquiring the current training data corresponding to the metaverse power integrated training scenario includes: If the metaverse power comprehensive training scenario is determined to be a metaverse power fault training scenario, then the current training data corresponding to the metaverse power fault training scenario is generated based on the preset power fault simulation production strategy. If the metaverse power integrated training scenario is determined to be the metaverse power equipment assembly training scenario, then the current training data corresponding to the metaverse power equipment assembly training scenario is generated based on the acquired power equipment assembly project information.

[0079] In this embodiment, the Metaverse Power Comprehensive Training Scenario is a first-level sub-scenario within the Metaverse Training Scenario, which further includes second-level sub-scenarios such as the Metaverse Power Fault Training Scenario and the Metaverse Power Equipment Assembly Training Scenario. When a user selects the Metaverse Power Fault Training Scenario as a second-level sub-scenario, it indicates that the user needs to perform training operations related to power fault repair. At this time, the Metaverse Comprehensive Training Platform generates the current training data corresponding to the Metaverse Power Fault Training Scenario based on a power fault simulation production strategy. For example, the power fault simulation production strategy is used to determine at least one target power fault type from the power fault simulation library based on the trainee's personnel tags (such as junior electrician, intermediate electrician, senior electrician, etc.) or based on the training popularity values ​​of various types of power faults in the power fault simulation library (such as the topK of training popularity values, K=3 or a positive integer set based on actual needs). Then, the current training data is determined based on the target power fault type. For example, if the virtual reality model objects to be included in the target power fault type are electricity meters, high-voltage cables, and distribution transformers, then the current training data displayed in the initial virtual reality scene will be generated based on the virtual reality model objects of electricity meters, high-voltage cables, and distribution transformers (each virtual reality model object has a corresponding initial three-dimensional spatial position and initial object state in the initial virtual reality scene).

[0080] When a user selects the "Metaverse Power Equipment Assembly Training Scenario," it indicates that the user needs to perform practical training operations related to power equipment assembly. At this point, the system can further retrieve information about a power equipment assembly project selected from the user interface's list of projects. This project information can include specific power equipment such as meters, high-voltage cables, distribution transformers, medium-voltage switchgear, and vacuum ring main units. The virtual reality model objects corresponding to the specific power equipment included in the project information are placed in their initial spatial positions within the initial virtual reality scene. Trainees need to assemble these virtual reality model objects using the power equipment assembly operation. After obtaining the power equipment assembly project information, the system generates the current training data displayed in the initial virtual reality scene based on the virtual reality model objects of each power equipment included in the project information.

[0081] Of course, in specific implementation, the first-level sub-scenario of the metaverse training scenario can include not only second-level sub-scenarios such as the metaverse power failure training scenario and the metaverse power equipment assembly training scenario, but also second-level sub-scenarios such as the metaverse power failure evacuation and escape training scenario and the metaverse power equipment emergency handling operation training scenario. The corresponding training scenarios can be set according to the actual needs of the users.

[0082] The VR headset 4 is used to receive the current training data and display the current virtual reality display data corresponding to the current training data.

[0083] In this embodiment, when the VR headset receives the current training data sent by the host via the local area network, it first obtains virtual reality model objects of various power devices included in the current training data, then combines the virtual reality model objects into current virtual reality display data, and displays it in the virtual reality scene displayed by the VR headset.

[0084] The force feedback haptic glove 5 is used to acquire the user's current training operation data in response to the current virtual reality display data, and send the current training operation data to the host for storage in real time until a training end command is detected.

[0085] The force feedback haptic glove model is mapped and displayed in the virtual reality scene displayed by the VR headset. The user's hand operation data collected by the force feedback haptic glove is synchronized to the virtual reality scene displayed by the VR headset to perform corresponding operations on the current virtual reality display data, and receives haptic data determined by the host according to the force feedback intensity determination strategy. The haptic data is the force feedback intensity data or force feedback intensity dataset generated and sent to the force feedback haptic glove when the VR headset performs corresponding operations on the current virtual reality display data based on the user's hand operation data.

[0086] In this embodiment, in order to simulate the user participating in the training performing corresponding operations on various virtual reality model objects in the virtual reality scene displayed by the VR headset device through the force feedback haptic gloves, it is first necessary to map and display the glove model corresponding to the force feedback haptic gloves in the virtual reality scene displayed by the VR headset device, more specifically, to integrate it into the virtual reality scene corresponding to the current virtual reality display data. Moreover, the operations performed by the user's hand driving the force feedback haptic gloves will be synchronized to the virtual reality scene in real time to perform corresponding operations on the current virtual reality display data, and receive the haptic data determined by the host according to the force feedback intensity determination strategy.

[0087] To enable the force feedback haptic glove to synchronize user hand operation data to the virtual reality scene for corresponding operations on the current virtual reality display data, and to receive tactile data determined by the host based on the force feedback intensity determination strategy, the force feedback haptic glove can specifically be the VR glove SenseGlove (which integrates active contact feedback, force feedback, and vibration haptic feedback functions; active contact feedback is used to sense the impact and gripping sensations of the palm; force feedback is used to sense the size and hardness of virtual objects; and vibration haptic feedback is used to sense signals and basic textures). It also features gesture recognition and an immersive tactile experience. The user hand operation data collected by the force feedback haptic glove can be synchronized in real time to the glove model in the virtual reality scene displayed on the VR headset. All operations performed by the user while wearing the force feedback haptic glove can be mapped to the glove model, and the glove model can then perform corresponding operations on the current virtual reality display data.

[0088] In one embodiment, the force feedback haptic glove 5 is specifically used for: Obtain the current operation model object in the current virtual reality display data for the user, and generate tactile data corresponding to the current operation model object to simulate real tactile sensation. The system acquires the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, and sends it to the host for storage.

[0089] In this embodiment, when a user wears a force feedback haptic glove for practical training, and the user's hand drives the glove to perform an action (such as moving the hand to touch a faulty cable), the hand model in the virtual reality scene will also move to the corresponding position of the current operation model object in the current virtual reality display data and perform the same action. Since the current operation model object (i.e., its specific physical type) is known, the tactile data corresponding to the current operation model object can be obtained from the background model object database of the Metaverse Integrated Training Platform to simulate the user's real tactile sensation towards the current operation model object. For example, if the current operation model object is a faulty cable, when the user operates the force feedback haptic glove to drive the glove model to move to the cable, the user can feel the cable's tension, etc., allowing the user to obtain a realistic force sensation when operating electrical equipment, thereby greatly enhancing the simulation effect of the training.

[0090] Subsequently, the force feedback haptic glove acquires the user's current training operation data on the current operation model object within the simulated haptic environment corresponding to the haptic data. This current training operation data represents the user's specific operation on the current operation model object within the current virtual reality display data. The force feedback haptic glove comprehensively captures and recognizes the user's hand movements and subtle gestures, ensuring that the user's operations in the virtual reality environment are accurately mapped to actual power operations (i.e., the user's operations in the virtual reality environment also provide the same operational experience as in actual power operations). Whether it's a simple switch operation or complex troubleshooting, accurate simulation is achieved. The obtained current training operation data is sent to the host computer in real time for storage. The above process saves the current training operation data of a user performing an operation while wearing the force feedback haptic glove. Subsequent operations are processed using the same process until a training end command is detected.

[0091] In one embodiment, acquiring the user's current training operation data on the current operation model object in the simulated haptic environment corresponding to the haptic data, and sending it to the host for storage, includes: The force feedback haptic glove acquires the user's current hand operation data on the current operation model object in the simulated haptic environment corresponding to the haptic data, and synchronously maps the current user's hand operation data to the glove model in the virtual reality scene of the VR headset, synchronously driving the glove model to perform corresponding operations on the current operation model object and generate the current training operation data.

[0092] In this embodiment, when the force feedback haptic glove acquires the user's current training operation data on the current operation model object in the simulated haptic environment corresponding to the haptic data, specifically, the physical device of the force feedback haptic glove first captures and recognizes the user's hand movements and subtle gestures, and maps these hand movements and subtle gestures onto the glove model in the virtual reality scene of the VR headset. The glove model serves as a digital twin model of the force feedback haptic glove to perform corresponding operations on the current operation model object and generate the current training operation data. This method provides the user with an immersive haptic experience, simulating the user being in a real operating environment, allowing them to intuitively receive physical feedback from each operation.

[0093] The host 1 is further configured to, if a training end instruction is detected, acquire the saved current training operation data and the current training operation data corresponding to the current training operation data and the current virtual reality display data, and form a current training dataset corresponding to the current training data.

[0094] In this embodiment, if the host detects a training end command, it indicates that the user's current round of metaverse comprehensive training has ended. First, it acquires the saved current training operation data and the current virtual reality display data corresponding to the current training operation data and the current training operation data, and forms a current training dataset corresponding to the current training data. Then, it saves the current training dataset in the host's background database. The current training dataset is bound to the user corresponding to the current training user information; that is, this current training dataset is generated by the user corresponding to the current training user information performing metaverse training operations within the user's operation time interval. The user operation time interval can start from the time the user logs in and end at the time the training end command is generated.

[0095] In one embodiment, the host 1 is further configured to: If the current training type selection information is determined to correspond to the metaverse training viewing scenario, then the currently viewable training operation video stream corresponding to the metaverse training viewing scenario is obtained, and the currently viewable training operation video stream is sent to the VR headset device.

[0096] In this embodiment, when a user selects the "Speed ​​Competition" virtual button on the main operation interface using the user input device, the corresponding current training type selection information is the Metaverse Training Viewing Scene. If the host is continuously receiving current training operation data from the current participating user, other users in the Metaverse Training Viewing Scene can obtain the current training operation data of the aforementioned participating user from the host and send it as the currently viewable training operation video stream to the VR headsets of other viewing users (e.g., teachers who comment on and operate the training user's operations). The current operation step information corresponding to the currently viewable training operation video stream is also displayed synchronously on the interface of the currently viewable training operation video stream, so that the VR headsets of other users can obtain the currently viewable training operation video stream in a timely manner, realizing real-time monitoring and transmission of training operation data.

[0097] Referring back to the example above, if the teacher wears both a VR headset and force feedback haptic gloves, the current training operation data of the participants can also be realistically simulated through the teacher's force feedback haptic gloves, enabling the teacher to quickly obtain the students' operation data and provide timely and accurate feedback.

[0098] In one embodiment, the host 1 is further configured to: If it is determined that the current training type selection information corresponds to a data observation scenario, then the relevant data of the project equipment corresponding to the data observation scenario is obtained and displayed through the display connected to the host.

[0099] In this embodiment, when a user selects the "Data Observation" virtual button on the main operation interface using the user input device, the corresponding current training type selection information is "Data Observation Scene." The user can obtain project-related data such as today's number of experiences, cumulative number of experiences, projects executed today, projects in progress, total number of devices, currently running devices, currently idle devices, currently unconnected devices, popular project rankings, and project categories, which are then displayed on the monitor connected to the host. Alternatively, the data can be sent to a VR headset for display.

[0100] When a user selects the "Active User Statistics" virtual button on the main operation interface using the user input device, the corresponding current training type is selected as a data analysis scenario. The user can view the active user statistics data corresponding to the data analysis scenario on the monitor or VR headset, allowing trainees to view training records and related training progress, and perform data analysis.

[0101] As can be seen, the implementation of this system, combining the host, VR headset, and force feedback haptic gloves, enables the timely collection of the user's current training operation data in the virtual reality scene displayed on the VR headset. Furthermore, during the operation, the system receives tactile data determined by the host based on the force feedback intensity determination strategy, which greatly enhances the simulation effect of the training.

[0102] The aforementioned teaching and training system based on metaverse force feedback technology can be implemented as a computer program, which can be used in, for example... Figure 5 It runs on the computer device shown.

[0103] Please see Figure 5 , Figure 5 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. This computer device integrates any of the teaching and training implementation systems based on metacosmic force feedback technology provided in this embodiment of the present invention.

[0104] See Figure 5 The computer device 400 includes a processor 402, a memory, and a network interface 405 connected via a system bus 401. The memory may include a storage medium 403 and internal memory 404.

[0105] The storage medium 403 can store an operating system 4031 and a computer program 4032. The computer program 4032 includes program instructions, which, when executed, cause the processor 402 to perform the aforementioned teaching and training implementation method based on metaverse force feedback technology.

[0106] The processor 402 provides computing and control capabilities to support the operation of the entire computer device.

[0107] The internal memory 404 provides an environment for the computer program 4032 in the storage medium 403 to run. When the computer program 4032 is executed by the processor 402, the processor 402 can execute the above-mentioned teaching and training implementation method based on the meta-cosmic force feedback technology.

[0108] This network interface 405 is used for network communication with other devices. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0109] The processor 402 is used to run the computer program 4032 stored in the memory to implement the teaching and training method based on the metaverse force feedback technology as described above.

[0110] It should be understood that, in this embodiment of the invention, the processor 402 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0111] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0112] Therefore, the present invention also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the teaching and training implementation method based on the metaverse force feedback technology described above.

[0113] The computer-readable storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0114] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0115] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0116] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0117] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0118] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A teaching and training implementation method based on metacosmic force feedback technology, applied to a teaching and training implementation system based on metacosmic force feedback technology, characterized in that, The teaching and training system based on metaverse force feedback technology includes a host, a monitor, a user input device, a VR headset, and a force feedback haptic glove; the monitor, the user input device, and the VR headset are all communicatively connected to the host, the VR headset is communicatively connected to the force feedback haptic glove, and the host is also connected to a cloud server via a network; The teaching and training implementation method based on metaverse force feedback technology includes: The host responds to the user login command and obtains the current training user information and the current training type selection information corresponding to the user login command; If the host determines that the current training type selection information corresponds to the metaverse training scenario, it sends the current training data corresponding to the current training type selection information to the VR headset; the host generates the current training data in the metaverse training scenario based on the fault type generation strategy or the power fault simulation production strategy. The VR headset receives the current training data and displays the current virtual reality display data corresponding to the current training data; The force feedback haptic glove acquires the user's current training operation data in response to the current virtual reality display data, and sends the current training operation data to the host for storage in real time until a training end command is detected. The glove model corresponding to the force feedback haptic glove is mapped and displayed in the virtual reality scene displayed by the VR headset. The user's hand operation data collected by the force feedback haptic glove is synchronized to the virtual reality scene displayed by the VR headset to perform corresponding operations on the current virtual reality display data, and receives haptic data determined by the host according to the force feedback intensity determination strategy. The haptic data is force feedback intensity data or force feedback intensity dataset generated and sent to the force feedback haptic glove when the VR headset performs corresponding operations on the current virtual reality display data based on the user's hand operation data. If the host detects a training end command, it acquires the saved current training operation data and the current training operation data corresponding to the current training operation data and the current virtual reality display data, and forms a current training dataset corresponding to the current training data.

2. The method according to claim 1, characterized in that, If it is determined that the current training type selection information corresponds to the metaverse training scenario, then the current training data corresponding to the current training type selection information is sent to the VR headset, including: If it is determined that the current training type selection information corresponds to the metaverse training scenario, then the information of the currently logged-in student is obtained; wherein, the current student information includes at least the student's unique user ID, the student's training skill level, and the student's training weakness information; The system obtains the training popularity value of each type of fault included in the preset fault database, the matching degree with the current trainee's skill level, the weight of the weak link corresponding to the fault, and the total number of matching fault types. Based on the preset fault type generation strategy, it determines the generation probability of each fault scenario that matches the current user. From the generation probabilities of each fault scenario matching the current user, determine the fault scenario with the highest generation probability, and generate the current training data accordingly. The current training data is sent to the VR headset.

3. The method according to claim 1, characterized in that, If the step of determining that the current training type selection information corresponds to the metaverse training scenario, then sending the current training data corresponding to the current training type selection information to the VR headset device includes: If the current training type selection information is determined to be the Metaverse Power Comprehensive Training Scenario in the Metaverse Training Scenario, then the current training data corresponding to the Metaverse Power Comprehensive Training Scenario is obtained, and the current training data is sent to the VR headset device.

4. The method according to claim 1, characterized in that, The force feedback haptic glove acquires the user's current training operation data in response to the current virtual reality display data, and sends the current training operation data to the host for storage in real time, including: The force feedback haptic glove acquires the user's current operation model object in the current virtual reality display data, and generates haptic data corresponding to the current operation model object to simulate real haptic sensation. The force feedback tactile glove acquires the user's current training operation data on the current operation model object in the simulated tactile environment corresponding to the tactile data, and sends it to the host for storage.

5. The method according to claim 4, characterized in that, After the force feedback haptic glove acquires the user's current training operation data on the current operation model object in the simulated haptic environment corresponding to the haptic data, and sends it to the host for storage, and before the host detects a training end command and acquires the saved current training operation data corresponding to the current training operation data and the current virtual reality display data, and forms a current training dataset corresponding to the current training data, the method further includes: The host obtains the student's hand operation force, the equipment material coefficient of the current operation model object, the scene difficulty coefficient and feedback calibration compensation factor corresponding to the current training operation data from the current training operation data, determines the strategy and the force feedback intensity dataset corresponding to the current training operation data according to the preset force feedback intensity, and sends it to the force feedback tactile glove.

6. The method according to claim 1, characterized in that, After the step of the host responding to the user login command and obtaining the current training user information and the current training type selection information corresponding to the user login command, the method further includes: If the host determines that the current training type selection information corresponds to the metaverse training viewing scenario, it obtains the currently viewable training operation video stream corresponding to the metaverse training viewing scenario and sends the currently viewable training operation video stream to the VR headset.

7. The method according to claim 1, characterized in that, After the step of the host responding to the user login command and obtaining the current training user information and the current training type selection information corresponding to the user login command, the method further includes: If the host determines that the current training type selection information corresponds to a data observation scenario, it acquires the relevant data of the project equipment corresponding to the data observation scenario and displays it through the display connected to the host.

8. A teaching and training system based on metacosmic force feedback technology, characterized in that, It includes a host, a display, a user input device, a VR headset, and a force feedback haptic glove; the display, the user input device, and the VR headset are all communicatively connected to the host, the VR headset is communicatively connected to the force feedback haptic glove, and the host is also connected to a cloud server via a network; The host is used to respond to a user login command by obtaining the current training user information and the current training type selection information corresponding to the user login command; The host is further configured to send the current training data corresponding to the current training type selection information to the VR headset if it is determined that the current training type selection information corresponds to the metaverse training scenario; The host generates the current training data in the metaverse training scenario based on a fault type generation strategy or a power fault simulation production strategy. The VR headset is used to receive the current training data and display the current virtual reality display data corresponding to the current training data; The force feedback haptic glove is used to acquire the user's current training operation data in response to the current virtual reality display data, and to send the current training operation data to the host for storage in real time until a training end command is detected. The glove model corresponding to the force feedback haptic glove is mapped and displayed in the virtual reality scene displayed by the VR headset. The user's hand operation data collected by the force feedback haptic glove is synchronized to the virtual reality scene displayed by the VR headset to perform corresponding operations on the current virtual reality display data, and receives haptic data determined by the host according to a force feedback intensity determination strategy. The haptic data is force feedback intensity data or force feedback intensity dataset generated and sent to the force feedback haptic glove when the VR headset performs corresponding operations on the current virtual reality display data based on the user's hand operation data. The host is further configured to, if a training end instruction is detected, acquire the saved current training operation data and the current training operation data corresponding to the current training operation data and the current virtual reality display data, and form a current training dataset corresponding to the current training data.

9. The teaching and training system based on metacosmic force feedback technology according to claim 8, characterized in that, The host is specifically used for: If the current training type selection information is determined to be the Metaverse Power Comprehensive Training Scenario in the Metaverse Training Scenario, then the current training data corresponding to the Metaverse Power Comprehensive Training Scenario is obtained, and the current training data is sent to the VR headset device.

10. The teaching and training system based on metacosmic force feedback technology according to claim 8, characterized in that, The force feedback tactile gloves are specifically used for: Obtain the current operation model object of the user in the current virtual reality display data, and generate tactile data corresponding to the current operation model object to simulate real tactile sensation; The system acquires the current training operation data of the user on the current operation model object in the simulated tactile environment corresponding to the tactile data, and sends it to the host for storage.

Images