Virtual reality-based field survival herbal simulation training system and method

Abstract

This invention relates to a virtual reality-based simulation training system and method for wild herbal survival. The system comprises, from bottom to top, a logic layer, an interaction layer, a 3D visual layer, and an application and data processing layer. The logic layer is used for designing logical relationships, interactive processes, implementing interactive methods, and integrating toolsets for the simulation training. The interaction layer, based on VR interaction technology, employs a virtual gesture adaptive algorithm to achieve refined hand gesture changes in the virtual world and the wild environment. The 3D visual layer uses 3ds Max, ZBrush, and Substance 3D tools to reverse engineer high-definition photos, achieving realistic rendering of various parts of wild herbal plants and realizing a VR harvesting scene for wild-grown herbal plants. The application and data processing layer stores and manages the 3D visual model, questions, knowledge points, task configurations, and student assessment results. The beneficial effect is a more three-dimensional and realistic teaching method, allowing students to feel as if they are actually there.

Description

[Technical Field]

[0001] This invention relates to the field of computer virtual reality technology, specifically to a virtual reality-based wilderness survival herbal medicine simulation training system and method. [Background Technology]

[0002] Traditional Chinese medicine (TCM) is one of the most important treatment methods in Traditional Chinese Medicine (TCM). Compared with modern chemical drugs, one of the advantages of TCM is that it is mainly derived from plants. Therefore, in situations such as sudden wilderness emergencies, it can be used to obtain food and save lives, proving invaluable.

[0003] For example, soldiers participating in combat might find themselves isolated in unfamiliar wilderness or on isolated islands, facing severe situations of food shortages and inadequate logistical support. Meanwhile, with the continuous improvement of people's living standards, outdoor sports have become a fashionable activity. However, outdoor sports are inherently dangerous; for instance, hikers might get lost in the wild or become separated from their group and encounter sudden danger. Furthermore, unexpected events such as floods and earthquakes can lead to supply shortages and delays in the availability of food and medicine. If soldiers, hikers, or even ordinary people encountering danger possess the ability to identify and utilize a certain number of wild plant resources, they can effectively avoid poisonous plants in the wild and collect edible, disease-preventing, and medicinal plants on-site. This can, to some extent, solve the problem of food and medicine shortages and improve their survival skills in the wild. Therefore, not only for combat personnel, but also for the general public, mastering a certain number of wild plants with survival value is essential and a vital skill for life.

[0004] Currently, there are no systematic courses or training methods for learning herbal medicine for wilderness survival. Furthermore, the variety of wild plants with survival value is vast, and the knowledge points are numerous, making it difficult to master solely through traditional media such as pictures, texts, and books. It requires combined with field research and learning. However, field learning is affected by factors such as time, space, and weather, not only requiring significant human, material, and financial resources, but also potentially posing considerable personal danger.

[0005] With the maturity and popularization of VR technology, virtual reality technology has become an indispensable part of the "metaverse" world. Combined with 3D modeling technology, PBR technology, AI algorithms, big data, etc., it constitutes the huge virtual world architecture of the "metaverse".

[0006] PBR: Physically Based Rendering. It is a method of shading and rendering that more accurately represents how light interacts with material properties.

[0007] TAA: Temporal Anti-Aliasing, one of the most commonly used image enhancement algorithms, is a shader-based algorithm that combines two frames using motion vectors to determine where to sample the previous frame. A dithering operation is performed on the pixels in the screen area in each frame, so that when the data of several consecutive frames is mixed, it is equivalent to sampling each pixel multiple times. He distributes the sampling points from a single frame to multiple frames, so that each frame does not need to increase the amount of calculation by multiple sampling, but TAA often blindly follows the motion vector of the moving object, resulting in unclear details on the screen.

[0008] Metaverse: A virtual world built by humans using digital technology, mapped from or beyond the real world, and can interact with the real world, with a new social system of digital living space.

[0009] The existing training technology is currently mainly in the form of books, videos and the like for teaching, and the practical aspect needs to be explored on site. The former is relatively flat and single through traditional text and video teaching methods, and it is difficult for students to intuitively and deeply understand the knowledge points. Furthermore, if the book teaching is separated from practice or seriously decoupled from practice, it is also difficult to strengthen and guarantee the retention rate of students' knowledge points. For practice, the category and quantity of field survival herbal medicine are large, distributed in various parts of the country, and some rare plants may grow in relatively remote areas. If it needs to be investigated on site, a lot of time needs to be spent on route planning, which involves a lot of time, cost, and is affected by uncertain factors such as region and weather, and is likely to be accompanied by high risk. Therefore, students often have difficulty getting real practice.

[0010] In addition, most of the field survival teaching content needs to be combined with practice and a large number of cases, and it is difficult for traditional teaching methods to perfectly integrate practice content and cases. Therefore, in view of the shortcomings of the existing technology, new technologies need to be used to make up for the shortcomings of traditional technology and bring new changes to teaching by integrating new technologies.

[0011] The present application aims to solve the technical problem that the existing training technology is mainly in the form of books, videos and the like for teaching, and the teaching method is relatively flat and single, making it difficult for students to intuitively and deeply understand the field survival herbal knowledge points which are large in quantity and distributed in various parts of the country. The field survival herbal training system and method have been technically improved.

SUMMARY

[0012] The purpose of the present application is to provide a training system that makes the teaching method three-dimensional and realistic, allowing students to intuitively and deeply understand field survival herbal knowledge.

[0013] In order to achieve the above object, the technical scheme adopted by the present application is a virtual reality-based field survival herbal simulation training system, comprising a training server; the software architecture of the training server comprises a logical layer, an interactive layer, a three-dimensional visual layer and an application and data processing layer from bottom to top; the logical layer is used for logical relationship design, interactive process design, interactive method implementation and integration of tool sets for field survival herbal simulation training; the interactive layer is based on VR interactive technology and uses a virtual gesture adaptive algorithm to realize fine posture changes of hands in a virtual world and a field scene; the three-dimensional visual layer realizes real rendering performance of each part of field survival herbal plants and a field growth herbal plant VR picking scene through reverse modeling of high-definition photos by means of 3Ds Max, ZBrush and Substance 3D tool flow; and the application and data processing layer performs background unified storage and management of three-dimensional visual models, questions, knowledge points, task configurations and assessment and scoring results of students.

[0014] Preferably, the application and data processing layer comprises a simulation simulation system module, a scene library, a task library, a case library, a model library and a student data record module; the simulation simulation system module comprises a field exploration sub-module, a plant collection sub-module, a plant viewing sub-module, a practical simulation sub-module and an evaluation and assessment sub-module; the scene library comprises an island map module and a jungle module; and the task library is used for storing various training tasks.

[0015] Preferably, the virtual reality-based field survival herbal simulation training system further comprises a scenario editing module; the scenario editing module is arranged on the training server or a front-end PC and is used for freely configuring various training tasks in scenario editing, setting corresponding task requirements from the task library and the case library, editing emergencies and cases in the current field survival exploration task, and transmitting scenario editing setting parameters into the task library.

[0016] Preferably, the virtual reality-based field survival herbal simulation training system further comprises a plurality of student terminals; the training server and the student terminals are connected in communication through a network; the training server comprises a task manager, and the student terminals comprise local task modules; the training server multicasts training tasks to the plurality of student terminals through the task manager; the student terminals complete a training task or trigger a new training task, notify the task manager, and simultaneously update a training task UI or trigger a new training task logic through the local task modules.

[0017] Preferably, the virtual reality-based field survival herbal simulation training system further comprises a resource management platform; the resource management platform is connected with the task library, the case library and the student data record module, is used for recording results of each student learning assessment, and analyzes learning results through a background.

[0018] Still another object of the present application is to provide a method for stereoscopic and realistic teaching, which enables students to intuitively and deeply understand field survival herbal medicine.

[0019] To achieve the above object, the technical scheme adopted by the present application comprises a three-dimensional modeling method for a virtual reality-based field survival herbal medicine simulation training system, a virtual reality-based field survival herbal medicine simulation training system based on the above method, which is used for three-dimensional modeling of field herbal plants and comprises the following steps:

[0020] J1, using a single-lens reflex camera to collect herbal plant photos;

[0021] J2, using Substance 3D tools to separate herbal plant photo data;

[0022] J3, making herbal plant map files;

[0023] J4, making high-precision three-dimensional models of herbal plants;

[0024] J5, finally completing the rendering of the three-dimensional model of the herbal plant.

[0025] Preferably, step J5 uses a PBR process to achieve realistic visual effects.

[0026] The present application also comprises a terrain modeling method for a virtual reality-based field survival herbal medicine simulation training system, which is based on the above method and is used for modeling the terrain where field herbal plants grow in the wild, and comprises the following steps:

[0027] D1, using WM terrain processing software to generate a map area after selecting a target terrain area;

[0028] D2, adjusting parameters to edit and set the topography;

[0029] D3, importing the generated height map of the topography into UE4 to automatically generate a scene;

[0030] D4, then drawing map details and setting the placement of herbal plants in the UE4 tool.

[0031] The present application also comprises a hand animation generation method for a virtual reality-based field survival herbal medicine simulation training system, which is based on the above method and is used for automatically generating detailed hand animations when training students to pick plants, pick up objects, and hold objects in virtual reality, and comprises the following steps:

[0032] S1, creating a hand model for gesture recognition;

[0033] S2, input real hand operation information into the hand model, and generate static gesture information;

[0034] S3, obtain gesture recognition information of the trainee;

[0035] S4, import the static gesture information and the gesture recognition information of the trainee into the preset field scene for gesture capture, and generate hand animation.

[0036] Preferably, the hand model comprises 24 hand skeleton key points.

[0037] The virtual reality-based field survival herbal simulation training system and method have the following advantages: the natural environment and topography, the three-dimensional model reconstruction of vegetation plants, the knowledge base, and the immersive interactive experience of the "meta universe" can be restored truly, the learning interest and the learning efficiency of the trainees are increased, the learning cost is reduced, the trainees can learn and practice in the virtual world in the classroom without being restricted by time and region, the trainees can understand and master common plants with field survival value (i.e., field survival herbs) that can be collected and applied on site, mainly understand the names and origins of the plants, be familiar with the basic knowledge of the plants, master the field identification points and field use methods of the plants, and enable the trainees to avoid toxic plants accurately and in time, solve the food shortage problem by using the resources around them, and lay a foundation for collecting and using the Chinese herbal medicines around them for epidemic prevention and treatment in the future work, especially in the field environment with no help and insufficient supply. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 It is a virtual reality-based field survival herbal simulation training system architecture diagram.

[0039] Figure 2 It is a virtual reality-based field survival herbal simulation training system data communication architecture diagram.

[0040] Figure 3 It is a virtual reality-based field survival herbal simulation training system local data class diagram.

[0041] Figure 4 It is a virtual reality-based field survival herbal simulation training system application and data processing layer module architecture diagram.

[0042] Figure 5 It is a virtual reality-based field survival herbal simulation training system resource management platform architecture diagram.

[0043] Figure 6 It is a virtual reality-based field survival herbal simulation training system scenario editing module software interface diagram.

[0044] Figure 7 It is a three-dimensional modeling method flow chart of a virtual reality-based field survival herbal simulation training system.

[0045] Figure 8 It is a terrain modeling method flow chart of a virtual reality-based field survival herbal simulation training system.

[0046] Figure 9 It is a hand skeleton key point schematic diagram of a hand animation generation method of a virtual reality-based field survival herbal simulation training system.

DETAILED DESCRIPTION

[0047] Features and exemplary embodiments of various aspects of the present application will be described in detail below. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one of ordinary skill in the art that the present application can be practiced without some of these specific details. The description of the embodiments is merely to provide a better understanding of the present application by showing examples of the present application. The present application is in no way limited to any specific configuration and algorithm set forth below, but covers any modification, replacement, and improvement of elements, components, and algorithms without departing from the concept of the present application. In the accompanying drawings and the following description, well-known structures and techniques are not shown in order to avoid unnecessary obscuring of the present application.

[0048] Embodiments

[0049] The present embodiment implements a virtual reality-based field survival herbal simulation training system and method thereof.

[0050] From the perspective of actual demand and application, the present embodiment designs a complete set of training software based on VR virtual reality technology and three-dimensional modeling technology. The software can combine all plant models with real geographical environment, allowing students to immerse themselves in the virtual world for exploration and learning. Through virtual interaction, students can closely observe three-dimensional models of various plants and learn relevant knowledge. The method software of the present embodiment, in cooperation with the resource management platform, can carry a powerful model library, image database, and case library, combine plant knowledge points with practice and actual combat cases, create a powerful learning closed loop, shorten the learning curve, strengthen the memory retention rate of students, break the time and space limitations, and allow students to learn and experience anytime and anywhere.

[0051] Figure 1 It is an architecture diagram of a virtual reality-based field survival herbal simulation training system. As shown in FIG. 1, the system includes a resource management platform, a three-dimensional modeling platform, a virtual reality platform, and a virtual reality platform. Figure 1As shown, 1, the underlying technical architecture of the system of the embodiment is a logic layer, which mainly realizes the logical relationship design, interactive process design, interactive method implementation, and integration of tool sets of different training processes; 2, the interactive layer of the system of the embodiment adopts a virtual gesture adaptive algorithm, which can realize fine posture changes of hands and objects in a virtual world; 3, the three-dimensional vision layer of the system of the embodiment realizes real rendering performance of each part of a plant by reverse modeling of high-definition photos through 3DsMax, ZBrush, and Substance 3D tool streams; 4, the application and data processing layer of the system of the embodiment uniformly stores and manages models, topics, knowledge points, task configurations, and assessment and scoring results of students in the background.

[0052] 3D Studio Max, commonly referred to as 3d Max or 3ds MAX, is a 3D modeling rendering and production software based on a PC system.

[0053] ZBrush software is a 3D design tool that gives artists the freedom to create without constraints, completely overturning the working mode of traditional three-dimensional design tools, liberating the hands and minds of artists, and saying goodbye to the past mode of awkward creation relying on a mouse and parameters, and fully respecting the creative inspiration and traditional working habits of designers.

[0054] The Substance 3D series of software will greatly reduce the complexity of 3D design technology, help creative people improve their workflow, and improve their 3D production capabilities.

[0055] Figure 2 It is a data communication architecture diagram of a virtual reality-based field survival herbal simulation training system. Figure 3 It is a local data class diagram of a virtual reality-based field survival herbal simulation training system. As shown in Figure 2 , and Figure 3 , the various data classes involved in the actual software development of the system of the embodiment, and the architecture of data exchange communication between data. The system of the embodiment is actually used for training, including a training server and a plurality of student terminals, the training server multicasts training tasks to the plurality of student terminals through a task manager; the student terminal notifies the task manager through a local task module after completing a training task or triggering a new training task, and simultaneously updates the training task UI or triggers new task logic.

[0056] Figure 4 It is a module architecture diagram of an application and data processing layer of a virtual reality-based field survival herbal simulation training system. As shown in Figure 4As shown, the application and data processing layer of the system of the embodiment includes a simulation system module, a scene library, a task library, a case library, a model library and a student data recording module. The simulation system module includes a field exploration sub-module, a plant collection sub-module, a plant viewing sub-module, a practical simulation sub-module and an evaluation and examination sub-module. The scene library includes island map modules, jungle modules and other field scene modules. The task library is used to store various training tasks.

[0057] Figure 5 A resource management platform architecture diagram of a virtual reality-based field survival herbal simulation training system is shown in FIG. 1. Figure 5 As shown, the system of the embodiment further includes a resource management platform that is connected with the task library, the case library and the student data recording module, and can record the results of each student's learning and examination, and analyze the learning results through the background.

[0058] Figure 6 A scenario editing module software interface diagram of a virtual reality-based field survival herbal simulation training system is shown in FIG. 2. Figure 6 As shown, the system of the embodiment further includes a scenario editing module. Through the scenario editing module of the system training server or the front end, the scenario editing module is used to freely configure various training tasks, can set the corresponding task requirements from the task library and the case library, edit the emergencies and cases in the current exploration task, and transmit the scenario editing setting parameters into the actual task system (task library).

[0059] The implementation of the system of the embodiment combines three-dimensional modeling technology, three-dimensional terrain generation technology and VR interaction technology.

[0060] Figure 7 A three-dimensional modeling method flowchart of a virtual reality-based field survival herbal simulation training system is shown in FIG. 3. Figure 7 As shown, the three-dimensional modeling method of the herbal plant of the system of the embodiment includes the following steps:

[0061] J1, collecting herbal plant photos by using a single-lens reflex camera;

[0062] J2, separating the herbal plant photo data by using a Substance 3D tool;

[0063] J3, making a herbal plant texture file;

[0064] J4, making a high-precision herbal plant model;

[0065] J5, completing the rendering of the final herbal plant model.

[0066] Figure 8 A terrain modeling method flowchart of a virtual reality-based field survival herbal simulation training system is shown in FIG. 4.Figure 8 As shown, the system of the embodiment includes the following steps for modeling the field terrain of wild herbs:

[0067] D1, through the WM terrain processing software, the target terrain area is selected to generate a map area;

[0068] D2, the editing and setting of the landform are performed by adjusting parameters;

[0069] D3, the landform is generated into a height map and then imported into UE4 for automatic scene generation;

[0070] D4, then the drawing of the map details and the placement of the plants are performed in the UE4 tool.

[0071] The system of the embodiment realizes the restoration and interaction of the plant VR scene.

[0072] The system of the embodiment is based on three-dimensional modeling technology and three-dimensional terrain technology, and generates a three-dimensional terrain of wild herbs by using a plant picking and collecting interaction module, to realize the restoration of a VR picking scene of wild herbs.

[0073] Gesture recognition technology is a technology for recognizing human gestures through algorithms. The key point information of the fingers is detected through visual detection, the complex movements of the five fingers of the human body are determined through the displacement information of the key point information of the fingers, and then the type of the gesture is judged.

[0074] Figure 9 A hand skeleton key point diagram of a hand animation generation method of a virtual reality-based wild survival herb simulation training system is shown in FIG. Figure 9 As shown, the embodiment adopts a Google virtual gesture adaptive algorithm including 21 hand skeleton key points. The interaction layer of the system of the embodiment is based on the hand animation generation method of the virtual gesture adaptive algorithm, and is used for automatically generating fine hand animation when picking plants, picking up objects, and taking objects.

[0075] 1. The hand animation production method creates a hand model for gesture recognition; 2. Then, real hand operation information is input into the hand model to generate static gesture information; 3. Then, gesture recognition object information is obtained; 4. Finally, the static gesture information and the gesture recognition object information are imported into a preset scene for gesture capture to generate hand animation. The originally static recorded gesture can be dynamically switched, the rules and conditions in the gesture recognition object information can be matched, the gesture can be adapted to various shapes of objects, new rules and conditions can be added, the expandability of the entire system is enhanced, the hand movement is no longer dependent on traditional animation files, various rules and condition constraints are used for gesture adaptive recognition, the complicated animation state machine logic is avoided, the reusability is improved, and the cost is saved.

[0076] Preferably, the embodiment adopts an improved Google virtual gesture adaptive algorithm, and the hand model required by the gesture making of the system interaction layer of the embodiment should at least include 24 bones, i.e., forearm root, wrist root, thumb joint 1, thumb joint 2, thumb joint 3, thumb joint 4, thumb tip, index finger joint 1, index finger joint 2, index finger joint 3, index finger tip, middle finger joint 1, middle finger joint 2, middle finger joint 3, middle finger tip, ring finger joint 1, ring finger joint 2, ring finger joint 3, ring finger tip, little finger joint 1, little finger joint 2, little finger joint 3, little finger joint 4, and little finger tip.

[0077] The system gesture interaction technology of the embodiment has the following advantages and highlights:

[0078] A. The number of bones that need to be controlled by the hand is more than 20. If the traditional animation making method is used, the workload is huge, and the final effect is not ideal because it cannot be previewed in VR. Using the method for making gestures, not only the efficiency is high, but also the accuracy is greatly improved.

[0079] B. The application of the gesture recognition object framework makes the originally static recorded gestures dynamically switchable, and the gestures can be adapted to various shapes of objects by matching rules and conditions.

[0080] C. The extensible gesture configuration framework can add new rules and conditions, and the whole system has strong extensibility.

[0081] D. The hand movement no longer depends on the traditional animation file, and the gesture adaptive recognition of various rules and conditions constraints eliminates the need to write complex animation state machine logic.

[0082] The system of the embodiment has been applied in university professional education, academic education related courses, and combat unit related field survival training subjects, and the teaching effect is good. For students, the immersive visual experience and game-like mode design have a comprehensive sense of involvement. Students can fully obtain field survival herbal knowledge points according to their own requirements in the virtual space, which can make up for the lack of spatial imagination caused by simple pictures and videos in traditional learning of field survival herbs, thereby realizing visual guidance and thinking guidance, greatly improving the students' participation awareness, stimulating their interest in learning, and improving their practical ability and comprehensive quality. It not only saves time and cost, but also promotes the development of modern teaching mode, greatly improves the quality and efficiency of education. In addition, the system of the embodiment is simple and portable, and is very suitable for training in grassroots units, communities, schools, etc.

[0083] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium, such as a magnetic disk, an optical disk, a Read-Only Memory (ROM) or a Random Access Memory (RAM).

[0084] The above only describes the preferred embodiments of the present application, and it should be pointed out that those skilled in the art can make several improvements and supplements without departing from the principles of the present application, and these improvements and supplements should also be considered as the protection scope of the present application.

Claims

1. A virtual reality-based wilderness survival herbal medicine simulation training system, comprising a training server, characterized in that: The training server software architecture, from bottom to top, includes a logic layer, an interaction layer, a 3D vision layer, and an application and data processing layer. The logic layer is used for the design of logical relationships, interaction flow design, interaction method implementation, and integration of toolsets in the simulation training of wilderness survival herbal medicine. The interaction layer is based on VR interaction technology and uses a virtual gesture adaptive algorithm to realize refined posture changes of hands in the virtual world and in the wild scene; the three-dimensional vision layer uses 3ds Max, ZBrush, and Substance 3D toolflow to reverse model high-definition photos to achieve realistic rendering of various parts of wild-grown herbal plants, realizing a VR harvesting scene of wild-grown herbal plants; the application and data processing layer stores and manages the three-dimensional vision model, questions, knowledge points, task configurations, and student assessment scores in a unified backend. The application and data processing layer includes a simulation system module, a scenario library, a task library, a case library, a model library, and a trainee data recording module; the simulation system module includes a field exploration submodule, a plant collection submodule, a plant viewing submodule, a combat simulation submodule, and an assessment submodule; the scenario library includes an island map module and a jungle module; the task library is used to store various training tasks. It also includes a scenario editing module; this module is set on the training server or front-end PC and is used to freely configure and edit various training tasks. It can set corresponding task requirements from the task library and case library, edit emergencies and cases in the current wilderness survival exploration task, and input the scenario editing settings into the task library. It also includes several student terminals, and the training server and student terminals communicate via a network connection; the training server includes a task manager, and the student terminals include a local task module; the training server multicasts training tasks to several student terminals through the task manager; after a student terminal completes a training task or triggers a new training task through the local task module, it notifies the task manager and simultaneously updates the training task UI or triggers the new training task logic.

2. The virtual reality-based wilderness survival herbal medicine simulation training system according to claim 1, characterized in that: It also includes a resource management platform, which is connected to the task library, case library and student data recording module to record the results of each student's learning assessment and to analyze the learning results through the backend.

3. A three-dimensional modeling method for a virtual reality-based wilderness survival herbal simulation training system, based on any one of claims 1 to 2, for three-dimensional modeling of wild herbal plants, characterized in that... Includes the following steps: J1. Use a DSLR camera to take photos of herbal plants; J2. Use Substance 3D tool to separate the data from herbal plant photographs; J3. Create herbal plant texture files; J4. Create high-precision 3D models of herbal plants; J5. Finally, complete the rendering of the 3D model of the herbal plant; Step J5 uses the PBR process to achieve realistic visual effects.

4. A terrain modeling method for a virtual reality-based simulation training system for wild herbal survival, based on any one of claims 1 to 2, used for modeling the terrain of wild herbal plant growth, characterized in that... Includes the following steps: D1. Using WM terrain processing software, select the target terrain area and generate a map area; D2. Edit and set the terrain by adjusting parameters; D3. After generating a height map of the terrain, import it into UE4 for automated scene generation; D4. Then, in the UE4 tool, we will draw the texture details and set the placement of the herbal plants.

5. A method for generating hand animations in a virtual reality-based wilderness survival herbal medicine simulation training system, based on any one of claims 1 to 2, for automatically generating detailed hand animations of trainees picking plants, gathering objects, and handling objects in virtual reality, characterized in that... Includes the following steps: S1. Create a hand model for gesture recognition; S2. Input and input the real hand operation information into the hand model to generate static gesture information; S3. Obtain gesture recognition information from trainees; S4. Import static gesture information and trainee gesture recognition information into a preset outdoor scene for gesture capture and generate hand animation. The hand model includes 24 key points of the hand skeleton.

Images