Wattle wound teaching and training system based on MR technology

Through the MR technology-based combat trauma teaching and training system, three-dimensional modeling and virtual reality technology are used to build multi-dimensional learning scenarios, which solves the problem that traditional training is difficult to simulate complex battlefields, realizes three-dimensional learning and personalized training, and improves students' Professional skills and emergency response capabilities.

CN118982940APending Publication Date: 2024-11-19THE 980TH HOSPITAL OF THE CHINESE PEOPLES LIBERATION ARMY JOINT LOGISTICS SUPPORT FORCE
View PDF 0 Cites 9 Cited by

Patent Information

Application Number
CN202411363796.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-28
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

Traditional combat trauma teaching and training is difficult to present trauma details and rescue scenarios in a comprehensive and three-dimensional manner, lacks a multi-dimensional learning experience, and cannot simulate complex and dangerous battlefield situations, which affects students' emergency response capabilities.

Method used

Adopt a combat trauma teaching and training system based on MR technology, including model library construction module, training system construction module and comprehensive evaluation module, to build multi-dimensional learning scenarios through three-dimensional modeling and virtual reality technology, simulate complex battlefield situations, and through microservices An online platform for architecture-designed virtual reality training system, providing interactive training tasks and real-time evaluation.

Benefits of technology

Provides a three-dimensional and multi-dimensional learning environment, deeply understands the trauma mechanism and treatment process, improves the professional knowledge and skills of students, enhances the authenticity and practicality of training, supports resource sharing and remote collaborative training, and realizes personalized training. Guidance and knowledge dissemination.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982940A_ABST
    Figure CN118982940A_ABST
Patent Text Reader

Abstract

The invention discloses a war wound teaching and training system based on an MR technology, and the system comprises a model library construction module which is used for constructing a wound model library and a medical knowledge base, and obtaining the incidence relation between wounds and treatment means based on the wound model library and the medical knowledge base; the training system construction module is used for designing a virtual reality training system online platform by adopting a micro-service architecture; the training module is used for training according to the online platform of the virtual reality training system and obtaining a comprehensive training score according to training data; the comprehensive evaluation module is used for obtaining student operation data, obtaining the level value of each key step of the student according to the student operation data, and updating a student training plan based on the level value and the training comprehensive score. The advanced MR technology and the artificial intelligence algorithm are applied to war wound teaching training, so that the learning efficiency and the operation skills of students are improved, the safety and the practicability of training are enhanced, and the method has important significance in improving the quality of war wound treatment training.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of simulation and in particular relates to a combat trauma teaching and training system based on MR technology. Background Art

[0002] Traditional combat trauma teaching and training usually uses two-dimensional pictures, videos and other materials, which makes it difficult to present trauma details and rescue scenes in a comprehensive and three-dimensional manner. The virtual reality function based on MR technology can break this limitation and provide students with a multi-dimensional learning experience. Through three-dimensional modeling and simulation rendering, students can observe and operate virtual trauma models and rescue scenes from different angles and levels, and deeply understand the trauma mechanism and treatment process. At the same time, virtual reality can also simulate various complex and dangerous battlefield situations, allowing students to practice repeatedly in a safe environment to improve their emergency response capabilities. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a combat trauma teaching and training system based on MR technology to solve the above problems existing in the prior art.

[0004] To achieve the above object, the present invention provides a combat trauma teaching and training system based on MR technology, comprising:

[0005] Model library building module, training system building module, training module, comprehensive evaluation module;

[0006] The model library construction module is used to construct a trauma model library and a medical knowledge library, and obtain the association between trauma and treatment methods based on the trauma model library and the medical knowledge library;

[0007] The training system building module is used to design a virtual reality training system online platform using a microservice architecture;

[0008] The training module is used to perform training according to the virtual reality training system online platform and obtain a comprehensive training score according to the training data;

[0009] The comprehensive evaluation module is used to obtain the trainee's operation data, obtain the level value of each key step of the trainee according to the trainee's operation data, and update the trainee's training plan based on the level value and the comprehensive training score;

[0010] The model library construction module, the training system construction module, the training module, and the comprehensive evaluation module are connected in sequence.

[0011] Optionally, the model library construction module includes a trauma model library construction submodule, a model matching submodule, and a solution generation module;

[0012] The trauma model library construction submodule is used to obtain anatomical result data of several trauma areas, divide them according to the trauma parts and trauma types, and construct trauma models based on the divided anatomical result data using a three-dimensional reconstruction algorithm to obtain a trauma model library;

[0013] The model matching submodule is used to sequentially extract individual trauma models from the trauma model library, align the individual trauma models with the human anatomical structure model, and obtain accurate position information of the trauma model in the human anatomical structure model through coordinate transformation after the alignment;

[0014] The plan generation module is used to build a medical knowledge base, generate a matching initial surgical plan based on each trauma model and the precise location information corresponding to the model and the medical knowledge base, verify and optimize the initial treatment plan through simulation, obtain a standard treatment plan, and store it in association with the trauma model.

[0015] Optionally, the training system construction module includes a scenario construction submodule, a communication platform construction submodule, and an interaction submodule;

[0016] The scene construction submodule is used to construct a three-dimensional model of a typical battlefield environment using three-dimensional modeling and render it to obtain a multi-dimensional virtual reality learning scene, wherein material properties and lighting parameters are adjusted by PBR physical rendering technology;

[0017] The communication platform construction submodule is used to construct a social discussion module and a real-time communication section; the social discussion module is used for students to interact with each other according to the preset communication section; the real-time communication section is used for real-time communication and interaction between students;

[0018] The interactive submodule is used to construct interactive scenarios and interactive rescue tasks.

[0019] Optionally, the interaction submodule includes a scene generation unit and a process generation unit;

[0020] The scene generation unit is used to fuse the trauma model with the anatomical structure model and combine it with the multi-dimensional virtual reality learning scene to obtain an interactive scene;

[0021] The task generating unit is used to generate an interactive rescue task.

[0022] Optionally, the task generation unit is used to extract a corresponding standard treatment plan according to the trauma model, and generate an interactive treatment task based on the standard treatment plan and the interactive scenario, in which the standard steps and precautions are split and embedded according to the links.

[0023] Optionally, the training module includes an environment control unit, a data acquisition submodule, and a training evaluation submodule;

[0024] The environment control unit is used to collect the behavior data of the trainees in the interactive scene in real time, analyze the behavior data using reinforcement learning, obtain the stress response level, and adjust the interactive scene parameters according to the stress response level;

[0025] The data acquisition submodule is used to obtain the trainee's operation trajectory and action through real-time tracking of motion capture, and obtain the trainee's operation data through real-time perception of force feedback of virtual reality equipment; the operation trajectory and action, operation data and operation process constitute the trainee's training data; wherein the operation data includes operation force, operation speed and operation angle;

[0026] The training evaluation submodule is used to obtain a comprehensive training score based on the training data, and obtain improvement suggestions based on the standard treatment plan, and synthesize the comprehensive training score and the improvement suggestions into a training report.

[0027] Optionally, the training evaluation submodule generates a three-dimensional treatment navigation model based on the standard treatment plan, trauma model and anatomical structure model, matches the operation trajectory and action with a pre-established virtual trauma model, and determines the degree of matching between the operation trajectory and action and the virtual trauma model; compares the operation trajectory and action with a preset operation process to obtain a standard score; obtains an accurate score based on the operation data; and calculates a weighted average of the accurate score, the standard score and the degree of matching to obtain a comprehensive training score.

[0028] Optionally, the comprehensive evaluation module includes an operation submodule, a data processing submodule, a model building submodule, and an evaluation submodule;

[0029] The operation submodule is used to operate in the interactive scene and obtain the trainee's operation data in real time;

[0030] The data processing submodule is used to clean the trainee training data;

[0031] The model building submodule is used to select a three-layer BP neural network to build a prediction model, input the processed student training data into the prediction model, and use the ReLU activation function for training;

[0032] The evaluation submodule is used to input the student's operation data into the trained prediction model, obtain the student's scores in multiple key steps, compare them with the standard scores, and obtain the level values ​​of each key step of the student; update the student's training plan according to the level value, comprehensive training score and number of training times.

[0033] Compared with the prior art, the present invention has the following advantages and technical effects:

[0034] The present invention can provide a three-dimensional and multi-dimensional learning environment, enabling trainees to observe and operate virtual trauma models and rescue scenes from different angles and levels, so as to deeply understand the trauma mechanism and treatment process; by constructing a trauma model library and a medical knowledge base, the association between trauma and treatment methods is realized, and rich learning resources and standardized treatment plans are provided for trainees, which is helpful to improve the trainees' professional knowledge and skills; the virtual reality training system online platform designed with microservice architecture supports resource sharing and remote collaborative training, allowing trainees to communicate and learn in a network environment, improving the flexibility and convenience of training; the interactive training tasks and real-time evaluation mechanism provided by the system can provide personalized feedback based on the trainees' operation data, helping trainees to timely understand their own operation level and improvement direction, effectively improving the pertinence and efficiency; through 3D modeling and physical rendering technology, this system constructs a realistic virtual reality learning scene, enhances the trainees' sense of immersion, enables trainees to train in a simulated battlefield environment, and improves the authenticity and practicality of the training; the comprehensive evaluation module can dynamically update the trainees' training plan based on the trainees' operation data and comprehensive training scores, realizes personalized training guidance, and helps trainees optimize their learning paths according to their own circumstances and improve learning effects; the construction of a communication platform supports real-time communication and social discussions among trainees, helps promote the sharing of rescue experience and the dissemination of knowledge, and builds a positive learning community; through motion capture and force feedback technology, the system can track and perceive the trainees' operations in real time, evaluate the accuracy and standardization of the operations, and provide improvement suggestions through the training evaluation sub-module to promote the improvement of trainees' operational skills. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0036] Figure 1 A schematic diagram of the system structure of an embodiment of the present invention;

[0037] Figure 2 This is a data processing flow chart of a training module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] Embodiment 1

[0041] like Figure 1-2 As shown, in this embodiment, a combat trauma teaching and training system based on MR technology is provided, including:

[0042] Model library building module, training system building module, training module, comprehensive evaluation module;

[0043] The model library construction module is used to construct a trauma model library and a medical knowledge library, and obtain the correlation between trauma and treatment methods based on the trauma model library and the medical knowledge library;

[0044] As a preferred embodiment, the model library construction module includes a trauma model library construction submodule, a model matching submodule, and a solution generation module;

[0045] The trauma model library construction submodule is used to obtain the anatomical result data of several trauma areas, divide them according to the trauma parts and trauma types, and construct the trauma model based on the divided anatomical result data using a three-dimensional reconstruction algorithm to obtain a trauma model library;

[0046] The model matching submodule is used to extract individual trauma models from the trauma model library in sequence, align the individual trauma models with the human anatomical structure model, and obtain the precise position information of the trauma model in the human anatomical structure model through coordinate transformation after alignment;

[0047] The plan generation module is used to build a medical knowledge base. Based on each trauma model and the precise location information corresponding to the model and the medical knowledge base, a matching initial surgical plan is generated. The initial treatment plan is verified and optimized through simulation to obtain a standard treatment plan, which is then stored in association with the trauma model.

[0048] Specifically, the anatomical data of the trauma area is first obtained through medical imaging equipment such as CT or MRI, and then the data is processed using 3D reconstruction algorithms such as Marching Cubes to construct a 3D model of the trauma area. According to the type of trauma, such as fracture, soft tissue injury, etc., the trauma model is registered with the preset human anatomical structure model using ICP, CPD and other registration algorithms, and the precise position information of the trauma model in the human anatomical structure model is obtained through coordinate transformation. Based on the geometric shape, position and other information of the trauma model, combined with the surgical specifications and expert experience in the medical knowledge base, the surgical plan is automatically generated through rule reasoning, decision tree and other algorithms, and the surgical process is simulated using virtual reality technology to verify the feasibility and safety of the surgical plan. If necessary, parameter optimization is performed to obtain the optimized surgical plan.

[0049] The training system building module is used to design an online platform for virtual reality training systems using a microservice architecture;

[0050] As a preferred embodiment, the training system construction module includes a scenario construction submodule, a communication platform construction submodule, and an interaction submodule;

[0051] The scene construction submodule is used to construct a 3D model of a typical battlefield environment using 3D modeling and render it to obtain a multi-dimensional virtual reality learning scene, in which material properties and lighting parameters are adjusted through PBR physical rendering technology;

[0052] The communication platform construction submodule is used to construct the social discussion module and the real-time communication section; the social discussion module is used for students to interact with each other according to the preset communication section; the real-time communication section is used for real-time communication and interaction between students;

[0053] The interactive submodule is used to construct interactive scenarios and interactive rescue tasks.

[0054] As a preferred embodiment, the interaction submodule includes a scene generation unit and a process generation unit;

[0055] The scene generation unit is used to fuse the trauma model with the anatomical structure model and combine it with a multi-dimensional virtual reality learning scene to obtain an interactive scene;

[0056] The task generation unit is used to generate interactive rescue tasks.

[0057] As a preferred embodiment, the task generation unit is used to extract the corresponding standard treatment plan according to the trauma model, generate an interactive treatment task based on the standard treatment plan and the interactive scene, and split and embed the standard steps and precautions in the interactive treatment task according to the links.

[0058] Specifically, Maya and other 3D modeling software are used to build high-precision 3D models of typical environments such as trenches, ruins, and the wild. For the constructed 3D models, the PBR physical rendering technology of the Unity3D engine is used to adjust the material properties, lighting parameters, etc. to achieve realistic rendering of the model and improve the visual quality of the scene. The rendered 3D model resources are imported into the online platform of the virtual reality training system, and the scene is rendered in real time in the browser using WebGL technology to build a multi-dimensional Web virtual reality learning scene. In the virtual reality learning scene, the explosion effect is simulated by the particle system, the shooting effect is achieved by ray detection, and the volume atomization technology is used to simulate battlefield smoke to enhance the immersion of the scene. The system collects the interactive behavior data of the trainees in the virtual reality learning scene in real time, such as head movement trajectory, gestures, etc., and inputs these data into the environment adaptation model and special effects optimization model based on deep learning. The dynamic adjustment values ​​of the scene environment parameters and special effects presentation are obtained through model reasoning to achieve personalized adaptation of the scene. Machine learning algorithms, such as SVM and decision trees, are used to analyze the user's behavior data in the virtual environment. By extracting the user's heart rate, breathing, behavior and other characteristics, the user's stress response level is obtained, such as the stress response level is 8. According to the user's stress response level, the virtual scene is dynamically adjusted. For example, when the stress response level is high, the intensity of special effects such as explosions and shooting is reduced to avoid excessive stress reactions in users; when the stress response level is low, the scene difficulty is appropriately increased to improve the training intensity.

[0059] The virtual reality training system online platform realizes training resource sharing and remote collaborative training, and trainees discuss rescue experiences through online communication.

[0060] The business needs of the virtual reality training platform are obtained, and the virtual reality training platform is built using WebVR technology. The virtual reality training platform is implemented based on the Web, and virtual scenes are rendered through browsers to achieve cross-platform access. In response to the needs of resource sharing and collaborative training, a distributed resource management system is built to uniformly manage and schedule virtual reality training resources to achieve resource sharing and collaborative training between at least two training nodes. In response to the needs of real-time communication, WebSocket technology is used to achieve real-time communication and interaction between trainees. If the network connection is unstable, the preset disconnection and reconnection mechanism is enabled. According to the needs of training communication and experience sharing, a social discussion module is built to support a preset number of communication and interaction forms between trainees to promote the sharing of rescue experience. In response to the needs of system scalability, a virtual reality training platform is designed using a microservice architecture, and the system functions are divided into multiple independent microservices to achieve modularization and scalability of the system. In response to the needs of trainees' skill improvement, machine learning algorithms are introduced, clustering algorithms are used to identify trainees' skill levels, and recommendation algorithms are used to recommend personalized training content and learning paths to trainees to promote the improvement of rescue skills. According to the needs of immersive training experience, realistic virtual reality training scenes and interactive training tasks are constructed through three-dimensional modeling and animation production technology to provide immersive training experience and enhance training effects.

[0061] Specifically, when WebVR technology is used to build a virtual reality training platform, three-dimensional scene rendering and interactive design can be performed based on WebGL libraries such as Three.js. In response to the needs of resource sharing and collaborative training, a resource management service based on distributed cache systems such as Redis can be built to achieve resource synchronization and load balancing between different training nodes. In order to achieve real-time communication and interaction between trainees, WebSocket technology can be used to achieve two-way communication, and a disconnection reconnection mechanism can be designed to automatically reestablish the connection when the network is unstable. In response to the needs of training communication and experience sharing, a social discussion module based on front-end frameworks such as Vue.js can be built to support multiple communication forms such as text chat, voice call, and screen sharing. In order to improve the scalability of the system, microservice architectures such as Spring Cloud can be used to divide system functions into multiple independent microservices, such as resource management services, training scene services, social services, etc., to achieve modularization and flexible expansion of the system. At the same time, through three-dimensional modeling and animation production tools such as Unity3D, realistic virtual reality training scenes and interactive training tasks can be built to provide an immersive training experience and improve training effects.

[0062] The training module is used to conduct training according to the virtual reality training system online platform and obtain a comprehensive training score according to the training data;

[0063] As a preferred embodiment, the training module includes an environment control unit, a data acquisition submodule, and a training evaluation submodule;

[0064] The environment control unit is used to collect the trainees' behavior data in the interactive scene in real time, analyze the behavior data using reinforcement learning, obtain the stress response level, and adjust the interactive scene parameters according to the stress response level;

[0065] The data acquisition submodule is used to obtain the trainee's operation trajectory and movements through real-time tracking of motion capture, and obtain the trainee's operation data through real-time perception of force feedback from virtual reality equipment; the operation trajectory and movement, operation data and operation process constitute the trainee's training data; among which, the operation data includes operation force, operation speed and operation angle;

[0066] The training evaluation submodule is used to obtain a comprehensive training score based on the training data, obtain improvement suggestions based on the standard treatment plan, and synthesize the comprehensive training score and improvement suggestions into a training report.

[0067] As a preferred embodiment, the training evaluation submodule generates a three-dimensional treatment navigation model based on the standard treatment plan, trauma model and anatomical structure model, matches the operation trajectory and action with the pre-established virtual trauma model, and determines the degree of matching between the operation trajectory and action and the virtual trauma model; compares the operation trajectory and action with the preset operation process to obtain a standard score; obtains an accurate score based on the operation data; and calculates the weighted average of the accurate score, the standard score and the matching degree to obtain a comprehensive training score.

[0068] Specifically, the surgical plan, trauma model and anatomical structure model are integrated to generate a three-dimensional rescue navigation model that contains information such as surgical path, surgical instruments, and surgical steps.

[0069] Specifically, in the virtual reality learning scenario, the physics engine and collision detection algorithm are used to realize the interactive operation of the trauma model. Students use the virtual reality headset and handle to select, move, zoom and other operations to explore the internal structure of the trauma.

[0070] Obtain the three-dimensional data of the trauma model, build a virtual reality scene, and import the trauma model into the scene. Use a physical engine to simulate the physical properties of the trauma model, such as mass, density, elasticity, etc., so that it can show real physical behavior in the virtual environment. Apply a collision detection algorithm to determine the collision between the trainee's virtual handle and the trauma model, and trigger corresponding interaction events based on the collision results. Obtain the trainee's operation input through the head display handle, such as selection, movement, scaling, etc., and transform and update the trauma model accordingly according to the operation type. Use graphics rendering technology to render the visual effect of the trauma model in real time, dynamically adjust the rendering perspective according to the changes in the trainee's perspective, and provide an immersive exploration experience. Use a layered rendering method to display the different organizational structures of the trauma model in layers. Trainees can control the display and hiding of the layers through the handle and explore the internal structure layer by layer. Apply force feedback technology to give trainees real tactile feedback based on the contact between the virtual handle and the model when they interact with the trauma model, enhancing the realism of the interaction.

[0071] Specifically, the 3D data of the trauma model was obtained and imported into the Unity engine using the FBX format to build a virtual reality scene. The physical properties of the trauma model were simulated using the PhysX physics engine, and the density of the model was set to 0.5 g / cm 3 , elastic coefficient of 4.2, friction coefficient of 3.7, to achieve realistic physical effects. The AABB bounding box algorithm is used for collision detection. When the virtual handle intersects with the bounding box of the trauma model, a collision event is triggered. The collision force is calculated according to the normal vector of the collision point, and the force is fed back to the handle to produce a vibration effect. The real-time position and rotation data of the HTC Vive headset and the handle are obtained through the OpenVR framework. The trainee's operation intention is judged according to the button status of the handle, such as pressing the trigger button to select the model, pressing and holding the touchpad to move the model, and double-clicking the touchpad to scale the model. The Graphics DrawMesh method of Unity is used to render the trauma model in real time. The camera angle is dynamically adjusted according to the position and rotation of the headset. MSAA 4X anti-aliasing is turned on when rendering each frame to improve the picture quality. The skin, muscle, bone and other tissue structures of the trauma model are rendered in layers through the Layer function of Unity. Trainees can control the display status of each layer through the menu buttons of the handle and analyze the internal structure layer by layer. When the virtual handle collides with the trauma model, the vibration frequency and amplitude of the handle are controlled by Unity's Haptic Pulse method according to the normal vector and penetration depth of the collision point to simulate real tactile feedback. The frequency range is 20Hz to 200Hz and the amplitude range is 0 to 1.

[0072] For each type of trauma, a corresponding interactive treatment task process is designed. The interactive treatment task process includes hemostasis, bandaging, fixation, and transportation. Standard operation steps and precautions are embedded in the interactive treatment task process. A trainee operation behavior collection and analysis model is established to collect trainee operation data in real time. The operation data includes operation trajectory, force, and completion time. By comparing with the preset standard operation data, the trainee's operation is evaluated and feedback is given.

[0073] Specifically, according to the virtual reality learning scenario and trauma type, a trauma model library containing 3D models and parameters of different trauma types is established. For example, for fracture trauma, 3D models of 10 common fracture types including closed fracture and open fracture are established, and each model contains parameters such as fracture site, fracture degree, and surrounding soft tissue injury. For each type of trauma, a corresponding interactive rescue task process is designed, including hemostasis, bandaging, fixation, and transportation. Taking open fracture as an example, a rescue process including compression hemostasis, debridement and disinfection, fracture reduction, external fixation, and bandaging is designed, and standardized operation steps and precautions are embedded in the process. For example, in the compression hemostasis step, it is prompted to "directly compress the wound with sterile dressings for 5-10 minutes." A model for collecting and analyzing trainees' operation behaviors is established to collect trainees' operation data in real time. The 3D coordinate data of trainees' hand movements is collected through the sensors of the virtual reality equipment to record the operation trajectory; the pressure data applied by the trainees is collected through the force feedback device to record the force; the system automatically records the start and end time of each operation step and calculates the completion time. The dynamic time warping algorithm (DTW) is used to compare the trainee's operation trajectory with the preset standard trajectory and calculate the similarity score; the collected pressure data is compared with the preset standard pressure range to evaluate whether the force is appropriate; the completion time is compared with the preset standard time to evaluate the operation speed. Based on the evaluation results of various indicators, the system automatically generates scores and improvement suggestions, such as "the force is insufficient when compressing to stop bleeding, it is recommended to increase the pressure to 2-3 kg", to achieve real-time evaluation and feedback on the trainee's operation. In this way, the virtual reality system can provide trainees with a highly simulated and interactive trauma treatment training experience, effectively improving the trainees' actual operation capabilities.

[0074] Utilizing the motion capture and force feedback functions of the virtual reality headset and handles, the trainees’ operations in interactive rescue tasks can be tracked and perceived in real time to evaluate the accuracy and standardization of their operations.

[0075] The data of the head display and handles of the trainees when they perform trauma treatment operations in the virtual reality environment are obtained, and the operation trajectory and movements of the trainees are tracked in real time through motion capture technology, and the operation trajectory and movements are matched with the pre-established virtual trauma model; the machine learning algorithm is used to determine the degree of matching between the operation trajectory and movement and the virtual trauma model, and according to the degree of matching, the force feedback function of the virtual reality device is used to perceive and feedback the trainees' operations in real time; the key characteristic parameters of the operation trajectory and movement are extracted, and the key characteristic parameters include operation speed, force and angle, which are used to evaluate the accuracy of the operation; the operation trajectory and movement are compared with the preset standard operation process, and the pattern recognition algorithm is used to determine the standardization of the operation trajectory and movement; the evaluation results of the degree of matching, accuracy and standardization are comprehensively evaluated, and the weighted average method is used to give an overall score to the trainees' trauma treatment operations; based on the overall score, a personalized training feedback report is generated, which points out the advantages and disadvantages in the operation and provides targeted improvement suggestions.

[0076] Specifically, in the virtual reality trauma treatment training system, the trainee operation data is collected through the head display and handle, and the Kinect depth camera and inertial measurement unit are used to track the trainee's movements in real time with an accuracy of up to millimeter level. The collected three-dimensional coordinate point cloud data is registered with the pre-established trauma model, and the iterative closest point algorithm is used, and the registration error is less than 0.5 cm. The support vector machine algorithm is used to determine the matching degree between the operation and the model. When the matching degree exceeds 85%, the force feedback device is triggered to generate corresponding tactile feedback. The characteristic parameters such as speed, acceleration, and angle are extracted from the operation trajectory to construct a feature vector. The feature vector is input into the pre-trained convolutional neural network to identify the standardization of the operation with an accuracy rate of up to 92%. The fuzzy comprehensive evaluation method is used to perform a weighted average of the matching degree, accuracy, and standardization of the operation to obtain an overall score of 0-100 points. Based on the scoring results, the system automatically generates a personalized feedback report, points out the specific problems of the trainees in the hemostasis bandage and other links, and gives targeted suggestions. For example, the system detects that the bandage of the trainee is not tight enough when bandaging, and will prompt in the report that "the bandage needs to be wound with increased tension to achieve the hemostasis effect." In this way, the system can comprehensively evaluate the trainees' operational skills and provide data support for improving training.

[0077] The comprehensive evaluation module is used to obtain the trainee's operation data, obtain the level value of each key step of the trainee according to the trainee's operation data, and update the trainee's training plan based on the level value and the comprehensive training score;

[0078] As a preferred embodiment, the comprehensive evaluation module includes an operation submodule, a data processing submodule, a model building submodule, and an evaluation submodule;

[0079] The operation submodule is used to operate in the interactive scene and obtain the trainee's operation data in real time;

[0080] The data processing submodule is used to clean the trainee training data;

[0081] The model building submodule is used to select a three-layer BP neural network to build a prediction model, input the processed student training data into the prediction model, and use the ReLU activation function for training;

[0082] The evaluation submodule is used to input the trainee's operation data into the trained prediction model, obtain the trainee's scores in multiple key steps, compare them with the standard scores, and obtain the level value of each key step of the trainee; update the trainee's training plan according to the level value, comprehensive training score and number of training times.

[0083] Specifically, the trainee's operation data is input into the trained model, and the output results are calculated through the forward propagation algorithm to determine the trainee's scores in key steps such as trauma treatment, hemostatic dressing, and airway management, and compared with the standard scores to determine the trainee's strengths and weaknesses. For example, the model analysis found that the trainee's accuracy in airway intubation operations was low, and the average time was 35% longer than the standard time. Through knowledge graph reasoning, the system automatically generates a training report, suggesting that trainees strengthen their learning of anatomical knowledge and repeatedly practice intubation operations in virtual scenes to improve the accuracy and speed of operations. In subsequent training, the trainee's operation data is continuously collected, and the model is incrementally trained using the gradient descent algorithm to continuously improve the generalization ability of the model. At the same time, according to the trainee's training stage and ability level, the difficulty and complexity of the virtual scene are dynamically adjusted, and targeted training projects are intelligently recommended to achieve personalized training guidance and optimize the trainee's training effect.

[0084] The model library construction module, the training system construction module, the training module, and the comprehensive evaluation module are connected in sequence.

[0085] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A war trauma teaching and training system based on MR technology, characterized in that: include: Model library building module, training system building module, training module, comprehensive evaluation module; The model library construction module is used to construct a trauma model library and a medical knowledge library, and obtain the association between trauma and treatment methods based on the trauma model library and the medical knowledge library; The training system building module is used to design a virtual reality training system online platform using a microservice architecture; The training module is used to perform training according to the virtual reality training system online platform and obtain a comprehensive training score according to the training data; The comprehensive evaluation module is used to obtain the trainee's operation data, obtain the level value of each key step of the trainee according to the trainee's operation data, and update the trainee's training plan based on the level value and the comprehensive training score; The model library construction module, the training system construction module, the training module, and the comprehensive evaluation module are connected in sequence.

2. The MR-based combat trauma teaching and training system according to claim 1 is characterized in that: The model library construction module includes a trauma model library construction submodule, a model matching submodule, and a solution generation module; The trauma model library construction submodule is used to obtain anatomical result data of several trauma areas, divide them according to the trauma parts and trauma types, and construct trauma models based on the divided anatomical result data using a three-dimensional reconstruction algorithm to obtain a trauma model library; The model matching submodule is used to sequentially extract individual trauma models from the trauma model library, align the individual trauma models with the human anatomical structure model, and obtain accurate position information of the trauma model in the human anatomical structure model through coordinate transformation after the alignment; The plan generation module is used to build a medical knowledge base, generate a matching initial surgical plan based on each trauma model and the precise location information corresponding to the model and the medical knowledge base, verify and optimize the initial treatment plan through simulation, obtain a standard treatment plan, and store it in association with the trauma model.

3. The combat trauma teaching and training system based on MR technology according to claim 1 is characterized in that: The training system construction module includes a scenario construction submodule, a communication platform construction submodule, and an interaction submodule; The scene construction submodule is used to construct a three-dimensional model of a typical battlefield environment by using three-dimensional modeling and render it to obtain a multi-dimensional virtual reality learning scene, wherein material properties and lighting parameters are adjusted by PBR physical rendering technology; The communication platform construction submodule is used to construct a social discussion module and a real-time communication section; the social discussion module is used for students to interact with each other according to the preset communication section; the real-time communication section is used for real-time communication and interaction between students; The interactive submodule is used to construct interactive scenarios and interactive rescue tasks.

4. The MR-based combat trauma teaching and training system according to claim 3 is characterized in that: The interaction submodule includes a scene generation unit and a process generation unit; The scene generation unit is used to fuse the trauma model with the anatomical structure model and combine it with the multi-dimensional virtual reality learning scene to obtain an interactive scene; The task generating unit is used to generate an interactive rescue task.

5. The MR-based combat trauma teaching and training system according to claim 4 is characterized in that: The task generation unit is used to extract the corresponding standard treatment plan according to the trauma model, and generate an interactive treatment task based on the standard treatment plan and the interactive scene. In the interactive treatment task, the standard steps and precautions are split and embedded according to the links.

6. The combat trauma teaching and training system based on MR technology according to claim 1 is characterized in that: The training module includes an environment control unit, a data acquisition submodule, and a training evaluation submodule; The environmental control unit is used to collect the trainee's behavior data in the interactive scene in real time, analyze the behavior data using reinforcement learning, obtain the stress response level, and adjust the interactive scene parameters according to the stress response level; The data acquisition submodule is used to obtain the trainee's operation trajectory and action through real-time tracking of motion capture, and obtain the trainee's operation data through real-time perception of force feedback of virtual reality equipment; the operation trajectory and action, operation data and operation process constitute the trainee's training data; wherein the operation data includes operation force, operation speed and operation angle; The training evaluation submodule is used to obtain a comprehensive training score based on the training data, and obtain improvement suggestions based on the standard treatment plan, and synthesize the comprehensive training score and the improvement suggestions into a training report.

7. The MR-based combat trauma teaching and training system according to claim 6 is characterized in that: The training evaluation submodule generates a three-dimensional treatment navigation model according to the standard treatment plan, the trauma model and the anatomical structure model, matches the operation trajectory and action with the pre-established virtual trauma model, and determines the matching degree between the operation trajectory and action and the virtual trauma model; Compare the operation track and action with the preset operation process to obtain a standard score; Obtaining an accurate score based on the operational data; Calculate the weighted average of the accurate score, standard score and matching degree to obtain the comprehensive training score.

8. The MR-based combat trauma teaching and training system according to claim 4 is characterized in that: The comprehensive evaluation module includes an operation submodule, a data processing submodule, a model building submodule, and an evaluation submodule; The operation submodule is used to operate in the interactive scene and obtain the trainee's operation data in real time; The data processing submodule is used to clean the trainee training data; The model building submodule is used to select a three-layer BP neural network to build a prediction model, input the processed student training data into the prediction model, and use the ReLU activation function for training; The evaluation submodule is used to input the student's operation data into the trained prediction model, obtain the student's scores in multiple key steps, compare them with the standard scores, and obtain the level values ​​of each key step of the student; update the student's training plan according to the level value, comprehensive training score and number of training times.

Citation Information

Cited By

  • Pediatric traditional Chinese medicine acupoint massage teaching and training system and method

    CN119400040A

  • A pediatric TCM acupoint massage teaching and training system and method

    CN119400040B

  • Method, system and equipment for analyzing operation training data of laparoscopic surgery robot

    CN119648487A

  • Laparoscopic surgery teaching auxiliary training system

    CN119649668A

  • A laparoscopic surgery teaching and assisting training system

    CN119649668B