VR multi-person cooperative training system for preoperative preparation

Abstract

The invention discloses a VR multi-person cooperative training system for preoperative preparation, and the system comprises a virtual scene construction module which is used for constructing a virtual scene, a virtual team module is electrically connected with the virtual scene construction module, and the virtual team module is used for constructing a virtual role template. The training task module carries out deep analysis on the operation field knowledge system through a big data technology, sorts out key points, and carries out training and assessment on trainees; the collaborative interaction module is electrically connected with the training task module, and the collaborative interaction module is used for adaptive interaction among various dependent devices or multi-role operation interaction or task allocation and adjustment of roles in a team; by means of laser scanning and high-resolution image acquisition, massive real operating room data are collected, a static milliliter graduation and dynamic fresh and vivid virtual space is constructed through professional 3D modeling, and students are wrapped with super-strong immersion at the moment of stepping in by wearing VR equipment and are systematically put into preoperative training.

Description

Technical Field

[0001] The present invention relates to the technical field of surgical collaborative training, and more specifically, to a VR multi-person collaborative training system for preoperative preparation. Background Art

[0002] In the global healthcare sector, the number of surgeries has shown a steady growth trend. According to authoritative medical industry statistics, in China alone, the average annual number of surgeries has remained at the 10 million level in recent years, and has been rising steadily year by year. This growth trend is mainly attributed to several key factors: First, the aging of the population is becoming more and more obvious, the physiological functions of the elderly are declining, and the risk of various chronic diseases has increased significantly, resulting in an increase in the demand for surgical intervention; second, with the continuous emergence and transformation of medical research results, cutting-edge methods such as minimally invasive technology and precision medicine continue to expand surgical indications, making more diseases possible to be cured by surgery.

[0003] As the core hub of medical services, some large comprehensive medical institutions have an astonishing daily average surgical capacity, and the peak of some top hospitals can even reach 1,000. According to the public data of the top ten hospitals in China in terms of surgical volume in 2023, high-intensity surgical task arrangements have become the norm, which undoubtedly puts the medical team in a high-load operation state for a long time, exacerbating the tension in professional doctor resources.

[0004] In this realistic scenario, the number of medical students and standardized training students (regular training students) as the new reserve force of the medical industry is quite considerable. However, from the perspective of capacity building and career development, they are generally trapped in the dilemma of lack of practical experience and insufficient experience in teamwork. The traditional "master-apprentice inheritance" model of old and new is limited by the limited energy and time of senior doctors, and it is difficult to achieve all-round and systematic guidance for multiple novices, resulting in limited training effectiveness. At the same time, the current mainstream surgery-related training methods are still relatively traditional, focusing more on one-way teaching of theoretical knowledge, practical video observation, and isolated single-person practice using physical models. The training model lacks immersion and interactivity.

[0005] In fact, preoperative preparation for surgery is a highly complex systematic project, involving close cooperation among multiple professional roles such as doctors, nurses, and anesthesiologists. The various links are closely intertwined, and the process is complicated and has many variables. In contrast, the existing virtual reality (VR) training system still has significant scientific shortcomings in key dimensions such as accurately replicating the preoperative preparation process, deeply stimulating the communication and collaboration efficiency of team members, and flexibly responding to complex emergencies. It is difficult to meet the high standards of preoperative preparation for team collaboration.

[0006] In view of this, this patent proposes a VR multi-person collaborative training system for preoperative preparation. With the help of virtual reality technology and high-speed network technology, the system can accommodate multiple novice trainees to participate in training at the same time with powerful multi-user concurrent processing capabilities and real-time data transmission technology. With the help of high-fidelity 3D modeling, physical simulation engine and other technical means, the system can realistically simulate various scenes of preoperative preparation, from the layout of the operating room environment, medical equipment operation feedback, to the dynamic changes of patients' vital signs, all of which can be reproduced with high precision, building an immersive virtual rehearsal space for trainees, prompting them to accelerate the accumulation of practical training experience through repeated practice. Summary of the invention

[0007] In order to solve at least one of the above technical problems, the present invention proposes a VR multi-person collaborative training system for preoperative preparation to solve the technical problems of lack of preoperative preparation training practice for surgeons, lack of practical training for team collaboration, and poor response ability of emergency groups, resulting in weak collaborative combat capability during surgery.

[0008] The first aspect of the present invention provides a VR multi-person collaborative training system for preoperative preparation, comprising:

[0009] A virtual scene construction module, which is used to construct a virtual scene. The virtual scene includes a dynamic scene and a static scene. The dynamic scene includes: light and shadow scenes, sound scenes and surgical operation animations inside an operating room, anesthesia preparation room and preoperative ward;

[0010] The static scenes include the structure, material and layout environment of the operating room, anesthesia preparation room, preoperative ward and various medical equipment;

[0011] A virtual team module, the virtual team module is electrically connected to the virtual scene construction module, the virtual team module is used to construct a virtual role template, and to set multiple roles and form at least one team according to the role template, wherein a team includes at least two roles;

[0012] The training task module uses big data technology to deeply analyze the knowledge system in the surgical field, sort out the key points, and then build a theoretical learning framework of three levels: basic, advanced, and high-level in a virtual scene, forming diversified learning resources, and pushing targeted emergency special training tasks to train and assess trainees;

[0013] A collaborative interaction module, the collaborative interaction module is electrically connected to the training task module, and the collaborative interaction module is used for adaptive interaction between multiple dependent devices or multiple role operation interaction or task allocation and adjustment of roles in a team.

[0014] In a preferred embodiment of the present invention, the virtual scene construction module includes a virtual space module and a shared space module, the virtual space module is electrically connected to the shared space module, the shared space module includes multiple VR device all-in-ones and a router, a local area network is established between the multiple VR device all-in-ones and the router, the VR device all-in-one starts a high-precision scanning process to capture real-world scene features and generate an independent space map that accurately maps virtual and reality.

[0015] In a preferred embodiment of the present invention, the virtual team module performs role setting according to the job segmentation of the actual surgical scene, and the role setting includes the general surgery chief surgeon, the cardiology assistant physician, the anesthesiologist of the anesthesia department, the operating room circulating nurse or the instrument nurse.

[0016] In a preferred embodiment of the present invention, the training task module includes a training server, and the training server is electrically connected to the collaborative interaction sub-module, the voice dialogue sub-module, the real-time operation message sub-module and the gesture and handle interaction sub-module. The training server is electrically connected to the preoperative training sub-module, and the preoperative training sub-module realizes signal connection with the collaborative interaction sub-module, the voice dialogue sub-module, the real-time operation message sub-module and the gesture and handle interaction sub-module.

[0017] In a preferred embodiment of the present invention, the collaborative interaction module includes a simulation server, a team role management submodule, a task allocation submodule and a collaborative task submodule.

[0018] In a preferred embodiment of the present invention, it also includes an AI auxiliary module, which is electrically connected to the training task module. The AI ​​auxiliary module includes an AI simulated nurse server, an AI player NPC and a task completion sub-module. The AI ​​auxiliary module is used for intelligent role switching and with the help of motion capture and voice recognition technology of VR equipment, real-time monitoring of the movements of surgical team roles and voice command auxiliary tool delivery, as well as preoperative item sorting and disinfection assistance.

[0019] In a preferred embodiment of the present invention, an evaluation and feedback module is further included, wherein the evaluation and feedback module collects multiple data indicators from multiple dimensions, analyzes them through AI algorithms, outputs team collaborative evaluation results, and generates customized reports;

[0020] The data indicators include the accuracy of the instrument retrieval sequence, the frequency of member communication and the completeness of information transmission.

[0021] The above technical solution of the present invention has the following advantages compared with the prior art:

[0022] (1) This application uses laser scanning and high-resolution image acquisition to collect a large amount of real operating room data. Through professional 3D modeling, physics-based rendering, real-time physical simulation, and high-fidelity sound effects, it constructs a virtual space with precise static and vivid dynamics. The moment students put on VR equipment and step into the room, they will be enveloped by a strong sense of immersion and fully devote themselves to preoperative training.

[0023] (2) The present invention relies on big data and artificial intelligence algorithms to deeply analyze the knowledge systems of various medical disciplines, break down the knowledge barriers between anatomy, pharmacology, and surgical practice, accurately disassemble and reorganize complex knowledge points according to the preoperative process, and seamlessly integrate them into all aspects of training in an intelligent push mode, so that students can efficiently absorb knowledge nutrients.

[0024] (3) This application utilizes the high controllability of virtual reality scenes to trigger dozens of preset emergency surgical situations, such as acute massive bleeding and anesthesia failure, with one click. Combined with environmental simulation and AI-driven dynamic changes in patient vital signs, trainees are deeply immersed in it, honing their emergency response, and improving their psychological and practical abilities.

[0025] (4) Through real-time collection and analysis of training data, we can accurately identify students’ shortcomings and provide personalized suggestions for improvement, so as to cultivate professional preoperative preparation talents with solid theoretical knowledge and strong practical skills for the medical industry.

[0026] (5) This application is connected with clinical practice, so that students can take their training results with them into the clinic. At the same time, a reverse optimization system for special clinical cases is established to ensure that the training is in line with actual practice and what they have learned is used. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, some of the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 It is a block diagram of a VR multi-person collaborative training system for preoperative preparation according to an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of map scanning and sharing according to an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of a local area network composed of multiple VRs and routers according to an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of multiple people training in the same space according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0033] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0034] Embodiment 1

[0035] See also Figure 1-Figure 4 As shown, the present invention proposes a VR multi-person collaborative training system for preoperative preparation. The VR multi-person collaborative training system for preoperative preparation integrates cutting-edge technology to build a set of precise and efficient training closed loops. In terms of virtual scene construction, high-resolution image acquisition and other technologies are used to construct the physical virtual space of the operating room and the details of medical equipment through software. After being carved by professional 3D modeling software and combined with physical-based rendering (PBR) technology, the static scene is given a hyper-realistic texture; with the help of real-time physical simulation algorithms and high-fidelity sound synthesis, fresh vitality is injected into the dynamic scene, and the actual operation feel of the operation and the environmental stress changes under sudden conditions are realistically reproduced; including:

[0036] Virtual scene construction module,The virtual scene construction module is used to construct virtual scenes.,Virtual scenes include dynamic scenes and static scenes.,Dynamic scenes include: light and shadow scenes, sound scenes and,surgical operation animations inside the operating room,,anesthesia preparation room, and preoperative ward.

[0037] Specifically, the virtual scene construction module creates a highly realistic and dynamic virtual environment for VR multi-person collaborative training before surgery. In this environment, doctors can become familiar with the real situation of the surgical site while training.

[0038] The static scene construction is as follows:

[0039] Data collection: We have in-depth cooperation with many tertiary hospitals to obtain detailed design drawings of operating rooms, anesthesia preparation rooms, preoperative wards and other places, including floor plans, cross-sections and installation layouts of various equipment. At the same time, we collect the material, color, light and shadow parameters of the real environment of the operating room on site, use high-definition cameras to build 3D images of the walls, floors, ceilings and various medical equipment in the operating room, and collect a large amount of hospital scene data with an accuracy of up to 1 mm to ensure the restoration of more details.

[0040] 3D modeling: Based on the collected data, professional 3D modeling software such as Maya, 3dMax, Blender, etc. are selected to build the basic framework first, accurately outline the room outline, door and window positions according to the actual size, and then gradually refine the internal facilities. For medical equipment, not only the appearance is shaped, but also the internal key mechanical structure and circuit layout are reproduced according to the equipment manual and disassembly data, the optical lens barrel of the surgical microscope, the arrangement of the bulbs of the shadowless lamp and the reflector design. The number of model faces is as high as millions, achieving fine restoration.

[0041] Material and texture processing: High-resolution real material images are processed by Photoshop, ZBrush and other software to give realistic textures to objects in the virtual scene. The reflectivity of the operating room wall tiles, the texture wrinkles of the operating table leather, and the metallic luster of the stainless steel instruments are all consistent with the characteristics of real materials. Then, by baking light maps, the diffuse reflection effect of ambient light on the material is simulated, making the scene visual effect more natural.

[0042] UI interface design and integration: Based on the actual operating habits of medical personnel, a simple and intuitive user interface is designed. The virtual UI panel in the operating room imitates the layout of the real equipment control panel, and clearly presents the functional divisions such as equipment switch, parameter adjustment, and medical record viewing. The button size and spacing are ergonomic, which is convenient for students to accurately touch and operate in the VR environment. HTML5 and CSS3 technology are used for front-end design, and then seamlessly integrated with 3D scenes through scripting languages ​​to ensure smooth interaction. At the same time, exclusive UI interfaces are customized for different roles (doctors, nurses, anesthesiologists) to display their respective key information and improve the efficiency of information acquisition.

[0043] Scene integration optimization: Integrate each built model and material into the game engine, such as Unity or UnrealEngine, adjust the collision relationship and hierarchy order between models, and optimize lighting baking and shadow calculation. Through LOD (level of detail) technology, the model accuracy is automatically switched according to the user's viewing angle, while ensuring the visual effect, the rendering efficiency is greatly improved to ensure the smooth operation of the scene.

[0044] The dynamic scene simulation is as follows:

[0045] Physical simulation: The latest VR physics engines such as Hapt i cs / OpenXR give objects in virtual scenes real physical properties. The operation of surgical instruments strictly follows the rules of gravity, friction, and elasticity. When holding a 150-gram scalpel, you can feel the grip resistance of 0.5-1N in your hand. When cutting simulated human soft tissue, the resistance will dynamically change between 2-5N according to the cutting depth; when simulating liquid flow, the flow rate and pressure changes during syringe suction and injection are accurately calculated. When injecting saline at a rate of 5 ml / s, you can see that the flow trajectory of the medicine in the catheter is natural and smooth, and the pressure simulation error is controlled within 5%; human tissue also simulates elasticity and toughness. When simulating suturing operations, the tissue will produce deformation feedback that conforms to physiological characteristics, and the stretching deformation amplitude can reach 1-3 cm.

[0046] Light and sound simulation: Utilize the powerful lighting system of the game engine to dynamically simulate the multi-angle illumination effect of the surgical lamp. When the surgical lamp is adjusted from vertical illumination to 30° inclined illumination, the shadow area of ​​the surgical area will increase by about 30%, and the blurriness of the shadow edge will increase by 20%, simulating the gradual process of natural light penetrating from the window. Depending on the time period, the light intensity can vary between 500-1500lux. With a professional sound effect library, the volume, tone, and reverberation are adjusted in real time according to the distance of the sound source and the spatial environment. The regular beeping sound of the monitor can reach 60 decibels within 1 meter from the sound source, and decays to about 30 decibels 3 meters away. The airflow sound of the anesthesia machine and the collision sound of surgical instruments are intertwined to enhance the immersive experience.

[0047] Animation design and implementation: Rich animation sequences are produced based on the preoperative preparation process and actual surgical operation specifications. From the advancement and push-out of the patient transfer bed, the standardized placement of surgical instruments, to the standard preoperative hand washing and disinfection actions of medical staff, all are presented with delicate animations. The use of bone binding and skinning technology makes the virtual character's movements smooth and natural, and with the facial capture technology, the character is given a vivid expression. The gentle expression of medical staff when comforting patients and the solemn expression in emergency situations can be accurately displayed. The frame rate of the animation is stable at 60fps to ensure smooth movements. The restoration degree of key actions exceeds 90%, and the real-time interaction of the scene injects fresh vitality into the virtual scene.

[0048] Emergency simulation: relying on massive medical knowledge graphs and clinical big data, carefully write emergency scripts. Comprehensively consider the patient's basic diseases, preoperative medication, family history and other factors, set a scientific and reasonable trigger probability. For example, patients with a specific history of allergies have a 15% probability of triggering an allergic reaction when using related allergenic drugs. After the trigger, the patient's vital signs data are updated synchronously, and key values ​​such as blood oxygen saturation, blood pressure, and heart rate are displayed in real time on the virtual monitor. The normal blood oxygen saturation is 95%-100%, and the allergic reaction can be reduced to 80% within 30 seconds. Combined with the patient's skin discoloration, shortness of breath and other visual changes in appearance, a tense and realistic emergency atmosphere is created in all directions.

[0049] Static scenes include the structure, material and layout environment of the operating room, anesthesia preparation room, preoperative ward and various medical equipment;

[0050] A virtual team module, the virtual team module is electrically connected to the virtual scene construction module, and the virtual team module is used to construct a virtual role template, and to set multiple roles and form at least one team according to the role template, wherein a team includes at least two roles;

[0051] Specifically, the virtual team module is used to scientifically construct, meticulously manage, and flexibly optimize virtual surgical teams, helping trainees to deeply integrate into the team collaboration process and effectively improve their collaborative combat capabilities.

[0052] The virtual team module specifically includes:

[0053] Virtual character template creation

[0054] Multiple role settings: Based on the job segmentation of real surgical scenarios, more than 20 key roles are set, including general surgery surgeons, cardiology assistants, anesthesiologists, operating room circulating nurses, instrument nurses, etc. For each role, its job responsibilities, necessary skills and knowledge scope are deeply analyzed. Neurosurgeons need to master more than 15 complex cranial surgery processes and be familiar with key anatomical knowledge such as the distribution of intracranial nerves and the direction of blood vessels; instrument nurses must be familiar with the names, uses and delivery specifications of more than 200 surgical instruments to ensure that the role settings are highly consistent with clinical reality.

[0055] Attribute quantification: The professional skills, knowledge reserves, and experience level of each role are quantified and presented. Skill proficiency adopts a scoring system of 1-10 points. An experienced ophthalmologist can achieve 9 points for cataract surgery skills. Knowledge reserves are counted by the number of knowledge points. Endocrinology assistants need to be proficient in more than 100 knowledge points related to endocrine diseases. Experience points are accumulated according to the number of simulated surgical cases. The starting experience value of a novice circulating nurse is 0, and each completed simulation task is appropriately accumulated according to the difficulty of the task.

[0056] Student competency assessment and mapping

[0057] Ability test: When students log into the system for the first time, they will be faced with a comprehensive ability test. The theoretical knowledge section has 80 multiple-choice questions and 30 short-answer questions, which are answered within 90 minutes, covering basic medical knowledge and key points of specialist surgery; the practical section requires students to accurately operate 8 commonly used instruments in simulated scenarios, and record the operation time and error frequency; the emergency scenario simulates 5 emergencies to consider students' response strategies and reaction speed.

[0058] Role matching: Based on the test results, the intelligent algorithm accurately matches the trainee's ability data with the attributes of the virtual role template. If the trainee's surgical theory score exceeds 85 points, the instrument operation proficiency reaches 8 points, and the emergency response is quick, it is likely to match the assistant doctor role; if the theory is solid but the practical operation is a little slow, it may be suitable for the circulating nurse role, which initially anchors the trainee's role positioning in the virtual team.

[0059] Smart team building

[0060] Big data analysis: Collect a large amount of past student training data and deeply study the performance of teams with different personalities and skill combinations in various preoperative preparation tasks. The data shows that the combination of calm and introverted members and lively and extroverted members can cooperate well in comforting the nervous family members of patients; the team of members with fine operation skills and strong overall coordination ability can efficiently cope with the preoperative preparation process of complex surgeries.

[0061] Team formation algorithm: input the role matching results and personality test data of the current students, run the intelligent team formation algorithm, fully weigh the factors of skill complementarity and personality integration, and automatically generate a surgical team. The goal is to maximize the comprehensive synergy potential of each team, try to control the skill overlap rate between members to less than 25%, maintain the personality conflict index at a low level, and ensure reasonable division of labor and harmonious communication in the team.

[0062] Team dynamic adjustment and optimization

[0063] Training process monitoring: The system captures data from all aspects of the team's preoperative preparation training, including core indicators such as the order of members' operation coordination, information exchange frequency, and task completion time. It also records the average time it takes for the instrument nurse to hand the instrument to the surgeon during orthopedic surgery, and the frequency of the anesthesiologist's communication with the assistant on key anesthesia parameters, to generate a real-time visual data dashboard.

[0064] Dynamic adjustment strategy: After completing a certain number of simulation tasks (such as 10 times), the system automatically analyzes the team data. Once it detects that a team member's skills have improved significantly or there is a blockage in collaboration, the team structure will be adjusted immediately. For example, if member B's orthopedic surgery skills have improved, he will be assigned to a more difficult orthopedic surgery team; if team communication is sluggish, a communication expert will be introduced to optimize the team atmosphere and continuously upgrade the team's collaborative efficiency.

[0065] Role lock and continuous training

[0066] Role selection login: When students log in to the system, the interface presents a clear role selection entrance, listing various doctor and nurse position options, such as "general surgery chief surgeon", "cardiology assistant doctor", "operating room instrument nurse", etc. Students carefully select roles based on their own career plans and interests. Once selected, the role will be locked by default in the subsequent series of training courses to form a coherent training path.

[0067] Role-specific training: The system pushes customized training tasks and learning resources that are in line with the growth of the role based on the role selected by the trainee. Trainees who choose "anesthesiologist" will receive targeted training content such as the design of complex surgical anesthesia plans and new anesthetic drug application cases, along with professional academic literature and expert explanation videos, to encourage trainees to conduct in-depth research and continuous accumulation in the field of their exclusive roles, and steadily improve their professional skills and teamwork capabilities.

[0068] The training task module uses big data technology to deeply analyze the knowledge system in the surgical field, sort out the key points, and then build a theoretical learning framework of three levels: basic, advanced, and high-level in a virtual scene, forming diversified learning resources, and pushing targeted emergency special training tasks to train and assess students;

[0069] Specifically, the training module is the core part of the VR multi-person collaborative training system for preoperative preparation. It is committed to creating an immersive and practical training experience for the surgical team and comprehensively improving the team's professional quality in dealing with real surgical scenarios.

[0070] The theoretical study of virtual space is as follows:

[0071] Hierarchical knowledge architecture construction:

[0072] With the help of big data, we deeply analyze the knowledge system in the field of surgery, accurately sort out the key points, and then build a theoretical learning framework of three levels: basic, advanced, and high-level in the virtual space. The basic level focuses on general knowledge such as human anatomy, physiology, and pathology, integrates authoritative medical textbooks and popular science literature, and extracts more than 50 core knowledge points. The key structures of the human skeleton, muscles, organs, etc. are transformed into fine 3D models, with dynamic annotations and professional voice explanations. Students wear VR glasses and can interact through the handle and rotate 360° to view the details of the model. The system detects learning results in real time, and pushes reinforcement content in a loop for the parts that do not meet the standards until the accuracy rate reaches more than 90%.

[0073] The advanced level focuses on the specialized knowledge of preoperative preparation for common surgeries. Experts from major hospital departments have been combined to sort out about 300 key items such as indications and contraindications for various surgeries. Case immersion teaching is adopted to deeply disassemble real surgical cases, integrate knowledge points into explanations, and match them with in-depth commentary audio recorded by experts. Students put on VR glasses and feel as if they are at the scene of the operation. The average explanation time for each case is 15 minutes, which is convenient for students to deeply understand the key points.

[0074] The advanced level digs deep into complex surgeries and interdisciplinary knowledge, works with medical research teams, selects the latest academic achievements and cutting-edge research trends, and summarizes more than 200 knowledge points. Upload international top academic papers and high-definition recordings of professional seminars, and students write their thoughts after reading or watching. The system uses natural language processing technology to evaluate the degree of understanding, and pushes targeted expansion materials based on this. When wearing VR glasses to view the materials, you can also use the virtual note function to record your thoughts at any time.

[0075] Integration of diverse learning resources:

[0076] The basic level integrates a large number of visual learning resources. In addition to static 3D models, it also introduces microscopic process animations such as cell metabolism and blood circulation presented by particle special effects and fluid simulation technology. The total duration is over 20 hours, which is cut into a series of 3-5 minute short videos to fit fragmented learning. The popular science explanation videos are recorded by well-known medical popular science bloggers, who interpret obscure concepts in a humorous way. Each episode is accompanied by a small test in class. Students wear VR glasses to answer the questions and get instant feedback on their learning effects.

[0077] The advanced level collects real surgical case videos from world-renowned hospitals, and records the key links of preoperative preparation from all angles, with a total of more than 100 cases. Classified by surgical type and difficulty, the corresponding surgical experts' real-time comments are embedded in the audio, with a total duration of about 5 hours. Students can watch with VR glasses, switch perspectives freely, immerse themselves in the surgical atmosphere, and learn from the experts' experience.

[0078] The high-level level connects to professional medical databases, updates cutting-edge information in real time, and builds an online exchange forum for students. Students put on VR glasses to enter the virtual space of the forum, and use virtual images to communicate and share insights on complex knowledge, stimulate deep thinking, and improve the efficiency of knowledge internalization.

[0079] Intelligent learning path planning: When students log in for the first time, they will first start a comprehensive test covering knowledge points at all levels, and evaluate their knowledge reserves and learning ability based on the accuracy and speed of answering questions. If the score of the basic anatomy part is less than 40%, the system will prioritize the basic level human structure intensive course, with daily learning time recommendations and review interval reminders; for students with fast learning progress and excellent test scores, advanced content will be unlocked, and relevant academic activities and online seminars will be recommended. The path will be dynamically optimized based on learning data throughout the process, allowing students to easily start their exclusive learning journey by wearing VR glasses.

[0080] Integration of multidisciplinary knowledge

[0081] Cross-integration of knowledge: When designing training tasks, accurately anchor the interdisciplinary knowledge integration points. Taking the preoperative preparation for heart bypass surgery as an example, we work with expert teams from multiple departments such as cardiology, respiratory medicine, and laboratory to customize detailed knowledge integration plans. Trainees put on VR glasses to enter the virtual scene to view the patient's medical records. The system automatically associates the corresponding department knowledge modules. The angiography interpretation section of the cardiology department presents the analysis process and quantitative standards for the degree of coronary artery blockage. Trainees can manipulate the handle to zoom and rotate the image. The respiratory department's pulmonary function assessment section displays dynamic charts of the relationship between various indicators and postoperative respiratory support, and different parameters can be switched to observe changes. The coagulation function data interpretation area of ​​the laboratory department provides a calculation formula for adjusting the dosage of anticoagulants based on real-time data, prompting trainees to think across disciplines and weave a systematic knowledge network.

[0082] Contingency simulations include:

[0083] Accident scenario preset: The medical expert team deeply analyzed the massive clinical medical records and literature, sorted out more than 20 common preoperative accidents, and set different trigger probabilities to implant training tasks based on the frequency of occurrence and degree of harm. In the preoperative preparation task for cesarean section, considering the rarity and severity of amniotic fluid embolism, its trigger probability is set to 3%; in the preoperative preparation for hip replacement, given the relatively high incidence of antibiotic allergies, the trigger probability is set to about 5%. Using the random number generation algorithm, combined with the task execution progress and the trainee's operation steps, the accident scene is intelligently triggered to add training uncertainty to the trainees wearing VR glasses.

[0084] Signs and environment change in tandem: At the moment of an accident, the system background quickly calls the medical simulation algorithm to synchronously update the patient's vital signs data. When simulating an allergic reaction, the patient's blood pressure drops by 30-50mm Hg at a rate of about 0.5-1mm Hg per second within 1 minute based on the physiological model, and the heart rate soars to 120-150 beats per minute at an increase of 10-15 times per minute. The area of ​​erythema on the body surface spreads at a rate of 10%-15% per minute with the help of the image rendering algorithm; the operating room environment also responds synchronously, and the monitor alarm sound is instantly increased to 80 decibels through the audio processing algorithm, and the flashing frequency of the lights is accelerated using the animation control script. The trainees put on VR glasses to experience the immersive and tense atmosphere in all directions, which fully tests the team's emergency response, and the system records these collaborative details.

[0085] Response strategy guidance and evaluation: Built-in medical knowledge base of over one million, covering emergency procedures for various diseases and drug use guidelines. When an accident occurs, the intelligent guidance assistant immediately pops up an emergency treatment idea window in the VR field of view. Taking allergic reactions as an example, it shows the standardized procedures such as discontinuation of sensitizing drugs and calculation method of epinephrine injection dosage in steps; at the same time, it records the team actions of trainees in multiple dimensions, quantitatively evaluates the response effect from the time deviation of the execution of emergency measures (required to be within ±30 seconds of the standard time), the fit of the division of labor among members (compared with the preset best division of labor plan for scoring), and the completeness of information communication (analyzing the omission of key information after voice recognition conversion to text), and generates a detailed report, focusing on the advantages and disadvantages of the team collaboration link.

[0086] Training Feedback and Progression

[0087] Real-time feedback presentation: During the whole training process, the system uses sensors, voice recognition and other technologies to capture operation data in real time. Trainees wear VR glasses and tactile feedback gloves to monitor the accuracy of equipment picking, spatial positioning tracks the deviation of operation time, and the voice interaction module counts more than 30 indicators such as the number of communication words between members and the completeness of key information transmission. Through the data visualization algorithm, it is instantly converted into charts and heat maps and presented in the VR field of view. Different colors indicate the advantages and disadvantages, allowing trainees to understand the training performance at a glance, especially to clearly see their contributions and areas for improvement in the team collaboration process.

[0088] Advanced path planning: At the moment of task completion, the system calculates the total score based on preset scoring rules, taking into account factors such as task completion time, operation accuracy, and emergency handling scores. If the team's emergency handling score is less than 60 points in a medium-difficulty task, 3-5 targeted emergency special training tasks will be automatically pushed to adjust the difficulty and increase the complexity of unexpected situations; if the overall performance is good, more difficult tasks will be unlocked, training scenarios will be updated, and more complex knowledge and operation requirements will be integrated to ensure the continuity and improvement of training. Trainees can seamlessly connect to subsequent training by wearing VR glasses, and continuously enhance team collaboration effectiveness.

[0089] The collaborative interaction module is electrically connected to the training task module, and is used for adaptive interaction between multiple dependent devices or operation interaction between multiple roles or task allocation and adjustment of roles in a team.

[0090] According to an embodiment of the present invention, it also includes a multi-person spatial positioning module, which performs real-time positioning of trainees. After the virtual scene is built, it is quickly activated, and a three-dimensional spatial coordinate system is constructed with the fixed point of the center of the head of the operating bed as the origin. The coordinate accuracy can reach 1 mm. With the help of 8-12 high-precision infrared cameras, combined with the positioning suits covered with more than 30 reflective marking points worn by the trainees, 360° all-round tracking is achieved. The system captures positioning data more than 60 times per second, and can update the spatial position of each trainee's head, hand and body trunk in real time. The position error is strictly controlled within 3 mm, and precise spatial anchor points are calibrated for subsequent interactive collaboration.

[0091] Specifically, the multi-person spatial positioning module uses cutting-edge consumer-grade VR device spatial positioning technology to achieve high-precision, real-time multi-person spatial positioning for the VR multi-person collaborative training system for preoperative preparation, allowing team members to collaborate accurately in the virtual operating room scene. This is achieved through the following methods:

[0092] Using Meta Oculus to share spatial positioning

[0093] Device pairing and integration: When students wear the Meta Oculus Quest 3 device to enter the training scene, the system automatically searches and pairs. With the help of Wi-Fi 6E or Bluetooth 5.0 high-speed communication links between devices, the underlying data of the devices can be shared. The high-precision IMU (inertial measurement unit) data of Meta Oculus Quest 3 and the spatial positioning data of HTC VIVE LBE / LBSS are integrated. The IMU data refresh rate is as high as 1000Hz, which supplements fast motion capture and fills the gaps in high-frequency changes in positioning data, making positioning smoother.

[0094] Implementation of shared positioning algorithm: A customized spatial positioning sharing algorithm is implanted, with Meta Oculus Quest3 as a distributed positioning node. When trainees move in the virtual scene, each device collects its own position change data in real time and aggregates it to the system server through an encrypted transmission channel. The server uses a time synchronization algorithm to align the collected data with the HTC VIVE LBE / LBSS main positioning data, unify the time reference, and ensure that the spatial position update of each trainee occurs at the same time, with the error controlled within 1 millisecond, to achieve true spatial positioning synchronization sharing, so that trainees participating in the team are in the same space.

[0095] Real-time dynamic tracking and calibration

[0096] Multi-part tracking accuracy assurance: With the collaboration of HTC VIVE LBE / LBSS and Meta Oculus Quest3, key parts of the trainees such as head, hands, and torso are tracked. The system collects more than 120 sets of positioning data per second, and uses deep learning gesture recognition algorithms to accurately analyze the spatial posture of each part. For example, when tracking hands, the finger joint motion capture accuracy can reach 2 mm, which can delicately restore the gesture of holding surgical instruments; the body rotation angle deviation is controlled within 1°, meeting the positioning requirements of surgical positions and other actions.

[0097] Dynamic calibration strengthens collaboration: During training, once the system detects abnormal fluctuations in the trainee's position data or a temporary signal loss due to occlusion, the system immediately starts the automatic calibration procedure. The coordinates are recalibrated based on the relative position relationship between the fixed landmarks in the scene and the surrounding trainees. The calibration process is completed within 0.5 seconds, and the position error quickly returns to within 3 mm, ensuring smooth and uninterrupted multi-person collaborative movements. It can also ensure accurate and correct cooperation such as the delivery of instruments during surgery, which requires extremely high position accuracy.

[0098] According to an embodiment of the present invention, the virtual scene construction module includes a virtual space module and a shared space module. The virtual space module is electrically connected to the shared space module. The shared space module includes multiple VR device all-in-ones and a router. A local area network is established between the multiple VR device all-in-ones and the router. The VR device all-in-one starts a high-precision scanning process to capture real-world scene features and generate an independent space map that accurately maps virtual and reality.

[0099] Specifically, this application uses a WiFi6 or WiFi 7 router to connect multiple VR devices to form a local area network. At the same time, the router is connected to multiple servers. The servers continuously transmit real-time data to each VR device to ensure that multiple devices in the same network can carry out training tasks synchronously in real time.

[0100] According to an embodiment of the present invention, the virtual team module sets roles according to the job segmentation of the actual surgical scene, and the role settings include the general surgery chief surgeon, the cardiology assistant physician, the anesthesiologist, the operating room circulating nurse or the instrument nurse.

[0101] According to an embodiment of the present invention, the training task module includes a training server, which is electrically connected to a collaborative interaction sub-module, a voice dialogue sub-module, a real-time operation message sub-module and a gesture and handle interaction sub-module. The training server is electrically connected to a preoperative training sub-module, and the preoperative training sub-module realizes signal connection with the collaborative interaction sub-module, the voice dialogue sub-module, the real-time operation message sub-module and the gesture and handle interaction sub-module.

[0102] According to an embodiment of the present invention, the collaborative interaction module includes a simulation server, a team role management submodule, a task allocation submodule and a collaborative task submodule.

[0103] Specifically, the collaborative interaction module is a key part of the VR multi-person collaborative training system for preoperative preparation to promote efficient collaboration and accurate communication among team members. With the help of cutting-edge technology, it replicates the interactive experience in real surgical scenes. The specific implementation steps are as follows:

[0104] Hardware Interaction Adaptation

[0105] Equipment access and calibration: When the system starts, an intelligent recognition algorithm is used to quickly identify the multiple VR hardware devices that are connected. For head-mounted displays, with the help of its built-in high-precision sensors, key parameters such as resolution (up to 8K and above), refresh rate (up to 120Hz-240Hz), and field of view (up to 200° in the horizontal direction) are accurately measured within 3 seconds, matching exclusive visual optimization solutions to ensure picture clarity and immersion. The tactile feedback gloves integrate more than 100 micro pressure sensors and vibration motors. After access, they automatically carry out 5 rounds of force sensing calibration. When simulating instruments of different weights, the error is strictly controlled within 3%. From soft medical cotton swabs to orthopedic electric drills weighing 500 grams, the touch can be accurately reproduced. The spatial positioning tracker relies on 8-12 positioning base stations distributed in the training space to emit infrared signals with millimeter-level accuracy. Within 10 seconds after the trainees put on the equipment, the position tracking initialization calibration of more than 20 key parts of the body is completed, and the motion capture accuracy can reach 0.1 mm.

[0106] Data transmission optimization: Relying on ultra-high-speed 5G or more cutting-edge next-generation communication networks, we build an ultra-low-latency data transmission "highway". Students' operations generate massive amounts of data, with a data transmission rate of up to 100Mbps per second. These data are processed by a lossless compression algorithm and sent to the system server, with the average transmission delay compressed to less than 5 milliseconds. The server uses the edge computing and cloud computing fusion architecture to process data at high speed, and the updated scene and member status data are transmitted back to the student's device. The packet loss rate of the entire round-trip transmission process is controlled below 0.01%, ensuring smooth interaction.

[0107] Operational interaction design

[0108] Instrument operation simulation: In the ultra-fine modeling environment of the virtual operating room, based on the CAD drawings provided by major medical equipment manufacturers, more than 200 commonly used surgical instruments are reproduced 1:1, ranging from surgical microscopes to micro suture needles, all of which are lifelike. The VR tactile feedback algorithm is used in combination with high-precision pressure sensors to simulate the feel of instrument operation. When picking up a 150-gram scalpel, the hand can feel a grip resistance of 0.8-1.2N. When cutting virtual human tissue, the system simulates the resistance changes of different tissues (fat, muscle, bone) based on the biomechanical model, with a simulation accuracy of up to 98%. When simulating complex surgical operations, such as heart bypass surgery, there are more than 30 operating steps. The system strictly follows the international authoritative surgical guidelines. Once an illegal operation occurs, not only will there be an immediate voice alarm, but a red warning sign will also light up in the field of view to indicate the details of the deviation.

[0109] Role positioning and action coordination: Based on massive surgical imaging data and expert experience, standard three-dimensional spatial positioning and dynamic range of activities are set for each role of the surgical team. The standing area of ​​the surgeon is accurately positioned within 0.5 square meters, and the positions of assistants, nurses and other roles are also precisely defined. With the help of multi-sensor fusion spatial positioning technology, the positions of members are tracked in real time. When the position deviation exceeds 5 cm, a vibration tactile feedback with a frequency of 10-20Hz and a red light flashing at 30 lumens brightness in the corresponding area in the virtual scene are used to remind. In the simulation of member collaborative actions, motion capture technology and human body dynamics models are used to simulate complex actions such as multiple people working together to carry patients. The deviation in the direction of force is controlled within 10°, and the delay in action connection does not exceed 50 milliseconds, replicating the collaborative fluency of real scenes.

[0110] Information interaction implementation

[0111] Voice communication construction: Integrate deep learning speech recognition and synthesis technology to create an ultra-clear team voice communication system. In a simulated operating room with an ambient noise of up to 60 decibels, the voice acquisition module is equipped with 4 directional microphones, and uses a beamforming algorithm to accurately focus on the direction of the members' voices, with a voice recognition accuracy of up to 95%. Voice transmission uses OPUS encoding technology, combined with an adaptive bit rate adjustment strategy to ensure real-time conversations. The average delay from a member speaking to the other party receiving the message is stable at 80 milliseconds. The voice transmission bandwidth is dynamically allocated, with a minimum guarantee of 32kbps clear sound quality. In addition, the system supports voice recognition of more than 50 commonly used surgical commands, such as "start the extracorporeal circulation machine" and "adjust the depth of anesthesia to level 3", with a command recognition success rate of more than 92%.

[0112] Data sharing and visualization: Build a virtual surgical information panel based on blockchain distributed storage to ensure data security, transparency and non-tamperability. Real-time synchronized patient medical records, imaging materials (high-definition X-rays and CT images with resolutions up to 1024×1024 and above), preoperative examination reports and other massive data are instantly pushed to team members. Using holographic projection and 3D visualization technology, complex patient vital signs data (such as heart rate, blood pressure, and blood oxygen saturation) are converted into dynamic color light bands and three-dimensional charts suspended in the air. The data refresh rate is 1 second / time. Members can retrieve key information to assist decision-making within 2-3 seconds with just eye contact and simple gestures.

[0113] Collaboration process guidance

[0114] Preoperative process guidance: In-depth analysis of the standard preoperative preparation process for surgery published by major medical associations around the world, breaking it down into more than 100 steps, and integrating it into an intelligent virtual guidance assistant. At the beginning of training, the assistant uses vivid 3D animations, clear voice narration, and eye-catching text prompts to prompt team members to perform their respective precise tasks step by step, from circulating nurses checking the completeness of more than 50 key parameters of 15 types of surgical instruments to anesthesiologists checking more than 20 items of patient anesthesia information, guiding the whole process in a standardized manner, with a probability of missing key steps less than 1%.

[0115] Emergency collaboration and dispatch: When encountering simulated emergencies, the system responds instantly. Based on the medical knowledge graph that integrates more than 1,000 real emergency cases, the adapted emergency collaboration process is activated within 1 second, and tasks are accurately assigned. For example, if a patient suddenly suffers from anaphylactic shock, the surgeon must suspend the operation within 10 seconds, the assistant must hand over the emergency medicine within 5 seconds, and the nurse must complete the preparation of emergency equipment within 30 seconds. At the same time, the emergency operation guide pops up in the center of the VR field of view, with a detailed level of more than 50 steps, guiding the team to work closely together to respond to the crisis.

[0116] Interaction effect evaluation

[0117] Quantitative indicator collection: AI big data analysis technology is used to collect trainee interaction data in an all-round and uninterrupted manner. More than 50 key quantitative indicators are collected, including the average time taken to transfer equipment (accurate to 0.1 seconds), information transmission accuracy (accurate to two decimal places), and instruction execution success rate (accurate to 1%). At the same time, the communication frequency between team members (number of exchanges per minute) and changes in tone and emotion (through voice emotion analysis to identify positive, neutral, and negative emotions) are monitored to build a multi-dimensional interaction portrait.

[0118] Feedback report generation: Based on the massive amount of data collected, the system's machine learning algorithm generates a detailed interaction effect evaluation report within 5 minutes after the training ends. The report gives a quantitative score of 0-100 for each link of operation interaction, information interaction, and collaboration process, and provides more than 20 customized improvement suggestions, supplemented by comparative data of similar teams, to help students clearly understand their own and their team's strengths and weaknesses in collaborative interaction, and point out the direction for subsequent training.

[0119] According to an embodiment of the present invention, it also includes an AI auxiliary module, which is electrically connected to the training task module. The AI ​​auxiliary module includes an AI simulated nurse server, an AI player NPC and a task completion sub-module. The AI ​​auxiliary module is used for intelligent role switching and motion capture and voice recognition technology with the help of VR equipment, real-time monitoring of the movements of the surgical team roles and voice command auxiliary tool delivery, as well as preoperative item organization and disinfection assistance.

[0120] Specifically, the AI ​​auxiliary module is a powerful assistant for the VR multi-person collaborative training system for preoperative preparation. It can temporarily replace some roles in specific scenarios, efficiently perform simple auxiliary work, and improve training efficiency and fluency. Its working principle is as follows:

[0121] Smart role switching

[0122] The system presets AI replaceable role templates, focusing on the role of nurses. Before the training starts, the instructor can activate the AI ​​nurse function with one click based on the team configuration and training needs. The AI ​​will then "take office" in the virtual scene. Its virtual image and action style simulate professional nurses, and it will fit in the team without any sense of disobedience. The entire switching process takes only 3-5 seconds.

[0123] Tool Handover Assist

[0124] Sensing needs: With the help of VR equipment's motion capture and voice recognition technology, AI nurses monitor the movements and voice commands of surgical team members in real time. When the surgeon reaches out and looks at a specific surgical instrument, or verbally says "pass me the scalpel", the AI ​​nurse can capture the demand signal within 1 second.

[0125] Precise delivery: Based on high-precision spatial positioning, the AI ​​nurse quickly locates the corresponding surgical instrument and moves to the storage location to pick it up. The moving speed can reach 1-1.5 meters per second, and the path planning avoids other members and obstacles. After picking up the instrument, it is delivered to the surgeon in a natural and smooth movement. The delivery position deviation is controlled within 5 cm, simulating the real delivery feel and ensuring the continuity of operation.

[0126] Basic assistance task execution

[0127] In addition to handing over tools, AI nurses can also undertake basic tasks such as preoperative item sorting and disinfection assistance. When sorting surgical instrument trays, AI can follow standard procedures and neatly arrange 10-15 types of instruments within 30 seconds; during disinfection, AI nurses hold disinfection equipment, and the movement covers a uniform range, with less than 5% of the area missed during disinfection, effectively sharing simple repetitive work and allowing team members to focus on key operations.

[0128] According to an embodiment of the present invention, an evaluation and feedback module is also included. The evaluation and feedback module collects multiple data indicators from multiple dimensions, analyzes them through AI algorithms, outputs team collaborative evaluation results, and generates customized reports;

[0129] Data indicators include the accuracy of the instrument retrieval sequence, the frequency of communication among members, and the completeness of information transmission.

[0130] Specifically, the evaluation and feedback module is a key part of the preoperative preparation VR multi-person collaborative training system. It accurately understands the capabilities of students and teams, efficiently guides learning, ensures the effectiveness of training, and connects theory with practice, helping to cultivate a high-level surgical team.

[0131] Data quantitative analysis: more than 50 data indicators are collected from dimensions such as operation process and communication. The accuracy of the order of taking instruments, the frequency of communication between members, and the completeness of information transmission are analyzed through algorithms to obtain quantitative results such as team coordination coefficient and individual operation standardization.

[0132] Report generation: The generated personalized report lists the problem points in detail and provides more than 10 targeted improvement suggestions. After feedback from the trainees and improvements based on the report, the team collaboration error rate in the next round of training was reduced by an average of 10%-20%.

[0133] The system sets "levels" for students to advance their abilities. If the results of five consecutive evaluation reports reach 90 points or above, it means that the student team has reached a very high level. At this time, students can have the opportunity to participate in real clinical surgery observation and learning. At the same time, the system starts the "clinical synchronous learning mode". Students will immediately feedback special situations encountered in the clinic to the system through the mobile APP. The system background will quickly organize medical experts and technical teams to discuss and complete the update and upgrade within 48 hours, so that the virtual training system can always keep up with the latest clinical trends and be infinitely close to real surgical scenes.

[0134] In the dimension of evaluation and feedback, with the help of data analysis solutions, key data such as operation process, communication frequency, emergency response, etc. are captured in an all-round way, and the AI ​​algorithm is used to analyze and output the results of team collaborative evaluation. The customized reports generated based on this anchor the improvement targets and guide the trainees to iteratively optimize; after reaching clinical practice, the outstanding trainees will feedback the special case data in real surgery to the system, drive the system upgrade and update, and always keep pace with the clinical frontier.

[0135] In summary, this application uses laser scanning and high-resolution image acquisition to collect massive amounts of real operating room data, and through professional 3D modeling, physics-based rendering, real-time physical simulation, and high-fidelity sound effects, it constructs a virtual space that is static and vivid, dynamic and realistic. The moment trainees put on VR equipment and step into it, they are wrapped in a super immersive feeling and fully devote themselves to preoperative training. Relying on big data and artificial intelligence algorithms, the present invention deeply analyzes the knowledge systems of various medical disciplines, breaks down the knowledge barriers between anatomy, pharmacology, and surgical practice, accurately disassembles and reorganizes complex knowledge points according to the preoperative process, and seamlessly integrates them into each link of training in the form of intelligent push, so that trainees can efficiently absorb knowledge. Identify nutrients, use the high controllability of virtual reality scenes, and trigger dozens of preset emergency situations in surgery with one click, such as acute massive bleeding, anesthesia failure, etc., with the help of environmental simulation and AI-driven dynamic changes in patient vital signs. Trainees are deeply involved in it, immersively honing their emergency response, and their psychological and practical abilities are greatly improved. By collecting and analyzing training data in real time, the shortcomings of trainees are accurately located, and personalized improvement suggestions are issued. In this way, professional talents with solid theoretical knowledge and strong practical skills are trained for the medical industry in preoperative preparation. This application is connected with clinical practice, and trainees can enter the clinic with the results of training. At the same time, the reverse optimization system of clinical special cases is used to ensure that the training is in line with actual combat and what they have learned is used.

[0136] In addition, this application relies on advanced high-precision spatial positioning systems, such as the use of multiple sets of ultra-high frequency infrared cameras and inertial measurement units (IMUs) to work together to capture the dynamics of various parts of the trainees' bodies more than 100 times per second, with a positioning accuracy of up to 0.1 mm, ensuring that the motion capture is accurate and without delay. The built-in intelligent collaboration algorithm quickly analyzes the current task within 0.5 seconds based on the division of surgical roles, and pushes accurate prompts such as position and operation sequence to the corresponding trainees. The trainees' operation information is transmitted in real time via the 5G high-speed network, with a bandwidth of over 1Gbps, ensuring lossless and instant sharing of massive action and status data. The delay in teammates receiving feedback is less than 10 milliseconds, and the interaction is smooth throughout the process. After multiple training cycles, the team's muscle memory gradually takes shape, the error rate of surgical cooperation can be reduced by 30%-50%, and the collaboration becomes more tacit and efficient.

[0137] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to the above embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the above embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0139] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A VR multi-person collaborative training system for preoperative preparation, characterized in that: include: A virtual scene construction module, which is used to construct a virtual scene. The virtual scene includes a dynamic scene and a static scene. The dynamic scene includes: light and shadow scenes, sound scenes and surgical operation animations inside an operating room, anesthesia preparation room and preoperative ward; The static scenes include the structure, material and layout environment of the operating room, anesthesia preparation room, preoperative ward and various medical equipment; A virtual team module, the virtual team module is electrically connected to the virtual scene construction module, the virtual team module is used to construct a virtual role template, and to set multiple roles and form at least one team according to the role template, wherein a team includes at least two roles; The training task module uses big data technology to deeply analyze the knowledge system in the surgical field, sort out the key points, and then build a theoretical learning framework of three levels: basic, advanced, and high-level in a virtual scene, forming diversified learning resources, and pushing targeted emergency special training tasks to train and assess trainees; A collaborative interaction module, the collaborative interaction module is electrically connected to the training task module, and the collaborative interaction module is used for adaptive interaction between multiple dependent devices or multiple role operation interaction or task allocation and adjustment of roles in a team.

2. The VR multi-person collaborative training system for preoperative preparation according to claim 1, characterized in that: The virtual scene construction module includes a virtual space module and a shared space module. The virtual space module is electrically connected to the shared space module. The shared space module includes multiple VR device all-in-ones and a router. A local area network is established between the multiple VR device all-in-ones and the router. The VR device all-in-one starts a high-precision scanning process to capture real-world scene features and generate an independent space map that accurately maps virtual and reality.

3. The VR multi-person collaborative training system for preoperative preparation according to claim 2, characterized in that: The virtual team module sets roles according to the job segmentation of the actual surgical scene, and the role settings include the chief surgeon of general surgery, assistant cardiologist, anesthesiologist of anesthesia department, circulating nurse or instrument nurse in the operating room.

4. The VR multi-person collaborative training system for preoperative preparation according to claim 1, characterized in that: The training task module includes a training server, which is electrically connected to a collaborative interaction sub-module, a voice dialogue sub-module, a real-time operation message sub-module and a gesture and handle interaction sub-module. The training server is electrically connected to a preoperative training sub-module, and the preoperative training sub-module realizes signal connection with the collaborative interaction sub-module, the voice dialogue sub-module, the real-time operation message sub-module and the gesture and handle interaction sub-module.

5. The VR multi-person collaborative training system for preoperative preparation according to claim 1, characterized in that: The collaborative interaction module includes a simulation server, a team role management submodule, a task allocation submodule and a collaborative task submodule.

6. The VR multi-person collaborative training system for preoperative preparation according to claim 1, characterized in that: It also includes an AI auxiliary module, which is electrically connected to the training task module. The AI ​​auxiliary module includes an AI simulated nurse server, an AI player NPC and a task completion sub-module. The AI ​​auxiliary module is used for intelligent role switching and motion capture and voice recognition technology with the help of VR equipment, real-time monitoring of the movements of surgical team roles and voice command auxiliary tool delivery, as well as pre-operative item sorting and disinfection assistance.

7. The VR multi-person collaborative training system for preoperative preparation according to claim 1, characterized in that: It also includes an evaluation and feedback module, which collects multiple data indicators from multiple dimensions, analyzes them through AI algorithms, outputs team collaborative evaluation results, and generates customized reports; The data indicators include the accuracy of the instrument retrieval sequence, the frequency of member communication and the completeness of information transmission.

Images