Virtual reality first aid trainer system

Abstract

A virtual reality first aid training system that includes a game engine; a virtual reality (VR) headset in communication with the game engine; and a controller in communication with the game engine, the game engine being configured to receive instructions from the controller, and being configured to present first aid scenarios to a user via the VR headset, each first aid scenario including a plurality of scenes sequentially presented to lead the user in a step by step fashion through a process of performing first aid for a particular type of emergency.

Description

Atty. Dkt. No. 88990-161VIRTUAL REALITY FIRST AID TRAINING SYSTEMCROSS-REFERENCE TO RELATES APPLICATIONS

[0001] The present application claims benefit of and priority to U.S. Provisional Patent Application No. 63 / 764,179 entitled VIRTUAL REALITY FIRST AID TRAINER SYSTEM filed February 27, 2025, the entire content of which is hereby incorporated by reference herein.BACKGROUNDField

[0002] The present invention relates to a serious game implemented on a virtual reality (VR) platform to train a person in first aid.Related Art

[0003] Proper first aid care may be crucial to saving lives when more advanced medical assistance is either not available or not immediately available. First aid training centers provide volunteers with the appropriate first aid skills, however, are limited by a variety of factors such as: inaccessibility to interested people, long mandatory hours, expensive equipment, or the lack of proper training tools and methods. Further, conventional programs typically required interested individuals to retain an overwhelming amount of information in a relatively short time and absorb this information sufficiently such that it is available to them in an emergency. To address the limitations of the traditional first aid training mobile applications, augmented reality (AR), and e-leaming platforms have been proposed for first aid training.

[0004] Accordingly, it would be advantageous to provide a first aid training program that avoids these and other problems.SUMMARY

[0005] A system for providing Virtual Reality (VR) based first aid training in accordance with an embodiment of the present disclosure includes: a VR platform configured to train “players” in first aid techniques, leveraging serious game concepts to mix outcome-oriented and process-oriented methods. It is an object of the present disclosure to provide an4912-3403-0738v.lAtty. Dkt. No. 88990-160enjoyable learning experience personalized and adaptable for each player through gameplay while effectively teaching first aid techniques with an immersive interactive experience.

[0006] The system, in embodiments, is beneficial for individual learners but also plays a crucial role in corporate and industrial training environments. In these settings, first aid training is important for ensuring the safety and well-being of employees. The system's ability to provide immersive, realistic, and personalized training experiences makes it an invaluable tool for organizations seeking to equip their workforce with vital life-saving skills. In embodiments, by integrating advanced technologies such as virtual reality, haptic feedback, and Al-driven personalization, the system can deliver consistent and effective training across large groups of employees, ensuring that they are prepared to respond effectively in emergency situations. Moreover, in embodiments, the scalable nature of the system allows for widespread adoption across various industries, from manufacturing to healthcare, where first aid knowledge is critical.

[0007] A virtual reality first aid training system in accordance with an embodiment of the present disclosure includes: a game engine generating virtual reality information associated with a first aid scenario; a virtual reality headset operably connected to the game engine; and a controller operably connected to the game engine, wherein the game engine provides the virtual reality information to the virtual reality headset and the virtual reality headset presents the first aid scenario on a display of the virtual reality headset, wherein the first aid scenario includes a plurality of scenes sequentially presented in the display of the virtual reality headset to lead a user in a step by step fashion through a process of performing first aid for a particular type of emergency simulated by the virtual reality headset based on the first aid scenario information, and wherein the controller provides input to the game engine to advance the user through the virtual reality scenario.

[0008] In embodiments, the virtual reality information includes visual information, audio information and feedback information provided to the user as the user advances through the first aid scenario.

[0009] In embodiments, the virtual reality headset includes one or more speakers configured to present audio based on the audio information.24912-3403-0738v.lAtty. Dkt. No. 88990-160

[0010] In embodiments, the virtual reality headset further comprises a microphone operable to provide voice information to the game engine.

[0011] In embodiments, the system includes a first feedback device operably connected to the game engine and configured to receive first feedback information to provide tactile feedback to the user as the user advances through the first aid scenario.

[0012] In embodiments, the system includes a second feedback device operably connected to the game engine and configured to provide olfactory feedback to the user as the user advanced through the first aid scenario.

[0013] In embodiments, the one or more speakers receive third feedback information that includes audio feedback information to provide audio feedback as the user moves through the scenario.

[0014] In embodiments, the first aid scenario includes a plurality of scenes to train the user in a first skill.

[0015] In embodiments, the first aid scenario includes a plurality of scenes to train the user in a second skill.

[0016] In embodiments, the virtual reality information includes video information, audio information, tactile feedback information and olfactory information that is presented to the user as the user advances through the first aid scenario.

[0017] In embodiments, the game engine generates a virtual hand controlled by inputs from the user using the controller to interact with virtual objects visually in one or more scenes the first aid scenario from the plurality of first aid scenarios.

[0018] In embodiments, the game engine utilizes artificial intelligence to generate the virtual reality information used to present the first aid scenario.

[0019] In embodiments, the game engine includes an artificial intelligence engine, the artificial intelligence engine implementing a scenario generator module that generates the virtual reality information including a virtual environment and tasks, a customization module that updates the virtual reality information to dynamically adjust scenario complexity and sensory feedback based on real-time user interactions, a feedback 34912-3403-0738v.lAtty. Dkt. No. 88990-160integration module that provides haptic, auditory, and olfactory feedback information, and an adaptation and monitoring module that continuously monitors user actions and makes scenario adaptations accordingly using the updated virtual reality information.

[0020] In embodiments, the scenario generator module uses a first machine learning algorithm that uses user information and first aid template information as an input and is trained using prior user information and prior first aid protocol information to provide an output related to virtual reality information.

[0021] In embodiments, the output is virtual reality information.

[0022] In embodiments, the output is used by the game engine to provide virtual reality information.

[0023] In embodiments, the user information includes a username, demographic information, training level information, first aid experience information, first aid protocol information and prior training outcome information.

[0024] In embodiments, the customization module uses a second machine learning algorithm that uses user information including activity information as an input and is trained using prior user information and activity information to provide a second output associated with updated virtual reality information to modify the first aid scenario in accordance with performance of the user.

[0025] In embodiments, the second output is provided to the game engine to provide updated virtual reality information that modifies the first aid scenario to aid the user in completing the first aid scenario.

[0026] In embodiments, the updated virtual reality information modifies the first aid scenario to make it more difficult for the user in completing the first aid scenario.

[0027] In embodiments, the scenario generator module selects or generates a virtual environment for the first aid scenario from a plurality of scenarios.

[0028] In embodiments, the updated virtual reality information adjusts scene complexity of the first aid scenario in real-time.44912-3403-0738v.lAtty. Dkt. No. 88990-160

[0029] In embodiments, the scenario generator module integrates haptic, auditory, and olfactory feedback into the virtual reality information in at least one scene of the first aid scenario.

[0030] In embodiments, the adaptation and monitoring module continuously monitors user actions and the customization module updates the first aid scenario information to make scenario adjustments in response to user actions.

[0031] In embodiments, the adaptation and monitoring module evaluates user actions to analyze performance of the user and generates a report that includes user strength information and user weaknesses information.

[0032] In embodiments, the system includes a user profile database containing data related to user skills, previous performance and learning objectives and a scenario database containing additional first aid scenarios, first aid protocols, and environmental templates.

[0033] In embodiments, the game engine includes a virtual environment Tenderer that generates virtual reality information associated with scenes of the first aid scenarios with all visual, auditory, and haptic elements.

[0034] In embodiments, the game engine further comprises a performance analysis module that evaluates user performance, generates reports and feeds data back into an artificial intelligence engine for training.

[0035] In embodiments, the virtual reality helmet includes an accelerometer and a gyroscope configured to measure rotation and orientation of the virtual reality headset to determine a sense of direction in the scene on the display.

[0036] In embodiments, the virtual reality helmet includes a magnetometer configured to determine orientation of the headset 14 relative to a magnetic field of the Earth.

[0037] In embodiments, the virtual reality helmet includes an internal camera positioned to track eye movement to determine progress in the first aid scenario.54912-3403-0738v.lAtty. Dkt. No. 88990-160

[0038] In embodiments, the virtual reality helmet includes an olfactory feedback element configured to provide olfactory feedback based on olfactory feedback information provided by the virtual game engine.

[0039] A method of providing first aid training in accordance with an embodiment of the present disclosure includes: providing virtual reality information associated with a first aid scenario to a virtual reality headset, wherein the first aid scenario is rendered on a display of the virtual headset; monitoring user activity information as a user progresses through the first aid scenario; providing updated virtual reality information based on user information including the user activity information; and providing the updated virtual reality information to the virtual reality headset to personalize the first aid scenario.

[0040] In embodiments, the virtual reality information includes visual information, audio information, olfactory information and haptic feedback information.

[0041] In embodiments, the step of providing the virtual reality information includes using artificial intelligence to generate the virtual reality information.

[0042] In embodiments, the step of providing virtual reality information includes receiving as an input to a first machine learning algorithm user information, wherein the first machine learning algorithm is trained using prior user information and first aid scenario template information and the first machine learning algorithm provides a first output associated with the virtual reality information to present the first aid scenario to a particular user associated with the user information.

[0043] In embodiments, the first output is provided to a virtual reality game engine and the virtual reality information is provided based on the first output.

[0044] In embodiments, the first output is the virtual reality information.

[0045] In embodiments, the step of updating the virtual reality information incudes receiving as an input to a second machine learning algorithm user information including user activity information in real time, wherein the second machine learning algorithm is trained using prior user activity information and first aid scenario template information and provides a second output associated with the updated virtual reality information.64912-3403-0738v.lAtty. Dkt. No. 88990-160

[0046] In embodiments, the second output is provided to a virtual reality game engine and the updated virtual reality information is provided based on the first output.

[0047] In embodiments, the second output is the updated virtual reality information.

[0048] In embodiments, the virtual reality information includes visual information, audio information and feedback information provided to the user as the user advances through the first aid scenario.

[0049] In embodiments, the virtual reality headset includes one or more speakers configured to present audio based on the audio information.

[0050] In embodiments, the virtual reality headset further comprises a microphone operable to provide voice information to the game engine.

[0051] In embodiments, a first feedback device is operably connected to the game engine and configured to receive first feedback information to provide tactile feedback to the user as the user advances through the first aid scenario.

[0052] In embodiments, a second feedback device operably connected to the game engine and configured to provide olfactory feedback to the user as the user advanced through the first aid scenario.

[0053] In embodiments, the one or more speakers receive third feedback information that includes audio feedback information to provide audio feedback as the user moves through the scenario.

[0054] In embodiments, the first aid scenario includes a plurality of scenes to train the user in a first skill.

[0055] In embodiments, the first aid scenario includes a plurality of scenes to train the user in a second skill.

[0056] In embodiments, the game engine generates a virtual hand controlled by inputs from the user using the controller to interact with virtual objects visually in one or more scenes the first aid scenario from the plurality of first aid scenarios.74912-3403-0738v.lAtty. Dkt. No. 88990-160

[0057] In embodiments, the game engine includes an artificial intelligence engine, the artificial intelligence engine implementing a scenario generator module that generates the virtual reality information including a virtual environment and tasks, a customization module that updates the virtual reality information to dynamically adjust scenario complexity and sensory feedback based on real-time user interactions to provide updated virtual reality information, a feedback integration module that provides haptic, auditory, and olfactory feedback information, and an adaptation and monitoring module that continuously monitors user actions and makes scenario adaptations accordingly using the updated virtual reality information.

[0058] In embodiments, the scenario generator module selects or generates a virtual environment for the first aid scenario from a plurality of scenarios.BRIEF DESCRIPTION OF THE DRAWINGS

[0059] FIG. 1 illustrates an exemplary block diagram of a system for providing virtual reality based first aid training in accordance with an embodiment of the present disclosure;

[0060] FIG. 2 illustrates an exemplary portal scene suitable for use in the system of FIG. 1 in accordance with an embodiment of the present disclosure;

[0061] FIG. 3 illustrates an exemplary form for collection of used information suitable for use in the system of FIG. 1 in accordance with an embodiment of the present disclosure;

[0062] FIG. 4 illustrates an exemplary user home screen suitable for use in the system of FIG. 1 in accordance with an embodiment of the present disclosure;

[0063] FIG. 5 illustrates an exemplary welcome scene suitable for use in the system of FIG. 1 in accordance with an embodiment of the present disclosure;

[0064] FIG. 6 highlights the exemplary anchor point that may be used in the welcome scene in accordance with an embodiment of the present disclosure;

[0065] FIG. 7 illustrates exemplary player progress through the welcome scene in accordance with an embodiment of the present disclosure;84912-3403-0738v.lAtty. Dkt. No. 88990-160

[0066] FIG. 8 illustrates an exemplary activity of the player as the player progresses through the welcome scene in accordance with an embodiment of the present disclosure;

[0067] FIG. 9 highlights an exemplary user interface that may be provided to the player as the player progresses through the welcome scene in accordance with an embodiment of the present disclosure;

[0068] FIG. 10 highlights a location of interest to the player as the player progresses through the welcome scene in accordance with an embodiment of the present disclosure;

[0069] FIG. 11 illustrates an exemplary activity of the player as the player progresses through the welcome scene in accordance with an embodiment of the present disclosure;

[0070] FIG. 12 highlights another location of interest to the player as the player progresses through the welcome scene in accordance with an embodiment of the present disclosure;

[0071] FIG. 13 illustrates an exemplary user interface instructing interaction with the location of interest to the player in accordance with an embodiment of the present disclosure;

[0072] FIG. 14 illustrates an exemplary user interface instructing how to use an object in accordance with an embodiment of the present disclosure;

[0073] FIG. 15 illustrates a result of the use the object in accordance with an embodiment of the present disclosure;

[0074] FIG. 16 illustrates an exemplary user interface providing a warning to the player of an error and instructions to remedy the error in accordance with an embodiment of the present disclosure;

[0075] FIG. 17 highlights an exemplary user interface to provide further information to the player regarding the error regarding the in accordance with an embodiment of the present disclosure;94912-3403-0738v.lAtty. Dkt. No. 88990-160

[0076] FIG. 18 illustrates an exemplary anchor point to end the home scene in accordance with an embodiment of the present disclosure;

[0077] FIG. 19 illustrates an exemplary user interface to provide information regarding the upcoming scenes in accordance with an embodiment of the present disclosure;

[0078] FIG. 20 illustrates other exemplary objects that the player may interact with in accordance with an embodiment of the present disclosure;

[0079] FIG. 21 illustrates an example of how information may be retrieved from certain objects while interacting with them in accordance with an embodiment of the present disclosure;

[0080] FIG. 22 illustrates an exemplary object that mat be used to exit the home scene in accordance with an embodiment of the present disclosure;

[0081] FIG. 23 illustrates a first scene of a first aid scenario that may be presented using the system of FIG. 1 in accordance with an embodiment of the present disclosure;

[0082] FIG. 24 highlights an object to be interacted with in the first scene in accordance with an embodiment of the present disclosure;

[0083] FIG. 25 highlights a user interface directing interaction with an object and the object in accordance with an embodiment of the present disclosure;

[0084] FIGS. 26-27 show the interaction with the object in accordance with an embodiment of the present disclosure;

[0085] FIG. 28-31 highlight a user interface directing interactions with a virtual patient in accordance with an embodiment of the present disclosure;

[0086] FIG. 32-33 illustrate exemplary interaction with a quiz that may be presented to the player to test knowledge in accordance with an embodiment of the present disclosure;104912-3403-0738v.lAtty. Dkt. No. 88990-160

[0087] FIG. 34 illustrates exemplary user interfaces indicating that the scenario has been passed by the user in accordance with an embodiment of the present disclosure;

[0088] FIG. 35 illustrates an exemplary object to be interacted with to leave the scenario in accordance with an embodiment of the present disclosure;

[0089] FIG. 36 illustrates a return to the welcome scene which includes a user interface with a grade for the player’s performance in accordance with an embodiment of the present disclosure;

[0090] FIG. 37 illustrates an exemplary first scene of another first aid scenario that may be presented using the system of FIG. 1 in accordance with an embodiment of the present disclosure;

[0091] FIGS. 38-40 illustrate exemplary interaction with objects in the first scene in accordance with an embodiment of the present disclosure;

[0092] FIGS. 41-42 highlight guidance to approach a virtual patient in accordance with an embodiment of the present disclosure;

[0093] FIG. 43 illustrates a player interacting with personal protective equipment prior to contacting the virtual patient in accordance with an embodiment of the present disclosure;

[0094] FIG. 44-46 illustrate treatment of the virtual patient in accordance with an embodiment of the present disclosure;

[0095] FIG. 47 illustrates exemplary interaction with a quiz that may be presented to the player to test knowledge in accordance with an embodiment of the present disclosure;

[0096] FIG. 48 illustrates an exemplary object to be interacted with to leave the scenario in accordance with an embodiment of the present disclosure;114912-3403-0738v.lAtty. Dkt. No. 88990-160

[0097] FIG. 49 illustrates a return to the welcome scene which includes a user interface with a grade for the players performance in the scenario in accordance with an embodiment of the present disclosure;

[0098] FIG. 50 illustrates an exemplary architecture that may be used to provide backend data management and player profiling in the virtual reality game engine of the system of FIG. 1 in the scenario in accordance with an embodiment of the present disclosure;

[0099] FIG. 51 illustrates exemplary interaction of Al agents with feedback modules to provide feedback information in the scenario in accordance with an embodiment of the present disclosure;

[0100] FIG. 52 illustrates operation of a game manager and integration of Al in the system of FIG. 1 in accordance with an embodiment of the present disclosure;

[0101] FIG. 53 illustrates am Al agent within virtual reality provided by the system of FIG. 1 in accordance with an embodiment of the present disclosure;

[0102] FIG. 54 is an exemplary bock diagram of an Al agent and its interactions with information in accordance with an embodiment of the present disclosure;

[0103] FIG. 55 illustrates the user activity information that is monitored, the Al processed used to analyze it and the resulting modifications in accordance with an embodiment of the present disclosure; and

[0104] FIG. 56 illustrated the user activity information that is gathers, modules in the game engine that process it and exemplary outputs in accordance with an embodiment of the present disclosure.DESCRIPTION OF THE EMBODIMENTS

[0105] Virtual reality (VR) is a potentially useful platform for first aid training due to its immersive nature and ability to replicate a variety of real-world scenarios. A VR platform is also advantageous because, in embodiments, it allows a feedback mechanism and may be quickly adapted, and provide a diversity of realistic scenarios, in a safe environment, and allows for training in a controlled environment. VR training, in embodiments, also gives 124912-3403-0738v.lAtty. Dkt. No. 88990-160the player repetitive practice opportunities to reinforce learning and boost confidence, lets a trainee set their own pace, and increases trainee engagement through the gamification of the process. The training may also be accessed from anywhere to anyone using an internet connection such that proximity of the trainee to a particular training center may be irrelevant

[0106] In embodiments, serious games, also known as “Applied games,” are games or game-like systems built using game technology and design principles for a purpose other than entertainment, like teaching a player tangible skills, changing consumer behavior or conducting research. In embodiments, serious games have an intrinsic value in their ingame mechanics, narrative, and design, which sets them apart from commercial entertainment video games, and helps them make a lasting impact on the player’s brain. The engagement in games usually follows a progression of increasing difficulty called cognitive flow; it is a predominantly cognitive state of deep concentration and task absorption that makes a person feel one with the activity.

[0107] Serious games can be played on a multitude of platforms. It is worth noting that the platform plays a crucial role in the effectiveness of the game, and each platform has its own advantages and disadvantages.

[0108] Serious games teach complex cognitive skills at relatively low cost compared to traditional methods of training, with an infrastructure that permits scalability for expansion.

[0109] In embodiments, feedback loops may be used to make sure that the player is taking the right path towards the intended outcome, and to allow the players to make corrections, preventing the development of bad habits in the player’s long-term memory. In addition, feedback loops make progress seem like a reward to support good habits.

[0110] In embodiments, another advantage serious games have over traditional training methods is the possibility to determine and measure the retum-on-investment in order to dive into the player’s engagement and achieve optimization of the games. The tracking and reporting of a considerable amount of data can be done automatically and quickly, reducing the amount of effort and time compared to traditional methods of training.134912-3403-0738v.lAtty. Dkt. No. 88990-160

[0111] Serious games offer a safe and controlled environment for players to experiment, discover, learn at their own pace, and explore different paths with different results without real-world consequences.

[0112] FIG. 1 illustrates an exemplary block diagram of a virtual reality (VR) first aid training system 10 that may be used to train a user to provide first aid. In embodiments, the system 10 utilizes a VR training platform to provide remote training. The system 10 includes a VR game engine 12, a VR headset 14, and at least one controller 16 operably connected to the VR game engine and the at least one controller, and a feedback device 17 operably connected to the engine. In embodiments, the system 10 may include two or more controllers 16. In embodiments, the system 10 may include multiple feedback devices 17. In embodiments, one feedback device 17 may provide haptic feedback. In embodiments, another feedback device 17 may provide olfactory feedback. In embodiments, audio feedback may be provided via one or more speakers provided in the VR headset 14 In embodiments, visual feedback may be provided via a display provided in the VR headset 14.

[0113] In embodiments, the VR game engine 12 may be or may be included in a computer such as a PC, a laptop, or a tablet. In embodiments, the VR game engine 12 may be implemented by a processor in such a computer system using processor executable code stored in a memory thereof operably connected to the processor. In embodiments the VR game engine 12 may be provided remotely on a server of a network and the memory may be provided on the network as well. In embodiments, the processor and memory may be cloud based and accessed by the VR helmet 12 or may be

[0114] a local computer system. In embodiments, the VR game engine 12 may be housed in the VR headset 14 itself. In embodiments, the game engine 12 may be or may be operably connected to a processor that is operably connected to memory including processor executable code that when executed by the processor generates virtual reality information associated with a first aid scenario, including audio and video information, that is presented to a player trainee via the VR headset 14. In embodiments, the player trainee may provide instructions to interact with the scenario via the controllers) 16, verbally or via another input device or element.

[0115] In embodiments, the word “computer” refers to an apparatus that has all the necessary components for performing the functions of the game engine 12 as described 144912-3403-0738v.lAtty. Dkt. No. 88990-160herein including a non-transitory memory, a CPU, or other processor, a RAM, a GRU, and so on. As noted above, this includes a processor, such as a CPU, that is operably connected to the memory that includes processor executable code that is executed by the processor to perform the functions of the gaming engine 12.

[0116] In embodiments, the controllers) 16 may be joysticks) or any other suitable input device that can provide electronic input instructions to the VR game engine 12. In embodiments, such an input device may be implemented by wearable items such as gloves, suits, or masks. In embodiments, the feedback device 17 may be a separate device. In embodiments, the feedback device 17 may be provided on or integrated into these input elements to provide tactile sensations, heat or cold sensations, and / or odors. In embodiments, multiple feedback devices 17 may be provided, one providing tactile sensations, for example a glove or sleeve, one providing hot and cold sensations, which may also be integrated into a glove or sleeve and one providing olfactory feedback, which may be integrated into the headset 14 to release odors. In embodiments, the headset 14 includes a visual display to provide visual information, including visual feedback as well as speakers to provided audio information, including audio feedback. Thus, in embodiments, when a player trainee interacts with a first scene of a first aid scenario including a plurality of scenes, for example, they may wear a glove or other wearable element that includes feedback devices 17 that provide a sensation, for example, that an object in the scene is soft, hot, cold, heavy, or light and so on. Similarly, when a player trainee enters a virtual area in the first scene, for example, an odor (e.g., gas odor) may be provided through the feedback device 17 configured to provide olfactory feedback. In embodiments, this feedback device 17 may be provided in the VR headset 14. In embodiments, the scene displayed on the display of the headset using video information provide in the virtual reality information. Audio information may be presented via a speaker in the headset 14.

[0117] In embodiments, The VR headset 14 may be used to immerse the player trainee in a 360 degrees virtual world. In embodiments, the VR headsets 14 provide six degrees of freedom, allowing the player to rotate and move in the virtual world for a complete immersion experience. In embodiments, the VR headset 14 and controllers) 16 preferably follow the IEEE P2048.1 - Standard for Virtual Reality and Augmented Reality. Other VR specifications may be used if desired.154912-3403-0738v.lAtty. Dkt. No. 88990-160

[0118] In embodiments, the system 10 may include sensors (integrated with the headset 14) to measure the movement of the headset 14 in all 3 dimensions (X, Y, and Z), and to sense the tilt, orientation, and movement of the headset 14 and thus the activity of the player trainee. In embodiments, the sensors may include an accelerometer and a gyroscope (integrated with the head set 14) to measure the rotation and the orientation of the headset 14, thereby helping the player to maintain a sense of direction in the virtual reality world projected on the display in the helmet 14. In embodiments, the sensors may include a magnetometer (integrated with the headset 14) to determine the orientation of the headset 14 relative to the magnetic field of the Earth, and in combination with the accelerometer and gyroscope, to provide six (6) degrees of freedom tracking in real time, which boosts the immersive VR experience. In embodiments, the sensors may include a proximity sensor (integrated with the headset 14) to detect the presence of the player’s face or eyes to determine if the player is wearing the headset 14 or not. In embodiments, the headset 14 may include an internal camera(s) in the headset 14 pointed towards the player’s eyes, and used for many purposes such as tracking the player’s movement without the need for external sensors, hand tracking, or face recognition. In embodiments, an IR proximity sensor may be integrated with the headset 14 that can be used to determine if the player trainee is wearing the headset and for positional tracking while external camera(s) outside the headset 14 may be used to track the position of the player in the player’s physical environment. This movement information may be used by the engine 12 to track the user’s progress in the scenario as well as the user’s interaction therewith which may be used to change the user’s point of view in the scene.

[0119] hi embodiments, a personal computer configured as the game engine 12 may be used because of its smooth graphics, accurate tracking in space, larger and better range of VR games, and other applications. A computer used for the game engine 12 preferably has a strong graphics card, enough RAM and a powerful CPU.

[0120] In embodiments, the Acer Windows Mixed Reality headset may be used as the headset 14 and is suitable to serve as the game engine 12 and includes two (2) controllers that provide haptic feedback and comply with IEEE P2048.6 - Standard for Virtual Reality and Augmented Reality: Immersive Player Interface. Other VR headsets may be used if desired.164912-3403-0738v.lAtty. Dkt. No. 88990-160

[0121] Described below is an example of how a game engine 12 may train a player in first aid techniques using a VR platform and the system 10.

[0122] In embodiments, the system 10 may first display a portal scene using the display of the headset 14 (FIG. 2, for example) illustrating basic player trainee data. In embodiments, this information may be collected via inputs to a form 18 (FIG. 3) that may be presented to a player trainee via the display on the headset 14 where information may be provided via the controller 16, verbally using a microphone provided in the headset 14, or elsewhere. In embodiment, this information may be provided via a keyboard or other input device operably connected to the engine 12 or may be retrieved from memory. In embodiments, new players may register in this manner and existing players may be identified. In embodiments, during this registration, in the portal scene, the player trainee may be presented visual elements to acclimate the player trainee to the game scenes of the first aid scenario although at this point the player will not interact with any objects in the scene.

[0123] In embodiments, the game engine 12 may generate for display using the headset 14 a welcome scene to a first time player. In embodiments, the welcome scene may be stored in memory and reused. In embodiments, the welcome scene:a) will guide the player trainee through a tutorial to teach the player how to use the VR headset 14 and controllers 16;b) will help the player trainee adapt to the ambiance and pace of the game;c) will give the player trainee a sense of navigation through the scenes;d) will help the player trainee develop a sense of intuition about interacting with interactable objects throughout all the scenes.

[0124] In embodiments, a player trainee who has played the game before and has viewed the welcome scene and the tutorial may be directed to a player’s personal home scene (FIG. 4). In embodiments, the personal home scene may be generated by video virtual reality information provided by the VR game engine 12 and may be stored for future use with the particular player. In embodiments, the player trainee is free to explore and try to interact with the game objects and the environment using the controllers 16. In embodiments, the 174912-3403-0738v.lAtty. Dkt. No. 88990-160personal home scene will be a central scene the player gets back to after finishing each level of the first aid scenario. The personal home scene will contain a “path” 20 through which the player can go from level to level to progress in the game.

[0125] In embodiments, the game engine 12 of a system 10 according to the present invention may utilize artificial intelligence (Al) to dynamically adapt the tutorial's pace and content based on the player's real-time engagement and comprehension levels. In embodiments, upon the successful sign-in of a new player, the system initiates an immersive welcome scene (FIG. 5, for example) specifically designed to orient the user within the virtual environment and prepare the user for the ensuing training experience. In embodiments, the scene may be generated by the game engine 12 and stored for reuse and may include a User Interface (UI) panel 22 that displays a first set of instructions, coupled with a highlighted anchor 24 strategically positioned within the player’s field of view. This initial interaction may be controlled or dynamically changed by an Al-driven guidance mechanism similar to the updating described below.

[0126] In embodiments the player trainee is prompted by the UI 22 to use the VR controller 16 to teleport to the highlighted anchor 24 using a thumbstick controller, for example. This teleportation mechanism, in embodiments, ensures that the player can only move to predefined locations, which are designed to introduce the interaction mechanics of the VR environment. This restriction is enforced to prevent the player trainee from navigating to unintended areas, thereby ensuring a structured and focused learning experience. The system 10 according to the present invention employs contextual visual cues in the form of dynamic textures on the UI panel to guide the player’s interaction with the controller 16, significantly reducing the learning curve and enhancing the onboarding process.

[0127] In embodiments, upon successful teleportation to the first anchor 24, the game engine 12 generates "Step Done" auditory feedback information and provides it to the headset’s integrated speakers, which is synchronized with a reorientation of the player's virtual view towards the next instructional target, such as a simulated forest. This combination of visual reorientation and auditory confirmation is part of the system’s multimodal feedback loop, which is designed to reinforce the learning objectives by engaging multiple senses.

[0128] In embodiments, system 10 may implement Al using the game engine 12, for example, that continuously monitors the player’s progress and activity and adapt the scenario in real-time. In embodiments, where an Al agent, for example, implemented by an 184912-3403-0738v.lAtty. Dkt. No. 88990-160Al engine 15, identifies hesitation or repeated errors, it can modify the scenario to intervene in the scenario , for example, to slow down the instruction pace or provide additional guidance, ensuring that the tutorial is personalized to the player’s unique learning needs. This adaptability allows for a highly individualized learning experience, which is useful in skillbased training like first aid.

[0129] As the player trainee progresses through the scenario, the game engine 12 dynamically generates haptic feedback information to provide haptic feedback through the controllers 16 or feedback device 17. For example, when the player uses a virtual hand 28 to grab a log 27 (FIG. 8), the haptic information is provided to simulate the texture and weight of the log through haptic responses delivered via the feedback device(s) 17, providing the player with a realistic tactile experience. This haptic feedback is complemented by auditory and visual cues provided via the headset 14, such as the deactivation of the previous UI panel 22 and the activation of a new one that provides instruction to the player to the next task — placing the log into a virtual campfire.

[0130] Throughout this process, the system 10 enhances the player immersion by integrating other contextual sensory feedback. For instance, in a first scene of the scenario, and others in the scenario, audio information associated with ambient sounds such as bird calls or a Christmas melody may be spatially rendered to correspond with the player’s location within the virtual environment. This use of spatial audio not only heightens the realism of the scenario but also aids in maintaining the player’s focus and engagement.

[0131] When the player trainee approaches a campfire in a scene, the system 10 may highlight a specific spot 30 (FIG. 10) in a fire pit, shaped to match the log, ensuring accurate placement. The Al-driven game engine 12 then snaps the log into place once the player releases it, followed by another "Step Done" sound effect, which signals the successful completion of the task. The system’s unique integration of haptic, auditory, and visual feedback in this sequence exemplifies its capability to deliver a holistic training experience.

[0132] In embodiments, as the player trainee advances in the scenario, the player trainee may be instructed to retrieve a virtual lighter 29 (FIG. 14) using virtual hand 28 and light the campfire. In embodiments, the UI 22 (FIG. 14) provides detailed visual instructions on operating the lighter, while the controller’s haptic feedback device 17 simulates the resistance of the lighter’s trigger, adding to the physical realism of the interaction. Upon lighting the campfire, the game engine 12 activates a fire particle effect (FIG. 15), along with synchronized crackling sounds, creating a fixlly immersive 194912-3403-0738v.lAtty. Dkt. No. 88990-160environment. This step also showcases the system’s ability to integrate multiple sensory modalities to reinforce the learning process.

[0133] In embodiments, the system’s Al-driven adaptive guidance mechanism ensures that the player trainee is immediately alerted if the player performs a potentially harmful action, such as burning the virtual hand while lighting the fire. In such cases, the system 10 may generate corrective feedback information associated with corrective feedback, for example, a specific instruction directing the player trainee to cool a bum under virtual running water, with the UI panel 22 turning red as a visual warning. This real-time adaptation and correction mechanism underscores the system’s effectiveness in preventing the reinforcement of incorrect behaviors.

[0134] In embodiments, as the scenario progresses, the player trainee is guided through a series of actions designed to solidify understanding of the VR controls and the physics of the virtual environment. The VR game engine 12 may adjust the difficulty and pace of these tasks based on Al intervention based on the player’s ongoing performance, ensuring that the tutorial remains challenging yet achievable. In embodiments, a final task may involve reaching a final anchor point 24 (FIG. 18), where the system generates a "Level Done" sound effect and deactivates all previous UI panels, signaling the completion of the tutorial.

[0135] In embodiments, at this point, the player trainee may be encouraged to explore the Welcome Scene further, interacting with various objects such as logs or a hammer 36 (FIG. 20), which reinforce the skills learned during the tutorial. Additionally, the system 10 provides contextual information, such as emergency contact numbers, by hovering over in-game objects like a book 38 (FIG. 21), making it a valuable resource for supplementary learning.

[0136] In embodiments, to transition between training modules or scenarios, the player follows a virtual path 20 (FIG. 4) that leads to various level stations. In embodiments, each station represents a specific first aid scenario, and by standing on a teleport anchor 24 and pressing a designated button 40, the player seamlessly advances to the next level. The Al-driven VR game engine 12 continues to personalize the experience by intervening to adjust the content of the virtual reality information used to present each subsequent level based on the player’s performance during the tutorial and during each level, ensuring a continuous and optimized learning journey. That is, the Al engine 15 used user204912-3403-0738v.lAtty. Dkt. No. 88990-160performance information along to determine whether to intervene in the scenario to customize the experience.

[0137] Tutorial. After a new player is signed in, the welcome scene (FIG. 5) is presented to the player. The welcome scene will have first instructions written on a UI panel 22 and a highlighted anchor 24 directly in front of the player. The player trainee is then prompted to teleport to the anchor 24 using any of the controller thumb sticks (note that a texture 26 above the UI panel 22 indicates to the player where the thumb stick is on the controller 16). In this scene, the player trainee can only move to the highlighted anchor 24, which is necessary to ensure that the player understands how to get to the anchor 24 and to ensure that the player cannot move to an unintended position without reading the instruction (FIG. 6). In this way, the player trainee will not be lost before the tutorial begins. When the player successfully teleports to the first anchor 24, a “Step Done” sound effect will be generated by the game engine 12 (which is played through the headset’s speakers for the player to hear), and the scene will be reoriented so that the player faces, for example, a forest. The anchor 24 and the UI panel 22 with the first instruction will also be disabled. A new instruction prompting the player to go into the forest will be activated in front of the player, and a next destination anchor will be activated in the forest. Once the player reaches the forest scene (FIG. 7), a “Step Done” sound effect is generated, and the scene will be reoriented so that the player faces a pile of wood, and the anchor and the instruction will be disabled. A new instruction panel containing instructions directing the player to pick a highlighted log will then appear (FIG. 7) in a UI 22.

[0138] In embodiments, after the player trainee successfully grabs a wood log 27 from the ground (FIG. 8) with a virtual hand(s) 28 generated by the game engine 12 and operated by controller(s) 16, a “Step Done” sound effect is generated, the previous instruction is deactivated, and new instructions prompting the player to place the wood log in the campfire will appear above the pile of wood logs in a UI 22 above the campfire (FIG. 9). The log will no longer be highlighted. Now the player must locate the campfire and teleport to it. Note that all the time the player is in or near the forest, bird sounds (simulating ambient nature sounds) may be generated and played through the headset’s speakers and while the player is near the Christmas tree next to the tent, a Christmas song may be generated and played through the speakers also. Indeed, all sounds and sound effects generated by the game engine 12 are played by the speakers integrated in the headset 14, for example, and are heard by the player.214912-3403-0738v.lAtty. Dkt. No. 88990-160

[0139] In embodiments, when the player trainee is close enough to the campfire, a spot 30 highlighted in a color (e.g., blue) with the shape of the log will appear in the campfire (FIG. 10). The player can then drop the virtual log from the virtual hand holding the log, which will be snapped into the spot 30. After the player trainee places the virtual log in the campfire, a “Step Done” sound effect is generated, the previous instruction is deactivated, and the logs in the forest will be shown again (FIG. 11). The player trainee may now go back into the forest to bring another log. Making the player go back and forth holding logs is helpful mainly because it forces the player to constantly navigate across the scene and interact with a log (picking it up and placing it in its highlighted place in the campfire), which in turn, helps the player memorize the teleportation and grab controls and get a sense of the physics mechanics of the game while interacting with the logs. When the player gets into the forest and grabs another log, a “Step Done” sound effect is generated, and the logs will no longer be highlighted. When the player is close enough to the campfire, a second spot 32 highlighted in color (e.g., blue) with the shape of the log will appear in the campfire (FIG. 12), and the player can drop the log from his virtual hand, and the log will be snapped into the second spot 32. After this step is completed, the instruction UI panel 22 above the campfire will change to ask the player to get a lighter 29 (FIG. 13) from the table to light the campfire (FIG. 14). When the player locates the table and grabs the lighter 29 a “Step Done” sound effect is generated, and an instruction UI panel 22 will appear directly above the table informing the player on how to use the lighter 29 (FIG. 14). Above this instruction UI panel 22, a trigger image 34 showing the controller indicates to the player what button to use (the trigger button) to light the lighter 29 (FIG. 14). When the player arrives to the virtual campfire site holding the virtual lighter, to light the campfire using the lighter the player must press the trigger button while holding the lighter (in the same virtual hand) and put it between the wood logs. Once the lighter is between the wood logs and the virtual lighter is turned on a virtual fire particle effect will be activated (FIG. 15), a “Step Done” sound effect is generated, sounds of fire crackling near the campfire are generated, two anchors will appear right in front of the player (behind the campfire) that will act as stairs to help the player reach the river, the instruction UI panel 22 will change to become a warning one (in red color, for example). The UI panel 22 now will inform the player that the player’s virtual hand has been burned while lighting the campfire and needs to be put under cool running water in the river (FIG. 16).224912-3403-0738v.lAtty. Dkt. No. 88990-160

[0140] In embodiments, the player trainee will need to use the steps 31 and go to the teleportation anchor right beside the river and put the burned virtual hand in the water, after which a “Step Done” sound effect is generated, the previous instruction UI panel 22 is deactivated, an information UI panel 22 (top UI in green, for example) will appear in front of the player that teaches the player about the types of bums (FIG. 17), an instruction UI panel 22 (middle UI in blue, for example) will appear below the green UI panel asking the player to locate the wooden path and teleport to the anchor 24 in front of it (FIG. 17), and the anchor 24 in front of the wooden path will appear (FIG. 18). Now, the player needs to only reach this last anchor 24 to complete the tutorial section of the Welcome Scene. When the last anchor 24 is reached a “Level Done” sound effect is generated, the pervious information and instruction UI panels are deactivated, two similarly colored (e.g., blue) UI panels 22 appear in front of the player informing the player about the learning upcoming journey and that the Welcome Scene will remain the player’s home throughout the journey (FIG. 19). At this point, the player has finished the tutorial section of the Welcome Scene. The player now can wander around the Welcome Scene and try to interact with the different game objects such as the logs, or the hammer 36 (FIG. 20). On the table next to the tent, there is an open book 38. By hovering over the book 38 a UI panel 22 is generated with a few phone numbers (e.g., Red Cross, civil defense, fire department, police department (FIG. 21). The above description provides an exemplary tutorial scenario that may be used to teach the player trainee the general use of the system 10. In embodiment, the game engine may generate a different tutorial scenario if desired. In embodiments, other colors may be used in the UI 22.

[0141] In embodiments, to move from one level (scenario) to another, the player trainee will have to follow the wooden path 20 (Fig. 4) mentioned above, and at the end of each section of the path, a station represents a level that the player can enter to play and learn first aid principles. To enter a given level, the player must stand on the Teleportation Anchor in front of the level’s station and push the red button 40 down. A colored blue UI panel 22 will be above the button 40 to inform the player about the level and what first aid skills will be presented.Level 1: Recovery Position Training

[0142] Level 1 is designed, for example, to immerse the player in a scenario where the player learns and practices the Recovery Position for aiding an unconscious but breathing individual. It is noted that Level 1 may be used to teach a different first aid skill if 234912-3403-0738v.lAtty. Dkt. No. 88990-160desired. The primary objective of this level is to teach the player trainee the critical steps necessary to assess and safely position an unconscious person, ensuring that the airway remains open and the person is protected from harm.

[0143] The training may begin with the player trainee entering a virtual apartment environment where the player is immediately informed of a gas leak through a strategically placed User Interface (UI) panel 22. The Al-driven engine provides virtual reality information to the headset 14 that generates a prompt to the player to ensure the environment's safety before attending to the unconscious individual. This step reinforces the importance of situational awareness in first aid procedures.

[0144] To initiate Level 1, the player may activate a red button 40 in front of a virtual wooden house (FIG. 22). Upon activation, the player trainee may be guided to the apartment door, where three distinct UI panels 22 appear sequentially (FIG. 23). These panels are color-coded to enhance clarity and urgency: a top red panel alerts the player to the gas leak, a bottom green panel instructs the player to check for hazards, and a bottom blue panel directs the player to locate and shut off the gas supply.

[0145] In embodiments, the system 10 utilizes augmented visual guidance by highlighting a virtual gas cylinder 42 in the kitchen with a rotating hologram (FIG. 24). In embodiments, this highlighting is included in the virtual reality information provided by the Al assisted engine 12. The player must teleport to the cylinder, using the VR controller 16 to simulate touching it with a virtual hand 28 (FIG. 25). Upon successful interaction, the system triggers a "Step Done" sound effect, deactivates the previous UI panels, and removes the gas cylinder hologram, dynamically updating the environment based on the players activity.

[0146] In embodiments, the player trainee is instructed to open a highlighted window 44 (FIGS. 26 and 27) to ventilate the area. The system 10 employs haptic feedback through the VR controller 16 which may include a feedback element 17 simulating the tactile sensation of sliding the window handle. Each successful action is met with auditory feedback, reinforcing correct behaviors and guiding the player to the next task.

[0147] In embodiments, the focus then shifts to the unconscious virtual person 46, where the player is prompted to check for responsiveness by squeezing the person’s shoulders (FIG. 28). The system 10 generates a haptic response information corresponding to the tactile interaction, followed by an auditory cue indicating task completion. The Al-driven engine updates the instructional UI panels 22, guiding the player trainee through the sequence of steps required to secure the airway, including chin lift and chest inspection (FIG. 30). The 244912-3403-0738v.lAtty. Dkt. No. 88990-160system 10 employs prolonged haptic feedback during the chest inspection to simulate the feel of checking for breathing.

[0148] In embodiments, the player trainee is then guided through the precise sequence of movements required to position the person into the PLS Recovery Position. This includes manipulating the virtual limbs (e.g., rotating the wrist, lifting the knee) which may include Al assistance that adapts the instructions based on the player’s performance, ensuring accurate execution (FIG. 31). The final step involves the player virtually grabbing the person’s hip, after which the system automatically places the virtual person into the correct recovery position. The successful completion of this sequence triggers a "Level Done" sound effect, updating the UI panel above the person’s head to signal the end of the training scenario.

[0149] In embodiments, to reinforce the learned concepts, the system 10 may present a quiz (FIGS. 32 and 33), where the player must answer questions by selecting the correct options on the UI panel using controller 16 or verbal responses via a microphone in the helmet 14. The system 10 may provide immediate feedback through color-coded panels and corresponding sound effects — green for correct and red for incorrect, for example — to ensure that the player understands the material. Upon completion, the player trainee may exit the level by pressing an exit button 40 on the kitchen counter (FIG. 34), at which point they are returned to the Welcome Scene. In embodiments, the quiz may use alternate feedback techniques. In embodiments the exit button may be positioned elsewhere.

[0150] In embodiments, the player’s performance is tracked throughout the level, with the player’s score displayed at the level’s station in the Welcome Scene (FIG. 36). This performance data may be utilized by the system’s Al engine 15 as user information to adjust future levels, providing a personalized learning path that adapts to the player's strengths and areas for improvement.Level 2: Car Crash Accident and Wound Treatment Simulation

[0151] Level 2 may introduce, for example, a highly immersive and technically advanced simulation focused on critical first aid procedures necessary in the event of a car crash, particularly addressing the treatment of a wounded and bleeding victim who is conscious and responsive. While a car crash scenario is discussed any suitable similar scenario may be used provided that it involved the same or similar first aid procedures. This simulation is designed to provide an interactive and realistic experience, leveraging advanced 254912-3403-0738v.lAtty. Dkt. No. 88990-160haptic feedback, olfactory feedback, and Al-driven personalization to optimize the learning process.

[0152] In one embodiment, the scenario begins with the player positioned on the side of a virtual highway bridge, confronted with a car crash incident. The Al-driven engine provides virtual reality information to present User Interface (UI) panels 22 that may be dynamically generated based on the player's actions and progress, sequentially providing critical information and instructions. The top UI panel 22, highlighted in red (for example), alerts the player to the presence of a car accident (FIG. 37). The second green (for example) panel instructs the player to survey the surroundings for potential hazards, ensuring the environment is safe. The final blue (for example) panel guides the player to locate traffic cones 48 and strategically place them in predesignated, color-highlighted spots 30 on the ground (FIGS. 38 and 39), thereby securing the area to prevent further accidents.

[0153] In embodiments, the system's haptic feedback design utilizes advanced actuators embedded in the VR controllers 16 to simulate realistic physical interactions. In embodiments, gloves may be used to provide haptic feedback. In embodiments, other standalone feedback devices 17 may be used. For example, as the player places each cone 48, the haptic system delivers precise vibrations that mimic the resistance and weight of handling real-world objects. This tactile feedback is complemented by auditory cues, such as the "Step Done" sound effect, which is triggered upon the successful placement of each cone and provided via a speaker in the helmet 14. Dynamic responses may be driven by Al assistance, using an Al engine 15 integrated into the agent or buy implementing one or more Al agents, to ensure that the player trainee remains engaged and follows the correct sequence of actions. Once all cones 48 are correctly positioned, the system 10 prompts the player to approach the injured virtual person (FIGS. 40 and 41), with new UI panels 22 appearing to guide the next steps. As noted above the UIs are generated by the Al assisted engine 12 and instructions may change based on activity of the player.

[0154] Upon reaching the injured person, the system 10 integrates olfactory feedback through a wearable olfactory device that releases specific scents to enhance realism. For instance, the player trainee may detect the faint smell of gasoline, indicating a potential hazard in the environment. This sensory cue adds another layer of immersion, helping the player to fully engage with the scenario. The system 10 also employs haptic feedback as the player interacts with the victim. When, for example, the player applies pressure to the person’s wound 54 (FIG. 44), the haptic feedback system generates information to simulate 264912-3403-0738v.lAtty. Dkt. No. 88990-160prolonged resistance, simulating the physical effort required to control bleeding. The AI-driven UI panels 22 provide step-by-step instructions, ensuring the player applies the correct amount of pressure and properly dresses the wound using virtual bandages 56 (FIGS. 45 and 46).

[0155] The system provides personalization by continuously monitoring the player's actions, receiving as input information their inputs and providing outputs to adapt the difficulty and guidance provided based on real-time performance data. The output may be updated virtual reality information to generate update instructions and / or to update the scene to reflect the progress of the individual player. For example, if the player struggles with a particular task, personalization can slow down the instructions, provide additional visual cues, or offer corrective feedback to help the player understand the correct procedure. This personalized adaptation ensures that each player receives a tailored training experience that addresses their specific strengths and weaknesses. In embodiments, the Al engine may continuously monitor performance to determine whether intervention is appropriate, and if so, to provide updated virtual reality information to aid the user, or make the scenario more challenging.

[0156] As the player trainee completes the wound treatment, the system 10 verifies the effectiveness of the action, for example visually reducing the blood flow and activating a white bandage 56 around the wound. In embodiment, a quiz (FIG. 47) may be generated to reinforce the learned concepts, dynamically adjusting the difficulty based on the player’s previous performance. The system 10 tracks the player's responses, providing immediate feedback through color-coded panels and corresponding sound effects — green for correct answers and red for incorrect, for example — ensuring comprehension of the material. The system also records the player's performance data, including the time taken to complete the level and the accuracy of the answers, which are used by an Al agent to further personalize future training sessions.

[0157] Throughout a scenario, the player trainee is exposed to various sensory feedback mechanisms designed to create a comprehensive learning environment. The olfactory feedback design continues to play a role, releasing scents that correspond with specific actions or environments, such as the smell of bandages or the scent of antiseptic after wound treatment. This multi-sensory approach, combined with haptic and auditory feedback, ensures that the training is not only realistic but also deeply engaging.274912-3403-0738v.lAtty. Dkt. No. 88990-160

[0158] As the player trainee exits the level (FIG. 48), they are seamlessly returned to the Welcome Scene, where they can review their results including highest scores, revisit level stations, or replay the level if desired, without needing to repeat the tutorial. In embodiments, the player trainee performance information to date is provided as an input to an Al personalization module to update subsequent scenarios for the player trainee offering a personalized and optimized learning path that adjusts to the player’s individual needs. This module may be implemented using the Al engine 15 which may be integrated into the engine 12.

[0159] The Car Crash Scene is generated to teach the player the basic steps that need to be taken in case of a car accident and how to help a wounded and bleeding victim, who is awake and responsive. The specifics of the scenario may be modified provided that it provides the same skill training. The scene may be presented with the following a storyline but other storylines could be used. Initially, the player will be taken to the side of the highway with three colored UI panels 22 (FIG. 37). The top panel 22 (red, for example) warns the player that there is a car accident in the middle of the highway. The middle panel 22 (green, for example) instructs the player to check the surroundings for hazards before trying to help other people. The bottom panel 22 (blue, for example) instructs the player to find cones and encircle the area to keep cars away from the crash area. Now, the player needs to locate the cones 48, which may be right behind the car, and put them in the colored (blue, for example) highlighted spots 30 on the ground (FIGS. 38 and 39). Upon placement of each cone 48, the “Step Done” sound effect is generated, the corresponding highlighted spot will disappear, and the following highlight spot will appear. When all three cones 48 are put into place, a top UI panel 22 (colored green, for example) and a bottom UI panel 22 (blue, for example) may appear above the last cone 48 asking the player to check the person and three more cones 48 are displayed on the other side of the car crash area (FIGS.40 and 41). When the player approaches the person, the “Step Done” sound effect is generated and bottom (blue, for example) and top (green, for example) UI panels 22 are displayed above the casualty's head (FIG. 42). By pressing the next button or providing other input, for example a verbal instruction to proceed, the player may be prompted to find the gloves 50 and wear them. The gloves 50 may appear in the area next to the player where a first aid kit 52 is shown (FIG. 43). The player may hover a hand or other indicator on the display of the headset over the gloves 50 to wear them. In embodiments, other inputs may be used to put the gloves on, for example a verbal instruction. Once the player wears the gloves284912-3403-0738v.lAtty. Dkt. No. 88990-16050 the “Step Done” sound effect is generated as audio feedback and the player’s virtual hands 28 change their color to white (to simulate wearing gloves), and the UI panels 22 above the person will be updated. The next step is to put pressure on the person’s wound 54 by hovering over it (FIG. 44). After the player puts pressure on the wound 54 the “Step Done” sound effect is generated, the amount of blood coming out of the wound decreases, and the UI panel instructs the player to get the bandage and perform pressure dressing. The bandages will appear near the previous location of the gloves 50. The final step is to grab the bandages 56 and put them on the wound 54 (FIGS. 45 and 46). After this step is completed, the player will see that no more blood is coming out of the wound 54, and a white bandage 56 has been activated around the wound 54. The player can now proceed to answer the pop quiz (FIG.47) regarding the steps he or she has just taken or other information regarding this type of treatment or scenario. At this point, the time for completing level two is saved as well as the time score. With every question that the player answers, the player’s score will be updated and saved so that the player can leave whenever, and the player’s work will already be saved. The quiz mechanism here is the same as in the previous level with the only difference being that it has many questions, so the player will have to navigate from one question to another using the next button 58 and the coefficients for the score will be 20% for every question and 40% for the time score.

[0160] In embodiments, every time the player exits a level (FIG. 48), the player will be brought back to the Welcome Scene (without, of course, having to redo the tutorial) and will be able to walk around the scene, go to the level stations, see the player’s highest score obtained (see Figure 107), or replay a level for a refresher or to improve the player’s score.

[0161] In addition to the car crash and wound treatment scenario, the system 10 according to the present invention is capable of implementing a wide range of first aid training scenarios. These include, but are not limited to, Cardiopulmonary Resuscitation (CPR), Automated External Defibrillator (AED) training, vital signs measurement, choking first aid, allergy response, and wound cleaning. Each scenario is designed to provide a comprehensive and immersive learning experience, leveraging the system’s unique combination of Al-driven adaptability, sensory feedback, and interactive virtual environments. Each scene is rendered using virtual reality information generated by the VR Game Engine 12 which may be customized using Al for each individual player trainee based on their activity in real time. That is, the Al engine may continuously monitor player activity294912-3403-0738v.lAtty. Dkt. No. 88990-160to determine whether intervention is appropriate, and will then provide updated virtual reality information to provide this customization. The updated virtual reality information may be provide using or assisted by the VR Game Engine 12.Al-assisted scenario or scene generation or modification

[0162] In embodiments, the game engine 12 in system 10 may include an Al or otherwise use Al Agents 68 that generate or modify first aid scenarios. In embodiments, the engine 12 may include an Al engine 15. In embodiments, Al plays a role not only in adapting the training experience to individual users but also in the design and dynamic generation of training scenarios such as the Car Crash Accident and Wound Treatment simulation as noted above. In embodiments, advanced algorithms that analyze user data, previous training outcomes, and real-world first aid protocols are implemented by the Al engine to generate or modify scenes for a first aid scenario. In embodiments, Al agents may be implemented to provide this functionality. This allows the system 10 to create highly realistic and contextually appropriate training environments that are tailored to the needs of different users or training objectives.Al Design Process

[0163] In embodiments, the Al-assisted scenario or scene generation or modification may start with a Scenario Analysis. To perform a scenario analysis the training objectives and user-specific data, such as skill level and previous performance are analyzed with reference to a database of real-world first aid protocols and scenarios to ensure that the training content is accurate and relevant. Thereafter, based on the analysis, environment generation may be implemented by selecting or generating virtual reality information used to render a virtual environment that correspond to the scenario. Such an environment may be a highway bridge setting for a car crash simulation. The Al driven VR engine 12 populates the environment with relevant elements such as vehicles, injured persons, hazards (like gasoline spills), and necessary first aid equipment by generating associated virtual reality information that may be provided to the helmet 14 where the environment is rendered on the display. The Al engine 15 also provides Dynamic Scene Customization by adjusting the complexity of the scene in real-time as the player trainee moves through the scene to provide more or less aid to 304912-3403-0738v.lAtty. Dkt. No. 88990-160the user, depending on their activity. For beginners, it may reduce environmental distractions or simplify the tasks. For advanced users, it might introduce additional challenges, such as multiple casualties or time-sensitive tasks, to enhance the training’s difficulty and realism. This customization may be implemented by an Al agent 68 which may be implemented by engine 15. Feedback Integration may also be provided using the Al engine 15 which integrates sensory feedback elements, such as visual cues, haptic responses, auditory cues, and olfactory feedback, into the scene as the player trainee advances by generating virtual reality information to provide this feedback. In embodiments, the may be accomplished via a feedback integration Al agent and ensures that these elements are contextually aligned with the training objectives, such as providing the correct haptic feedback when applying pressure to a wound or releasing a specific scent when a hazard is detected. During the scenario, the Al engine may perform Real-Time Adaptation by continuously monitoring the user's actions and adapting the scene or the scenario accordingly. For example, if the user activity, which is an input, indicated the user is struggling with a particular task, which may be determined based on prior user interaction information, the updated virtual reality information may be generated to adjust the instructions, provide additional visual or auditory cues, or modify the environment to facilitate learning. In embodiments, a machine learning algorithm may be used with the user information as an input where the algorithm is trained using prior user information to provide updated virtual reality information personalized to the user in the scenario. In addition, after the training session is over, the Al engine may perform evaluation and reporting based on by evaluating the user's performance, and generating reports that highlight areas of strength and those needing improvement. It also uses this data to further refine future scenarios, ensuring that each training session is progressively more effective. In embodiments, an Al agent may be used to provide this functionality as well.1. Design Architecture

[0164] In embodiments, a User Profile Database 64 (FIG. 50), stores data related to each user’s skills, previous performance, and learning objectives. In embodiments, a scenario database stores a library of first aid scenarios, real-world protocols, and environmental templates. In embodiments, the Al engine 15 may interact with a Virtual Environment Renderer in the VR game engine 12, which renders the generated scene, including all visual, auditory, and haptic elements as noted above. A Performance Analysis314912-3403-0738v.lAtty. Dkt. No. 88990-160Module, may evaluate user performance, generate reports and feed data back into the Al Engine for future optimization.

[0165] The Al engine 15 may include or may work with a Scenario Generator Module of the engine 12 that uses user data and scenario templates to generate a virtual environment and tasks for generating virtual reality information for a particular scenario. In embodiments, a machine learning algorithm may be used with the user data and template as inputs. The machine learning algorithm may be trained using prior user information, templates and virtual environment information to output a virtual reality information to render a current environment, including visual, audio and olfactory information. In embodiments, a Customization Module dynamically adjusts the virtual reality information based on the output of a second machine learning algorithm the provides updates in real time based on user information, including user activity in the scenario where sensory feedback may be used to track activity of the player trainee. In embodiments, the user activity information may include:• xtposeinformation associated with User Pose State: This variable represents the physical posture and movements of the user at time t in the scenario which should be maintained in a safe alignment while moving through smoke for example.• xtmteractioninformation associated with a User Interaction State: This variable captures how the user interacts with virtual objects and the environment, that is objects being touched or manipulated.• ptprotocoi information associated with Protocol Progress State: This variable indicates the current step of the medical or first-aid protocol being executed.• eterrorsinformation associated with Error and Safety State: This variable records detected mistakes and safety related events, including unsafe behaviors or rule violations, for example pressure is applied at the wrong location or with insufficient duration.• ctcontextassociated with the Environmental and Scenario Context - This variable represents the current scenario conditions, such as: type of training scene (e.g., fire, trauma, clinical setting), presence of hazards (smoke, blood, debris) and condition of the virtual patient or environmentThe second machine learning algorithm may be trained by prior user activity information to provide update information to modify the scenario or scene based on the user’s activity to 324912-3403-0738v.lAtty. Dkt. No. 88990-160date. As noted above, this may result in slowing down instruction, providing additional guidance, etc.

[0166] In embodiments, a feedback integration module may integrate visual, haptic, auditory, and olfactory feedback into the scenario as the player trainee progresses, and may use the same inputs as the second machine learning algorithm as well as prior feedback information to output current feedback information associated with feedback to be provided to the player trainee in the scenario now. The module may be included in the engine 15 or may connected to the engine 15 to provide updated virtual reality information to modify the scenario based on user progress.

[0167] In embodiments, these same inputs may be provided to an Adaptation and Monitoring Module that continuously monitors user actions and records progress. As noted above, the scenario may be modified as well depending on the player trainee activities and progress,1. Innovative Backend Integration for Enhanced Immersive Learning

[0168] While engaging with the system 10, the player trainee progresses through a series of meticulously designed steps within each scene of the scenario, ultimately achieving the specific objectives of the level. For instance, learning the correct procedure for placing someone in the Recovery Position is one such example.

[0169] In embodiments, the game engine 12 may include or provide a Game Manager 60 (FIG. 50) to ensure that a game object is embedded in every scene to which a sophisticated Game Manager script is attached. This script serves as the command center, maintaining references to the majority of game objects within the scene and dynamically managing them as the player advances through the level and the state of the game evolves. That is, the script is used to track objects and activities in the scene as the player trainee move through the scene.Game Manager’s Role and Al Integration

[0170] In embodiments, the Game Manager script is equipped with or triggers execution of algorithms that monitor events triggered by the player’s interactions with game objects. As the player completes steps towards level completion, the Game Manager 60 either334912-3403-0738v.lAtty. Dkt. No. 88990-160relays this information to other game objects, triggering state changes and corresponding events, or directly alters the state of those objects. In embodiments, the Game Manager 60 script plays a role in dynamically adjusting the difficulty and pace of the game using the Al assisted engine 12 discussed above. In embodiments, the Al engine 15 may analyze real-time player data, using the parameters discussed above, for example, to personalize the gameplay experience. For instance, if a player struggles with a task, the Al agent can modify the game’s environment or instructions to better suit the player's skill level, ensuring an optimized and tailored learning experience.Backend Data Management and Player Profiling

[0171] In embodiments, the system 10 may include a backend data management facility 62 (FIG. 50), which may be integrated into the game engine 12. This facility meticulously tracks each player's activity throughout the game, recording critical metrics such as the time taken to complete each level, overall scores, and the effectiveness of the training. The data collected provides valuable insights into how quickly a player is adapting to the game mechanics and the VR simulation, allowing the system 10 to gauge the player's performance accurately. This information may also be used by the Al engine 15 or an Al agent to provide personalization. In embodiments, this information may be used to generate reports or logs of activity which may be used to training purposes for the Al engine.

[0172] In embodiments, the backend system stores detailed player profiles in the Player Profile Database 64 (FIG. 50) which may be provided in a memory operably connected to the game engine 12. A typical player profile may include demographic information, previous VR and first aid experience, and performance metrics across different levels. This data is continuously analyzed using advanced analytics tools, such as Unity Gaming Services’ (UGS) Analytics integration, to refine the player's training path. At least one Al agent may use this data to dynamically adjust the game's difficulty, as noted above, advancing players through scenes or levels at an appropriate pace based on their performance, thereby creating a personalized learning trajectory that adapts to the individual needs of each player. Other analytics tools may be used.

[0173] In embodiments, an analytics engine 65 acts as the core processor of player performance data, enables Al-driven customization of training by identifying userspecific needs, enhances learning efficiency by providing real-time insights to the Al 344912-3403-0738v.lAtty. Dkt. No. 88990-160Personalization Module, and ensures progressive training through long-term performance tracking.Haptic and Smell Feedback Processing

[0174] To further enhance the immersive experience, system 10 integrates sophisticated haptic processing and smell simulation technologies. The haptic feedback information is provided to haptic feedback device 17, which may be integrated into controller 16 or headset 14, or included in a glove or other garment, to provide realistic tactile sensations that correspond with in-game actions. For example, when a player applies pressure to a virtual wound, the haptic feedback information is provided to the haptic feedback device 17, which may be a glove, for example, to simulate the resistance felt, offering a true-to-life experience that deepens the learning process. The olfactory feedback device 17 releases specific scents at moments in the game, such as the smell of gasoline in a car crash scenario or antiseptic during wound treatment. This multi-sensory feedback may be managed by the Game Manager 60 of the game engine 12, enriches the player’s engagement, making the virtual training environment as close to reality as possible.Adaptive Audio Management and Timer Integration

[0175] In embodiments, system 10 may include audio management module for managing all audio elements, including sound effects and voice instructions, which are crucial for guiding the player through each level. An audio management script may be associated with each scene to ensure that sound is contextually accurate and synchronized with the player’s actions, enhancing both the realism and educational value of the training.

[0176] Moreover, each level may be equipped with a timer implemented as a game object, with a timer script tracking the duration of the player’s engagement within the scene. The timer not only records the total time spent in each level but also interacts with other game objects through its getter methods, providing real-time data that informs the Al agents' adaptive decisions. A timer script may include an update function that increments the time variable every frame, ensuring precise timekeeping that is critical for evaluating the player's performance and adjusting the pacing of the training.AI-Driven Personalization354912-3403-0738v.lAtty. Dkt. No. 88990-160

[0177] In embodiments, the Al agents 68 (FIG. 51) may receive historical or prior information from a Backend Data Feed 76 (FIG. 51) and continuously monitors haptic feedback from a Haptic Module 70 (Fig. 51), olfactory feedback information an olfactory Module 72 (FIG. 51), audio cues from an Audio Module 74 (FIG. 51) and timing data, ensuring that each element is fine-tuned to maximize the player’s learning experience. By integrating these innovative technologies into the backend architecture, the system 10 not only delivers a highly immersive training environment but also provides a scalable solution capable of adapting to the unique needs of each user, enhancing the overall effectiveness of VR-based first aid training. This same activity may be provided using one or more machine learning algorithms implemented in the Al engine 15 of the engine 12.

[0178] Fig. 52 illustrates an adaptive Al-driven process within the virtual reality (VR) first aid training system 10, where an intelligent Al agent dynamically modifies training conditions based on real-time player performance, optimizing learning outcomes and ensuring effective skill acquisition. The Al agent continuously evaluates user inputs using the parameters discussed above, analyzes responses, and adjusts the difficulty level, pacing, and feedback delivery accordingly by providing updated virtual reality information.

[0179] The following is a general description of Fig. 52:1. Player Actions• Represents the user interacting with the VR training system, performing simulated first aid procedures such as CPR, wound dressing, or airway clearance.• The player's actions provide activity information used as an input for the Al-driven adaptation process.2. Game Manager• Manages scenario flow, user inputs, and VR information generation and Al-based decision-making.• Ensures seamless training progression by monitoring user performance and triggering appropriate system responses.3. State Change364912-3403-0738v.lAtty. Dkt. No. 88990-160• Tracks player status and environmental conditions in real-time.• Modifies the difficulty, scenario complexity, and feedback mechanisms dynamically based on Al assessment.4. Al Design Process• The Al engine is responsible for processing player data, performance metrics, and training progress.• Uses machine learning algorithms to evaluate skill levels and determine necessary adjustments.5. Adaptation• Implements Al-generated modifications to training difficulty, pacing, and response mechanisms.• Adjusts training scenarios in real-time to align with individual learning needs.6. Player Trainee• Represents the trainee engaging in the simulation, receiving personalized feedback based on their actions and decisions.7. Initiate Action (Player inputs and actions)• The player's first action (e.g., performing CPR or bandaging a wound) triggers the Al evaluation process.• The system records the user’s input and activities for further analysis.8. Evaluate Condition• The Al and Engine assesses the player's performance in the given first aid task. • Determines if the action was executed correctly, delayed, or requires intervention.9. Response Generation• Based on the evaluation, the Al determines if feedback is required.374912-3403-0738v.lAtty. Dkt. No. 88990-160• If errors are detected, the system initiates corrective measures and provided feedback to player like a warning or a consequence.10. Analyze Inputs• The Al and engine 12 processes user actions, reaction times, and decision-making patterns.• This data is used to adjust the training parameters dynamically.11. Al Decision• Based on data analysis and pattern recognition, the Al determines whether the training needs updating (intervention):o Increased difficulty for advanced learners.o Simplified instructions for struggling trainees.o Additional practice opportunities to reinforce skills.12. Adjust Difficulty• The Al modifies task complexity based on user performance:o If the user excels, the system introduces time constraints and realistic emergencies.o If the user struggles, the system slows down and offers additional hints.13. Adjust Pacing• The Al alters task speed based on user response times:o Fast learners progress at a quicker pace.o Slow learners receive step-by-step guidance with additional time.14. Feedback• The system provides real-time feedback via:o Visual cues (e.g., color-coded instructions).o Auditory guidance (e.g., Al-generated coaching).o Haptic feedback (e.g., force resistance in VR gloves for CPR compression). o Ensures immediate learning reinforcement for skill improvement.384912-3403-0738v.lAtty. Dkt. No. 88990-160In Fig. 52, the following in the adaptive Al driven process:1. Player Interaction and Input Processing• The player engages in a virtual first aid training scenario, performing simulated procedures such as cardiopulmonary resuscitation (CPR), wound dressing, airway clearance, and fracture stabilization.• The Al-driven monitoring system continuously tracks user action information including response times, procedural accuracy, ensuring real-time data collection for adaptive adjustments.2. AI-Drivcn Adaptation Mechanism• The Al adaptation module receives real-time player input and evaluates the player’s performance against a predefined criteria.• If the player performs a task incorrectly, the Al identifies the deviation and triggers an intervention.• If the player demonstrates proficiency, the Al increases the challenge level by introducing new variables, such as time constraints, distractions, or complex patient conditions.3. Al Analysis & Decision-Making• The Al decision engine continuously processes collected data, executing:o Performance Evaluation: Al assesses player skill level based on accuracy, reaction time, and procedural efficiency.o Pattern Recognition: Al identifies common mistakes and determines whether corrective feedback is needed.o Adaptive Adjustment: Al modifies training intensity by increasing or decreasing scenario complexity.4. Feedback Generation and Adaptive Responses• The Al agent delivers real-time feedback to guide player improvement, utilizing: o Visual cues and instructions: Color-coded indicators showing errors and corrections.394912-3403-0738v.lAtty. Dkt. No. 88990-160o Auditory instructions: Al-generated verbal guidance correcting improper actions.o Haptic feedback: Simulated tactile responses (if integrated with VR gloves or controllers).• The Al can reinforce learning by prompting repetition of key steps or introducing alternative first aid techniques.5. Game Manager & Al Design Process• The game manager serves as the system coordinator, ensuring seamless synchronization between player actions, Al-driven analysis, and scenario updates. • The Al design process incorporates reinforcement learning models, improving system accuracy and adaptability with continued use.6. Scenario Modification Based on Al Insights• The Al dynamically adjusts pacing and difficulty based on real-time player interactions.• If a player struggles, the Al slows down the training, adds step-by-step guidance, or introduces practice rounds.• If a player excels, the Al accelerates the scenario, introduces more advanced case simulations, or requires the player to respond under pressure.

[0180] FIG. 53 shows an exemplary schematic Al agent 68 within a virtual reality system 10 according to the present invention, and FIG. 54 shows a block diagram of the Al agent and indicates how it functions. As noted above Al engine 15 may implement the Al Agent using one or more machine leaning algorithms to output virtual reality information to customize the scenario as well as updated virtual reality information in conjunction with the VR engine 12.Game Set up

[0181] The game engine 12 may be implemented using suitable game engine software. Some popular game engine software products are Godot, and Unreal Engine and Unity3d, which may be used in the engine 12 with an editor version v2021.3.18fl.404912-3403-0738v.lAtty. Dkt. No. 88990-160Development Kit OpenXR may be used to provide VR operations. OpenXR is an open standard for VR and AR developed by Khronos Group, and has a set of tools, libraries, sample codes and documentation to help developers create applications that work seamlessly with different VR and AR platforms. OpenXR is suitable for use with a variety of headsets. The XR Interaction Toolkit package version v2.4.1 (provided by Unity Technologies), which has different functionalities to interact with the Unity Input System through a componentbased interaction system, may be employed to provide certain functions such as basic hovering, grab and select functions, haptic feedback, visual feedback, cross-platform XR controller integration, and XR Origin implementations. Other game engine software may be used.

[0182] The Unity Engine is a suitable software package for configuring the game engine 12. Unity offers a multitude of systems for good game mechanics implementation namely the UI (player interface) systems, the Event System, the Physics System and the Particle System. The Unity UI panels display to the player the information, score, and tasks to be performed and serve as the communication bridge between the player (humans) and the game engine (computer). A typical UI panel has four components. The first component may include navigational elements to help the player navigate an interface. Examples of navigation elements are a slide bar, a search field, and a back arrow. The second component may be an input control to enable the player to input information / instruction, such as a button, checkboxes, and a text field. One example could be the “next” button that informs the player of the casualty state being responsive but bleeding severely. The selection of the “next” button displays the succeeding instruction. Another example is a quiz where the player has to select / click a button corresponding to an answer. The third component may be an informational component used to communicate information to the player. An example of an informational component is a progress bar display along with a video or a tutorial. The fourth component may be containers for organizing content into easily digestible sections, like an accordion menu that hides or shows content. Other engines may be used.

[0183] The UI panels 22 are preferably color-coded to signify something to the player. Organizing and associating colors with certain items can help the player easily locate and anticipate the kind of information presented in a scene before the player actually reads the information. Also, the locations of the UI panels become more memorable when the player sees the color. The UI panels may be presented in four colors, namely red, green, blue and orange. The color red may signify danger (e.g., burnt hand, a gas leak or a car crash) to414912-3403-0738v.lAtty. Dkt. No. 88990-160grab attention. The color green may present information or steps that should be memorized by the end of each level (e.g., bum types and cooling, scene safety assurance, AVPU and ABC scales, PLS visual steps, application of pressure on a bleeding wound with clean gloves). The color blue may explain in detail the general information written in green by adding a step-by-step approach to the task. The color orange may represent the quizzes at the end of each level. Position of the UI panels 22 may also be used to convey information.

[0184] The Unity Event System is a feature that allows the developer to trigger events when certain conditions are met in the game. Events can be triggered by a keyboard, a mouse, a touch pad, or a custom input (e.g., the buttons in a UI). UI panels and Event System go hand in hand. In fact, at the moment of creation of a UI panel, an Event System component is automatically generated alongside it. This component contains several Input Modules that are responsible for the way game objects inside the UI respond to input, as they encompass the main logic of the system. A button used in a quiz, the clicking of which should indicate to the player if the chosen answer was correct or false, is a great example.

[0185] Collision with a game object leading to a response from the game engine 12 is another example.

[0186] The primary roles of the Event System are managing which game object is considered selected, managing which input module is in use, managing raycasting (if required), and updating all input modules as required. A raycaster is commonly used by input modules to calculate what the pointing device is over. The Event System is responsible for several functions within the Unity Engine, including determining which game object is currently selected by the player, selecting and managing the appropriate input module (such as keyboard, mouse, or VR controller) to process user inputs, and performing raycasting when necessary. Raycasting is a technique where an invisible ray is projected from a pointing device (like a mouse cursor or VR controller) into the game world to detect which object it intersects with. This allows the system to calculate and determine which object the player is pointing at or interacting with. Additionally, the Event System ensures that all input modules are regularly updated to reflect the latest interactions, providing a seamless and responsive user experience. Three Raycasters exist by default, namely the Graphic Raycaster that allows the communication of UI panels with each other by forming a connection called a “hit” between a pointer position (mouse or touch on a screen) and an object in the scene. The other two raycasters are the physics 2D Raycaster for 2D physics elements, and the Physics Raycaster for 3D physics elements.424912-3403-0738v.lAtty. Dkt. No. 88990-160

[0187] A Physics System of Unity helps simulate physics to ensure that the objects correctly accelerate and respond to collisions, gravity, and various other forces. Unity provides different physics engine implementations namely 3D, 2D, object-oriented, or data-oriented. Unity’s built-in 3D physics engine integrates the Nvidia PhysX engine. It can be used in object-oriented 3D projects, and includes Rigidbody Physics, which enables physicsbased behavior such as movement, gravity, and collision, Collisions to manage collision events, and Joints that can affect the movement of rigid bodies by applying forces. The Unity’s Particle System makes it possible to simulate fluid intangible effects like moving liquids, smoke clouds, magic spells, flames, bleeding and so on.

[0188] Unity provides two animation systems, which can be used to animate objects. In a system according to the present invention, the Unity’s animation system can be employed to animate the virtual hand 28 or virtual hands 28 with which a player can engage a game object. The virtual hand(s) 28 can be used by the player to carry out tasks examples of which are provided above. The virtual hands 28 may serve as the player’s left hand and right hand each under the control of a respective controller 16, for example.

[0189] The Unity’s animation rigging package may be used to integrate procedural animation to humanoid characters. For example, a humanoid object representing an accident victim 46 could be implemented using the animation rigging package permitting player interaction (e.g., examination or manipulation using the virtual hands 28). For an interaction to happen, there should be interactors and interactables. An interactor is an object that handles the interactions and creates a list of all the interactables as valid targets with interaction priorities. The priorities can be managed and changed using the Target Filters. An interactable is an object that changes states (hover, select, focus and / or activate) whenever the interactor is in the vicinity. An interaction could be anything like a hover, selection, focus, or activation.

[0190] In a system 10 according to the present disclosure there could be four fundamental interactors on each controller 16, namely a direct interactor, a poke interactor, a teleport interactor and a socket Interactor. The direct interactor enables the player to interact directly with a game object. The poke interactor enables the player to interact with a UI panel 22 directly. The teleport interactor implements a teleport movement rather than a continuous motion system as it was discovered that continuous motion may cause motion sickness. A system according to the present invention may allow the player to teleport to a game object at a teleportation area, or to a teleportation anchor 24 component allowing the 434912-3403-0738v.lAtty. Dkt. No. 88990-160player to only teleport to a specific area defined by the anchor. To optimize the UI of the teleport interactor, a line visual could be set to a projectile curve, which can turn green, for example, when it is a valid area to teleport to, or red when it is not a valid area.

[0191] A system 10 according to the present invention may be implemented based on the Singleton Pattern. The singleton pattern is a design pattern that ensures a class can only instantiate itself once (only one object can be created using this class), and the class gives global access to that object (also called instance). In a system according to the present invention, the Singleton pattern ensures that one, and only one, object interacts at a time.

[0192] In a system according to present invention the observer pattern could be used to implement a one-to-many dependency relationship between the game objects, allowing one object to notify other objects of state changes automatically and without directly referencing them (this creates unnecessary dependencies). The main advantage of the observer pattern is that it decouples the classes such that the Subject class triggers events without really knowing who is listening to the events (the observers) and the observers themselves do not know one another (although they know who the subject is). The observer pattern could be implemented using events. These events may be used to construct a game logic in general. More specifically, an event may be triggered every time the player completes a step in any of the scenes. Then, this event notifies the Game Manager (discussed below) and any other Game Object to update their states and to perform any related logic. This implementation is handy because anytime the player does something (completing a step, a level, or interacting with its environment), many behaviors, coming from many different objects need to be executed, e.g., playing a sound effect, updating a UI panel, activating / deactivating a game object, playing a visual effect, etc.

[0193] The system 10 provides feedback and cues (including olfactory cues) according to the situational context. For example, in the car accident scene, environmental context includes traffic noise, debris, and fuel odor cues to ensure the player trainee remaining immersed in the experience for improved learning.

[0194] In embodiments, as noted above, user actions are continuously monitored to provide updated virtual reality information suitable for the user. In embodiments, in each first aid scenario, the system may analyze user’s physical posture, object interactions, procedural progress, detected errors, and environmental conditions to enable context-aware, safety-constrained adaptive coaching and assessment. As noted above virtual reality information may be updated to aid the user in completing the scenario, or where the user is excelling, the 444912-3403-0738v.lAtty. Dkt. No. 88990-160updated information may make the scenario more challenging to further sharpen the user’s skills.

[0195] FIG. 55 shows the above parameters that are continuously monitored and provided to the artificial intelligence engine 15. In embodiments, the artificial intelligence engine may use a variety of machine learning algorithms to generate or update the virtual reality information based on user information, including user activity information to customize the first aid scenario for the user. As indicated a convolutional neural network may be used generate and update the virtual reality information. In embodiments, a rule based engine may be used to assess the user’s procedural progress. In embodiments, a reinforcement learning agent may be used to provide appropriate feedback information to be included in the virtual reality information.

[0196] In embodiments, as noted above, the artificial intelligence engine provides the virtual reality information and update virtual reality information. In embodiments, this may include visual information, audio information and feedback information. As noted above updated virtual reality information may be updated to adapt the scenario the personal activities. In embodiments, olfactory feedback information may also be provided. As noted above, the artificial intelligence engine may also collect information and create session logs and reports including analysis of the user’s skills and progress in the training.

[0197] FIG. 56 similarly indicates the reception of user’s physical posture, object interactions, procedural progress, detected errors, and environmental conditions. The artificial intelligence engine may implement, based on these parameters, a protocol engine to determine the user’s process in the training protocol, a safety constraint module that detects errors in safety by the player trainee while proceeding through the scenario, a learning state estimator determining the skill level of the player based on their actions, an adaptation policy module indicting update to the scenario and a feedback selector that may select and generate feedback information to be included in the updated virtual reality information. As noted above, the virtual reality information or updated virtual reality information included visual, audio and feedback information that provide for changes in the scenario including interventions, in some cases, including guidance or warnings and other immersive techniques like olfactory feedback. In embodiments, the user information and the virtual reality information may be stored for training purposes and / or may be provided in a log or other records. In embodiments, the records may also be used to determine a skill level of the user.454912-3403-0738v.lAtty. Dkt. No. 88990-160

[0198] The system 10 offers a number of improvements not available in conventional VR systems as follows:Feature Your Adaptive Coaching Agent Multimodal In ut Integrates pose, motion, object use, protocol context, and scene data in real timeUses neural networks to classify learner skill level and Personalized Skill EstimationprogressSafety-Constrained Integrates a rule-based safety module that halts or changes Adaptation tasks if danger is detectedUses reinforcement learning to optimize feedback per RL-Based Policy Adaptationindividual user in real-timeOlfactory Feedback Novel addition of smell-based triggers for realism / training Integration immersionCross-Modal Feedback Chooses audio, visual, haptic feedback dynamically based Control on performanceBuilt for structured task protocols (e.g., medical, industrial) Protocol-Centric Coachingwith rule enforcementSession Logging for AfterLogs full sensor + action context for later analysis and skill Action Review assessment

[0199] The use of olfactory feedback provides for further immersion of the user which leads to better absorption. The scents used should align with training goals to reinforce memory, alert users, or simulate realism:Scent Type Purpose I Psychological CueGasoline I Burning Alerts the player to hazardous environments (fire, gas leaks, car Rubber crash).Reinforces medical context and procedural realism during wound Antiseptic / Alcoholcleaning or dressing.Smoke I CharredAssociated with campfire or bum injury tutorial.Wood

[0200] In embodiments a scent emitting device 17 may be used as one of the feedback devices 17 and may be a compact, programmable hardware unit designed to release precise doses of aroma compounds into the air based on digital signals or triggers from the game engineer 12 (or specifically from the Al engine).

[0201] The system 10 provides certain serious game features that are also not typically available in conventional VR training systems:Mechanism DescriptionReal-time customization of environment and Al-driven scenario adaptationfeedback464912-3403-0738v.lAtty. Dkt. No. 88990-160Mechanism Description Multisensory feedback (smell +Engages olfactory and tactile senses for realism touch)Teleportation anchors Guided, anchor-based progressionAdaptive corrective feedback Immediate behavioral correction via AlColor-coded cognition system Uses color and positioning to maintain flowAl performance profiling Continuous learning analytics loop

[0202] The system 10 introduces an Al-adaptive, multi-sensory interaction loop that merges player action, Al guidance, and real-time sensory correction within the same gameplay moment.

[0203] Unlike conventional VR simulations the present system:• Reacts dynamically to each player’s behavioral pattern and error type.• Provides tactile, visual, auditory, and even olfactory feedback based on the player’s specific interaction.• Integrates Al-driven guidance that changes the pace and content of training scenes as the player performs actions.

[0204] For example, in a scene of a scenario, when a player bums their virtual hand while lighting a campfire, the system 10 immediately detects the error, displays a red UI warning, generates a pain-related haptic pulse, and instructs the player to cool the injury under virtual water reinforcing correct first-aid procedure through embodied learning rather than simple on-screen messaging.

[0205] Thus, even errors in the first aid protocol may be turned into usefill training using sensory, cognitive, and procedural feedback in a virtual embodiment of real-world learning consequences.

[0206] A novel gameplay interaction within the system allows players to experience real-time Al-guided corrective feedback through combined haptic, auditory, and olfactory feedback responses. This mechanism transforms mistakes into embodied learning moments This adaptive, multisensory feedback loop is not present in conventional VR learning systems and enables personalized, realistic first aid training

[0207] While playing the game, the player will be completing a chain of steps in every scene to achieve the goal of the respective scene. Learning when and how to put someone in the PLS recovery position is an example. The Game Manager includes a game 474912-3403-0738v.lAtty. Dkt. No. 88990-160object in every scene of the game, to which a Game Manager script is attached. This script holds references to the majority of the game objects present in the scene and manages them while the player is progressing through the level and the state of the game is changing. More specifically, the Game Manager script monitors the events triggered by the Game Objects in the scene once the player has interacted with them and completes a step towards the level completion and either channels the information to other game objects (also by triggering events indicating a state change in the level) to act accordingly or changes their state itself.

[0208] Now that embodiments of the present invention have been shown and described in detail, various modifications and improvements thereon can become readily apparent to those skilled in the art. Accordingly, the exemplary embodiments of the present invention, as set forth above, are intended to be illustrative, not limiting. The spirit and scope of the present invention is to be construed broadly.484912-3403-0738v.l

Claims

Atty. Dkt. No. 88990-160What is claimed is:

1. A virtual reality first aid training system, comprising:a game engine generating virtual reality information associated with a first aid scenario;a virtual reality headset operably connected to the game engine; anda controller operably connected to the game engine,wherein the game engine provides the virtual reality information to the virtual reality headset and the virtual reality headset presents the first aid scenario on a display of the virtual reality headset,wherein the first aid scenario includes a plurality of scenes sequentially presented in the display of the virtual reality headset to lead a user in a step by step fashion through a process of performing first aid for a particular type of emergency simulated by the virtual reality headset based on the first aid scenario information, andwherein the controller provides input to the game engine to advance the user through the virtual reality scenario.

2. The virtual reality first aid training of system claim 1 , wherein the virtual reality information includes visual information, audio information and feedback information provided to the user as the user advances through the first aid scenario.

3. The virtual reality first aid training system of claim 2, wherein the virtual reality headset includes one or more speakers configured to present audio based on the audio information.

4. The virtual reality first aid training system of claim 2, wherein the virtual reality headset further comprises a microphone operable to provide voice information to the game engine.

5. The virtual reality first aid training system of claim 2, further comprising a first feedback device operably connected to the game engine and configured to receive first feedback information to provide tactile feedback to the user as the user advances through the first aid scenario.494912-3403-0738v.lAtty. Dkt. No. 88990-1606. The virtual reality first aid training system of claim 2, further comprising a second feedback device operably connected to the game engine and configured to provide olfactory feedback to the user as the user advanced through the first aid scenario.

7. The virtual reality first aid training system of claim 3, wherein the one or more speakers receive third feedback information that includes audio feedback information to provide audio feedback as the user moves through the scenario.

8. The virtual reality first aid training system of claim 1 , wherein the first aid scenario includes a plurality of scenes to train the user in a first skill.

9. The virtual reality first aid training system of claim 1, wherein the first aid scenario includes a plurality of scenes to train the user in a second skill.

10. The virtual reality first aid training system of claim 1, wherein the virtual reality information includes video information, audio information, tactile feedback information and olfactory information that is presented to the user as the user advances through the first aid scenario.

11. The virtual reality first aid training system of claim 1, wherein the game engine generates a virtual hand controlled by inputs from the user using the controller to interact with virtual objects visually in one or more scenes the first aid scenario from the plurality of first aid scenarios.

12. The virtual reality first aid training system of claim 1, wherein the game engine utilizes artificial intelligence to generate the virtual reality information used to present the first aid scenario.

13. The virtual reality first aid training system of claim 12, wherein the game engine includes an artificial intelligence engine, the artificial intelligence engine implementing a scenario generator module that generates the virtual reality information including a virtual environment and tasks, a customization module that updates the virtual reality information to dynamically adjust scenario complexity and sensory feedback based on real-time user504912-3403-0738v.lAtty. Dkt. No. 88990-160interactions, a feedback integration module that provides haptic, auditory, and olfactory feedback information, and an adaptation and monitoring module that continuously monitors user actions and makes scenario adaptations accordingly using the updated virtual reality information.

14. The virtual reality first aid training system of claim 13, wherein the scenario generator module uses a first machine learning algorithm that uses user information and first aid template information as an input and is trained using prior user information and prior first aid protocol information to provide an output related to virtual reality information.

15. The virtual reality first aid training system of claim 13, wherein the output is virtual reality information.

16. The virtual reality first aid training system of claim 13, wherein the output is used by the game engine to provide virtual reality information.

17. The virtual reality first aid training system of claim 13, wherein the user information includes a username, demographic information, training level information, first aid experience information, first aid protocol information and prior training outcome information.

18. The virtual reality first aid training system of claim 13, wherein the customization module uses a second machine learning algorithm that uses user information including activity information as an input and is trained using prior user information and activity information to provide a second output associated with updated virtual reality information to modify the first aid scenario in accordance with performance of the user.

19. The virtual reality first aid training system of claim 18, wherein the second output is provided to the game engine to provide updated virtual reality information that modifies the first aid scenario to aid the user in completing the first aid scenario.514912-3403-0738v.lAtty. Dkt. No. 88990-16020. The virtual reality first aid training system of claim 19, wherein the updated virtual reality information modifies the first aid scenario to make it more difficult for the user in completing the first aid scenario.

21. The virtual reality first aid trainer system of claim 13, wherein the scenario generator module selects or generates a virtual environment for the first aid scenario from a plurality of scenarios.

22. The virtual reality first aid training system of claim 19, wherein the updated virtual reality information adjusts scene complexity of the first aid scenario in real-time.

23. The virtual reality first aid training system of claim 13, wherein the scenario generator module integrates haptic, auditory, and olfactory feedback into the virtual reality information in at least one scene of the first aid scenario.

24. The virtual reality first aid training system of claim 13, wherein the adaptation and monitoring module continuously monitors user actions and the customization module updates the first aid scenario information to make scenario adjustments in response to user actions.

25. The virtual reality first aid training system of claim 13, wherein the adaptation and monitoring module evaluates user actions to analyze performance of the user and generates a report that includes user strength information and user weaknesses information.

26. The virtual reality first aid training system of claim 1, further comprising a user profile database containing data related to user skills, previous performance and learning objectives and a scenario database containing additional first aid scenarios, first aid protocols, and environmental templates.

27. The virtual reality first aid training system of claim 1, wherein the game engine includes a virtual environment Tenderer that generates virtual reality information associated with scenes of the first aid scenarios with all visual, auditory, and haptic elements.524912-3403-0738v.lAtty. Dkt. No. 88990-16028. The virtual reality first aid training system of claim 1, wherein the game engine further comprises a performance analysis module that evaluates user performance, generates reports and feeds data back into an artificial intelligence engine for training.

29. The virtual reality first aid training system of claim 1, wherein the virtual reality helmet includes an accelerometer and a gyroscope configured to measure rotation and orientation of the virtual reality headset to determine a sense of direction in the scene on the display.

30. The virtual reality first aid training system of claim 29, wherein the virtual reality helmet includes a magnetometer configured to determine orientation of the headset 14 relative to a magnetic field of the Earth.

31. The virtual reality first aid training system of claim 30, wherein the virtual reality helmet includes an internal camera positioned to track eye movement to determine progress in the first aid scenario.

32. The virtual reality first aid training system of claim 30, wherein the virtual reality helmet includes an olfactory feedback element configured to provide olfactory feedback based on olfactory feedback information provided by the virtual game engine.

33. A method of providing first aid training comprises:providing virtual reality information associated with a first aid scenario to a virtual reality headset, wherein the first aid scenario is rendered on a display of the virtual headset;monitoring user activity information as a user progresses through the first aid scenario;providing updated virtual reality information based on user information including the user activity information; andproviding the updated virtual reality information to the virtual reality headset to personalize the first aid scenario.

34. The method of claim 33, wherein the virtual reality information includes visual information, audio information, olfactory information and haptic feedback information.534912-3403-0738v.lAtty. Dkt. No. 88990-16035. The method of claim 33, wherein the step of providing the virtual reality information includes using artificial intelligence to generate the virtual reality information.

36. The method of claim 35, wherein the step of providing virtual reality information includes receiving as an input to a first machine learning algorithm user information, wherein the first machine learning algorithm is trained using prior user information and first aid scenario template information and the first machine learning algorithm provides a first output associated with the virtual reality information to present the first aid scenario to a particular user associated with the user information.

37. The method of claim 36, wherein the first output is provided to a virtual reality game engine and the virtual reality information is provided based on the first output.

38. The method of claim 36, wherein the first output is the virtual reality information.

39. The method of claim 33, wherein the step of updating the virtual reality information incudes receiving as an input to a second machine learning algorithm user information including user activity information in real time, wherein the second machine learning algorithm is trained using prior user activity information and first aid scenario template information and provides a second output associated with the updated virtual reality information.

40. The method of claim 39, wherein the second output is provided to a virtual reality game engine and the updated virtual reality information is provided based on the first output.

41. The method of claim 39, wherein the second output is the updated virtual reality information.

41. The method of claim 33, wherein the virtual reality information includes visual information, audio information and feedback information provided to the user as the user advances through the first aid scenario.544912-3403-0738v.lAtty. Dkt. No. 88990-16042. The method of claim 33, wherein the virtual reality headset includes one or more speakers configured to present audio based on the audio information.

43. The method of claim 33, wherein the virtual reality headset further comprises a microphone operable to provide voice information to the game engine.

44. The method of claim 33, wherein a first feedback device is operably connected to the game engine and configured to receive first feedback information to provide tactile feedback to the user as the user advances through the first aid scenario.

45. The method of claim 33, wherein a second feedback device is operably connected to the game engine and configured to provide olfactory feedback to the user as the user advanced through the first aid scenario.

46. The method of claim 42, wherein the one or more speakers receive third feedback information that includes audio feedback information to provide audio feedback as the user moves through the scenario.

47. The method of claim 33, wherein the first aid scenario includes a plurality of scenes to train the user in a first skill.

48. The method of claim 33, wherein the first aid scenario includes a plurality of scenes to train the user in a second skill.

49. The method of claim 37, wherein the game engine generates a virtual hand controlled by inputs from the user using the controller to interact with virtual objects visually in one or more scenes the first aid scenario from the plurality of first aid scenarios.

50. The method of claim 37, wherein the game engine includes an artificial intelligence engine, the artificial intelligence engine implementing a scenario generator module that generates the virtual reality information including a virtual environment and tasks, a customization module that updates the virtual reality information to dynamically adjust scenario complexity and sensory feedback based on real-time user interactions to554912-3403-0738v.lAtty. Dkt. No. 88990-160provide updated virtual reality information, a feedback integration module that provides haptic, auditory, and olfactory feedback information, and an adaptation and monitoring module that continuously monitors user actions and makes scenario adaptations accordingly using the updated virtual reality information.

51. The method of claim 50, wherein the scenario generator module selects or generates a virtual environment for the first aid scenario from a plurality of scenarios.564912-3403-0738v.l

Images