Automatically generating audio responses for simulation systems

Abstract

A simulation system may automatically generate an audible response for a non-playable character ("NPC") of a simulation provided via the simulation system. The audible response may be provided in accordance with a dialog response for the NPC. The dialog response may comprise a computer-generated dialog response. The dialog response may be generated in accordance with determining a predetermined dialog response does not match a user input. A natural language understanding analysis may be applied to an audio input to generate intent information usable to determine whether the predetermined dialog response matches the user input. The dialog response may be generated in accordance with simulation information associated with the simulation. The dialog response may be automatically generated using a generative large language model executed by a generative computing device.

Description

TITLE: AUTOMATICALLY GENERATING AUDIO RESPONSES FOR SIMULATION SYSTEMSFIELD OF THE INVENTION

[0001] Embodiments according to various aspects of the present disclosure relate to simulation systems usable to provide training. Particularly, various embodiments relate to systems, devices, and methods for generating an audio response for a virtual character in a simulation provided via a simulation system.BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The subject matter of the present disclosure is particularly pointed out and distinctly claimed in the concluding portion of the specification. A more complete understanding of the present disclosure, however, may best be obtained by referring to the detailed description and claims when considered in connection with the following illustrative figures. In the following figures, like reference numbers refer to similar elements and steps throughout the figures.

[0003] FIG. 1 illustrates a simulation system, in accordance with various embodiments described herein;

[0004] FIG. 2 illustrates a schematic diagram of devices of a simulation system, in accordance with various embodiments described herein;

[0005] FIG. 3 illustrates an image of an example simulation provided by a simulation system, in accordance with various embodiments described herein;

[0006] FIG. 4 illustrates a method of automatically generating an audio response in accordance with various embodiments described herein;

[0007] FIG. 5 illustrates a method of automatically generating an audio response in accordance with various embodiments described herein; and

[0008] FIG. 6 illustrates a method of automatically generating a response for a non-playable character in accordance with various embodiments described herein.

[0009] Elements and steps in the figures are illustrated for simplicity and clarity and have not necessarily been rendered according to any particular sequence. For example, steps that may be performed concurrently or in different order are illustrated in the figures to help to improve understanding of embodiments of the present disclosure.DETAILED DESCRIPTION

[0010] The detailed description of exemplary embodiments herein makes reference to the accompanying drawings, which show exemplary embodiments by way of illustration. While these embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosures, it should be understood that other embodiments may be realized and that logical changes and adaptations in design and construction may be made in accordance with this disclosure and the teachings herein. Thus, the detailed description herein is presented for purposes of illustration only and not of limitation.

[0011] The scope of the disclosure is defined by the appended claims and their legal equivalents rather than by merely the examples described. For example, the steps recited in any of the method or process descriptions may be executed in any order and are not necessarily limited to the order presented. Furthermore, any reference to singular includes plural embodiments, and any reference to more than one component or step may include a singular embodiment or step. Also, any reference to attached, fixed, coupled, connected, or the like may include permanent, removable, temporary, partial, full, and / or any other possible attachment option.

[0012] Enforcement personnel (e.g., law enforcement officers, police officers, border patrol officers, security guards, etc.) are issued equipment in order to perform their duties. Such equipment may include lethal weapons (e.g., firearms, guns, handguns, etc.), less lethal weapons (e.g., conducted electrical weapons [“CEWs”], etc.), communication devices (e.g., radios, mobile phones, etc.), and recording devices (e.g., wearable cameras, body-worn cameras, wireless microphones, etc.). Enforcement personnel are required to practice with equipment in order to improve situational awareness, decision making and problem-solving skills, stress management, and safe and effective use of the equipment in varying situations. Preferably, training is performed on equipment as similar as possible to the equipment enforcement personnel use in the field.

[0013] Enforcement personnel may also encounter a variety of situations during the course of performing their duties. Certain situations may be stressful, unfamiliar, or otherwise challenging for various reasons. Written training materials may educate and prepare personnel for these situations. However, reading about situations may fail to convey an extent of challenges that personnel may encounter during these situations. Training using of role playing may be used toconvey auditory and physical challenges of certain situations, though such training may be difficult to repeat for multiple users and it may be difficult to find live actors that are familiar with the physical, emotional, and auditory cues that may be associated with different situations.

[0014] Simulation based training provides a means for enforcement personnel to repeatably practice with equipment in a variety of situations without the need for using live actors. Simulation based training may use simulated environments to advance the degree of environmental and psychological fidelity available in training. Simulated environments may include augmented reality (AR) environments and virtual reality (VR) environments.

[0015] An augmented reality (AR) environment may refer to a user’s perception of a real, physical environment with the addition of generated virtual, two- or three-dimensional objects in the real environment. Virtual objects may be perceived to exist in the real space as if they were real objects, with the ability of users to move around the virtual objects, see the virtual objects from different angles, and interact with the virtual objects, as appropriate.

[0016] A virtual reality (VR) environment may include a wholly generated virtual environment which may include images, sounds, and other sensations intended to replicate a real or imaginary environment. A virtual environment may simulate a user's physical presence in the virtual environment and enable the user to interact with the virtual space. Virtual environments may be implemented to provide an extensive, seemingly infinite number of environments and training simulations.

[0017] Establishing an augmented reality environment within a real space may include projecting computer generated virtual objects into the space, where the virtual objects behave as if they are physically in the space. In an augmented reality environment, one or more users may be able to see each other (i.e., actual or virtual representations of each other) and the virtual objects, and interact with the virtual objects and each other.

[0018] Virtual reality environments may be established independent of the space and include an entirely virtual space and virtual objects. Virtual representation of one or more users may interact with each other and with the virtual space and virtual objects therein. In a virtual reality system, an entirely virtual image may be simulated for display to a user, and in an augmented reality system, a simulated image may be overlaid or otherwise incorporated with an actual image for display to the user.

[0019] A real-world experience may be simulated using an immersive virtual reality or augmented reality. Simulations of real-world experiences may offer enforcement personnel vital training in responding to various simulations they may encounter in their jobs. Simulations may present multiple decision-making steps, allowing the user to rapidly iterate and learn from the outcomes of each of their choices.

[0020] A simulation system may provide feedback (e.g., immediate feedback and / or delayed feedback) based on the user's selections and actions during the simulation. The feedback may include virtual representations of non-player characters (NPCs) during the simulation, and / or a generated summary at the end of the simulation. The simulation system may also compute and provide various metrics and statistics of performance. Additionally, or alternatively, the feedback may include visual, audible, or haptic signals during the simulation and / or at the end of the simulation.

[0021] A simulation system may provide a platform on which enforcement personnel may experience and train in various simulations. The simulation system may compute and display a visual virtual or augmented reality model of an environment, and in accordance with the user's gestures, actions, and / or interactions with equipment, may provide feedback, such as visual, audible, or haptic signals.

[0022] A simulation system may include a simulator and a simulation equipment. The simulator may comprise a display. The simulation equipment may comprise a weapon. The simulation equipment may be configured to be virtually projected as virtual equipment in a simulated environment that is generated by the simulator and projected on the display. The display may include an augmented reality display device that allows virtual objects to be represented in a real space, or a virtual reality display device that visually immerses a user in an entirely generated virtual environment. The simulator may be configured to receive inputs from the simulation equipment to track virtual equipment corresponding with the equipment in a simulated environment.

[0023] Simulation equipment may include real equipment adapted to be used in a simulation system. Simulation equipment may be configured to provide visual, audible, or haptic inputs to a simulator. Simulation equipment may be displayed as virtual equipment in a simulated environment. A user’s interactions with simulation equipment in real space may correspond with the user’s interactions with virtual equipment in a simulated environment. At least one of achange in position, a change in orientation, an initial orientation, and an operation of the simulation equipment in a real environment may correspond with at least one of a change in position, a change in orientation, an initial orientation, and an operation of the virtual equipment in the simulated environment.

[0024] A real-world experience may be further provided by enabling interaction with a simulated environment in a natural, intuitive manner. Such interaction may include audio input from a user. For example, embodiments according to various aspects of the present disclosure enable a user to interact with a simulated environment using their voice. Such interaction may also include hand gestures, bodily movement, and simulated physical contact with objects in a simulated environment. Devices in a simulation system may avoid wired connections between devices, enabling use of equipment in wireless manner, consistent with how such equipment may be used in a real-world situation. Embodiments according to various aspects of the present disclosure provide these and other improvements to the technical field of simulation-based training.

[0025] In embodiments, simulation provided via simulation system may involve a virtual character. The virtual character may comprise a non-playable character or “NPC”. The virtual character, or NPC, may not be controlled by a user of the simulation system. The virtual character may be designed to replicate a person a user may encounter in a real-world environment. For example, in a training simulation for a law enforcement officer, a virtual character provided in a training simulation may be represent a real-world person that is experiencing a specific medical issue or exhibits an uncommon behavior. Based on their experience with the training simulation, a user may be better prepared to address a similar situation involving a real person in a real-world environment.

[0026] In embodiments, a simulated experience may comprise a narrative. A narrative may relate to an intended goal or end state of the simulated experience. A narrative may comprise audible and / or visual information provided by the simulation system in accordance with a set of rules. The set of rules may determine how a simulated environment may change responsive to a user input. For example, the set of rules may define changes in a visual appearance and / or audible response of an NPC responsive to a user input. A simulation system may apply the set of rules to the user input to determine one or more changes in visual appearance and / or audible response of the NPC.

[0027] Tn embodiments, audible information provided by an NPC may comprise an audible response from the NPC. The audible response may comprise a spoken response from the NPC. The audible response may be provided to a user via an audio transducer. For example, a simulation system may comprise a headset by which an audible response and other audible information for a simulated experience may be provided to a user.

[0028] In embodiments, text of an audible response may be predetermined. For example, a narrative may comprise a number of manually generated text responses for a given NPC. The text responses may be designated for a predetermined user input. Upon receipt of the predetermined user input, a processor of a simulation system may select a predetermined text response from a plurality of text responses to provide as a change in audible information responsive to the predetermined user input.

[0029] In embodiments, user inputs for a simulation system may comprise a variety of different types of inputs. For example, user inputs may include one or more of a verbal or audible input, a motion-based input, and / or an equipment-related input. A user input may comprise a combination of such different types of inputs.

[0030] In embodiments, a complexity of user inputs may make reliance of predetermined audible responses from an NPC time prohibitive. For example, the complexity of user inputs for an increasing combination of different types of inputs may require an extensive number of bespoke, predetermined audible responses to be generated. Further, a plurality of user inputs may not be related to a full range of user inputs that may be provided via a simulation system. For example, a user input may comprise an audible input referring to a virtual object in the simulated experience. A plurality of predetermined audible responses may not include an audible response related to the virtual object, causing an unrelated response or no response to be provided by the NPC. Such a response may be detrimental to the otherwise immersive experience. Such a response may also indicate to a user that provided the input that the user input was incorrectly received and / or the simulation system may not otherwise be functioning as intended. The plurality of predetermined audible responses may also require a substantial amount of computer-readable storage for storage on a computing device, even though none or few of the responses may be provided in given simulated experience. Embodiments according to various aspects of the present disclosure address technical limitations and deficiencies of a simulation system that enables a range of user inputs. For example, embodiments according tothe present disclosure decrease a number of predetermined audible responses that may be necessary to provide a simulated experience. Alternately or additionally, embodiments according to various aspects of the present disclosure enable a variety of non-predetermined user inputs to be provided to simulation system. The simulation system may further provide a related response to each of the variety of non-predetermined user inputs in a controlled manner, despite the simulation system not being previously programmed to specifically address the full variety of user inputs.

[0031] Embodiments according to various aspects of the present disclosure enable a simulation system to be provided with or without additional devices. For example, embodiments according to various aspects of the present disclosure may enable user input through the use of existing equipment and / or exact replicas of existing equipment. The devices may enable a training simulation to be provided to a user with or without additional equipment placed on a hand of a user. The devices may enable a training simulation to be provided without and / or independent of equipping a physical space (e.g., training room) with various external sensors, thereby enabling training to be performed in variety of physical location and / or with minimal setup.

[0032] In various embodiments, and with reference to FIG. 1, a simulation system 100 is disclosed. Simulation system 100 may be similar to, or have similar aspects and / or components with, the simulation systems previously and / or subsequently discussed herein. It should be understood by one skilled in the art that FIG. l is a schematic representation of simulation system 100, and one or more of the components of simulation system 100 may be located in one or more suitable positions within or external to simulation system 100. Simulation system 100 may a comprise a headset 110, at least one equipment 120, a server 130, a trainer computing device 150. Such devices may be in communication via one or more communication networks. For example, headset 110, server 130, and trainer computing device 150 may be in communication via network 140. Network 140 may comprise one or more wired and / or wireless connections. Alternately or additionally, headset 110 and equipment 120 may be in direct communication via short range wireless connection. Headset 110, alone or in combination with server 130, may perform operations of a simulation system as referenced herein.

[0033] In various embodiments, headset 110 may be configured to receive a user input. The user input may comprise a signal associated with an action of a user. Headset 110 may comprisea VR headset and / or an AR headset. Headset 1 10 may comprise at least one camera 112 configured to capture image data (e.g., an image, video, video data, video signal etc.) in which a physical input (e.g., motion, movement, pose, location, etc.) of a portion of a user is represented. Headset 110 may comprise at least one microphone 114 configured to capture an audio input (e.g., audio data, audio signal, etc.) in which an audio input from a user is provided. The user input received by headset 110 may comprise the audio input. Headset 110 may comprise at least one wireless communication interface (not shown) configured to receive a physical input corresponding to user control of equipment 120.

[0034] While camera 112 and microphone 114 are illustrated as part of headset 110 in system 100, such components may alternately or additionally be provided in other computing devices of a simulation system according to various aspects of the present disclosure. For example, one or more image sensors may be employed by a simulation system, in addition to or as an alternative to camera 112. One or more audio transducers may be employed by a simulation system, in addition to or as an alternative to microphone 114. Such image sensor(s) and / or audio transducer(s) may be positioned at alternate location on a user and / or disposed separately within a proximate physical environment in which a headset of a simulation system is employed by a user.

[0035] In various embodiments, headset 110 may be configured to provide a simulated environment to a user. Headset 110 may comprise display 118 to visually output (e.g., present) the simulated environment to the user. In some embodiments, display 118 may be disposed within an entire field of view of an eye or eyes of a user on which headset 110 is disposed.

[0036] In various embodiments, headset 110 may be configured to provide an audio output. The audio output may provide audio signals associated with the simulated environment. The audio signals may comprise speech signals associated with a virtual object in the simulated environment. Headset 110 may comprise at least one output speaker 116 configured to output audio signals to a user. Headset speaker 116 may be positioned on a housing of headset 110 proximate an ear or ears of a user. In other embodiments, one or more other audio transducers may alternately or additionally be provided for a simulation system according to various aspects of the present disclosure. Such transducers may or may not be wearable or non-wearable audio transducers.

[0037] In embodiments, a headset may be configured to communicate with a remote computing device to generate a simulated environment for the user. For example, headset 110may comprise a wired or wireless communication interface configured to communication with server 130 to generate a simulated environment based on user input.

[0038] In various embodiments, equipment 120 may comprise a wearable and / or handheld device used by enforcement personnel. Equipment 120 may be adapted to be used in simulation system 100. Equipment 120 may comprise a wireless device. Equipment 120 may comprise a same form factor as a corresponding real-world device.

[0039] In various embodiments, equipment 120 may comprise one of a CEW or a firearm. In real-world usage, a CEW may be used to deliver a current (e.g., stimulus signal, pulses of current, pulses of charge, etc.) through tissue of a human or animal target. The stimulus signal may carry a charge into target tissue. The stimulus signal may interfere with voluntary locomotion of the target. The stimulus signal may cause pain. The pain may also function to encourage the target to stop moving. The stimulus signal may cause skeletal muscles of the target to become stiff (e.g., lock up, freeze, etc.). The stiffening of the muscles in response to a stimulus signal may be referred to as neuromuscular incapacitation (“NMI”). NMI disrupts voluntary control of the muscles of the target. The inability of the target to control its muscles interferes with locomotion by the target. When used with simulation system 100, the CEW may be adapted (e.g., configured, modified, etc.) to deploy non-functional electrodes and / or provide audible or visual feedback that simulates operations associated with providing a stimulus signal. Alternately or additionally, feedback associated with a status of equipment 120 may be provided via headset 110. In embodiments, equipment 120 may alternately or additionally comprise one of a firearm or a bola launcher. In various embodiments, multiple equipment 120 may be provided, wherein each equipment 120 is configured to provide one or more operations of equipment 120 disclosed herein.

[0040] In various embodiments, equipment 120 may be configured to transmit an input signal corresponding to at least one of a position (e.g., orientation, pose, motion, movement, rotation, etc.) or a status (e.g., operation, activation, actuation of user interface, etc.) of equipment 120 to headset 110. Headset 110 may perform operations on the input signal and provide an output signal to display 118 to proj ect virtual equipment in a simulated environment corresponding to equipment 120.

[0041] In various embodiments, headset 110 may be configured to receive an input signal corresponding with at least one of a motion and an operation of equipment 120. Responsive to the input signal, headset 110 may project a virtual equipment corresponding with the input signal ondisplay 1 18 of headset 110. Headset 1 10 may comprise a processing circuit and a display configured to generate a simulated environment. In embodiments, the processing circuit and the display may be integrated into headset 110. In other embodiments, the processing circuit and / or display may be integrated in a separate computing device communicatively coupled with headset 110. Headset 110 may comprise a wearable unit configured to be worn on a head of a user to position the display close to eyes of a user. Headset 110 may be fully supported (e.g. mounted, worn, etc.) on the head of the user.

[0042] In various embodiments, simulation system 100 may comprise a server configured to aid in analyzing a user input to generate output data to be displayed by a display of headset 110. Server 130 may be in wired and / or wireless communication with one or more of headset 110 and equipment 120 via a network, such as network 140. Network 140 may comprise a wireless communication network using one or more wireless communication protocols, such as WIFI, 2G, 3G, 4G, 5G, LTE, WIMAX, BLUETOOTH, and the like.

[0043] In various embodiments, a user input (e.g., user input signal, user input data, etc.) may be uploaded to server 130. Various methods may be used to upload an input signal from a headset 110 and / or equipment 120 to server 130. In an implementation, an equipment may upload an input signal to a server via a wireless connection to a network. For example, equipment 120 may wirelessly transmit an input signal corresponding to a user input to server 130 via network 140. In another implementation, equipment 120 may transmit an input signal to server 130 via a wired connection to network 140.

[0044] A server may store a user input. A server may analyze a user input. For example, a server may store and / or analyze an input provided by a CEW. A server may analyze the user input from one or more simulation equipment to determine one or more of a change in position, a change in orientation, an initial orientation, and an operation of the one or more simulation equipment. A server may be configured to generate a virtual equipment in a virtual environment, wherein movement and / or operation of the virtual equipment corresponds with movement and / or operation of a simulation equipment.

[0045] A server may generate a virtual environment, virtual objects, or a combination of a virtual environment and virtual objects. The generated virtual environment and / or virtual objects may be projected on a display. At least one of a change in position, change in orientation, initial orientation, and operation of a virtual object may correspond with a real object. For example, animage of a virtual equipment presented in display 1 18 of headset 110 may correspond to a real equipment 120 in communication with headset 110.

[0046] In embodiments, simulation system 100 may comprise a generative computing device 150. Generative computing device 150 may be configured to generate media presented via headset 110. Prompts may be received via a network interface of generative computing device 150. Responsive to the prompts, generative computing device 150 may generate media. The media may be further provided to one or more of server 130 or headset 110 for inclusion in a virtual environment presented via a display of headset. In embodiments, generative computing device 150 may be configured to receive information regarding operation of headset 110 and / or equipment 120. For example, generative computing device 150 may receive a prompt comprising text data associated with an audible input received via microphone 114 of headset 110 so that generative computing device 150 may generate media related to a user input provided a user of headset 110.

[0047] In embodiments, a simulation system may comprise a plurality of computing devices configured to present a simulation. The simulation may comprise a training simulation. For example, and with reference to FIG. 2, simulation system 200 may comprise wearable computing device 210 and server computing device 230. Simulation system 200 may further include, or at least be communicatively coupled to equipment 220 and / or generative computing device 250. Wearable computing device 210 and a server computing device 230 may be configured to receive one or more user inputs and, responsive to the one or more user inputs, present a training simulation comprising one or more visual simulation states via wearable computing device 210. In embodiments, simulation system 200 may comprise equipment 220 and wearable computing device 210, and server computing device 230, and may be configured to detect equipment 220 and present a virtual equipment corresponding to equipment 220 via wearable computing device 210. In embodiments, simulation system 200 may perform one or more operations of simulation system 100 (with brief reference to FIG. 1). In other embodiments according to various aspects of the present disclosure, one or more elements of wearable computing device 210 and server computing device 230 may be combined into a common wearable or non-wearable computing device. In such embodiments, the computing device may be further communicatively coupled with equipment 220 and / or other user interface devices to receive user inputs. The computing device in such device may also be communicatively coupled to audio / visual equipment in orderto provide audio-visual information to a user of the system. For example, such a device may be communicatively coupled to an image sensor and audio sensor to receive user inputs, as well as communicatively coupled to a speaker and display on order to provide audible and visual outputs.

[0048] In embodiments, wearable computing device 210 may be configured to receive user inputs and present one or more visual simulations states to a user. Wearable computing device 210 may comprise a headset. Wearable computing device 210 may comprise a VR headset. A housing of wearable computing device 210 may be configured to be mounted on a head of a user. Wearable computing device 210 may comprise one or more hardware and / or software components. For example, wearable computing device 210 may comprise processor 212, memory 214, wireless communication interface 216, display 218, input audio interface 211, image sensor 213, output audio interface 215, position sensor 217, object detection module 219A, and / or equipment position detection module 219B. In embodiments, and with brief reference to FIG. 1, wearable computing device 210 may perform one or more operations of headset 110.

[0049] In various embodiments, processor 212 may be similar to other processing unit or units, processor, or the like described herein. Processor 212 may comprise circuitry, electrical components, electronic components, software, and / or the like configured to perform various operations and functions discussed herein. For example, processor 212 may comprise a processing unit, a processing circuit, a processor, a digital signal processor, a microcontroller, a microprocessor, an application specific integrated circuit (ASIC), a programmable logic device, logic circuitry, state machines, MEMS devices, signal conditioning circuitry, communication circuitry, a computer, a computer-based system, a radio, a network appliance, a data bus, an address bus, and / or combination(s) of such components.

[0050] Processor 212 may be configured to provide and / or receive electrical signals whether digital and / or analog in form. Processor 212 may provide and / or receive digital information via a data bus using a protocol. Processor 212 may receive information, manipulate the received information, and provide the manipulated information. Processor 212 may store information and retrieve stored information. Information received, stored, and / or manipulated by processor 212 may be used to perform a function, control a function, and / or to perform an operation or execute a stored program.

[0051] Processor 212 may control the operation and / or function of other circuits and / or components of wearable computing device 210. Processor 212 may receive status information regarding the operation of other components, perform calculations with respect to the status information, and provide commands (e.g., instructions) to one or more other components. Processor 212 may command another component to start operation, continue operation, alter operation, suspend operation, cease operation, or the like. Commands and / or status may be communicated between processor 212 and other circuits and / or components via various types of electronic communication.

[0052] In various embodiments, processor 212 may also comprise (or be in electronic communication with) a memory unit capable of storing and maintaining data. The memory unit may comprise any memory unit, database, data structure, memory component, or the like disclosed herein.

[0053] In various embodiments, wearable computing device 210 may comprise a memory. For example, wearable computing device 210 may comprise memory 214. The memory may comprise non-transitory computer-readable storage medium comprising computer-executable instructions that when executed by a processor (e.g., processor 212), perform one or more actions. The non- transitory computer-readable storage medium may be a tangible, non-transitory memory configured to communicate with the processor. In one embodiment, non-transitory computer- readable storage medium contains computer-executable instructions that enable user inputs to be received and one or more simulation states to be presented via a display. Those skilled in the art will appreciate that one or more processors and / or computer-readable storage mediums may be provided as part of one or more various components recited herein.

[0054] In various embodiments, wearable computing device 210 may comprise a wireless communication interface. For example, wearable computing device 210 may comprise wireless communication interface 216. Wireless communication interface 216 may comprise suitable hardware and / or software components capable of enabling the transmission and / or reception of data. Wireless communication interface 216 may enable electronic communications between devices and systems. Wireless communication interface 216 may be configured to communicate via a wireless protocol such as an 802.1 la / b / g / n / ac signal (e.g., WI-FI), a wireless communications protocol using short wavelength UHF radio waves and defined at least in part by IEEE 802.15.1 (e.g., the BLUETOOTH® protocol maintained by Bluetooth Special Interest Group), a wirelesscommunications protocol defined at least in part by IEEE 802.15.4 (e.g., the ZigBee® protocol maintained by the ZigBee alliance), a cellular protocol, an infrared protocol, an optical protocol, or any other protocol capable of transmitting information via a wireless connection. In embodiments, wireless communication interface 216 may enable wearable computing device 210 to communicate with server computing device 230 and / or generative computing device 250 via network 240 using a first type of wireless connection (e.g., a WIFI connection, long-range wireless connection, etc.) and communicate with equipment 220 via a second type of wireless connection (e g., a short-range wireless connection). In embodiments, wireless communication interface 216 may be configured to receive position information from equipment 220.

[0055] In various embodiments, wearable computing device 210 may comprise a display. A display may be configured to present visual information to a user. For example, wearable computing device 210 may comprise display 218. Display 218 may include an augmented reality display (e.g., AR display) or a virtual reality display (e.g., VR display). Display 218 may comprise a head-mounted display. Display 218 may include a light emitting diode (LED) display, liquid crystal display (LCD), organic liquid crystal display (OLED), cathode ray tube (CRT) display, plasma display, quantum dot display, projection display, stereoscopic display, holographic display, near-eye display or other display configured to display visual data to a user. In embodiments, visual information presented to a user via display 218 may comprise a visual representation of an NPC.

[0056] In various embodiments, wearable computing device 210 may comprise at least one input audio interface. An input audio interface may be configured to receive (e.g., detect, transduce, etc.) an audio input from a user. The audio input may comprise an audio signal. The audio input may comprise one or more of an analog audio signal and / or a digital audio signal. The audio input may comprise sounds vocalized by a user. The sounds may comprise one or more words spoken by the user. For example, wearable computing device 210 may comprise input audio interface 211. Input audio interface 211 may comprise a transducer or a connector configured to be coupled to a transducer. In embodiments, input audio interface 211 may comprise at least one microphone, such as microphone 114 with brief reference to FIG. 1. Input audio interface 211 may be positioned proximate to a mouth of a user when wearable computing device 210 is worn by the user.

[0057] In various embodiments, wearable computing device 210 may comprise at least one output audio interface. An output audio interface may be configured to transmit (e g., play, produce, emit, etc.) an audio output to a user. The audio output may comprise an audio signal. The audio output may comprise one or more of an analog audio signal and / or a digital audio signal. The audio output may comprise sounds that may be heard by a user. The sounds may comprise one or more words spoken by a non-playable character or other virtual object or entity in a simulated environment. For example, wearable computing device 210 may comprise output audio interface 215. Output audio interface 215 may comprise a transducer or a connector configured to be coupled to a transducer. In embodiments, output audio interface 215 may comprise at least one speaker (e.g., loudspeaker, headphone, earphone, etc.). Output audio interface 215 may be positioned proximate to an ear of a user when wearable computing device 210 is worn by the user. For example, output audio interface 215 may comprise headset speaker 116 with brief reference to FIG. 1.

[0058] In various embodiments, wearable computing device 210 may comprise at least one position sensor. A position sensor may be configured to detect a position and / or changes in position of a wearable computing device. A position sensor may include a radar-based sensor, an infrared sensor, microwave sensor, gyroscope, ultrasonic detector, acoustic sensor, optical sensor, vibration detector, electromagnetic sensor, accelerometer, inertial measurement unit (IMUs), and / or other device or component capable of detecting movement. For example, wearable computing device 210 may comprise at least one position sensor 217. In embodiments, wearable computing device 210 may comprise more than one position sensor 217. For example, position sensor 217 may comprise an accelerometer and gyroscope that are used to detect movement of wearable computing device 210. Position information detected by position sensor 217 may be provided to processor 212 to enable a simulation state presented via display 218 to be updated in accordance with movement of wearable computing device 210 on a user.

[0059] In various embodiments, wearable computing device 210 may comprise at least one image sensor. An image sensor may be configured to capture image data. The image data may be generated by the image sensor based on light detected by the image sensor. Capturing image data may comprise converting received light into the image data. The image sensor may comprise one of a charge-coupled device and / or an active-pixel sensor. For example, wearable computing device 210 may comprise image sensor 213. In embodiments, image sensor 213alone or in combination with memory 214 and / or processor 212, may perform one or more operations of camera 112 with brief reference to FIG. 1. Image sensor 213 may be configured to capture image data of a real-world environment of a user of simulation system 200 and provide the image data for further processing, including detection a position of equipment 220 as further discussed herein. Image sensor 213 may comprise a forward-facing image sensor oriented to receive light in a direction ahead or forward a user when wearable computing device 210 is mounted on the user. Image sensor 213 may comprise an inward-facing image sensor oriented to receive light in a direction toward an eye of a user when wearable computing device 210 is mounted on the user. In embodiments, wearable computing device 210 may be comprise multiple image sensors.

[0060] In various embodiments, wearable computing device 210 may comprise an object detection module. An object detection module may comprise one or more software and / or hardware components capable of detecting one or more objects. The object detection module may detect specific objects in image data. For example, an object detection module may be configured to detect portion (e.g., body part) of a user. The portion of the user may comprise an arm, wrist, hand, and / or finger(s) of the user. The object detection module may comprise a trained machine learning model configured to receive image data and detect the one or more objects in the image data. The object detection module may provide detection information in accordance with received image data, wherein the detection information indicates the one or more objects are detected in the image data. The detection information may include a location within the image data of the one or more objects. In embodiments, the detection information may comprise three-dimensional information regarding a physical location of an object of the one or more objects in three- dimensional space. The detection information may comprise three degrees of freedom information of the object. The detection information may comprise three degrees of translational position information of the object. In embodiments, the image data may comprise image data from a plurality of image sensors in a wearable computing device that enables the object detection module to triangulate a position of the object in three-dimensional space relative to wearable computing device. For example, wearable computing device 210 may comprise object detection module 219A. In embodiments, object detection module 219A may be executed by processor 212 and / or implemented in wearable computing device 210 via processor 212 and memory 214. For example, object detection module 219A may comprise computer-readable instructions stored in memory214 that, when executed by processor 212, cause wearable computing device 210 to perform one or more operations for detecting one or more objects disclosed herein.

[0061] In various embodiments, wearable computing device 210 may comprise an equipment position detection module. An equipment position detection module may comprise one or more software and / or hardware components capable of detecting a position of one or more pieces of equipment. The equipment position detection module may detect a position of an equipment in accordance with detection information associated with an object and position information received from the equipment. The equipment position detection module may be configured to generate virtual position information in accordance with the detection information and the position information. The virtual position information may comprise an image of the equipment. The virtual position information may comprise an image of the object. The virtual position information may comprise a location for rendering the image(s) of the equipment and / or the object in a simulated environment. The virtual position information may be provided to a processor configured to render the object and / or equipment in a simulated environment via a display of a wearable computing device. For example, wearable computing device 210 may comprise equipment position detection module 219B. In embodiments, equipment position detection module 219B may be executed by processor 212 and / or implemented in wearable computing device 210 via processor 212 and memory 214. For example, equipment position detection module 219B may comprise computer-readable instructions stored in memory 214 that, when executed by processor 212, cause wearable computing device 210 to perform one or more operations for detecting a virtual position of an equipment as further disclosed herein. For example, an equipment position detection module may be configured to detect a position of equipment 220. The equipment position detection module may receive position information from equipment 220 and detection information from object detection module 219A. In accordance with the position information and the detection information, the equipment position detection module may provide virtual position information to processor 212 for rendering virtual equipment in a simulated environment presented via display 218.

[0062] Equipment 220 may be configured to receive one or more user inputs and transmit the one or more user inputs to a wearable computing device. Equipment 220 may comprise nonfunctional or semi-functional training equipment configured to recreate operation of corresponding, functional equipment. Relative to functional equipment, equipment 220 may lackone or more functions. For example, equipment 220 may comprise a simulated firearm that is unable to deploy a projectile. Such an arrangement may increase safety of such equipment 220 as used in a training environment. Such an arrangement may also enable equipment 220 to be modified for specific use with a simulation system. For example, equipment 220 may correspond to a firearm, CEW, handcuffs, pepper spray, a land mobile radio, or a recording device. Equipment 220 may be configured to be carried by a user and / or mounted to a user. Equipment 220 may be configured to be manually positioned by a user. Equipment 220 may comprise various software and / or hardware components. For example, equipment 220 may comprise a processor 222, memory 224, wireless communication interface 226, and / or a user interface 228. In embodiments, and with brief reference to FIG. 1, equipment 220 may perform one or more operations of equipment 120.

[0063] In embodiments, simulation system 200 may comprise multiple equipment. For example, a first equipment of simulation system 200 may comprise a training firearm and a second equipment of simulation system 200 may comprise a training CEW. Each such equipment may respectively comprise one or more components of equipment 220.

[0064] In embodiments, user interface 228 may be configured to receive a user input from a user. The user input may comprise a physical input. For example, user interface 228 may comprise one or more buttons, switches, triggers, or other actuatable physical elements and the manual input may comprise a physical actuation of one or more such elements. In embodiments, user interface 228 may comprise a housing and one or more position sensors coupled to the housing, and the manual input may comprise a physical input applied to the housing that is further detected by the one or more sensors. In embodiments, processor 222 may receive the user input via user interface 228.

[0065] In embodiments, user interface 228 may be configured to generate one or more indications regarding a status of equipment 220. User interface 228 may be configured to provide one or more of audible, visual feedback regarding a status of equipment 220. The feedback may be responsive to a user input received via user interface 228. For example, when a physical input is applied to a trigger of equipment 220, an audible output (e.g., sound) and visual output (e.g., light) may be provided via user interface 228 in accordance with the physical input. In embodiments, inputs and outputs received and provided by user interface 228 may correspond to respective inputs and outputs of equipment 220 when equipment 220 is used in a non-simulated, real-world environment, thereby enabling a user to be trained in a realistic, authentic manner.

[0066] In various embodiments, equipment 220 may comprise at least one sensor. For example, equipment 220 may comprise at least one sensor 229. At least one sensor 229 may be separate from, or provided in addition to, one or more sensors of user interface 228.

[0067] In embodiments, a sensor may comprise a position sensor configured to detect a position and / or changes in position of an equipment. A position sensor may include a radarbased sensor, an infrared sensor, microwave sensor, gyroscope, ultrasonic detector, acoustic sensor, optical sensor, vibration detector, electromagnetic sensor, accelerometer, inertial measurement unit (IMUs), and / or other component capable of detecting movement. Position information detected by position sensor of sensor 229 may be provided to processor 222 to enable a simulation state presented via display 218 to be updated in accordance with movement of equipment 220 by a user.

[0068] In embodiments, a sensor may alternately comprise a physiological sensor configured detect a physiological signal of a user. A physiological sensor may include a heart rate sensor, a pulse sensor, a blood pressure sensor, a temperature sensor, an oxygen saturation sensor, a sweat sensor, and / or another sensor configured to detect a physiological property of a user. The physiological signal detected via a physiological sensor may comprise one or more of a physical biosignal, an electrophysiological signal, and / or an electrochemical sensor. In embodiments, equipment 220 comprising a physiological sensor may comprise a wearable health sensor. For example, equipment 220 may comprise a watch or wrist band comprising one or more physiological sensors. Alternately or additionally, equipment 220 comprising at least one physiological sensor may comprise training equipment in which the at least one physiological sensor has been integrated. For example, a training firearm and / or a training CEW may comprise at least one integrated physiological sensor. Physiological information detected by a physiological sensor of sensor 229 may be provided to processor 222 to enable a simulation state presented via display 218 to be updated in accordance with one or more biological states of a user detected via of equipment 220.

[0069] In embodiments, processor 222 may be configured to receive signals and provide signals that cause equipment 220 to perform one or more operations. Processor 222 may comprise circuitry, electrical components, electronic components, software, and / or the likeconfigured to perform various operations and functions discussed herein. For example, processor 222 may comprise a processing unit described herein, a processing circuit, a processor, a digital signal processor, a microcontroller, a microprocessor, an application specific integrated circuit (ASIC), a programmable logic device, logic circuitry, state machines, MEMS devices, signal conditioning circuitry, communication circuitry, a computer, a computer-based system, a radio, a network appliance, a data bus, an address bus, and / or any combination thereof.

[0070] In various embodiments, equipment 220 may comprise a memory 224. For example, equipment 220 may comprise memory 224. The memory may comprise non-transitory computer- readable storage medium comprising computer-executable instructions that when executed by a processor (e.g., processor 222), perform one or more actions. The non-transitory computer- readable storage medium may be a tangible, non-transitory memory configured to communicate with the processor. In one embodiment, non-transitory computer-readable storage medium contains computer-executable instructions that enable user inputs to be received via user interface 228. The instructions, when executed may cause one or more indications of the user inputs to be output via user interface 228. The non-transitory computer-readable storage medium may comprise computer-executable instructions that, when executed, cause the user inputs to be transmitted from equipment 220 via wireless communication interface 226. For example, the user inputs transmitted via wireless communication interface 226 may comprise status information associated of equipment 220 and / or position information of equipment 220.

[0071] In accordance with instructions stored in memory 224, processor 222 may be configured to receive user inputs via user interface 228 and transmit status information in accordance with the user inputs to another device (e.g., wearable computing device 210). For example, processor 222 may receive user inputs corresponding to actuation of one or more of a safety switch, trigger, warning button, mode selection switch, or other user interface element and determine a change in status of equipment 220 instructed by a user in accordance with the user input. Responsive to the user input, processor 222 may control other components of equipment to change a status of equipment 220. For example, processor 222 may enable deployment of a projectile, deploy a projectile, generate an audible and / or visible warning, change a mode of operation in accordance with a user input received via user interface 228. Processor 222 may further cause status information indicating the change in status of equipment to be transmitted via wireless communication interface 226.

[0072] Tn accordance with instructions stored in memory 224, processor 222 may be configured to receive user inputs via user interface 228 and transmit position information in accordance with the user inputs to another device (e.g., wearable computing device 210). For example, processor 222 may receive user inputs corresponding to movement of a housing of equipment 220 and determine a position and / or change in position of equipment 220 applied by a user in accordance with the user input. Responsive to the user input, processor 222 may determine position information of equipment 220. The position information may comprise a pose of equipment 220. The position information may comprise a rotation of equipment in three-dimensional space. The position information may comprise three degrees of freedom information. The three degrees of freedom information may indicate a yaw, pitch, and roll of equipment 220. Processor 222 may further cause position information indicating the position and / or change in position of equipment to be transmitted via wireless communication interface 226. In embodiments, equipment 220 may comprise sensor 229 communicatively coupled to processor 222, wherein processor 222 is configured to determine a position and / or change in position of equipment 220 in accordance with information received by the processor 222 from sensor 229. Sensor 229 may comprise one or more features or components as discussed with regards to position sensor 217. In embodiments, user interface 228 and / or sensor 229 may comprise at least one position sensor communicatively coupled to processor 222 to enable position information to be generated by equipment 220.

[0073] Wireless communication interface 226 may comprise suitable hardware and / or software components capable of enabling the transmission and / or reception of data. Wireless communication interface 226 may enable electronic communications between devices and systems. Wireless communication interface 226 may be configured to communicate via a wireless protocol such as an 802.1 la / b / g / n / ac signal (e.g., Wi-Fi), a wireless communications protocol using short wavelength UHF radio waves and defined at least in part by IEEE 802.15.1 (e.g., the BLUETOOTH® protocol maintained by Bluetooth Special Interest Group), a wireless communications protocol defined at least in part by IEEE 802.15.4 (e.g., the ZigBee® protocol maintained by the ZigBee alliance), a cellular protocol, an infrared protocol, an optical protocol, or any other protocol capable of transmitting information via a wireless connection. In embodiments, wireless communication interface 226 may enable equipment 220 to communicate with wearable computing device 210 and / or server computing device 230. In embodiments, wireless communication interface 226 may transmit a controller input from equipment 220 via ashort-range wireless connection. The controller input may comprise status information and / or position information. A controller input comprising the status information and / or position information may be transmitted from equipment 220 and received by wearable computing device 210 and / or server computing device 230.

[0074] Server computing device 230 may be configured to receive one or more user inputs, perform an analysis on the one or more user inputs, and / or transmit the analysis. The user inputs may be received from wearable computing device 210 and / or equipment 220. The analysis may be provided to wearable computing device 210 and / or generative computing device 250. Server computing device 230 may comprise various software and / or hardware components. For example, server computing device 230 may comprise processor 232, memory 234, network interface 236, transcription module 231, sentiment detection module 233, target detection module 235, motion detection module 237, and / or intent detection module 238. In embodiments, and with brief reference to FIG. 1, server computing device 230 may perform one or more operations of server 130.

[0075] Computing devices may be appropriate for use in accordance with various embodiments of the present disclosure. While discussed in the context of server computing device 230, the accompanying description of a computing device may be applicable to servers, personal computers, mobile phones, smart phones, tablet computers, embedded computing devices, and other devices that may be used in accordance with embodiments of the present disclosure, including wearable computing device 210 and / or generative computing device 250.

[0076] Server computing device 230 may include processor 232 and a memory 234 connected by a communication bus (not shown). Memory 234 may comprise a system memory. Depending on the exact configuration and type of computing device, the system memory may be volatile or nonvolatile memory, such as read only memory (“ROM”), random access memory (“RAM”), EEPROM, flash memory, or other memory technology. Those of ordinary skill in the art and others will recognize that the system memory typically stores data and / or program modules that are immediately accessible to and / or currently being operated on by processor 232. In this regard, processor 232 may serve as a computational center of server computing device 230 by supporting the execution of instructions. Processor 232 may comprise one or more processing units, as discussed further herein. System memory may comprise one or more memory units, as discussed further herein.

[0077] Server computing device 230 may include a network interface 236 comprising one or more components for communicating with other devices and systems over a network. Embodiments of the present disclosure may access basic services that utilize network interface 236 to perform communications using network protocols.

[0078] Memory 234 may also include a storage medium. However, services may be accessed using a computer-based system that does not include means for persisting data to a local storage medium. The storage medium may be volatile or nonvolatile, removable or nonremovable, implemented using any technology capable of storing information such as, but not limited to, a hard drive, solid state drive, CD-ROM, DVD, or other disk storage, magnetic tape, magnetic disk storage, and / or the like. The storage medium may include one or more memory units, as discussed further herein.

[0079] As used herein, the term “computer-readable medium” includes volatile and nonvolatile and removable and nonremovable media implemented in any method or technology capable of storing information, such as computer-readable instructions, data structures, program modules, or other data. In this regard, system memory and storage medium of memory 234 are examples of computer-readable storage media.

[0080] For ease of illustration and because it is not important for an understanding of the claimed subject matter, FIG. 2 does not show some of the typical components of many computer- based systems. In this regard, server computing device 230 may include input devices, such as a keyboard, keypad, mouse, trackball, microphone, video camera, touchpad, touchscreen, electronic pen, stylus, and / or any other input device described herein. Such input devices may be coupled to server computing device 230 by wired or wireless connections including RF, infrared, serial, parallel, BLUETOOTH®, USB, or other suitable connection protocols using wireless or physical connections.

[0081] In any of the described examples, data can be captured by input devices and transmitted or stored for future processing. The processing may include encoding data streams, which can be subsequently decoded for presentation by output devices. Media data can be captured by multimedia input devices and stored by saving media data streams as files on a computer-readable storage medium (e.g., in memory or persistent storage on a client device, server, administrator device, or some other device). Input devices can be separate from and communicatively coupled to server computing device 230 (e.g., a client device), or can be integral components of servercomputing device 230. In some embodiments, multiple input devices may be combined into a single, multifunction input device (e.g., a video camera with an integrated microphone).

[0082] Server computing device 230 may also include output devices such as a display, speakers, printer, and / or any other output device described herein. The output devices may include video output devices such as a display or touchscreen. The output devices also may include audio output devices such as external speakers or earphones. The output devices can be separate from and communicatively coupled to server computing device 230 or can be integral components of server computing device 230. Input functionality and output functionality may be integrated into the same input / output device (e.g., a touchscreen). A suitable input device, output device, or combined input / output device may be used with described systems.

[0083] In various embodiments, a “processor” or “processing unit” as described herein may comprise any suitable hardware and / or software-based processing component. For example, a processing unit may comprise one or more of a processing circuit, a processor, an application specific integrated circuit (ASIC), a controller, a microcontroller, a microprocessor, a programmable logic device, logic circuitry, and / or the like.

[0084] In various embodiments, a “communication interface” or “network interface” (collectively, “communications unit”) as described herein may comprise suitable hardware and / or software components capable of enabling the transmission and / or reception of data. A communications unit may enable electronic communications between devices and systems. A communications unit may enable communications over a network. Examples of a communications unit may include a modem, a network interface (such as an Ethernet card), a communications port, etc. Data may be transferred via a communications unit in the form of signals which may be electronic, electromagnetic, optical, or other signals capable of being transmitted or received by a communications unit. A communications unit may be configured to communicate via any wired or wireless protocol such as a CAN bus protocol, an Ethernet physical layer protocol (e.g., those using 10BASE-T, 100BASE-T, 1000BASE-T, etc.), an IEEE 1394 interface (e.g., FireWire), Integrated Services for Digital Network (ISDN), a digital subscriber line (DSL), an 802.1 la / b / g / n / ac signal (e.g., Wi-Fi), a wireless communications protocol using short wavelength UHF radio waves and defined at least in part by IEEE 802.15.1 (e.g., the BLUETOOTH® protocol maintained by Bluetooth Special Interest Group), a wireless communications protocol defined at least in part by IEEE 802.15.4 (e.g., the ZigBee® protocol maintained by the ZigBee alliance), acellular protocol, an infrared protocol, an optical protocol, or any other protocol capable of transmitting information via a wired or wireless connection. In embodiments, server computing device 230 may comprise at least one network interface 256 in communication with other devices of simulation system 200 via network 240.

[0085] Two or more of the system components may be in electronic communication via a network. For example, two or more of wearable computing device 210, server computing device 230, and generative computing device 250 may communicate via network 240. As used herein, the term “network” may further include cloud, cloud computing system, or electronic communications system or method that incorporates hardware and / or software components. Communication amongst the devices and systems over a network may be accomplished through any suitable communication channel, such as, for example, a telephone network, an extranet, an intranet, the internet, a wireless communication, local area network (LAN), wide area network (WAN), virtual private network (VPN), and / or the like.

[0086] Electronic communications between the systems and devices may be unsecure. A network may be unsecure. Electronic communications disclosed herein may utilize data encryption. Encryption may be performed by way of any of the techniques now available in the art or which may become available — e.g., Twofish, RSA, El Gamal, Schorr signature, DSA, PGP, PM, GPG (GnuPG), HPE Format-Preserving Encryption (FPE), Voltage, Triple DES, Blowfish, AES, MD5, HMAC, IDEA, RC6, and symmetric and asymmetric cryptosystems. Network communications may also incorporate SHA series cryptographic methods, elliptic-curve cryptography (e.g., ECC, ECDH, ECDSA, etc.), and / or other post-quantum cryptography algorithms under development.

[0087] For the sake of brevity, conventional data networking, application development, and other functional aspects of system may not be described in detail herein. Furthermore, the connecting lines shown in the various figures contained herein are intended to represent exemplary functional relationships and / or electronic communications between the various elements. It should be noted that many alternative or additional functional relationships or electronic communications may be present in a practical system.

[0088] In various embodiments, a “memory” or “memory unit” as discussed herein may comprise any hardware, software, and / or database component capable of storing and maintaining data. For example, a memory unit may comprise a database, data structure, memory component,or the like. A memory unit may comprise any suitable non-transitory memory known in the art, such as, an internal memory (e.g., random access memory (RAM), read-only memory (ROM), solid state drive (SSD), etc.), removable memory (e.g., an SD card, an xD card, a CompactFlash card, etc.), or the like.

[0089] Any database discussed herein may include relational, hierarchical, graphical, distributed ledger, blockchain, object-oriented structure, and / or any other database configurations, unless otherwise specified. Any database may also include a flat file structure wherein data may be stored in a single file in the form of rows and columns, with no structure for indexing and no structural relationships between records. For example, a flat file structure may include a delimited text file, a CSV (comma-separated values) file, and / or any other suitable flat file structure. Moreover, a database may be organized in any suitable manner, for example, as data tables or lookup tables. Each record stored in a database may be a single file, a series of files, a linked series of data fields, and / or any other data structure or schema.

[0090] Any database, system, device, server, or other components of the system described herein may consist of any combination thereof at a single location or at multiple locations. For example, any database described herein may comprise a single database or a plurality of databases (virtual partitions or physically distinct). Each database or system may include any of various suitable security features, such as firewalls, access codes, encryption, decryption, compression, decompression, and / or the like.

[0091] In various embodiments, an “input device” as discussed herein may comprise hardware and / or software used to provide data, inputs, control signals, and the like to a computer-based system, software application, etc. For example, an input device may include a pointing device (e.g., mouse, joystick, pointer, etc.), a keyboard (e.g., virtual or physical), a touchpad or touchscreen interface, a video input device (e g., camera, scanner, multi-camera system, etc.), a virtual reality system, an audio input device (e.g., microphone, digital musical instrument, etc.), a biometric input device (e.g., fingerprint scanner, iris scanner, etc.), a composite device (e.g., a device having a plurality of different forms of input), and / or any other input device. Examples of input devices may include an input audio interface, an image sensor, or a user interface.

[0092] In various embodiments, an “output device” as discussed herein may comprise hardware and / or software configured to convert information into a human-accessible form, for display, projection, or physical reproduction. For example, an output device may include a display device(e g., monitor, monochrome display, colored display, CRT, LCD, LED, projector, video card, etc.), an audio output device (e.g., speaker, headphones, sound card, etc.), a location services system (e.g., global positioning system (GPS), etc.), a printer (e.g., dot matrix printer, inkjet printer, laser printer, 3D printer, wide-format printer, etc.), a braille reader, a composite device (e.g., a device having a plurality of different forms of output), and / or any other output device. Examples of output devices may include an output audio interface, a display, or a user interface.

[0093] In embodiments, server computing device 230 may comprise one or more modules configured to perform various operations of server computing device 230. The one or more modules may comprise separate hardware components relative to processor 232 and memory 234. For example, a module may comprise a graphics processor or other specialized electronic circuit configured to perform operations. Alternately or additionally, the one or more modules may be implemented wholly or at least in part via processor 232, memory 234, and / or computer-readable instructions stored on memory 234 that, when executed by processor 232, cause server computing device 230 to perform various operations.

[0094] In embodiments, server computing device 230 may comprise transcription module 231. Transcription module 231 may be configured to receive audio input and generate text data in accordance with information included in the audio input. The text data may comprise sets of words detected in the audio input by transcription module 231. The text data may comprise a confidence score for each word representing a likelihood that the word was included the audio input from which the text data was generated. Transcription module 231 may comprise a trained machine learning module configured to receive the audio input and generate the text data. In embodiments, the audio inputs may be received at wearable computing device 210. Wearable computing device 210 may record audio data in which the audio inputs are represented. Wearable computing device 210 may then transmit this audio data to transcription module 231 via wireless communication interface 216, network 240, and / or network interface 236. Transcription module 231 may perform a speech-to-text recognition operation on the audio data to generate text data reflecting one or more words detected in the text data. The text data may be provided to other components of simulation system 200 for further processing, including one or more of target detection module 235 and intent detection module 238.

[0095] In embodiments, server computing device 230 may comprise sentiment detection module 233. Sentiment detection module 233 may be configured to receive user input and generatesentiment information in accordance with the user input. The sentiment information may comprise a sentiment of the user input. The sentiment information may comprise an indication of polarity of the user input. For example, the sentiment information may include an indication of whether a user input appears to convey a positive, neutral, or negative attitude. Alternately or additionally, the sentiment information may identify an emotional state among a plurality of emotional states as been represented in the user input. For example, and in embodiments, sentiment detection module 233 may classify a user input as one of a group of emotional states comprising anger, disgust, calmness, fear, sadness, anxiety, confusion, or joy. Alternately or additionally, the sentiment information may comprise a measurement of each of one or more properties of the user input. For example, the sentiment information may comprise an indication of volume and / or rate of information (e.g., text or gesture) represented in the user input. In embodiments, the user input may comprise an audio input, physical input, and / or a combination of the audio input and the physical input provided by a user. For example, sentiment detection module 233 may perform sentiment detection on an audio input received by audio input interface 211, a physical input represented in image data captured by image sensor 213, a physical input received via user interface 228, or combinations of such user inputs to generate sentiment information. In embodiments, sentiment detection may be applied to text and / or gestures represented in user input. The sentiment information may be provided to other components of simulation system 200 for further processing, including intent detection module 238.

[0096] In embodiments, server computing device 230 may comprise motion detection module 237. Motion detection module 237 may be configured to receive user input and generate motion information identifying motion events indicated in the user input. The motion events may comprise a gesture indicated in the user input. For example, motion detection module 237 may receive image data from image sensor 213 and detect a hand gesture made by a user of wearable computing device 210 as indicated in the image data. The gesture may comprise a type selected among a group comprising a lateral waving gesture, a downward waving gesture, a thumbs up gesture, a thumbs down gesture, a stop gesture, and a pointing gesture. Alternately or additionally, motion detection module 237 may receive physical input from user interface 228 and detect a gesture (e.g., aim, pattern of movement, orientation) made with equipment 220 as indicated in the user input.

[0097] Tn embodiments, motion events alternately or additionally detected by an operation of motion detection module 237 may comprise a simulated contact with an object in a simulated environment. For example, motion detection module 237 may receive physical input and object information regarding a location of an object (e.g., door, window, vehicle, NPC, etc.) in a simulated environment. Based on a comparison between the physical input and the object information, motion detection module 237 may generate a motion event indicating virtual contact between the physical input and the virtual object. For example, the motion event may comprise virtual contact in the simulated environment between the user and the object. The virtual contact may comprise indication of one or more of an intensity, location, and duration contact between a user and the object. The motion event may be provided to other components of simulation system 200 for further processing, including intent detection module 238. As

[0098] In embodiments, server computing device 230 may comprise target detection module 235. Target detection module 235 may be configured to receive user input and generate target information (e.g., target, target data, etc.) in accordance with the user input. The target information may comprise a target of the user input. The target may comprise an entity in a simulated environment. For example, the target information may include one of an NPC presented in the simulated environment, an equipment, or a non-visible entity represented in the simulated environment. The equipment may comprise a recording device (e.g., body-worn camera) represented in the simulated environment. The non-visible entity may comprise a virtual dispatcher accessible to the user via a communication device (e.g., two-way radio) represented in the simulated environment. The target may be detected in accordance with an audio input. For example, target detection module 235 may detect a target in accordance with one or more words in the audio input that match a name of the target. Alternately or additionally, the target may be detected in accordance with a physical input of the user input. For example, target detection module 235 may detect a target in accordance with a simulated spatial relationship between an equipment and the target. The simulated spatial relationship may be determined based on an orientation between the equipment and a simulated location of the target. For example, when a CEW is aimed toward an NPC in a simulated environment, the NPC may be detected as the target of the user input. As another example, an alignment between an orientation of a field of view of a wearable computing device 210 and an NPC may indicate a user of wearable computing device 210 is looking at the NPC and, accordingly, the NPC may be selected as the target. The targetinformation may be provided to other components of simulation system 200 for further processing, including intent detection module 238.

[0099] In embodiments, server computing device 230 may comprise intent detection module 238. Intent detection module 238 may be configured to detect an intent (e.g., intent information, intent data, etc.) in accordance with input data. The intent may indicate a purpose of the user input. The intent may indicate a desired change in a simulation state of a simulated environment. For example, an audio input may comprise a question, indicating a desired auditory response from an entity in a simulated environment. The intent may comprise an action. For example, the action may comprise an action of questioning, instructing, informing, reassuring, listening, arresting, acknowledging, moving, contacting or other actions. The intent may comprise a target. For example, the target may comprise aNPC, an equipment, or a non-visible entity as discussed above. Detecting the intent may comprise applying natural language processing to an audio input of the user input. Detecting the intent may comprise analyzing one or more of text data, sentiment information, motion events, and / or target information discussed above. Accordingly, the intent may be detected in accordance with audio input, physical input, or a combination of audio input and physical input of user input. In embodiments, the intent may be provided to other components of simulation system 200. For example, the intent may be transmitted to processor 212 to selectively change a simulation state presented via display 218 of wearable computing device 210.

[0100] In embodiments, processor 212 may be configured to change a simulated environment in accordance with an intent. Changing the simulated environment may comprise performing a comparison between the intent and one or more predetermined intents associated with a simulation state of the simulated environment. Changing the simulated environment may comprise determining the intent matches zero or one predetermined intent of the one more predetermined intents. Accordingly, the comparison may indicate that the intent does match (e.g., has one matching intent) or does not match (e.g., has zero matching intents) the one or more predetermined intents. In accordance with the comparison, the simulation state may be selectively changed.

[0101] In embodiments, modules 231, 233, 235, 237, and 238 may be implemented via server computing device 230 of simulation system 200. Alternately or additionally, such modules may be wholly or at least partially implemented in other computing devices, including wearable computing device 210 or generative computing device 250.

[0102] Network 240 may comprise one or more devices configured to enable communication between devices of simulation system 200. For example, network 240 may comprise one or more routers, modems, and / or access points configured to enable wired and / or wireless connections between devices. In embodiments, and with brief reference to FIG. 1, network 240 may perform one or more operations of network 140.

[0103] In embodiments, generative computing device 250 may be configured to generate media for a simulated environment provided by simulation system 200. For example, generative computing device 250 may receive a prompt associated with a simulated environment presented via wearable computing device 210. The prompt may comprise simulation data associated with a simulated experience provided via simulation system 200. The prompt may comprise text data. Responsive to the prompt, generative computing device 250 may generate a dialog response. The dialog response may be associated with an NPC in the simulated environment. Generative computing device 250 may comprise various software and / or hardware components. For example, generative computing device 250 may comprise processor 252, memory 254, and / or network interface 256. In embodiments, and with brief reference to FIG. 1, generative computing device 250 may perform one or more operations of generative computing device 150.

[0104] Network interface 256 may comprise suitable hardware and / or software components capable of enabling the transmission and / or reception of data. Network interface 256 may enable electronic communications between devices and systems. Network interface 256 may enable electronic communications between generative computing device 250 and network 240. Network interface 256 may enable wired and / or wireless connections and corresponding protocols as further discussed above. In embodiments, network interface 256 may transmit dialog response data to wearable computing device 210 and / or receive simulated environment data from wearable computing device 210 and / or server computing device 230 to enable a dialog response to be generated.

[0105] In embodiments, generative computing device 250 may comprise a processor 252 and memory 254 configured to perform one or more operations for generating dialog responses. Processor 252 and memory 254 may be similar to one or more other processors and / or memories disclosed herein. Memory 254 may store instructions that, when executed by processor 252, cause generative computing device 250 to receive a prompt from wearable computing device 210 and / or server computing device 230 to enable a dialog response to be generated. The instructions, whenexecuted by processor 252 may further enable generated dialog responses to be transmitted to a computing device from which corresponding prompts were received.

[0106] In embodiments, instructions stored in memory 254 may comprise a large language model (LLM). The LLM may comprise an artificial neural network trained to generate output text data responsive to input text data applied to the LLM. The output text data may comprise text data engineered to comprise human-readable text information. The output text data may comprise words, as well as syntax, associated with human language.

[0107] In embodiments, instructions stored in memory 254 may comprise at least one generative model 258 configured to generate a dialog response. Generative model 258 may comprise a LLM trained to receive a prompt and, responsive to the prompt, generate a dialog response. The prompt may comprise text data indicating one or more features that should be included in the dialog response. In some embodiments, the prompt may comprise text data indicating text information provided to a simulation system by a user. Generative model 258 may be applied to the prompt to generate a dialog response. The dialog response may be machinegenerated in accordance with a manner in which generative model 258 was trained. For example, one or more words and / or a syntax of the generated dialog response may be at least partially determined in accordance with a manner in which the generative model 258 was trained. At least part of the dialog response may be different from the prompt. For example, and relative to the prompt., a generate dialog response may comprise a different syntax and / or different word(s). In some embodiments, the generated dialog response may comprise additional subject matter information that generative model 258, in accordance with its training, may determine to be related to the prompt. In accordance with various aspects of the present disclosure, generative model 258 may enable dialog responses to be automatically generated, in addition to or as an alternative dialog responses that may be predetermined for simulation system 200.

[0108] In embodiments, generative model 258 may comprise a plurality of models. Each model of the plurality of models may be associated with a different NPC of a simulation provided by a simulation system. For example, a first model of generative model 258 may be associated with a first NPC of a simulation and a second model of generative model 258 may be associated with a second NPC of a simulation. Each model of the plurality of models 258 may comprise a copy of a common LLM. For use with a simulation provided by simulation system 200, each separate model may receive NPC information associated with respective NPCs. Forexample, a first model of generative model 258 may receive information that a first NPC is Canadian and a second model of generative models 258 may receive prompt information indicating that a second NPC is from the United Kingdom. In accordance with the respective, different prompt information, dialog responses generated for the first NPC by the first model may comprise a lexicon, grammar, and style associated with a person from Canada, while dialog responses generated for the second NPC by the second model may comprise a lexicon, grammar, and style associated with a person from the United Kingdom.

[0109] In embodiments, separate models of generative model 258 may further receive different prompt information while a simulation is provided by a simulation system. For example, a first model may receive a prompt indicating that an emotional state of a first NPC is angry and a second model may receive a prompt indication that an emotional state of a second NPC is afraid. Upon the receipt of a subsequent prompt, the first model may generate a first dialog response associated with a person that is angry. For example, text data may comprise a relatively lower number of words than text data associated with other emotional states. Upon the receipt of a subsequent prompt, the second model may generate a second dialog response associated with a person that is afraid. For example, text data may comprise a higher number of incomplete sentence than text data associated with other emotional states. Each model of model 258 may reply upon previously received prompt information when processing a new prompt. Accordingly, for a series of prompts pertaining to the first NPC, dialog responses generated for a first NPC may reflect information provided in prior prompts received by the first model. For a series of prompts pertaining to the second NPC, dialog responses generated for a second NPC may reflect information provided in prior prompts received by the second model. The use of respective models for separate NPCs may enable separate dialog responses to be generated for each respective NPC. The separate responses may comprise distinct text data in accordance with distinct prompt information being previously provided to each of the plurality of models. Alternately or additionally, the use of a respective models for separate NPCs may decrease an amount of information that needs to be provided to and / or processed by a respective model. For example, NPC information may not need to be provided anew each time a dialog response is requested to be generated for a given NPC among a plurality of NPCs. This decreased amount of information may decrease a response time necessary to process a new prompt and generate a corresponding dialog response. Such decreased response time may provide a particular technicalbenefit for systems in which a generative model is executed on a different computing device from which a prompt is generated.

[0110] In embodiments, a simulation system may be configured to render a simulated environment on a display. The simulated environment may include one or more virtual objects. The simulated equipment may also include rendered, virtual versions of equipment used by a user, as well as parts of a body of the user. For example, and with reference to FIG. 3, an image 300 for an example simulated environment may be presented via a display in accordance with one or more user inputs received by simulation system. The simulation system may comprise one or more components or features of system 100 and / or system 200 with brief reference to FIG. 1-2. Image 300 may be rendered via a display such as display 118 or 218 with brief reference to FIG. 1-2. The example simulated environment of FIG. 3 is provided for purposes of illustration and further discussion of various aspects of the present disclosure.[0U1] As shown in image 300, a simulated environment may comprise one or more virtual objects, one or more virtual persons, as well as a virtual environment. Based on changes in the simulated environment, a user of a simulation system by which the simulated environment is presented may gain exposure to a situation similar what the user may encounter in real-world environments. Particularly, the user may gain experience in responding to persons in the real- world environments that exhibit behaviors similar to those presented by NPCs in the simulated environment. The user may also gain experience in identifying objects in an environment that may indicate one or more activities that may have occurred prior to the user being engaged with the simulated environment. As represented in image 300, the example simulated environment of FIG. 3 includes an NPC 310, a first user object 320-1, a second user object 320-2, a virtual equipment 330, a virtual setting 340, a first virtual object 350, and a second virtual object 352. In the simulation state depicted in image 300, NPC 310 is positioned inside virtual setting 340 comprising an inside of a house. Within virtual setting 340, a first object 350 comprising a spilled cup and a second object 352 comprising an exercise bicycle may be provided. Each of NPC 310, virtual setting 340, first object 350, and second object 352 may be fully simulated in accordance with a simulation intended to be provided via the simulated environment. Each such object may lack a physical, real -world counterpart in a physical environment in which a simulation system that provides the simulated environment is used. Each such object may bepresented via image 300 independent of a physical environment in which a simulation system that generates image 300 is used.

[0112] In embodiments, objects in the simulated environment may correspond to physical object involved with the simulation system. For example, first user object 320-1 may represent a user’s left hand, second user object 320-2 may represent a user’s right hand, and virtual equipment 330 may represent a training firearm employed by a user. A manner in which each of these objects is represented in image 300 may reflect a manner in which corresponding, real- world objects involved with a simulation system are provided. For example, an appearance, pose, and / or position of each of first user object 320-1, second user object 320-2, and virtual equipment 330 are depicted in image 300 may correspond to an appearance, pose, and / or position of a user’s left hand, a user’s right hand, and an equipment physically used with the simulation system. The simulation system by which simulated environment, including as represented in image 300, may comprise one or more components by which an appearance, pose, and / or position of such real-world objects is detected in a physical environment for subsequent representation in a simulated environment. For example, and with reference to FIG. 2, an appearance, pose, and / or position of portions of a user and / or equipment may be detected by one or more of image sensor 213, position sensor 217, object detection module 219A, equipment position detection module 219B, processor 212, processor 222, and / or user interface 228.

[0113] In embodiments, a simulation may comprise an audio response. An audio response may audibly convey information regarding the simulation. Particularly, an audio response for an NPC may convey information regarding the NPC and / or information regarding an event in the simulation that is simulated to be known by the NPC. In some embodiments, an object of the simulation may include causing, via one or more user inputs, the NPC to share information regarding the NPC and / or information regarding the event that is simulated to be known by the NPC.

[0114] In embodiments, an audio response may comprise a dialog response of an NPC. A dialog response may correspond to part of a conversation between the NPC and a user of a simulation system. A dialog response may comprise a manner in which an NPC may communicate with a user of a simulation system. A dialog response may comprise one or more words simulated to be spoken by the NPC to provide the audio response. A dialog response may comprise text information that identifies one or more words to be simulated as spoken by theNPC to provide the audio response. The dialog response may comprise text data that may be converted to audio data for output as the audible response.

[0115] For a simulation, one more dialog responses for an NPC may be predetermined. The predetermined dialog responses may provide information to enable a simulation to progress. For example, a predetermined dialog response may provide information regarding a reason why a simulated call for service was placed to initiate the simulation. In embodiments, a plurality of predetermined dialog responses may be created for a simulation in order to enable information for progressing through the simulation to be shared by an NPC to a user of the simulation system. In embodiments, the predetermined dialog responses may be manually generated and provided as part of a simulation.

[0116] In embodiments, an example simulation system by which the simulated environment of FIG. 3 may be provided may comprise a number of predetermined dialog responses for NPC 310. The predetermined dialog responses may be associated with predetermined user inputs. For example, for a user input comprising an audio input asking about a name of NPC 310, a simulation system may comprise a predetermined dialog response comprising text data that represents a statement of “My name is Bob.” Predetermined dialog responses may also be provided for objects that are represented or not represented in image 300. For example, predetermined dialog responses may also be provided pertaining to first object 350 and / or an appearance of NPC 310. Responsive to audio inputs from a user directed toward the spilled cup of first object 350 or the facial expression of NPC 310, for example, the simulation system may be configured to provide the predetermined dialog responses pertaining to first object 350 and the appearance of NPC 310 respectively.

[0117] In embodiments, a simulation system may lack predetermined dialog responses for other user inputs. For example, the simulation system may lack predetermined dialog responses for a pose of equipment 330 as shown in image 300 and / or second object 352. Such other user inputs may be enabled to be received by the simulation system. By enabling such user inputs to be provided, an interaction between a user and the simulation system may better represent a range of user actions that may be enabled in a real-world situation. However, the other user inputs may not fall within a range of expected, predetermined user inputs for which a predetermined dialog response is stored by the simulation system. Embodiments according to various aspects address this technical deficiency of such a simulation system. Embodimentsaccording to various aspects also enable a dialog response to be automatically generated in a manner that confirms to a current simulation state provided by a simulation system.

[0118] Embodiment according to various aspects of the present disclosure comprise computer-implemented methods for automatically generating dialog responses for simulation systems. For example, and with brief reference to FIG. 4, an example method 400 for automatically generating a dialog response for a simulation system is provided. As a further example, and with brief reference to FIG. 5, example operations of an example method 500 for automatically generating a dialog response by one or more computing devices of a simulation system is provided. One or more operations of method 400 and / or one or more operations of method 500 may be performed by one or more devices of a simulation system. For example, and with brief reference to FIG. 1 and 2, one or more operations of method 400 and / or method 500 may be performed by headset 110, server 130, wearable computing device 210, and / or server computing device 230. In some embodiments according to various aspects of the present disclosure, such operations may alternately or additionally be performed by other computing devices. For example, and in some embodiments, one or more operations of method 400 and / or method 500 may be performed by generative computing device 150 and / or generative computing device 250 with brief reference to FIG. 1 and 2.

[0119] In embodiments, method 400 may comprise one or more of receiving user input 410, classifying user input to generate signal information 420, determining user input matches a predetermined dialog response 430, selecting a predetermined dialog response 440, selecting simulation information 450, transmitting a prompt 460, receiving a generated dialog response 470, and / or providing an audio response 480.

[0120] In embodiments, method 500 may comprise various operations that may be applied to various information of a simulation system. The information may be received and / or generated by one or more computing devices. The operations may be performed by one or more computing devices. For example, and with brief reference to FIG. 5, method 500 may comprise one or more operations for automatically recognizing speech 510, detecting one or more features of a speech signal 512, detecting a natural language intent 514, detecting an equipment state 522, detecting a position and / or movement of a user 524, detecting a biological feedback of a user 526, managing a narrative 540, generating an NPC dialog response 552, and generating a text-to-speech signal 560. In embodiments, method 400 may comprise one or more operations of method 500 as further discussed below.

[0121] In embodiments, one or more operations of method 400 and / or 500 may be performed by at least one computing device. The computing device may comprise a processor configured to execute instructions. The computing device may further comprise a non-transitory computer- readable storage medium configured to store the instructions. Upon execution of the instructions by the processor, the computing device may be configured to perform the one or more operations of method 400 and / or method 500. For example, and with brief reference to FIG. 2, the processor may comprise one or more of processor 212, processor 222, processor 232, and / or processor 252. Alternately or additionally, the computer-readable storage medium may comprise one or more of memory 214, memory 224, memory 234, and / or memory 254. In some embodiments, one or more operations may be at least partially performed by one or more modules 219A-B and / or modules 231-238 according to various aspects of the present disclosure.

[0122] In embodiments, one or more operations of method 500 may be applied to various input information. The input information may comprise analog and / or digital information. The one or more operations applied to the input information. For some operations, the input information may comprise one or more user inputs. The operations may detect one or more properties of the one or more user inputs and generate new information in accordance with processing the one or more user inputs. For example, and in accordance with various embodiments, automatically recognizing speech 510 may be applied to audio input 502; detecting one or more features of a speech signal 512 may be applied to audio input 502; detecting an equipment state 522 may be applied to at least one controller input 504; detecting a position and / or movement of a user 524 may be applied to at least one position input 506; and / or detecting a biological feedback of a user 526 may be applied to at least one physiological input 508.

[0123] In embodiments, one or more operations of method 500 may comprise generating new information. The new information may be related to input information to which the one or more operations may be applied. For example, and in accordance with various embodiments, automatically recognizing speech 510 may generate text information representing one or more words detected in audio input 502; detecting one or more features of a speech signal 512 may generate acoustic information detected in audio input 502; detecting an equipment state 522 maygenerate equipment status information detected from at least one controller input 504; detecting a position and / or movement of a user 524 may generate user information detected from at least one position input 506; and / or detecting a biological feedback of a user 526 may generate biological response information from at least one physiological input 508.

[0124] In embodiments, a set of information generated by a simulation system may be stored for subsequent processing. The set of information may comprise information collectively generated by simulation system relative to one or more user inputs received by the system. For example, signal information 520 may comprise one or more of intent information generated by natural language intent 514, text information generated by automatically recognizing speech 510, acoustic information generated by detecting one or more features of a speech signal 512; equipment status information generated by detecting an equipment state 522; position information generated from detecting a position and / or movement of a user 524; and / or biological response information generated from detecting a biological feedback of a user 526. In embodiments, the set of information may be provided for subsequent processing one or more operations of method 500. In embodiments, the set of information may be stored for subsequent access by one or more operations of method 500. For example, signal information 520 may be stored in memory 214, memory 224, and / or memory 234 in accordance with various aspects of the present disclosure.

[0125] In embodiments, generated information may be applied to one or more operations of method 500. For example, signal information 520 may be provided for managing a narrative 540. In some embodiments, signal information 520 may be provided to managing 540 responsive to each change in signal information 520.

[0126] In embodiments, one or more operations may be applied to predetermined information. The predetermined information may define a simulation provided via a simulation system. The predetermined information may define an initial set of visual and / or audio information provided for the simulation. The predetermined information may identify one or more changes in visual information and / or audio information that may be provided for the simulation. In embodiments, the predetermined information may be stored for access by one or more operations of method 500. The predetermined information may at least be partially manually determined. The predetermined information may be generated prior to execution of one or more operations of method. In some embodiments, at least some of the predeterminedinformation may be updated in accordance with one or more changes provided during execution of method 500. In some embodiments, at least some of the predetermined information may remain static during execution of method 500. In embodiments, predetermined information may be stored in memory 214, memory 224, and / or memory 234 in accordance with various aspects of the present disclosure. For example, and in accordance with various embodiments, predetermined information of method 500 may comprise one or more of narrative rules 542, NPC information 544, predetermined NPC dialog responses 546, and / or simulation state information 548.

[0127] In embodiments, one or more operations may be applied by at least one computing device. For example, one or more operations of method 400 may be performed by at least computing device comprising headset 110, equipment 120, server 130, generative computing device 150, wearable computing device 210, equipment 220, server computing device 230, and / or generative computing device 250 with brief reference to FIG. 1-2.

[0128] In embodiments, one or more operations may be performed by at least two computing devices communicatively coupled to each other. For example, one or more operations of method 500 may be performed by at least one simulation computing device 530 communicatively coupled to generative computing device 550. In embodiments, simulation computing device 530 may comprise one or more computing devices. The one or more computing devices may be communicatively coupled to each other. For example, simulation computing device 530 may comprise one or more of comprising headset 110, server 130, wearable computing device 210, and / or server computing device 230 with brief reference to FIG. 1-2. Two or more computing devices of simulation computing device 530 may be coupled via network 140 and / or network 240 with brief reference to FIG. 1-2. Generative computing device 550 may comprise generative computing device 150 and / or generative computing device 250 with brief reference to FIG. 1-2.

[0129] In embodiments, generative computing device 550 may be in communication with simulation computing device 530 via a network. For example, generative computing device 550 may be in communication with simulation computing device 530 via network comprising network 140 and / or network 240 with brief reference to FIG. 1-2. In other embodiments, one or more operations of method 400 and / or method 500 may be performed by a same computing device according to various aspects of the present disclosure.

[0130] In embodiments, two or more computing devices of a simulation system may exchange information in order to perform one or more operations. For example, responsive to managing a narrative 540 of simulation computing device 530, a prompt may be transmitted via network to generative computing device 550. In some embodiments, generative computing device 550 may provide an application programming interface (API) by which requests may be transmitted to generative computing device 550. Transmitting the prompt may comprise providing, by simulation computing device 530, a request comprising a prompt to generative computing device 550 in accordance with an API of generative computing device 550.

[0131] In embodiments, an exchange of information may comprise providing information responsive to a request. The request may comprise a prompt. For example, responsive to receiving a prompt, generative computing device 550 may automatically generate a dialog response. The dialog response may be generated in accordance with prompt information of the prompt. Responsive to the dialog response being generated, the dialog response may be transmitted to a source of the prompt. For example, responsive to execution of automatically generating an NPC dialog response 552, generative computing device 550 may be configured to transmit text data comprising the computer-generated dialog response to simulation computing device 530.

[0132] In embodiments, one or more operations for method 500 may be applied to a user input received via a user interface. The user interface may comprise at least one input audio transducer. The input audio transducer may comprise a microphone. For example, and with brief reference to FIG. 1-2, audio input 502 may be received via one or more of microphone 114 and / or an input audio transducer of input audio interface 211 according to various aspects of the present disclosure.

[0133] In embodiments, one or more operations for method 500 may be performed to provide audio output 516 via a user interface. The user interface may comprise at least one output audio transducer. The output audio transducer may comprise an audio speaker. For example, and with brief reference to FIG. 1-2, an output audio signal may be provided from one or more of headset speaker 116 and / or an output audio transducer of output audio interface 215 according to various aspects of the present disclosure.

[0134] In embodiments, at least one user input may be received by a simulation system. The at least one user input may comprise a plurality of user inputs. The at least one user input maybe associated with an interaction between the simulation system and a user. The at least one user input may be received responsive to a simulation presented to the user by the simulation system. For example, the at least one user input may be received responsive to audible information and / or visual information of the simulation being presented to the user via the simulation system. The at least one user input may be received after audible information and / or visual information of the simulation is presented via at least one user interface of the simulation system. For example, method 400 may comprise receiving at least one user input 410. Prior to receiving 410, audible and / or visual information of a training simulation may be provided via one or more headset speaker headset speaker 116, display 118, output audio interface 215, and / or display 218 with brief reference to FIG. 1-2.

[0135] In embodiments, a user input may comprise one or more of a plurality of different types of inputs for a simulation system. The plurality may be received via different input elements of the simulation system. Different combinations of the plurality may be provided over time. Different numbers of the plurality may be received at a given time. For example, receiving 410 may comprise receiving a first user input at a first time and receiving two or more second user inputs at a second time, wherein the first user input is different from each of the two or more second user inputs.

[0136] In embodiments, a user input may comprise a controller input. For example, receiving 410 may comprise receiving controller input 504. The controller may comprise an equipment used with the simulation system. The equipment may comprise a training equipment. For example, the equipment may comprise equipment 120 and / or equipment 220 with brief reference to FIG. 1-2. At least one controller input 504 may indicate a user action applied to a controller of the system. Controller input 504 may comprise status information indicating that a user interface of an equipment has been actuated. For example, controller input 504 may indicate that one or more of a safety switch, trigger, warning button, mode selection switch, or other user interface element of an equipment comprising a weapon has been actuated. Controller input 504 may indicate that a safety of an equipment has been engaged or disengaged, a trigger of the equipment has been depressed or released, a body of the equipment is received in or removed from aa holster, and / or whether a switch, button, lever, touchscreen or other input element of a user interface has been actuated. Alternately or additionally, controller input 504 may comprise position information indicating a position of the equipment has been provided by aforce manually applied to the equipment. The position information may indicate one or more of a relative position, an orientation, and / or an absolute position of the equipment. An equipment may comprise one or more components configured to detect a user action applied to the equipment. For example, and with brief reference to FIG. 1-2, receiving 410 may comprise receiving controller input 504 in accordance with manual interaction detected by one or more of user interface 228 and / or sensor 229. Alternately or additionally, controller input 504 may be received via at least one sensor integrated with a computing device. For example, receiving 410 may comprise capturing an image via image sensor 213 in which equipment 220 is represented. In embodiments, a computing device of a simulation system may further comprise one or more components by which controller input 504 may be received. For example, receiving 410 may comprise receiving a wireless signal comprising status information and / or position information via wireless communication interface 216. The status information and / or position information may be transmitted via wireless communication interface 226 of equipment 220 according to various aspects of the present disclosure.

[0137] In embodiments, a user input may comprise a position input. The position input may indicate a position of at least a portion of a user of a simulation system. For example, the portion may comprise a hand or head of a user. The position may comprise a change in the position.The position may comprise a motion of the portion of the user over time. For example, receiving 410 may comprise receiving at least one position input 506. At least one position input 506 may comprise an input signal in which at least a portion of the user is represented. For example, the input signal may comprise one or more images in which the portion of the user is represented. Alternately or additionally, the input signal may be received from an external device configured to detect the position of the portion of the user. For example, the input signal may comprise one or more proximity signals indicating a relative position of the user within a physical space in which a simulation system is being used. Position input 506 may comprise one or more of a relative position, absolute position, and / or orientation of the portion of the user. Position input 506 may comprise position information generated by one or more components of a simulation system. For example, and with brief reference to FIG. 1-2, receiving 410 may comprise receiving at least one position input 506 via one or more of position sensor 217, image sensor 213, and / or wireless communication interface 216. In embodiments, at least one position input 506 may be received via wireless communication interface 216 from another second computingdevice that comprises an image sensor, proximity sensor, or other form of position sensor configured to detect position information regarding at least portion of a user and provide this information to wearable computing device 210 and / or server computing device 230.

[0138] In embodiments, a user input may comprise a physiological input. For example, receiving 410 may comprise receiving at least one physiological input 508. At least one computing device of a simulation system may comprise a at least one physiological sensor by which physiological information may be detected. For example, sensor 229 of equipment 220 may comprise a physiological sensor. At least one physiological input 508 may indicate a physiological response of a user relative to a simulation. At least one physiological input 508 may comprise physiological information indicating that a physiological state has changed and / or falls outside of a predetermined range. For example, at least one physiological input 508 may comprise physiological information indicating that a heart rate has increased and / or exceeds a predetermined range. In embodiments, an equipment and / or a computing device may comprise one or more sensor configured to detect at least one physiological input 508. For example, and with brief reference to FIG. 1-2, receiving 410 may comprise receiving at least one physiological input 508 via sensor 229 of equipment and / or a physiological sensor integrated with wearable computing device 210. In embodiments, a computing device of a simulation system may further comprise one or more components by which at least one physiological input 508 may be received. For example, receiving 410 may comprise receiving a wireless signal comprising physiological information via wireless communication interface 216. The physiological information and / or position information may be transmitted via wireless communication interface 226 of equipment 220 according to various aspects of the present disclosure.

[0139] In embodiments, one or more operations may be performed to generate signal information from a user input. The signal information may be associated with one or more features of the user input. The signal information may comprise secondary information associated with a user input. The signal information may indicate one or more predetermined, isolated characteristics of the user input. The signal information may associate one or more predetermined properties with the user input. The signal information may indicate whether a property of the predetermined properties is represented or not represented in the user input. The signal information may indicate a degree and / or subset of a property of the predeterminedproperties is represented in the input signal. For example, method 400 may comprise classifying user input to generate signal information 420.

[0140] In embodiments, classifying 420 may comprise classifying an audio input. Classifying the audio input may comprise identifying one or more predetermined properties of speech in the audio signal. The predetermined properties may comprise one or more linguistic units in the audio input. For example, classifying 420 may comprise identifying one or more words and / or morphemes represented in an audio input. Classifying 420 may comprise identifying an order of linguistic units represented in an audio input. Classifying 420 may comprise performing one or more operations to an audio input to generate one or more of text information and / or acoustic information associated with an audio input.

[0141] In embodiments, classifying 420 may comprise automatically recognizing speech 510 from audio input 502. Recognizing 510 may comprise transcribing audio input 502.Recognizing 510 may comprise detecting sounds associated with one or more words in audio input 502. Recognizing 510 may comprise providing text information identifying words detected in audio input 502. Recognizing 510 may comprise generating text data indicating one or more words detected from audio input 502. Recognizing 510 may comprise performing one or more operations of transcription module 231 with brief reference to FIG. 2.

[0142] In embodiments, classifying 420 may comprise detecting one or more features of a speech signal 512. Audio input 502 may comprise the speech signal to which detecting 512 may be applied. Detecting 512 may comprise detecting one or more acoustic features of audio input 502. For example, detecting 512 may comprise detecting one or more of a tone, frequency, amplitude and duration of a speech signal captured in audio input 502. Detecting 512 may comprise providing acoustic information indicating the one or more features detected in audio input 502. Detecting 512 may comprise generating data indicating the acoustic information detected from audio input 502.

[0143] In embodiments, classifying 420 may comprise detecting a natural language intent 514 of audio input 502. Detecting 514 may comprise performing a natural language understanding analysis to audio input 502. Detecting 514 may comprise one or more naturallanguage processing operations configured to detect intent information for audio input 502. For example, the one or more operations may comprise syntactic analysis, morphological analysis, lexical semantics analysis, relational semantics analysis, and / or discourse analysis. The one ormore operations may be applied to input data associated with audio input 502. For example the input data may comprise text data generated by recognizing 510 and / or data comprising acoustic information generated in accordance with detecting 512. In some embodiments, detecting 514 may comprise one or more operations of intent detection module 238 with brief reference to FIG.2. While recognizing 510, detecting 512, and detecting 514 are represented as separate operations in FIG. 5, in embodiments according to various aspects of the present disclosure, applying a natural language understanding analysis to audio input 502 may comprise collectively applying operations of recognizing 510, detecting 512, and detecting 514 to audio input 502 to generate intent information for audio input 502 in embodiments according to various aspects of the present disclosure.

[0144] In embodiments, intent information may comprise a target of 502. The target may indicate an object of a simulation to which audio input 502 is directed. In embodiments, the target may comprise an NPC of the simulation. Alternately or additionally, the target may comprise a virtual remote entity associated with a simulation. For example, a virtual remote entity may comprise a simulated dispatcher in communication via a simulated communication equipment of a simulation system. In embodiments, detecting 514 may comprise one or more operations of a target detection module 235 with brief reference to FIG. 2.

[0145] In embodiments, intent information may comprise a speech type of audio input 502. For example, the speech type may comprise one or more of a request for information, a request for action, an observation, an announcement, and an action statement. The speech types may comprise predetermined speech types. For example, detecting 514 may comprise associating one or more predetermined speech types to audio input 502. Detecting 514 may comprise applying natural language processing operations to information generated by recognizing 510 and / or detecting 512 to generate intent information comprising one more speech types.

[0146] In embodiments, intent information may comprise a subject. The subject may comprise subject matter to which audio input 502 pertains. For example, the subject may comprise an object or person represented in a simulation. For example, audio input 502 may comprise a question related to first object 350 with brief reference to FIG. 3. As another example, a audio input 502 may comprise a request for action pertaining toward a portion of NPC 310. The portion of NPC 310 may comprise, for example, hands of the NPC 310.Alternately or additionally, the subject may comprise an information pertaining to the simulation. For example, 503 may comprise a request for information comprising a name of NPC 310.

[0147] In embodiments, classifying 420 may comprise classifying a controller input. For example, classifying 420 may comprise classifying controller input 504. Classifying 420 may comprise generating equipment status information. The equipment status information may indicate a state of use of the equipment. For example, the equipment status information may indicate a mode of operation of the equipment and / or whether the equipment is being used during a simulation.

[0148] In embodiments, classifying 420 may comprise detecting an equipment state from controller input 504. Detecting 522 may comprise processing controller input 504 to detect one or more of status information indicated by controller input 504 and / or position information indicated by controller input 504. Detecting 522 may comprise processing the status information and / or position information to detect an equipment state. The equipment state may comprise a state of use. For example, an equipment state may comprise one or more of whether an equipment is activated, deactivated, bolstered, drawn, aimed, and / or deployed. Detecting 522 may comprise generating equipment status information that indicates the equipment state. Detecting 522 may comprise generating data that identifies the equipment status information. In embodiments, detecting 522 may comprise one or more operations of equipment position detection module 219B.

[0149] In embodiments, classifying 420 may comprise detecting a position and / or movement of a user 524 from at least one position input 506. Detecting 524 may generate user information detected from at least one position input 506. Detecting 524 may comprise processing at least one position input 506 to detect user information indicated by at least one position input 506. The user information may comprise, for example, a pose, location, and / or movement of the user or a portion of the user. Detecting 524 may generate, for example, user information indicating that a user made a waving motion with their left hand. In embodiments, detecting 524 may comprise one or more operations of object detection module 219A and / or motion detection module 237. Detecting 524 may comprise generating user information that indicates the position and / or movement of the user. Detecting 524 may comprise generating data that identifies the user information.

[0150] In embodiments, classifying 420 may comprise detecting a biological feedback of a user 526 from at least one physiological input 508. Detecting 526 may comprise processing at least one physiological input 508 to detect physiological information indicated 508. Detecting 522 may comprise processing at least one physiological input 508 to detect the biological feedback. The biological feedback may comprise a physiological state of a user. For example, biological feedback may comprise a emotional state of a user comprising a level of one or more of anger, fear, disgust, or other emotions of a user. Alternately or additionally, biological feedback may comprise a physiological stress level of a user, including where the stress level falls withing a range of predetermined stress levels. In some embodiment, biological feedback may indicate a level of biological response to a portion of a simulation, separate from a physical input or audio input that may also be provided by the user. Detecting 526 may comprise generating physiological information that indicates the biological feedback. Detecting 526 may comprise generating data that identifies the physiological information.

[0151] In embodiments, classifying 420 may comprise storing signal information. The signal information may be generated in according with one or more operations applied to at least one user input. Storing the signal information may comprise at least temporarily storing the signal information for subsequent operations. For example, classifying 420 may comprise storing signal information 520. Signal information 520 may comprise one or more of text information, acoustic information, intent information., equipment status information, user information, and / or biological response information. In some embodiments, simulation computing device 530 may comprise at least text information generated by recognizing 510, intent information generated by detecting 514, and equipment status information generated by detecting 522. In embodiments, simulation computing device 530 may be updated upon receipt, recognition and / or detection of each user input of one or more user inputs.

[0152] In embodiments, a simulation or simulated experience presented via a simulation system may comprise a plurality of simulation states. Each simulation state of the simulation states may be associated with at least one change in visual and / or audible information presented to a user. The plurality of simulation states may comprise an initial state comprising an initial set of audiovisual information provided to a user for the simulation. The plurality of simulation states may comprise one or more end states. Each end state of the one or more end states may be associated with one or more final changes in the simulation. For example, a training simulationmay comprise a first end state and a second end state. The first state may comprise, for example, a deescalated state and the second state may comprise escalated state.

[0153] In embodiments, a simulation or simulated experience presented via a simulation system may comprise a narrative. A narrative may define a sequence of simulation states. A narrative may define a change in simulation state that may occur upon receipt of a user input. For example, a first user input may correspond to a change from a first simulation state to a second simulation state and a second user input may correspond to a change from the first simulation state to a third simulation state different from the second simulation state. In accordance with a narrative of a simulation, a user may gain familiarity with how their actions, as represented to a computing device in the form of user inputs, may result in either the second state or the third state. The simulation may enable a user to identify and employ user inputs that enable the second state to be obtained instead of the third state.

[0154] In embodiments, a simulation state of a plurality of simulation states may comprise a dialog response. A dialog response may comprise one or more words. The one or more words may define one or more utterances associated with an NPC. The dialog response may comprise text data identifying the one or more words. Upon transition to the simulation state, a simulation system may provide an audio output comprising the dialog response. The audio output may be provided in a manner that indicates that the dialog response is associated with the NPC. For example, providing an audio output comprising a dialog response for NPC 310 may comprise providing the audio signal with a simulated voice or vocal characteristics previous employed for an utterance associated with NPC.

[0155] In some embodiments, a simulation state may comprise a predetermined dialog response. The predetermined dialog response may comprise predetermined information selected for a simulation. For example, a predetermined dialog response may comprise one or more of a name and address of an NPC. The predetermined dialog response may comprise, for example, an utterance of “My name is Bob” or “My address is 123 Main Street”.

[0156] In some embodiments, a simulation state comprising a predetermined dialog response may be associated with a predetermined user input. For example, simulation states comprising a predetermined dialog response indicating an NPC may be associated with a user input comprising a request for information regarding a name. A simulation may comprise simulation states with predetermined dialog responses for key simulation states of the simulation. Forexample, a narrative for a simulation may define a sequence of simulation states comprising a first state with a first predetermined dialog response provided responsive to a first predetermined user input. The first predetermined user input may comprise a request for information regarding a name. However, as noted above, a variety of user inputs may be received by a simulation system in order to replicate a range of user actions possible in a similar, real-world situation. The variety of user inputs that may be provided for a simulation system may not be predictable and / or generating a predetermined response for each potential user input may be resource intensive in terms of an amount of computer processing and / or storage required in order to enable a large number of predetermined dialog responses available for a simulation system. Embodiments according to various aspects of the present disclosure enable dialog responses related to a simulation to be automatically generated.

[0157] In embodiments, a simulation may lack a predetermined dialog response for a user input. For example, the user input may be associated with a simulation state that does not include a predetermined dialog response. Alternately or additionally, the simulation may lack a simulation state associated with the user input. For example, and with brief reference to FIG. 3, a simulation may comprise a first simulation state associated with a request for information regarding a name of NPC and a second simulation state associated with a request for information regarding first object 350. However, for a third user input comprising request for information regarding second object 352, the simulation may lack a predetermined dialog response. Embodiments according to various aspects of the present disclosure may enable a dialog response to be provided responsive to a user input, despite the lack of a predetermined dialog response associated with the user input. By providing dialog responses for such an input, a simulation may more accurately simulate a real-world simulation. Providing such dialog responses may also indicate that a user input was detected by the simulation system, despite a predetermined dialog response not being available for the user input.

[0158] In embodiments, a simulation system may determine whether a predetermined dialog response exists for a user input. For example, method 400 may comprise determining whether the user input matches a predetermined dialog response 430. Determining 430 may comprise comparing the user input to one or more predetermined user inputs. In some embodiments, the predetermined user inputs may be associated with one or more simulation states to which asimulation may advance relative to a current simulation state. Determining 430 may comprise employing the user input as an index relative to a set of predetermined user inputs.

[0159] In embodiments, determining 430 may comprise determining the predetermined dialog response matches the user input. For example, the user input may be determined to logically match a predetermined input of a plurality of predetermined user inputs. In some embodiments, determining the predetermined dialog response matches the user input may comprise determining the predetermined user input is associated with a simulation state comprising the predetermined user input.

[0160] In embodiments, determining 430 may comprise determining a predetermined dialog response does not match the user input. Determining 430 may comprise determining a plurality of predetermined dialog responses do not match the user input. For example, the user input may be determined to not logically match each predetermined input of a plurality of predetermined user inputs. In some embodiments, determining the predetermined dialog response does not match a user input may comprise determining the predetermined user input is not associated with a simulation state comprising a predetermined user input. In some embodiments, determining the predetermined dialog response does not match a user input may comprise determining the predetermined user input is associated with a simulation state that does not comprise a predetermined dialog response.

[0161] In embodiments, determining 430 may comprise managing a narrative 540. Managing 540 may comprise comparing signal information to a narrative rules. The signal information may comprise signal information 520. The narrative rules may comprise narrative rules 542. Signal information 430 may identify a received user input. The received user input may comprise a combination of one more user inputs comprising audio input 502, controller input 504, at least one position input 506, and / or at least one physiological input 508. Managing 540 may comprise comparing one or more of text information, acoustic information, intent information, equipment status information, user information, and / or biological response information to the narrative. Managing 540 may comprise comparing a subset of text information, acoustic information, intent information., equipment status information, user information, and / or biological response information of signal information 520 to the narrative.

[0162] In embodiments, the narrative may comprise narrative rules 542. Narrative rules may comprise executable instructions that, when applied to signal information 520, apply one or morelogical operations to signal information 520 to identify a next simulation state of a simulation. The rules may comprise one or more predetermined values of signal information that may be received via user inputs. The one or more predetermined values of signal information may be compared to corresponding values of signal information 520. For example, a first simulation state may be associated with values of signal information comprising a user input type of a “request for information” and a subject of “name.” In embodiments, signal information 520 may comprise matching values derived from audio input 502. Managing 540 may comprise applying the narrative rules 542 to signal information 520 to determine whether one or more values of narrative rules 542 match corresponding one or more values of signal information 520.

[0163] In embodiments, managing 540 may comprise identifying a predetermined dialog response is associated with a user input. Managing 540 may comprise performing a comparison to determine the user input matches the predetermined dialog response. For example, managing 540 may comprise applying narrative rules 542 to signal information 520 associated with the user input to determine one or more values of narrative rules 542 match corresponding one or more values of signal information 520 associated with the user input. For example, narrative rules 542 may comprise a first rule comprising values of a user input type of a “request for information” and a subject of “name.” The first rule may further comprise a predetermined dialog response. Alternately, the first rule may identify a predetermined dialog response of a plurality of predetermined NPC dialog responses 546. A first user input indicated by signal information 520 may be associated with values of signal information 520 comprising values of a user input type of a “request for information” and a subject of “name.” Managing 540 may comprise determining the values of the first rule of narrative rules 542 match the values of signal information 520. In accordance with the matching, the predetermined dialog response of the first rule may be determined to match the user input. In accordance with the matching, the predetermined dialog response may be associated with the user input via the first rule.

[0164] In embodiments, managing 540 may comprise identifying a predetermined dialog response is not associated with a user input. Managing 540 may comprise performing a comparison to determine the user input does not match at least one predetermined dialog response of a plurality of predetermined dialog responses. The comparison may indicate the user input does not match any dialog response of the plurality of predetermined NPC dialog responses 546 as defined by narrative rules 542. For example, managing 540 may comprise applyingnarrative rules 542 to signal information 520 associated with the user input to determine one or more values of narrative rules 542 do not match corresponding one or more values of signal information 520 associated with the user input. For example, narrative rules 542 may comprise a first rule comprising values of a user input type of a “request for information” and a subject of “name.” The first rule may further comprise a predetermined dialog response. A second user input may be associated with values of signal information 520 comprising values of a user input type of a “request for information” and a subject of “exercise bicycle.” Managing 540 may comprise comparing the values of the first rule of narrative rules 542 the values of signal information 520 to determine the respective values do not match. In accordance with the matching, the predetermined dialog response of the first rule may be determined to not match the user input. In embodiments, the matching may be performed relative to each rule of a plurality of rules of narrative rules 542. Matching 542 may comprise determining that at least one value of each rule of the plurality of rules does not match at least one corresponding value of signal information for a user input. In accordance with the matching, the predetermined response may be determined to not match a predetermined response.

[0165] In some embodiments, identifying a predetermined dialog response is not associated with a user input may comprise matching the user input to a simulation state that does not include a predetermined dialog response. For example, managing 540 may comprise applying narrative rules 542 to signal information 520 associated with the user input to determine one or more values of a second rule of narrative rules 542 match one or more values of signal information 520 associated with the user input. For example, narrative rules 542 may comprise a second rule comprising values of a user input type of a “request for information” and a subject of “exercise bicycle” A second user input may be associated with values of signal information 520 comprising values of a user input type of a “request for information” and a subject of “exercise bicycle.” Managing 540 may comprise comparing the values of the first rule of narrative rules 542 the values of signal information 520 to determine the respective values match. However, second rule may lack a predetermined dialog response. The second rule may not comprise a predetermined dialog response. The second rule may not identify a predetermined dialog response. Second rule may comprise other information associated with next simulation state. However, the second rule may lack a predetermined dialog response. In accordance with the matching, no predetermined dialog response may be determined to match the user input.Matching 542 may comprise determining that value of at least one rule of the plurality of rules matches at least one corresponding value of signal information for a user input, but lacks a predetermined dialog response. In accordance with the matching, the predetermined response may be determined to not match a predetermined response.

[0166] In embodiments, a simulation system may comprise a plurality of predetermined NPC dialog responses 546. Each predetermined response of the predetermined NPC dialog responses 546 may be associated with a rule of narrative rules 542. For example, a first rule of narrative rules 542 may be logically associated with a first predetermine dialog response of predetermined NPC dialog responses 546 and a second rule of narrative rules 542 may be logically associated with a second predetermine dialog response of predetermined NPC dialog responses 546. In some embodiments, a predetermined dialog response may be uniquely associated with a rule of a narrative. In other embodiments, multiple rules associated with different user inputs may be logically associated with a same predetermined dialog response. In embodiments, narrative rules 542 and predetermined NPC dialog responses 546 may be stored in separate, logically associated data stores. In other embodiments, narrative rules 542 and predetermined NPC dialog responses 546 may be stored in a same data store.

[0167] In embodiments, managing 540 may comprise updating simulation information in accordance with a user input. The simulation information may be updated independent of whether a predetermined dialog response is determined to correspond or not correspond to the user input. In some embodiments, a rule of one or more rules may comprise information identifying a value of a simulation information to be set responsive to user input associated with the rule being received. For example, a first rule may comprise at least one value indicating a user input comprising a request for action. The first rule may further comprise values indicating an emotional state of angry. Managing 540 may comprise determining at least one audio input 502 comprises a request for action. Managing 540 may further perform a comparison between the at least one audio input 502 and narrative rules 542 comprising the first rule to determine the at least one audio input matches the first rule. Responsive to the comparison indicating the match, managing may comprise setting simulation information for an NPC to comprise an emotional state of angry in accordance with the first rule. Updating the simulation information may enable a simulation to provide consistent dialog responses, independent of whether predetermined dialog responses or computer-generated dialog responses are provided. Forexample, a first rule of narrative rules 542 may be associated with a predetermined dialog response and specific simulation information. Upon receipt of a subsequent user input, and in accordance with updating simulation information, a computer-generated dialog response may be generated relative to the specific simulation information, despite the simulation information being modified relative to a separate, prior user input.

[0168] In embodiments, a simulation system may select a predetermined dialog response. For example, method 400 may comprise selecting a predetermined dialog response 440. Selecting 440 may be performed responsive to performing determining 430 to determine the predetermined dialog response matches the user input. Selecting 440 may comprise selecting the predetermined dialog response determined to match the user input. Selecting 440 may comprise identifying the predetermined dialog response among a plurality of predetermined responses for subsequent processing.

[0169] In embodiments, selecting 440 may comprise managing a narrative 540 to select a predetermined response of predetermined NPC dialog responses 546. Managing 540 may comprise identifying the predetermined dialog response in accordance with a rule of narrative rules 542. The rule may be determined to match a received user input. The rule may comprise the predetermined dialog response. The rule may indicate the predetermined dialog response. For example, the rule may comprise a unique identifier associated with a predetermined dialog response of predetermined NPC dialog responses 546. Managing 540 may comprise selecting a predetermined dialog response of predetermined NPC dialog responses 546 associated with a rule of narrative rules 542 that is associated with the predetermined dialog response. Managing 540 may comprise providing the predetermined dialog response for subsequent processing.

[0170] In embodiments, a simulation system may perform one or more operations to automatically generate a dialog response. The dialog response may comprise a computergenerated dialog response. The computer-generated dialog response may be generated by a generative computing device. Automatically generating the dialog response may comprise applying information associated with a simulation. In embodiments, automatically generating the dialog response may comprise one or more of selecting simulation information 450, transmitting a prompt 460, receiving a generated dialog response 470, and / or providing an audio response 480.

[0171] In embodiments, a simulation system may be configured to automatically generate a dialog response. For example, method 400 may comprise selecting simulation information 450. Selecting 450 may comprise identifying simulation information of a simulation usable for generating a dialog response. Selecting 450 may comprise selecting simulation information to provide prompt information included in a prompt. For example, the simulation information of a simulation may comprise information regarding one or more of an NPC, a virtual setting, and / or a virtual object.

[0172] In embodiments, selecting 450 may comprise selecting a subset of the simulation information for inclusion in a prompt. For example, selecting may comprise selecting information regarding an NPC and a first virtual object presented via a simulation system for a simulation. Selecting 450 may comprise not selecting other available information of the simulation. For example, selecting 450 may comprise not selecting information regarding a second virtual object and / or a virtual setting.

[0173] In embodiments, selecting 450 may comprise generating a prompt. The prompt may comprise selected simulation information. The prompt may comprise text data. In embodiments, generating the prompt may comprise generating text data indicating the selected simulation information.

[0174] In embodiments, selecting 450 may comprise reformatting the simulation information. For example, signal information generated in accordance with detecting 524 may comprise data indicating “heart rate = 100 bpm.” The data may be provided in a computational format.However, in embodiments, a computer-generated dialog response may be configured to receive prompts in a natural language format. Accordingly, selecting may comprise reformatting information provided in a first format into corresponding information provided in a natural language format. In some embodiments, selecting may comprise combining simulation information. For example, selecting 450 may comprise combining simulation information regarding a name of a user and a physiological information regarding the user. Selecting 450 may comprise combining this simulation information into a single prompt. For example, the information may be combined into single prompt in a natural language format. Text data for such a single prompt may comprise, for example, a sentence of “Trainee Joe’s heart rate is 100 beats per minute.”

[0175] Tn embodiments, a simulation for presentation via a simulation system may comprise NPC information 544. NPC information 544 may comprise information associated with a NPC of a simulation. NPC information 544 may comprise information indicating how an NPC may engage with a user during presentation of the simulation. NPC information 544 may comprise information indicating a manner in which an NPC may communicate with a user of a simulation system during a simulation. For example, NPC information 544 may comprise one or more of identity information and / or behavior information. In embodiments, NPC information 544 may comprise a separate set of NPC information for each NPC provided with a simulation. In embodiments, NPC information 544 may be used to generate a dialog response that reflects a manner in which a similar, real-word person may communicate during a simulation.

[0176] In embodiments, identify information may comprise information regarding how an NPC may communicate relative to a designated identity. The designated identify may comprise a representative identity of a simulated person. Identify information may comprise static information. Identity information may not change during presentation of a simulation. Identify information may comprise information regarding in inherent nature of a simulated person represented by an NPC. For example, identify information may comprise one or more of a quality, a belief, and / or a personality trait that characterizes a simulated person represented by an NPC. For example, a quality of an NPC may comprise a nationality of an NPC. The nationality may be used to modify a dialog response for the NPC. For example, NPC information for an NPC may indicate the NPC is Canadian. In accordance with this NPC information, a dialog response may be generated that incorporates characteristics of a Canadian English language. For example, a belief of an NPC may comprise a fear of weapons. In accordance with this NPC information, a dialog response may be generated that incorporates characteristics that change depending a state of an equipment employed with the simulation. Identify information may be predetermined for a simulation. Identify information may employed to present a simulation independent of the user input(s) received by a simulation system while the simulation is provided to a user.

[0177] In embodiments, identify information may comprise information regarding how an NPC may communicate relative to a designated identity. The designated identify may comprise a representative identity of a simulated person. Identify information may comprise static information. Identity information may not change during presentation of a simulation. Identifyinformation may comprise information regarding in inherent nature of a simulated person represented by an NPC. For example, identify information may comprise one or more of a quality, a belief, and / or a personality trait that characterizes a simulated person represented by an NPC. For example, a quality of an NPC may comprise a nationality of an NPC. The nationality may be used to modify a dialog response for the NPC. For example, NPC information for an NPC may indicate the NPC is Canadian. In accordance with this NPC information, a dialog response may be generated that incorporates characteristics of a Canadian English language. For example, a dialog response may comprise one or more words that are not included in vocabularies of other varieties of the English language. As another example, a belief of an NPC may comprise a fear of weapons. In accordance with this NPC information, a dialog response may generated that incorporates characteristics that change depending a state of an equipment employed with the simulation. For example, and in accordance with identify information comprising a fear of weapons a number of words in a computer-generated dialog response may decrease when an equipment simulating a firearm is drawn relative to when the equipment is not drawn.

[0178] In embodiments, identify information may comprise a unique NPC identifier. For example, the NPC identifier may comprise one or more of a name and / or address of the NPC. The NPC identifier may identify an individual NPC of a plurality ofNPCs included in a simulation. Selecting 450 may comprise providing identify information comprising a unique NPC identifier in prompt information for a prompt.

[0179] In embodiments, NPC information 544 may comprise behavior information. Behavior information may comprise information regarding how an NPC may respond to one or more user inputs. Behavior information may comprise variable information. Behavior information associated with an NPC may vary in accordance with one or more user inputs received for a simulation. For example, behavior information may comprise an emotional state. The emotional state may comprise one or more of happy, sad, angry, anxious or other emotional states. Alternately or additionally, behavior information may comprise a level of compliance. The level of compliance may comprise one of non-compliant, neutral, or non-compliant. Alternately or additionally, behavior information may comprise a level of trust. For example, a level of trust may indicate a state of trust the NPC exhibits in communication toward a user. The level of trust may comprise, for example, the NPC does not trust the user, the NPC trusts the user, or the NPCneither trusts nor distrusts the user. In accordance with the behavior information, different dialog responses may be automatically generated that reflect a current set of behavior information. For example, one or more of a vocabulary and / or number of words of a computer-generated dialog response may be selected in accordance with a current set of behavior information an NPC for which the computer-generated dialog response is generated.

[0180] In embodiments, behavior information of NPC information 544 may comprise an initial set of behavior information. The initial set of behavior information may comprise initial values for one or more behavior characteristics. For example, an initial set of bhavious information may comprise a level of compliance of non-compliant and an emotional state of anxious. Narrative rules 542 may comprise one or more logical associations between a user input and a change in such behavior information. A rule of rules may comprise a behavior information for an NPC associated with a user input. For example, a first rule may comprise a user input of a hand wave gesture and an emotional state of calm. Managing 540 may comprise determining at least one position input 506 is associated with signal information comprising a hand wave gesture. Responsive to matching the user input to the first rule, managing 540 may modify behavior information of an NPC to indicate an emotional level of calm in accordance with the first rule.

[0181] In embodiments, a simulation for presentation via a simulation system may comprise simulation state information 548. Simulation state information 548 may comprise information regarding a simulated environment of a simulation. For example, simulation state information 548 may comprise information describing a virtual setting and / or one or more virtual objects. In some embodiments, simulation state information 548 may comprise a log of user inputs previous provided during a simulation and / or a log of prior dialog responses provided during a simulation. Such state information may enable a computer-generated dialog response to refer to the simulated environment and / or one or more prior interactions between a user and one or more NPCs for a simulation.

[0182] In embodiments, simulation information may be manually provided to a simulation system. The simulation information may be selected in order to determine a desired progression through simulation states in response to various user inputs received by the simulation system. For example, at least a portion of each of narrative rules 542, NPC information 544,predetermined NPC dialog responses 546, and simulation state information 548 may be provided to a simulation system in order to enable a simulation to be provided by the simulation system.

[0183] In embodiments, selecting 450 may comprise managing 540 a narrative to select simulation information upon which a dialog response is subsequently generated. In embodiments, may comprise selecting simulation information in accordance with a rule of narrative rules 542. For example, a rule of narrative rules 542 may comprise a logical association between simulation information and a user input. The logical association may indicate a subset of the simulation information to be selected for further processing to generate a computer-generated dialog response. For example, a rule of narrative rules 542 may indicate that behavior information and identity information from NPC information 544 should be selected to generate a dialog response relative to a user input. The rule may indicate that simulation state information 548 should be excluded from a selection of prompt information. In some embodiments, the rule may lack a predetermined dialog response. The rule may be a same rule by which matching of managing 540 may determine that a predetermined dialog response is not associated with the user input. In other embodiments, a default rule for selecting simulation information may be employed. The default rule may be applied in accordance with determining the user input does not match a predetermined dialog response in accordance with narrative rules 542.

[0184] In embodiments, selecting 450 may comprise managing 540 to select signal information 520 as prompt information for generating a dialog response. For example, selecting prompt information may comprise selecting one or more of text information, intent information, equipment status information, user information, and / or biological response information of signal information 520. Such signal information may enable a computer-generated dialog response to relate to one or more user inputs provided to a simulation system. However, such user inputs alone may be insufficient to cause a dialog response to be generated that is related to a simulation. For example, the computer-generated dialog response may be irrelevant and / or conflict with predetermined simulation information for a simulation. By including simulation information with a prompt, a dialog response may be automatically generated within predetermined constraints or bounds of a simulation. Embodiments according to various aspects of the present disclosure enable generative large language models to be used to generate dialog responses, while ensuring that the generated dialog responses remain relevant to an underlyingsimulation presented via a simulation system. Such embodiments also provide the technical benefit of enabling dialog responses related to a simulation to be generated, without sharing the simulation information with a user of simulation system and / or requiring the user to both provide user inputs and simulation information to a simulation system during a simulation. In embodiments, selected prompt information may comprise simulation information or a combination of simulation information and signal information.

[0185] In embodiments, a simulation system may transmit a prompt to a generative computing device to generate a dialog response. The prompt may comprise information selected upon selecting 450. For example, method 400 may comprise transmitting a prompt 460. Transmitting 460 may comprise providing a prompt comprising simulation information to a large language model configured to generate dialog responses. For example, transmitting 460 may comprise providing a prompt generated upon selecting 450 to a generative computing device on which a large language model is executed. The generative computing device may be configured to apply the large language model to the prompt to generate a computer-generated dialog response. In embodiments, the generative computing device may be further configured to, responsive to generating the computer-generated dialog response, transmit the computergenerated dialog response to a computing device from which the prompt was received.

[0186] In embodiments, transmitting 460 may comprise transmitting a prompt from simulation computing device 530 to generative computing device 550. Generative computing device 550 may be configured to generate a dialog response in accordance with the prompt. For example, generative computing device 550 may be configured to perform one or more operations of generative computing device 150 and / or generative computing device 250 with brief reference to FIG. 1-2. The prompt may be transmitted over a network. For example, the prompt may be transmitted over a network comprising network 140 and / or network 240 with brief reference to FIG. 1-2. The prompt may comprise simulation information. For example, the prompt may at least comprise one of NPC information 544 and / or simulation state information 548. In some embodiments, the prompt may further comprise signal information in addition to simulation information. For example, and in addition to simulation information comprising at least one of NPC information 544 and / or simulation state information 548, the prompt may comprise one or more of text information, intent information, equipment status information, user information, and / or biological response information of signal information 520.

[0187] Tn embodiments, automatically generating a dialog response may comprise generating an NPC dialog response 552. Generating 552 may comprise applying a trained LLM to a prompt. The prompt may be provided in a natural language format. The trained LLM may be configured to detect an intent represented by text data of the prompt. The trained LLM may be configured to generate text data representing the intent of the prompt. The prompt may further comprise simulation information. Generating 552 may comprise automatically generating a computer-generated dialog response in accordance with the simulation information. For example, generating 552 may comprise generating a dialog response in accordance with one or more if identity information and / or behavior information associated with an NPC. In embodiments, a computer-generated dialog response may be generated in accordance with one or more of information identifying a belief of an NPC, information identifying a personal quality of the NPC, information identifying a personality trait of the NPC, information identifying an emotional state of the NPC, a level of compliance of the NPC, and / or a level of trust of the NPC. An LLM to which the prompt information is applied may be trained to generate text data for a dialog response in accordance with one or more of each information of the different types of simulation information provided in a prompt.

[0188] In embodiments, generating 552 may comprise generating separate computergenerated dialog responses for separate NPCs of a simulation. For example, generating 552 may comprise generating a first computer-generated dialog response for a first NPC and generating a second computer-generated dialog response for a second NPC. The separate responses may be generated responsive to a separate prompts. For example, generating 552 may comprise generating the first computer-generated dialog response responsive to a first prompt and generating the second computer-generated dialog response responsive to a second prompt. In embodiments, the separate responses may be generated using separate instances of an LLM. For example, generating 552 may comprise generating the first computer-generated dialog response using a first LLM that receives prompts associated with the first NPC but not the second NPC and generating the second computer-generated dialog response using a second LLM that receives prompts associated with the second NPC but not the first NPC. In embodiments, generative computing device 550 may comprise multiple instances of an LLM by which prompts for separate NPCs may be received in order to generate separate, respective dialog responses for each of the separate NPCs.

[0189] Tn embodiments, a simulation system may be configured to receive a computergenerated dialog response. For example, method 400 may comprise receiving a generated dialog response 470. Receiving 470 may comprise receiving text data indicating one or more words of a computer-generated dialog response. Receiving 470 may comprise receiving the computergenerated dialog response from a separate computing device. For example, receiving 470 may comprise receiving, by simulation computing device 530, a computer-generated dialog response from generative computing device 550. Receiving 470 may be performed by a same computing device from which a prompt was transmitted in accordance with transmitting 460. For example, simulation computing device 530 may transmit a prompt for an NPC and, responsive to transmitting the prompt, receive a computer-generated dialog response for the NPC. In embodiments, generative computing device 550 may be configured to automatically transmit a computer-generated dialog response generated by 550 to a computing device from which a corresponding prompt was received.

[0190] In embodiments, generative computing device 550 may correspond to one or more of generative computing device 150 and / or generative computing device 250. Generating 552 may comprise, for example, applying generative model 258 to a prompt comprising simulation information to generate a computer-generated dialog response for a NPC. In some embodiments, generating 552 may comprises multiple instances of generative model 258, wherein an instance of generative model 258 is separately employed to generate a dialog response for each NPC of a plurality of NPCs included in a simulation.

[0191] In embodiments, a simulation system may be configured to provide an audio response. The audio response may comprise a dialog response determined in accordance with a user input. For example, method 400 according to various aspects of the present disclosure may comprise providing an audio response 480. The audio response may correspond to a dialog response. The audio response may be associated with an NPC. The audio response may be associated with a user input received by the simulation system prior to providing 480. The audio response may at least partially identify the user input. For example, text data of the audio response provided upon providing 480 may comprise one or more words detected in an audio input received by a simulation system and / or one more words indicating a motion of an equipment detected via one or more sensors and / or one or more user interfaces of the simulation system. Providing the audio response 480 may enable a user of a simulation system todetermine an effect imparted by a user input previously provided by a user. Providing the audio response 480 may enable a user of a simulation system to determine a user input previously provided by a user was properly received by the simulation system. Providing the audio response 480 may enable a user of a simulation system to receive information associated with a simulation.

[0192] In embodiments, providing 480 may comprised providing an audio response comprising a dialog response. Providing 480 may comprise one or more operations to a dialog response to generate the audio response. Providing 480 may be performed independent of a source of the dialog response. For example, providing 480 may performed in accordance with a predetermined dialog response selected upon selecting predetermined dialog response 440. Alternately or additionally, providing 480 may performed in accordance with a computergenerated dialog response received upon receiving 470. The audio response provided upon providing 480 may be associated with one of one of a predetermined audio response and a computer-generated audio response. In embodiments, providing an audio response 480 may comprise generating an audio response in accordance with one of a predetermined audio response and a computer-generated audio response.

[0193] In embodiments, providing 480 may comprise providing respective audio responses in accordance with each execution of method 400. The respective audio responses may correspond to different types of dialog responses. For example, a first iteration of method 400 may comprise providing 480 a first audio response associated with a predetermined audio response. A second, sequential iteration of method of 400 may comprise providing 480 a second audio response associated with a computer-generated audio response. In embodiments, providing audio responses for a simulation presented via a simulation system may comprise repeatedly providing 480 for each iteration of method 400, wherein an audio response associated with each instance of providing 480 is associated with a same or different a type of dialog response generated by a preceding instance of providing 480. For example, a first operation comprising providing 480 may generate a first audio response associated with one of a predetermined dialog response or a computer-generated dialog response, while a second, sequential operation comprising providing may comprise generating a second audio response associated with another of a computer-generated dialog response or a predetermined dialog response different from the first audio response.

[0194] In embodiments, providing the audio response 480 may comprise outputting an audio output 516 associated with the audio response. Audio output 516 may comprise and audio signal. The audio signal may comprise one or more sounds corresponding to one or more words of a dialog response. Audio output 516 may enable a user to receive a dialog response in an intuitive manner. Audio output 516 may enable a user to receive the dialog response in a manner that reflects how the user may communication with a person in a real-world simulation.

[0195] In embodiments, providing the audio response 480 may comprise providing audio output 516 via at least one output audio transducer of a simulation system. The output audio transducer may be integrated with the simulation system. For example, providing the audio response 480 may comprise providing audio output 516 via one or more of headset speaker 116 and / or output audio interface 215 with brief reference to FIG. 1-2.

[0196] In embodiments, providing the audio response 480 may comprise generating text-to- speech information 560. Generating 560 may comprise performing one or more operations to convert text data to audio information. The text data may comprise text data of a dialog response. Audio output 516 may comprise the generated audio information. The audio information may comprise one or more audible words. The audible words may correspond to the words in the text data to which generating 560 is applied. In some embodiments, an operation of generating 560 may comprise conversion of text data to digital audio data in which one or more words of the text data are converted into audio data that represents corresponding sound for each of the one or more words. In some embodiments, an operation of generating 560 may further comprise digital-to-analog conversion of digital audio data into a corresponding speech signal. Audio output 516 may comprise an audio signal that includes audio information associated with a dialog response. The audio information may correspond to an audible format of the dialog response.

[0197] In embodiments, and as further discussed below with regards to FIG. 6, a simulation system may be configured to provide a video response. The video response may comprise a visual representation of an NPC corresponding to a dialog response determined in accordance with a user input. For example, method 400 according to various aspects of the present disclosure may comprise providing a video response 490. The video response may correspond to a dialog response. The video response may be associated with an NPC. The video response may be associated with a user input received by the simulation system prior to providing 490. Thevideo response may at least partially represent the dialog response. Providing the video response 490 may comprise modifying an appearance of the NPC in accordance with the dialog response. For example, text data of the audio response provided upon providing 480 may be associated with an emotion of sadness and a pose comprising slumped shoulders. Providing the video response 490 may comprise visually representing an NPC to have a sad facial expression and a pose that includes slumped shoulders. Providing the video response 490 may enable a user of a simulation system to determine an effect imparted by a user input previously provided by a user. Providing the audio response 490 may enable a user of a simulation system to determine a user input previously provided by a user was properly received by the simulation system. Providing the audio response 490 may enable a user of a simulation system to identify information associated with a simulation. Providing the audio response 490 may provide non-verbal feedback to a user regarding a simulated state of an NPC responsive to a recently provided user input.

[0198] In embodiments, text data to which generating 560 may be applied may comprise a text data of a dialog response. The dialog response may be associated with an NPC. For example, a dialog response may be associated with NPC 310 with brief reference to FIG. 3.

[0199] In embodiments, the dialog response to which generating 560 is applied may comprise a predetermined dialog response. For example, generating 560 may comprise one or more operations to generate text-to-speech information from a predetermined dialog response selected by managing a narrative 540. The predetermined dialog response may comprise a predetermined dialog response of a plurality of NPC dialog responses 546. Generating 560 may comprise applying one or more operations to a predetermined dialog response to generate an audio response comprising information included in the predetermined dialog response.

[0200] In embodiments, the dialog response to which generating 560 is applied may comprise a computer-generated dialog response. For example, generating 560 may comprise one or more operations to generate text-to-speech information from a computer-generated dialog response generated by generating NPC dialog response 552. Generating 560 may comprise applying one or more operations to a computer-generated dialog response to generate an audio response comprising information included in the computer-generated dialog response.

[0201] In embodiments, providing a dialog responses for a simulation system may comprise repeating one or more operations of method 400 and / or method 500. For example, providingdialog responses may comprise repeating one or more operations 410-430 for each user input received by a simulation system. A first iteration of method 400 may comprise selecting 440, while a second iteration of method may alternately comprise one or more operations 450 - 470 of method 400. Providing a first dialog response of a plurality of dialog responses may comprise providing 480 a first audio response associated with a predetermined dialog response selected via selecting 440, while providing a second dialog response of the plurality of dialog responses may comprise providing 480 a second audio response associated with a computer-generated dialog response generated via one or more of operations 450 - 470 and / or generating 552.

[0202] In embodiments, generating a response may comprise generating a video output. The video output may be generated in addition to an audio output. Alternately, the video output may be generated instead of, or independent of, generation of an audio output. For example, providing an output for an NPC may comprise providing at least one of an automatically generated audio output and / or an automatically generated video output. In embodiments, a method for generating an output for an NPC may comprise various operations performed by one or more computing devices. For example, and with brief reference to FIG. 6, method 600 may comprise one or more operations for managing a narrative 540, generating an NPC dialog response 552, generating an emotion 654, generating an animation 656, generating a text-to- speech signal 560, generating an NPC image 670, outputting an audio output 516, and / or outputting a video output 618. In some embodiments, the one more operations may be performed by computing device 630 and / or generative computing device 650. Computing device 630 may be configured to implement one or more operations of a computing device discussed herein. Generative computing device 650 may be configured to implement one or more operations of a generative computing device discussed herein. For example, computing device 630 may implement one or more operations of server computing device 230 and / or simulation computing device 530 with brief reference to FIG. 2 and 5. Generative computing device 650 may implement one or more operations of generative computing device 250 and / or generative computing device 550 with brief reference to FIG. 2 and 5.

[0203] In embodiments, managing a narrative 540 may comprise one or more operations associated with providing a simulation in accordance with a user input. The one or more operations may be applied to signal information corresponding to the user input. The simulation may be further provided in accordance with various simulation information, including one ormore of narrative rules 542, NPC information 544, predetermined NPC dialog responses, and simulation state information 548. Managing 540 may comprise one or more operations as discussed with regards to FIG. 5. Managing 540 may comprise determining that a user input is not associated with a predetermined NPC dialog response of responses 546. In embodiments, managing 540 may comprise instructing generation of a response for an NPC in accordance with a user input.

[0204] In embodiments, managing 540 may comprise instructing generation of a response for an NPC from a generative computing device. The instructing may comprise sending a prompt from computing device 630 to generative computing device 650. In other embodiments, a same computer may perform one or more operations on a same computing device. Instructing generation of a response for an NPC may comprise providing a prompt with simulation information to a trained, executable module implemented on a same or different computing device relative to a computing device from which the prompt is provided.

[0205] In embodiments, generating an output for an NPC may comprise automatically generating an NPC response. The NPC response may be generated in accordance with the prompt provided upon managing the narrative 540. Generating the NPC response may comprise providing the prompt comprising simulation information to a generative artificial intelligence model. The model may be applied to the prompt to generate the output for the NPC response. The model may be executable. The model may be trained on patterns and structure of predetermined data such that, when a new input is applied to the model, a corresponding output may be generated in a manner consistent with the patterns and structure on which the model was trained.

[0206] In embodiments, generating the NPC response may comprise applying a plurality of trained models. A model of the models may be applied to a received prompt and / or an output of another model of the plurality of models. For example, a first model may be applied to a prmpt comprising signal information associated with a user input and simulation information to generate a first media output. A second model of the plurality of models may be further applied to the first media output to generate a second media output. In some embodiments, secondary prompt information may be applied with the first media output as input to the second model. The secondary prompt information may comprise supplementary information that indicates a type of the second media output to be generated by the second model relative to the first mediaoutput. For example, the second prompt information may indicate that the second media output is required to include one or more of emotion information and / or animation information. The model of the models may generate information used to update an NPC presented via a simulation system in response to a user input. For example, and with brief reference to FIG. 6, generating the NPC response may comprise automatically generating an NPC dialog response 652, automatically generating an emotion 654, and / or automatically generating an animation 656.

[0207] In embodiments, generating the NPC response may comprise automatically generating an NPC dialog response 552. The dialog response may comprise text to be spoken by an NPC in response to a user input, including as further discussed above with regards to FIG. 5. By enabling a dialog response to be generated, a simulation system may be enabled to provide a wider and more relevant number of responses to greater diversity of user inputs relative to what may be provided as predetermined NPC dialog responses 546. In embodiments, the dialog generated upon generating 552 may be subsequently provided for automatic text-to-speech generation 560 and output as an audio output 516, including as further discussed with regards to FIG. 5.

[0208] In embodiments, generating an NPC response may further comprise generating visual information to be provided in addition to, or as an alternative to, audio output 516. For example, generating an NPC response may comprise generating an emotion 654. Generating 654 may comprise generating emotion information for an NPC response. The emotion information may indicate an emotion to be replicated as part of the NPC response. In some embodiments, the emotion information may comprise text data that indicates the emotion. For example, the emotion information may indicate an emotion comprising one of neutral, joy, anger, anger, surprise, sadness, disgust, or fear. In embodiments, the generating 654 may comprise generating the emotion information in accordance with a dialog response generated upon generating 652. For example, generating 652 may generate text information comprising the statement, “I spilled my cup.” Generating 654 may comprise applying the text information to a previously trained model configured to identify an emotion associated with such a statement. For example, generating 654 may comprise generating emotional information that indicates “sadness” in accordance with text information. Generating 654 may further comprise providing generated emotion information for automatically generating an NPC image in accordance with the emotion information.

[0209] Tn some embodiments, the emotion information may alternately or additionally be provided for modification of an audio output. For example, generating 654 may comprise providing generated emotion information for generating text-to-speech 560. Generating 560 may comprise generating audio output 516 such that audio output indicates the emotion indicated by the emotion information. For example,, generating text-to-speech 560 may comprise selecting a volume, timbre, pace, and / or pitch of a generated audio output 516 in accordance with emption information generated upon generating 654.

[0210] In embodiments, generating an NPC response may alternately or additionally comprise generating an animation 656. Generating an animation 656 may comprise generating animation information for an NPC response. The animation information may indicate a physical position or movement to be replicated as part of the NPC response. In some embodiments, the animation information may comprise text data that indicates the physical position or movement. For example, the animation information may indicate an animation comprising one of a facial expression, a facial movement or position, an arm movement or position, a leg movement or position, or a combination of such expressions, movements, and / or positions. In embodiments, the generating 656 may comprise generating the animation information in accordance with a dialog response generated upon generating 652. For example, generating 652 may generate text information comprising the statement, “I spilled my cup.” Generating 656 may comprise applying the text information to a previously trained model configured to identify an animation associated with such a statement. For example, generating 656 may comprise generating animation information that indicates “slumped shoulders and downward tilted head” in accordance with text information. Generating 656 may further comprise providing animation information for automatically generating an NPC image in accordance with the animation information. In embodiments, animation information and / or emotion information may comprise text data that respectively identifies at least one animation or at least one emotion.

[0211] In some embodiments, automatically generating an emotion 654 and / or automatically generating an animation 656 may comprise updating a simulating state in accordance with the generated information. For example, an emotional state of an NPC and / or a position of an NPC maintained in one or more of simulation state information 548 and / or NPC information 544 may be updated in accordance with the emotion information and / or the animation information. By updating such information 544,548, a feedback loop may be established, thereby ensuringcontinuity of a simulation in response to a sequence of user inputs. By updating the information 544,548, an NPC response may be sequentially updated in accordance with a sequence of NPC response that may be automatically generated or, alternately, comprise both predetermined responses and automatically generated responses. The updated information may enable a simulation to be accurately modified in accordance with a second, subsequently received user input.

[0212] In embodiments, generating an NPC response may comprise transmitting one or more of a dialog response, emotion information, and / or animation information to a source of a prompt upon which the NPC response was generated. For example, generating the NPC response may comprise transmitting the dialog response, emotion information, and / or animation information from generative computing device 650 to computing device 630. In other embodiments, such information may be applied to one or more additional operations on a same computing device on which such information was generated.

[0213] In embodiments, generating an NPC response may comprise generating an image of the NPC 670. The image of the NPC may comprise one or more images. The one or more images may comprise video. The one or more images may be output as video output 618. For example, the one or more images may be provided via display 218 of wearable computing device 210 to provide visual information regarding the NPC. The image of the NPC may visually indicate an NPC response to a user input.

[0214] In embodiments, the image of the NPC may be generated in accordance with information identifying an NPC response. For example, an NPC response may comprise emotion information generated upon generating 654 and / or animation information generated upon generating 656. By generating the image of the NPC in accordance with such an information, visual output 618 of an NPC of a simulation may be coordinated with an audio output 516 for the NPC of the simulation. For example, a movement or position of the NPC in video output 618 may visually match an audible dialog response in audio output 516, thereby providing a uniform multisensory NPC response to a user input.

[0215] In embodiments, generating an NPC image 670 may comprise modifying a visual representation of an NPC. The representation may comprise an appearance of the NPC. The representation of the NPC may be modified relative to a representation of the NPC in a preceding video output 618. Modifying the visual representation may comprise modifying a two or threedimensional representation of the NPC. For example, a current representation of the NPC may be indicated in simulation state information 548 and / or NPC information 544. Generating the NPC image 670 may comprise further modifying this information in accordance with information associated with an NPC response. For example, simulation information may indicate that an NPC is represented in a first state in a prior video output. In the first state, the NPC may have a first pose of a plurality of poses. In the first state, the NPC may have a first facial expression of a plurality of facial expressions. Generating the NPC image may comprise modifying the representation of the NPC from the first state to a second state. In the second state, the pose of the NPC may be changed from the first pose to a second pose different from the first pose. In the second state, the facial expression of the NPC may be changed from the first facial expression to a second facial expression different from the first facial expression. The first pose and / or the first facial expression may be indicated in one or more of NPC information 544 and / or simulation state information 548. Upon generating the NPC image, NPC information 544 and / or simulation state information 548 may be updated to indicate a second, different state of the NPC as represented in the image generated upon generating 670.

[0216] In embodiments, generating an NPC image may comprise modifying a pose 672 of the NPC. The pose of the NPC may comprise an arrangement of a part of a body of the NPC, including as further represented in a video output of the simulation system. For example, the part of the body may comprise a limb or limbs, torso, or head of the NPC. Modifying the pose 672 may comprise repositioning the part of the body of the NPC from a first position to a second position. In some embodiments, modifying the pose may comprise updating a wireframe of the NPC from which a visual representation of the NPC is subsequently rendered. In other embodiments, modifying the pose 672 may comprise modifying a two dimensional representation of the NPC. Alternate or additional animation techniques may be applied as well. In some embodiments, modifying the pose 672 may comprise modifying the pose over time. The pose modified over time may represent a gesture or movement made by the NPC. Modifying the pose 672 may enable a simulation to reflect how a real-world person may move, thereby improving the realistic nature of the simulation.

[0217] In embodiments, modifying the pose 672 may comprise modifying the pose in accordance with animation information. The animation information may be generated upon generating 654. Modifying the pose 672 may comprise obtaining a current pose as indicated inone or more of NPC information and / or simulation state information 548. Modifying the pose 672 may comprise modifying the pose relative to narrative rules 542. The narrative rules 542 may limit an extent to which the pose 672 may be modified. For example, narrative rules 542 may require an NPC to maintain their right hand in a coat pocket during a simulation. Accordingly, modifying the pose 672 may comprise adjusting a position of a left hand of the NPC in accordance with animation information, but maintaining the right hand of the NPC in the coat pocket. By applying narrative rules 542 upon modifying the pose 672, an NPC response may be automatically generated and not predetermined, while ensuring an underlying narrative or goal remains reflected in the response of the NPC.

[0218] In embodiments, generating an NPC image may comprise modifying a facial expression 676 of the NPC. The facial expression of the NPC may comprise an arrangement of facial features of the NPC, including as further represented in a video output of the simulation system. For example, a facial expression may comprise a particular position of each of one or more of an eyebrow, an eyelid, and a jawbone of an NPC. The particular position may be selected from a plurality of positions for each respective facial feature. Modifying the facial expression 676 may comprise repositioning at least one facial feature of the NPC from a first position to a second position. In some embodiments, modifying the facial expression may comprise updating a wireframe of the NPC from which a visual representation of a face the NPC is subsequently rendered. In other embodiments, modifying the facial expression 676 may comprise modifying a two dimensional representation of a face of the NPC. Other animation techniques may alternately or additionally be applied as well. In some embodiments, modifying the facial expression 676 may comprise modifying the facial expression over time. Modifying the facial expression 676 may enable a simulation to reflect how a real-world person may express their response visually, thereby improving the visual accuracy of the simulation relative to a real-world event.

[0219] In embodiments, modifying the facial expression 676 may comprise modifying the facial expression in accordance with emotion information. The emotion information may be generated upon generating 652. Modifying the facial expression 676 may comprise obtaining a current facial expression as indicated in one or more of NPC information 544 and / or simulation state information 548. For example, NPC information 544 may indicate that a current facial expression of the NPC is neutral. Modifying the facial expression 676 in accordance withemotion information may comprise modifying the facial expression from neutral to a different, second facial expression as represented in the emotion information.

[0220] In embodiments, modifying the facial expression 676 may comprise modifying the facial expression relative to narrative rules 542. The narrative rules 542 may limit an extent to which the facial expression 676 may be modified. For example, narrative rules 542 may restrict a facial expression of an NPC to a subset of permitted facial expressions, wherein the subset includes a neutral facial expression. In accordance with emotion information that corresponds to an emotion outside of this subset and narrative rules 542, modifying the facial expression 676 may limit changing the facial expression of the NPC to neutral, rather than a non-permitted facial expression associated with the emotional information. By applying narrative rules 542 upon modifying the facial expression 676, an NPC response may be automatically generated and not predetermined, while ensuring an underlying narrative or goal remains reflected in the response of the NPC.

[0221] In embodiments, generating the image of the NPC 670 may comprise syncing lips 674 of the NPC with an audio output. The audio output may represent a dialog response generated for the NPC. A position of each lip of lips 674 an NPC may be modified over time in coordination with a dialog response provided via audio output 516. In some embodiments, the dialog response may have been previously modified for audio output 516 in accordance with emotion information generated for the NPC. In accordance with synchronizing lips 674 of the NPC with the audio output 516, a visible response provided for an NPC may be aligned with an audible response provided by the NPC.

[0222] In embodiments, generating an output for an NPC may comprise outputting a video output 618. Outputting the video output 618 may comprise rendering an NPC in accordance with various information for display via a wearable computing device of a simulation system. In some embodiments, for example, outputting the video output 618 may comprise rendering an NPC for display via the display. Outputting the video output 618 may comprise combining visual information representing a virtual environment and / or one or more other NPCs with visual information representing the NPC for which a response was automatically generated. Outputting the video output 618 may comprise providing video output 618 to a display of a simulation system. For example, and with brief reference to FIG. 2, outputting the video output 618 may comprise providing the video output 618 to display 218 of wearable computing device 210 inorder to provide visual feedback associated with the response of the NPC to a user of wearable computing device 210.

[0223] In various embodiments, “satisfy,” “meet,” “match,” “associated with,” or similar phrases used herein may include an identical match, a partial match, meeting certain criteria, matching a subset of data, a correlation, satisfying certain criteria, a correspondence, an association, an algorithmic relationship, and / or the like. Similarly, as used herein, “authenticate,” “verify,” “validate,” or similar terms may include an exact authentication, verification, or validation; a partial authentication, verification, or validation; authenticating, verifying, or validating a subset of data; satisfying certain criteria; an association; an algorithmic relationship; and / or the like.

[0224] In various embodiments, an automatic operation comprises performance of a subsequent operation upon performance of an initial operation. The automatic operation may be performed by a computing device. The automatic operation may be performed without, or independent of, manual input that may or may not be received the initial operation. For example, an operation may be automatically performed to provide an output after a user input is received. The operation may be performed without requiring, or independent of, whether any subsequent user input is received. For an operation comprising multiple sub-operations, each sub-operation of the multiple sub-operations may be performed in series or in parallel without, or independent of, whether a manual input is subsequently received after the operation is initiated. For example, and with brief reference to FIG. 6, each operation of method 600 may be automatically performed without human intervention.

[0225] Benefits, other advantages, and solutions to problems have been described herein with regard to specific embodiments. Furthermore, the connecting lines shown in the various figures contained herein are intended to represent exemplary functional relationships and / or physical couplings between the various elements. It should be noted that many alternative or additional functional relationships or physical connections may be present in a practical system. However, the benefits, advantages, solutions to problems, and any elements that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as critical, required, or essential features or elements of the disclosures. The scope of the disclosure is accordingly to be limited by nothing other than the appended claims and their legal equivalents, in which reference to an element in the singular is not intended to mean “one and only one” unlessexplicitly so stated, but rather “one or more.” Moreover, where phrase similar to “at least one of A, B, or C”, “at least one of A, B, and C”, or “and / or” is used in the claims, it is intended that the phrase be interpreted to mean that A alone may be present in an embodiment, B alone may be present in an embodiment, C alone may be present in an embodiment, or that any combination of the elements A, B and C may be present in a single embodiment; for example, A and B, A and C, B and C, or A and B and C.

[0226] Systems, methods and apparatus are provided herein. In the detailed description herein, references to “various embodiments,” “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described. After reading the description, it will be apparent to one skilled in the relevant art(s) how to implement the disclosure in alternative embodiments. Furthermore, no element, component, or method step in the present disclosure is intended to be dedicated to the public regardless of whether the element, component, or method step is explicitly recited in the claims. No claim element is intended to invoke 35 U.S.C. 112(f) unless the element is expressly recited using the phrase “means for.” As used herein, the terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented method for automatically generating an audio response for a non-playable character (“NPC”) of a simulation presented by a simulation system, the method comprising: providing, by a computing device of the simulation system, a prompt comprising simulation information of the simulation to a generative computing device configured to automatically generate a computer-generated dialog response for the NPC in accordance with the prompt; receiving, by the computing device of the simulation system, the computer-generated dialog response for the NPC generated by the generative computing device; and providing, by the computing device of the simulation system, the audio response in accordance with the computer-generated dialog response.

2. The method of claim 1, wherein the simulation information comprises identity information for the NPC.

3. The method of claim 2, wherein the identity information comprises one or more of information identifying a belief of the NPC, information identifying a personal quality of the NPC, and / or information identifying a personality trait of the NPC.

4. The method of claim 1, wherein the simulation information comprises behavior information for the NPC.

5. The method of claim 4, wherein the behavior information comprises one or more of information identifying an emotional state of the NPC, information identifying a level of compliance of the NPC, and / or information identifying a level of trust of the NPC.

6. The method of claim 1, further comprising receiving, by the computing device of the simulation system, a user input, wherein the audio response is provided responsive to the user input.

7. The method of claim 6, wherein the user input comprises an audio input and the method further comprising classifying the audio input to detect intent information associated with the audio input.

8. The method of claim 7, wherein classifying the audio input comprises performing a natural language understanding analysis of the audio input.

9. The method of claim 1, further comprising: receiving, by the computing device of the simulation system, a first user input; and performing a first comparison to determine whether the first user input matches at least one predetermined dialog response of a plurality of predetermined dialog responses, wherein the prompt is provided responsive to the first comparison indicating the first user input does not match the at least one predetermined dialog response of the plurality of predetermined dialog responses.

10. The method of claim 9, further comprising receiving, by the computing device of the simulation system, a second user input; performing a second comparison to determine whether the second user input matches a predetermined dialog response of the plurality of predetermined dialog responses; and providing a second audio response corresponding to the predetermined dialog response of the plurality of predetermined dialog responses in accordance with the second comparison indicating the second user input matches the predetermined dialog response of the plurality of predetermined dialog responses.

11. The method of claim 9, wherein the prompt further comprises signal information associated with a user input.

12. The method of claim 9, wherein the first user input comprises one or more of an audio input, a controller input, a position input, and / or a physiological input.

13. The method of claim 1, wherein the simulation comprises a plurality of NPCs and the method further comprises providing separate prompts for each NPC of the plurality of NPCs.

14. A simulation computing device of a simulation system for automatically generating audio responses for a non-playable character (“NPC”) of a simulation presented by the simulation system, the simulation computing device comprising: an input audio transducer configured to receive audio inputs; an output audio transducer configured to provide an audio response of the audio responses; at least one non-transitory computer-readable storage medium storing instructions and a plurality of predetermined dialog responses; andat least one processor communicatively coupled to the input audio transducer, the output audio transducer, and the storage medium, wherein the instructions, when executed by the processor, cause the simulation computing device to perform operations comprising: receiving, via the input audio transducer, a first audio input and a second audio input; determining the first audio input matches a predetermined dialog response of the plurality of predetermined dialog responses; responsive to determining the first audio input matches the predetermined dialog response of the plurality of predetermined dialog responses, providing, via the output audio transducer, a first audio response in accordance with the predetermined dialog response; determining the second audio input does not match at least one predetermined dialog response of the plurality of predetermined dialog responses; providing a prompt comprising simulation information of the simulation; receiving a computer-generated dialog response for the NPC generated in accordance with applying the prompt to a large language model trained to automatically generate the computer-generated dialog response for the NPC in accordance with the prompt; and providing, via the output audio transducer, a second audio response in accordance with the computer-generated dialog response, wherein the second audio response is different from the first audio response.

15. The simulation system of claim 14, wherein the simulation information comprises at least one of identity information for the NPC and / or behavior information for the NPC.

16. The simulation system of claim 14, wherein the simulation information comprises one or more of information identifying a belief of the NPC, information identifying an emotional state of the NPC, information identifying a level of compliance of the NPC, and / or information identifying a level of trust of the NPC.

17. The simulation system of claim 14, wherein the prompt further comprises signal information detected from the second audio input.

18. A non-transitory computer-readable storage medium storing instructions for automatically generating an audio response for a non-playable character (“NPC”) of a simulationpresented by a simulation system that, when executed by a processor of at least one computing device of the simulation system, cause the at least one computing device to perform operations comprising: receiving a user input; determining the user input does not match at least one predetermined dialog response of a plurality of predetermined dialog responses provided for the simulation; and in accordance with the determining, generating a computer-generated dialog response for the NPC in accordance with at least one of identify information for the NPC and / or behavior information for the NPC; and providing the audio response in accordance with the computer-generated dialog response.

19. The storage medium of claim 18, wherein determining the user input does not match at least one predetermined dialog response of the plurality of predetermined dialog responses provided for the simulation comprises: classifying the user input to generate signal information associated with the user input; and comparing the signal information to one or more narrative rules for the simulation.

20. The storage medium of claim 19, wherein the user input comprises audio input and classifying the user input comprises detecting intent information for the audio input in accordance with a natural language understanding analysis of the audio input.

21. A computer-implemented method for automatically generating a response for a non- playable character (“NPC”) of a simulation presented by a simulation system, the method comprising: providing, by a computing device of the simulation system, a prompt comprising simulation information of the simulation to a generative computing device configured to automatically generate a computer-generated response for the NPC in accordance with the prompt; receiving, by the computing device of the simulation system, the computer-generated response for the NPC generated by the generative computing device; andproviding, by the computing device of the simulation system, the response for the NPC in accordance with the computer-generated response, wherein the response for the NPC comprises at least one of an audio response and / or a video response.

22. The method of claim 21, wherein the response comprises the video response.

23. The method of claim 22, wherein providing the video response for the NPC comprises generating emotion information associated with the NPC.

24. The method of claim 23, wherein the computer-generated response for the NPC comprises a dialog response and the emotion information is generated in accordance with the dialog response.

25. The method of claim 22, wherein providing the video response for the NPC comprises generating animation information associated with the NPC.

26. The method of claim 25, wherein the computer-generated response for the NPC comprises a dialog response and the animation information is generated in accordance with the dialog response.

27. The method of claim 22, wherein the computer-generated response for the NPC comprises a dialog response and providing the video response comprises modifying an appearance of the NPC in accordance with the dialog response.

28. The method of claim 27, wherein modifying the appearance comprises at least one of modifying a pose and / or a facial expression of the NPC.

Images