Simulator for navigating monitored operational technology environments for cybersecurity events using contextualized simulations

Abstract

Embodiments of the invention are directed to a computer-implemented method that includes detecting, using a processor system, a monitored environment (ME) event based at least in part on information of an operational technology (OT) network that monitors the ME. The computer-implemented method further includes generating, using the processor system, a contextualized simulation of the ME event based at least in part on the information of the OT network, information of the ME, and information of the ME event.

Description

202318192 SIMULATOR FOR NAVIGATING MONITORED OPERATIONAL TECHNOLOGY ENVIRONMENTS FOR CYBERSECURITY EVENTS USING CONTEXTUALIZED SIMULATIONS TECHNICAL FIELD

[0001] The present embodiments relate to computing systems, computer- implemented method, and computer program products configured and arranged to implement simulation systems operable to navigate monitored-environment (ME) events using contextualized simulations of the ME, the ME event, and an operational technology (OT) network that monitors the ME. In some embodiments, the ME event is an anomalous ME event, and the simulations include three-dimensional (3D) replays of the ME event. BACKGROUND

[0002] An OT network is a configuration of hardware and software tools used to monitor and control MEs. An ME can include a wide range of physical processes, devices, and infrastructure found across multiple different asset-intensive sectors, including but not limited to manufacturing, oil & gas, electrical generation / distribution, aviation, maritime, rail, and utilities. OT networks perform a wide variety of tasks ranging from monitoring critical infrastructure to controlling robots on a manufacturing floor.

[0003] A security operations center (SOC) is a command center facility in which a configuration of computer detection and analysis technologies (i.e., SOC computing systems) receive and evaluate outputs from an OT network to electronically differentiate normal or non-anomalous OT network (or ME) activity from abnormal or anomalous OT network (or ME) activity. The SOC computing system generates a notification that alerts an SOC analyst / user that the SOC computing system has detected anomalous OT activity. In known configurations, SOC computing systems can be located remotely from its202318192 associated ME and electronically coupled to the OT network that is monitoring the ME. The SOC computing system electronically coupled to the OT network is often referred to as an OT-SOC system. A team of information technology (IT) professionals (i.e., the SOC analysts / users) with expertise in information security (infosec) review SOC computing system notifications to confirm whether or not they are valid and accurate.

[0004] In OT-SOC systems, internet traffic, networks, desktops, servers, endpoint devices, databases, applications and other systems of the OT network are continuously monitored at the SOC and analyzed for signs of a potential or actual security incident (i.e., an ME event). OT-SOC systems are an integral part an organization's data protection strategy. They help organizations respond to intrusions quickly while also improving detection and prevention processes.

[0005] Although existing SOC computing systems improve SOC operations by identifying anomalous OT activity / events that would be impossible to detect manually, there are still challenges, including specifically challenges associated with providing an efficient and effective interface between SOC computing systems and SOC users. Accordingly, there is a need in the relevant art to provide an interface between SOC computing systems and SOC users that improves the ability of SOC users to interpret and evaluate an SOC computing system’s determinations about the security status of an OT network that is monitoring and controlling the activities of an ME. BRIEF SUMMARY

[0006] Embodiments of the invention are directed to a computer-implemented method that includes detecting, using a processor system, a monitored environment (ME) event based at least in part on information of an operational technology (OT) network that monitors the ME. The computer-implemented method further includes generating, using the processor system, a contextualized simulation of the ME event based at least in part on the information of the OT network, information of the ME, and information of the ME event.202318192

[0007] In addition to any one or more of the features described herein, the contextualized simulation of the ME event includes a simulation of the ME event; ME activity associated with the ME event; and OT network activity associated with ME event.

[0008] In addition to any one or more of the features described herein, the ME event includes an anomalous ME event.

[0009] In addition to any one or more of the features described herein, the contextualized simulation of the ME event includes a three-dimensional (3D) representation of the contextualized simulation of the ME event.

[0010] In addition to any one or more of the features described herein, the contextualized simulation has a time duration.

[0011] In addition to any one or more of the features described herein, the time duration includes a time duration of the ME event.

[0012] In addition to any one or more of the features described herein, the time duration further covers prior to and subsequent to the ME event.

[0013] In addition to any one or more of the features described herein, the computer- implemented method further includes generating, using the processor, a replay of the contextualized simulation of the ME event.

[0014] In addition to any one or more of the features described herein, the computer- implemented method further includes providing to a display the contextualized simulation of the ME event; the display is part of a head mounted display; and the display includes a three-hundred and sixty degree (360-degree) video frame.

[0015] In addition to any one or more of the features described herein, the information of the OT network includes information of a digital twin of the OT network; the information of the ME includes information of a digital twin of the ME; the202318192 contextualized simulation of the ME event further includes natural language text annotations associated with the ME event; and the contextualized simulation of the ME event further includes one or more audible generated natural language ME event descriptions associated with the ME event.

[0016] Embodiments of the invention are also directed to computer systems and computer program products having substantially the same features as the computer- implemented method described above.

[0017] Additional features and advantages are realized through techniques described herein. Other embodiments and aspects are described in detail herein. For a better understanding, refer to the description and to the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The subject matter which is regarded as embodiments is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other features and advantages of the embodiments are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:

[0019] FIG. 1 depicts a simplified block diagram illustrating a system according to embodiments;

[0020] FIG. 2 depicts a simplified block diagram illustrating additional details of a subsystem of the system shown in FIG. 1 in accordance with embodiments;

[0021] FIG. 3 depicts a simplified block diagram illustrating additional details of a subsystem of the system shown in FIG. 1 in accordance with embodiments;

[0022] FIG. 4 depicts a simplified block diagram illustrating a head-mounted-device (HMD) according to embodiments;202318192

[0023] FIG. 5 depicts a simplified block diagram illustrating a full 360-degree video frame according to embodiments;

[0024] FIG. 6 depicts a simplified block diagram illustrating a contextualized event simulation on a full 360-degree video frame according to embodiments;

[0025] FIG. 7 depicts a simplified block diagram illustrating temporal features of a contextualized event simulation according to embodiments;

[0026] FIG. 8 depicts a simplified block diagram illustrating a digital twin processing system in accordance with embodiments;

[0027] FIG. 9 depicts simplified block diagram and a flow diagram illustrating hardware components, software components, and a computer-implemented methodology according to embodiments;

[0028] FIG. 10 depicts a simplified block diagram illustrating a machine learning system that can be utilized to implement embodiments;

[0029] FIG. 11 depicts a simplified diagram illustrating a learning phase that can be implemented by the machine learning system shown in FIG. 10; and

[0030] FIG. 12 depicts a simplified block diagram illustrating a programmable computer system operable to implementing embodiments.

[0031] In the accompanying figures and following detailed description of the disclosed embodiments, the various elements illustrated in the figures are provided with three digit reference numbers. In some instances, the leftmost digits of each reference number corresponds to the figure in which its element is first illustrated. DETAILED DESCRIPTION

[0032] Embodiments provide a novel interface between SOC computing systems and SOC users that improves the ability of SOC users to interpret and evaluate an SOC202318192 computing system’s determinations about the security status of an OT network that is monitoring and controlling the activities of an ME. More specifically, embodiments of the invention provide a novel simulator that can be implemented as part of an OT-SOC system. The novel simulator is configured and arranged to, in response to the OT-SOC system detecting an ME event, generate a novel contextualized simulation of the ME event based at least in part on information of an OT network, information of an ME that is being monitored by the OT network, and information of the ME event itself. The simulation is contextualized in that it simulates both the ME event and realistic representations of activities of the ME in which the ME event occurs. For example, where the ME is an automobile manufacturing plant, the contextualized simulation of the ME event will include a realistic representation of the various portions of the ME that are relevant to the ME event, including but not limited to automobiles, assembly lines, manufacturing equipment, ME computer control components, and ME computer controlled processes. Additionally, where the ME is an automobile manufacturing plant, the contextualized simulation of the ME event will include a realistic representation of the various portions of the OT network that are relevant to the ME event, including but not limited to OT network computer control components and OT network computer controlled processes.

[0033] The novel contextualized simulation enables an SOC analyst to leverage simulation technology in a novel way. Conventional simulation technology is used to test hypothetical or experimental operating conditions on an electronic representation of a system-under-development (SUD) to provide an estimate of how the SUD is likely to respond. In contrast to using simulation technology to perform forward-looking evaluations of experimental or hypothetical conditions, the novel contextualized simulation of ME events in accordance with embodiments use its simulation technology to perform evaluations of ME events that have actually occurred, along with evaluations of the surrounding circumstances (i.e., the context) of the ME event. Accordingly, in embodiments, the contextualized simulation of the ME event includes a simulation of the202318192 ME event, ME activity associated with the ME event, and OT network activity associated with ME event.

[0034] In some embodiments, the contextualized simulation of the ME event can be displayed as a two-dimensional representation, a three-dimensional (3D) representation, or a three-hundred and sixty degree (360-degree) virtual reality representation. In some embodiments, the contextualized simulation includes an ability to replay the simulation of the ME event multiple times, and the original or replay of the contextualized simulation of the ME event has a time duration that covers any combination of the ME event, prior to the ME event, and subsequent to the ME event.

[0035] In some embodiments of the invention, the ability of the novel contextualized simulation of ME events to use its simulation technology to perform evaluations of ME events that have actually occurred, along with evaluations of the surrounding circumstances (i.e., the context) of the ME event is enabled by generating a digital twin of the OT network and a digital twin of the ME, then using novel simulator to create the novel contextualized simulations based on the digital twins.

[0036] In some embodiments, machine learning (ML), artificial intelligence (AI), and generative AI technologies are used to provide the contextualized simulations with the ability to display natural language text annotations associated with the ME event, as well as one or more audible generated natural language ME event descriptions associated with the ME event.

[0037] Turning now to a more detailed description of the aspects of the invention, FIG. 1 depicts a simplified block diagram illustrating an OT-SOC system 100 according to embodiments of the invention. In embodiments, the OT-SOC system 100 includes an SOC 110, an ME 120, and historical events / activities 130 configured and arranged as shown. In some embodiments, the SOC 110, the ME 120, and the historical events / activities 130 can be configured in any combination of the same or physically separate locations. A cloud computing system 50 is in wired or wireless electronic202318192 communication with the OT-SOC system 100. The cloud computing system 50 can supplement, support or replace some or all of the functionality (in any combination) of the OT-SOC system 100. Additionally, some or all of the functionality of the OT-SOC system 100 can be implemented as a node of the cloud computing system 50. Additional details of cloud computing functionality that can be used in connection with embodiments are depicted by the computing system 1200 shown in FIG. 12 and described in greater detail subsequently herein.

[0038] The ME 120 includes an OT network 124 operable to monitor operations performed at the ME 120; an OT sensor network 126B operable to sense activities of the OT network 124; ME assets 122A and ME processes 122B operable to perform the operations the ME 120 is designed to perform (e.g., manufacture automobiles); and an ME sensor network 126A operable to sense activities of the ME assets 122A and ME processes 122B.

[0039] The historical events / activities 130 accumulate events and activities that of the ME 120 that occur over time and provide the same to the SOC 110 for use in training various ML / AI / Generative-AI (e.g., ML / AI 216 and NL generative AI 220 shown in FIG. 3) that support simulation operations performed at the SOC 110. In embodiments, the historical events / activities 130 is implemented as a repository. In embodiments, the repository can be implemented as a searchable database operable to organize and store data / information of activities and / or events of the ME 120 in segments or regions of the repository. The repository can be any form of database, including but not limited to, relational SQL databases, noSQL unstructured databases, unstructured data lakes, time- series databases, and the like. In some embodiments, the repository can include features and functionality of a relational database operably controlled by the computing system 1200 (shown in FIG. 12). In general, a database is a means of storing information in such a way that information can be retrieved from it, and a relational database presents information in tables with rows and columns. A table is referred to as a relational table in the sense that it is a collection of objects of the same type (rows). Data in a table can be202318192 related according to common keys or concepts, and the ability to retrieve related data from a table is the basis for the term relational database. A database management system (DBMS) of the computing system 1200 controls the way data in the historical events / activities 130 in the repository are stored, maintained, and retrieved. A database management system of the computing system 1200 performs the tasks of determining the way data and other information are stored, maintained, and retrieved from the repository.

[0040] In embodiments, the SOC 110 is a command center facility in which a configuration of computer detection and analysis technologies (i.e., SOC computing systems) receive and evaluate outputs from the ME 120, including the ME sensor network 126A, the OT sensor network 126B, and / or the historical events / activities 130 to electronically differentiate normal or non-anomalous events / activities of the OT network 124 (or ME 120) from abnormal or anomalous events / activities of the OT network 124 (or ME 120). The SOC 110 implements a novel simulator (not shown separately from the SOC 110) is configured and arranged to, in response to the SOC 110 detecting an ME event, use a contextualized simulation system 112 to generate a novel contextualized simulation (e.g., the contextualized event simulation 604 shown in FIG. 6) of the ME event based at least in part on information of the OT sensor network 126B, information of the ME sensor network 126A, and information of the ME event itself. The simulation is contextualized in that it simulates both the ME event and realistic representations of activities of the ME 120 in which the ME event occurs. For example, where the ME 120 is an automobile manufacturing plant, the contextualized simulation of the ME event will include a realistic representation of the various portions of the ME 120 (e.g., ME assets 122A and ME processes 122B) that are relevant to the ME event, including but not limited to automobiles, assembly lines, manufacturing equipment, ME computer control components, and ME computer controlled processes. Additionally, where the ME 120 is an automobile manufacturing plant, the contextualized simulation of the ME event generated by the contextualized simulation system 112 will include a realistic representation of the various portions of the OT network 124 that are relevant to the ME202318192 event, including but not limited to computer control components and computer controlled processes of the OT network 124.

[0041] FIG. 2 depicts a non-limiting example of how the contextualized simulation system 112 shown in FIG. 1 can be implemented as a contextualized simulation system 112A. The contextualized simulation system 112A includes an ME digital twin 210, an OT digital twin 212, simulation functionality 214, ML / AI functionality 216, ME / OT annotations 218, NL generative AI 220, and 3D functionality 230, configured and arranged as shown.

[0042] Referring now to elements depicted in FIGS. 1 and / or 2, the novel contextualized simulation (e.g., the contextualized event simulation 604 shown in FIG. 6) generated by the contextualized simulation system 112, 112A enables an SOC analyst / user 140 to leverage simulation technology in a novel way. Conventional simulation technology is used to test hypothetical or experimental operating conditions on an electronic representation of a system-under-development (SUD) to provide an estimate of how the SUD is likely to respond. In contrast to using simulation technology to perform forward-looking evaluations of experimental or hypothetical conditions, the novel contextualized simulation system 112, 112A in accordance with embodiments use its simulation technology to perform evaluations of ME events (e.g., ME event 620 shown in FIG. 6) that have actually occurred, along with evaluations of the surrounding circumstances (i.e., the context) of the ME event. Accordingly, in embodiments, the contextualized simulation of the ME event generated by the contextualized simulation system 112, 112A includes a simulation of the ME event, a simulation of activity of the ME 120 associated with the ME event, and a simulation activity of the OT network 124 associated with ME event, all presented together (e.g., the contextualized event simulation 604 shown in FIG. 6).

[0043] In some embodiments, the contextualized simulation of the ME event generated by the contextualized simulation system 112, 112A can be displayed as a two- dimensional representation, a three-dimensional (3D) representation, or a three-hundred202318192 and sixty degree (360-degree) virtual reality representation. In some embodiments, the contextualized simulation generated by the contextualized simulation system 112, 112A includes an ability to replay the simulation of the ME event multiple times (e.g., using the simulation functionality 214 shown in FIG. 2), and the original or replay of the contextualized simulation of the ME event has a time duration (e.g., Time duration-1 and / or Time duration-2 shown in FIG. 7) that covers any combination of the ME event, prior to the ME event, and subsequent to the ME event.

[0044] In some embodiments of the invention, the ability of the novel contextualized simulation of ME events to use its simulation technology to perform evaluations of ME events that have actually occurred, along with evaluations of the surrounding circumstances (i.e., the context) of the ME event is enabled by generating a digital twin of the OT network 124 (e.g., OT digital twin 212 shown in FIG. 2) and a digital twin of the ME 120 (e.g., ME digital twin 210 shown in FIG. 2), then using the novel simulator (not shown separately from the SOC 110) to create the novel contextualized simulations based on the digital twins.

[0045] In some embodiments, the contextualized simulation system 112 includes ML, AI, and generative AI technologies (e.g., ML / AI 216 and / or NL generative AI 220 shown in FIG. 2) that are used to provide the contextualized simulations with the ability to display natural language text annotations (e.g., ME / OT annotations 622 shown in FIG. 6) associated with the ME event, as well as one or more audible generated natural language ME event descriptions (e.g., audible generated natural language event descriptions 624 shown in FIG. 6) associated with the ME event.

[0046] FIG. 3 depicts a non-limiting example of how the SOC 110 can be provided with a display 348 that can be implemented in a variety display formats, including but not limited to 2D, 3D and immersive virtual reality (VR) displays. The SOC 110 is in communication one or more displays 348 over a network 320. The network 320 can be local or remote (e.g., the Internet), which means that the SOC user 140 can be local with the SOC 110 or remote from the SOC 110. In some embodiments of the invention, one202318192 or more instances of the displays 348 can be implemented as a head mounted display (HMD) 350 or a non-HMD (NHMD) 360. The NHMD 360 can be a stand-alone flat panel display or a flat panel display integrated with another device such as a smartphone or a laptop. The HMD 350 is configured to be worn by the SOC user 140 (or the remote users). Both the HMD 350 and the NHMD 360 can be in wired or wireless communication with manipulation device(s) 352 (e.g., a three-dimensional mouse, data gloves, etc.) configured to be worn by and / or otherwise controlled / used by the SOC user 140.

[0047] In some embodiments, the SOC 110 is configured and arranged to use the contextualized simulation system 112 to generate the contextualized simulations of ME events and transmit the same over the network 320. Input data stream 316 received from the SOC user 140 (via display(s) 348) is used to control aspects of how the contextualized simulation system 112 to generates the contextualized simulations of ME events and provides the same in the output data stream 318 and to the display(s) 348. In accordance with embodiments, the output data stream 318 includes a full 360-degree video frame 500 shown at a time denoted as TimeN in FIG. 5. The full 360-degree video frame 500 is depicted in FIG. 5 as an equirectangular mapped 360-degree video frame where the yaw angle (-180 to +180 degrees) and the pitch angle (-90 to +90 degrees) are mapped to the x-axis and the y-axis, respectively. The full 360-degree video frame 500 can be a video recording where a view in every direction is recorded at the same time, shot using an omnidirectional camera or a collection of cameras. During playback on a normal flat display (e.g., the NHMD 360), the SOC user 140 has control of the viewing direction like a panorama. The full 360-degree video frame 500 can also be played on displays or projectors arranged in a sphere or some part of a sphere (not shown). The displayed region 502 (also known as the visible area or the user’s viewpoint) of the full 360-degree video frame 500 can be displayed on the displays 348. In embodiments where the display 348 is incorporated within the HMD 350, immersive (i.e., 3D) views of the full 360-degree video frame 500 can be displayed to the SOC user 140 on a display (e.g., display 206 shown in FIG. 2) of the HMD 350, which places tiny screens and lenses202318192 close to the eyes of the SOC user 140 to simulate large screens. As the SOC user 140 performs actions like walking, head rotating (i.e., changing the point of view), data describing behavior of the SOC user 140 is fed through the input data stream 316 to the SOC 110 from the HMD 350 and / or the manipulation devices 352. The SOC 110 processes the information in real-time and generates appropriate feedback that is passed back to the SOC user 140 by means of the output data stream 318.

[0048] FIG. 4 depicts an HMD 350A, which is a non-limiting example of how the HMD 350 (shown in FIG. 1) can be implemented. In accordance with embodiments, the HMD 350A includes control circuitry 402 and input-output circuitry 404, configured and arranged as shown. The input-output circuitry 404 includes display(s) 406, optical components 408, input-output devices 410, and communications circuitry 418, configured and arranged as shown. The input-output devices 410 include sensors 412 and audio components 414, configured and arranged as shown. The various components of the HMD 350A can be supported by a head-mountable support structure such as a pair of glasses; a helmet; a pair of goggles; and / or other head-mountable support structure configurations.

[0049] In embodiments of the invention, the control circuitry 402 can include storage and processing circuitry for controlling the operation of the HMD 350A. The control circuitry 402 can include storage such as hard disk drive storage, nonvolatile memory (e.g., electrically-programmable-read-only memory configured to form a solid-state drive), volatile memory (e.g., static or dynamic random-access-memory), etc. Processing circuitry in the control circuitry 402 can be based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio chips, graphic processing units, application specific integrated circuits, and other integrated circuits. Computer program instructions can be stored on storage in the control circuitry 402 and run on processing circuitry in the control circuitry 402 to implement operations for HMD 350A (e.g., data gathering operations, operations202318192 involving the adjustment of components using control signals, image rendering operations to produce image content to be displayed for a user, etc.). S equipment

[0050] Display(s) 406 of the input-output circuitry 404 can be used to display images (e.g., the full 360-degree video frame 500 (shown in FIG. 5)) to the SOC user 140 (shown in FIG. 3) of the HMD 350A. The display(s) 406 can be configured to have pixel array(s) to generate images that are presented to the SOC user 140 through an optical system. The optical system can, if desired, have a transparent portion through which the SOC user 140 (viewer) can observe real-world objects while computer-generated content is overlaid on top of the real-world objects by producing computer-generated images (e.g., the full 360-degree video frame 500) on the display(s) 406. In embodiments of the invention, the display(s) 406 are immersive views of the full 360-degree video frame 500, wherein the display(s) 406 place tiny screens and lenses close to the user’s eyes to simulate large screens that encompass most of the user’s field of view. As the SOC user 140 performs actions like walking, head rotating (i.e., changing the point of view), data describing behavior of the SOC user 140 (shown in FIG. 3) is fed to the SOC 110 (shown in FIG. 1) from the HMD 350A and / or the manipulation devices 352 (shown in FIGS. 3 and 4).

[0051] The optical components 408 can be used in forming the optical system that presents images to the SOC user 140. The optical components 408 can include static components such as waveguides, static optical couplers, and fixed lenses. The optical components 408 can also include adjustable optical components such as an adjustable polarizer, tunable lenses (e.g., liquid crystal tunable lenses; tunable lenses based on electro-optic materials; tunable liquid lenses; microelectromechanical system tunable lenses; or other tunable lenses), a dynamically adjustable coupler, and other optical devices formed from electro-optical materials (e.g., lithium niobate or other materials exhibiting the electro-optic effect). The optical components 408 can be used in receiving and modifying light (images) from the display 406 and in providing images (e.g., the full202318192 360-degree video frame 500) to the SOC user 140 for viewing. In some embodiments of the invention, one or more of the optical components 408 can be stacked so that light passes through multiple of the components 408 in series. In embodiments of the invention, the optical components 408 can be spread out laterally (e.g., multiple displays can be arranged on a waveguide or set of waveguides using a tiled set of laterally adjacent couplers). In some embodiments of the invention, both tiling and stacking configurations are present.

[0052] The input-output devices 410 of the input-output circuitry 404 are configured to gather data and user input and for supplying the SOC user 140 (shown in FIG. 3) with output. The input-output devices 410 can include sensors 412, audio components 414, and other components for gathering input from the SOC user 140 and / or or the environment surrounding the HMD 350A and for providing output to the SOC user 140. The input-output devices 410 can, for example, include keyboards; buttons; joysticks; touch sensors for trackpads and other touch sensitive input devices; cameras; light- emitting diodes; and / or other input-output components. For example, cameras or other devices in the input-output circuitry 404 can face the eyes of the SOC user 140 and track the gaze of the SOC user 140. The sensors 412 can include position and motion sensors, which can include, for example, compasses; gyroscopes; accelerometers and / or other devices for monitoring the location, orientation, and movement of the HMD 350A; and satellite navigation system circuitry such as Global Positioning System (GPS) circuitry for monitoring location of the SOC user 140. The sensors 412 can further include eye- tracking functionality. Using the sensors 412, for example, the control circuitry 402 can monitor the current direction in which a user's head is oriented relative to the surrounding environment. Movements of the user's head (e.g., motion to the left and / or right to track on-screen objects and / or to view additional real-world objects) can also be monitored using the sensors 412.

[0053] In some embodiments of the invention, the sensors 412 can include ambient light sensors that measure ambient light intensity and / or ambient light color; force202318192 sensors; temperature sensors; touch sensors; capacitive proximity sensors; light-based proximity sensors; other types of proximity sensors; strain gauges; gas sensors; pressure sensors; moisture sensors; magnetic sensors; and the like. The audio components 414 can include microphones for gathering voice commands and other audio input and speakers for providing audio output (e.g., ear buds, bone conduction speakers, or other speakers for providing sound to the left and right ears of a user). In some embodiments of the invention, the input-output devices 410 can include haptic output devices (e.g., vibrating components); light-emitting diodes and other light sources; and other output components. The input-output circuitry 404 can include wired and / or wireless communications circuitry 416 that allows the HMD 350A (e.g., using the control circuitry 402) to communicate with external equipment (e.g., remote controls, joysticks, input controllers, portable electronic devices, computers, displays, and the like) and that allows signals to be conveyed between components (circuitry) at different locations in the HMD 350A.

[0054] FIG. 6 depicts a simplified example of a VR environment 602 that can be displayed to the SOC user 140 on the full 360-degree video frame 500A using the SOC 110, including specifically the contextualized simulation system 112. The VR environment 602 displays a contextualized event simulation 604 of an automobile assembly plant 610, which is an example of the ME 120. The contextualized event simulation 604 include major components of the automobile assembly plant 610, including an automobile assembly line 612, automobiles 614 that are being manufactured, a robotic arm 616 that is performing manufacturing operations on the automobiles 614, a controller 618 that is controlling how the robotic arm 616 performs manufacturing operations on the automobiles 614, events (or ME events) 620, ME / OT annotations 622 that describe aspects of the ME event 620, and audible generated NL event descriptions 624 of various aspects of the ME event 620.

[0055] In accordance with embodiments, the contextualized event simulation 604 generated by the contextualized simulation system 112 provides a novel interface between the SOC 110 and SOC users 140 that improves the ability of the SOC users 140202318192 to interpret and evaluate determinations by the SOC 110 about the security status of OT network 124 (shown in FIG. 1) that is monitoring and controlling the activities of the automobile assembly plant 610. More specifically, the SOC 110 provides a novel simulator (not shown separately from the SOC 110) that can be implemented as part of the OT-SOC system 100. The novel simulator is configured and arranged to, in response to the OT-SOC system 100 detecting the ME event 620, generate the novel contextualized simulation 604 of the ME event 620 based at least in part on information of OT network 124, information of the automobile assembly plant 610 that is being monitored by the OT network 124, and information of the ME event 620 itself. The simulation is contextualized in that it simulates both the ME event 620 and realistic representations of components and activities of the automobile assembly plant 610 ME in which the ME event 620 occurs.

[0056] FIG. 7 depicts a simplified block diagram illustrating temporal features of the VR environment 602 (shown in FIG. 6) generated by the SOC 110 (shown in FIG. 1) using the contextualized simulation system 112, 112 (shown in FIGS. 1 and 2) according to embodiments. As depicted in FIG. 7, various ME and / or OT activities are represented by Activity-1 through Activity-N, where N is a whole number greater than four (4)) are depicted along a timeline 702, which shows T-1 through T-N, where N is a whole number greater than six (6). The “Activity-1 through Activity-N” depicted in FIG. 7 correspond to the various activities of the ME 120 and the OT network 124 shown in FIG. 1. In accordance with embodiments, the contextualized event simulation 604 can be configured and arranged to simulate activities and events at the ME 120 around a user- selected time frame, examples of which are shown in FIG. 7 as Time duration-1 and Time duration-2.

[0057] FIG. 8 depicts additional details of how the ME digital twin 210 and the OT digital twin 212 can be generated using a digitization module 820 and a digital twin processor module 810. The ME 120 through the ME sensor network 126A provides real time sensed information about activities and operations of the ME 120. Similarly, the OT202318192 sensor network 126B provides real time sensed information about the OT network 124 performing monitoring operations on the ME 120. The outputs from the ME sensor network 126A and the OT sensor network 126B are provided to the digitization module 820. The digitization module 820 provides the digitized data to the digital twin processor module 810 where known technologies are used to create and continuously update the ME digital twin 210 and the OT digital twin 212.

[0058] FIG. 9 is a simplified block diagram and a flow diagram illustrating hardware / software components 910 and a computer-implemented methodology 900 according to embodiments. The hardware / software components 910 include the ME 120, the OT network 124, the ME / OT digital twins 210, 212, and the ME / OT sensor networks 126A, 126B, configured and arranged as shown. The functions of the hardware / software components 910 have been previously-described herein. The computer-implemented methodology 900 includes Step-1, Step-2, Step3, and Step-4, configured and arranged as shown. In Step-1, the hardware / software components 910 track OT / ME digital twins using the simulator of the SOC 110. In Step-2, ME events (e.g., ME events 620 shown in FIG. 6) are mapped in simulator of the SOC 110. In Step-3, the hardware / software components 910 map ME events, ME annotations (e.g., ME annotations 622 shown in FIG. 6), and ME generated descriptions (e.g., audible generated NL event descriptions 624) using the simulator of the SOC 110. In Step-4, the hardware / software components 910, working with the SOC 110, render simulations (e.g., the contextualized event simulation 604 shown in FIG. 6) and enable navigation of the simulations by the SOC user 140.

[0059] It should be noted that, while there is some overlap, an SOC is different from a network operating center (NOC). Unlike an SOC, an NOC primarily handles issues related to network performance, reliability and availability. This includes implementing processes for network monitoring, device malfunctions and network configuration. An NOC is also in charge of ensuring the network meets service-level- agreement requirements, such as minimum downtime and network latency. SOCs and202318192 NOCs respond to very different types of incidents. Network issues are typically operational events, such as a switching system malfunction, traffic congestion and a loss of transmission facilities. By contrast, SOC events (e.g., cybersecurity events) come from sources both inside and outside the organization's control. SOC events use existing networks to gain unauthorized access to company resources, which includes social engineering attacks, as well as using coercion or tricks to get people to share confidential information. Rogue employees present a serious security risk, especially if they know security access codes, so they also under the SOC computing system's purview.

[0060] Embodiments disclosed herein implement virtual reality (VR) environments. 360-degree videos, also known as immersive videos or spherical videos, are video recordings where a view in every direction is recorded at the same time using, for example, an omnidirectional camera or a collection of cameras. An immersive 360- degree video system can be implemented as a computer system operable to generate and display immersive 360-degree video images that simulate a real world experience. A person can enter and leave the simulated real world experience at any time using technology. The basic components of a 360-degree video system include a display; a computing system; and various feedback components that provide inputs from the user to the computing system.

[0061] An example of machine learning techniques that can be used to implement aspects of the invention will be described with reference to FIGS. 10 and 11. Machine learning models configured and arranged according to embodiments of the invention will be described with reference to FIG. 10. Detailed descriptions of an example computing system 1200 and network architecture capable of implementing embodiments of the invention described herein will be provided with reference to FIG. 12.

[0062] FIG. 10 depicts a block diagram showing a classifier system 1000 capable of implementing various aspects of the invention described herein. More specifically, the functionality of the system 1000 is used in embodiments of the invention to generate various models and / or sub-models that can be used to implement computer functionality202318192 in embodiments of the invention. The system 1000 includes multiple data sources 1002 in communication through a network 1004 with a classifier 1010. In some aspects of the invention, the data sources 1002 can bypass the network 1004 and feed directly into the classifier 1010. The data sources 1002 provide data / information inputs that will be evaluated by the classifier 1010 in accordance with embodiments of the invention. The data sources 1002 also provide data / information inputs that can be used by the classifier 1010 to train and / or update model(s) 1016 created by the classifier 1010. The data sources 1002 can be implemented as a wide variety of data sources, including but not limited to, sensors configured to gather real time data, data repositories (including training data repositories), and outputs from other classifiers. The network 1004 can be any type of communications network, including but not limited to local networks, wide area networks, private networks, the Internet, and the like.

[0063] The classifier 1010 can be implemented as algorithms executed by a programmable computer such as the computing system 1200 (shown in FIG. 12). As shown in FIG. 10, the classifier 1010 includes a suite of machine learning (ML) algorithms 1012; natural language processing (NLP) algorithms 1014; and model(s) 1016 that are relationship (or prediction) algorithms generated (or learned) by the ML algorithms 1012. The algorithms 1012, 1014, 1016 of the classifier 1010 are depicted separately for ease of illustration and explanation. In embodiments of the invention, the functions performed by the various algorithms 1012, 1014, 1016 of the classifier 1010 can be distributed differently than shown. For example, where the classifier 1010 is configured to perform an overall task having sub-tasks, the suite of ML algorithms 1012 can be segmented such that a portion of the ML algorithms 1012 executes each sub-task and a portion of the ML algorithms 1012 executes the overall task. Additionally, in some embodiments of the invention, the NLP algorithms 1014 can be integrated within the ML algorithms 1012.

[0064] The NLP algorithms 1014 includes text recognition functionality that allows the classifier 1010, and more specifically the ML algorithms 1012, to receive natural202318192 language data (e.g., text written as English alphabet symbols) and apply elements of language processing, information retrieval, and machine learning to derive meaning from the natural language inputs and potentially take action based on the derived meaning. The NLP algorithms 1014 used in accordance with aspects of the invention can also include speech synthesis functionality that allows the classifier 1010 to translate the result(s) 1020 into natural language (text and audio) to communicate aspects of the result(s) 1020 as natural language communications.

[0065] The NLP and ML algorithms 1014, 1012 receive and evaluate input data (i.e., training data and data-under-analysis) from the data sources 1002. The ML algorithms 1012 include functionality that is necessary to interpret and utilize the input data’s format. For example, where the data sources 1002 include image data, the ML algorithms 1012 can include visual recognition software configured to interpret image data. The ML algorithms 1012 apply machine learning techniques to received training data (e.g., data received from one or more of the data sources 1002) in order to, over time, create / train / update one or more models 1016 that model the overall task and the sub-tasks that the classifier 1010 is designed to complete.

[0066] Referring now to FIGS. 10 and 11 collectively, FIG. 11 depicts an example of a learning phase 1100 performed by the ML algorithms 1012 to generate the above- described models 1016. In the learning phase 1100, the classifier 1010 extracts features from the training data and converts the features to vector representations that can be recognized and analyzed by the ML algorithms 1012. The feature vectors are analyzed by the ML algorithm 1012 to “classify” the training data against the target model (or the model’s task) and uncover relationships between and among the classified training data. Examples of suitable implementations of the ML algorithms 1012 include but are not limited to neural networks, support vector machines (SVMs), logistic regression, decision trees, hidden Markov Models (HMMs), etc. The learning or training performed by the ML algorithms 1012 can be supervised, unsupervised, or a hybrid that includes aspects of supervised and unsupervised learning. Supervised learning is when training data is202318192 already available and classified / labeled. Unsupervised learning is when training data is not classified / labeled so must be developed through iterations of the classifier 1010 and the ML algorithms 1012. Unsupervised learning can utilize additional learning / training methods including, for example, clustering, anomaly detection, neural networks, deep learning, and the like.

[0067] When the models 1016 are sufficiently trained by the ML algorithms 1012, the data sources 1002 that generate “real world” data are accessed, and the “real world” data is applied to the models 1016 to generate usable versions of the results 1020. In some embodiments of the invention, the results 1020 can be fed back to the classifier 1010 and used by the ML algorithms 1012 as additional training data for updating and / or refining the models 1016.

[0068] In aspects of the invention, the ML algorithms 1012 and the models 1016 can be configured to apply confidence levels (CLs) to various ones of their results / determinations (including the results 1020) in order to improve the overall accuracy of the particular result / determination. When the ML algorithms 1012 and / or the models 1016 make a determination or generate a result for which the value of CL is below a predetermined threshold (TH) (i.e., CL < TH), the result / determination can be classified as having sufficiently low “confidence” to justify a conclusion that the determination / result is not valid, and this conclusion can be used to determine when, how, and / or if the determinations / results are handled in downstream processing. If CL > TH, the determination / result can be considered valid, and this conclusion can be used to determine when, how, and / or if the determinations / results are handled in downstream processing. Many different predetermined TH levels can be provided. The determinations / results with CL>TH can be ranked from the highest CL>TH to the lowest CL>TH in order to prioritize when, how, and / or if the determinations / results are handled in downstream processing.

[0069] In aspects of the invention, the classifier 1010 can be configured to apply confidence levels (CLs) to the results 1020. When the classifier 1010 determines that a202318192 CL in the results 1020 is below a predetermined threshold (TH) (i.e., CL < TH), the results 1020 can be classified as sufficiently low to justify a classification of “no confidence” in the results 1020. If CL > TH, the results 1020 can be classified as sufficiently high to justify a determination that the results 1020 are valid. Many different predetermined TH levels can be provided such that the results 1020 with CL>TH can be ranked from the highest CL>TH to the lowest CL>TH.

[0070] FIG. 12 illustrates an example of a computer system 1200 that can be used to implement the computer-based components of the neural network system described herein. The computer system 1200 includes an exemplary computing device (“computer”) 1202 configured for performing various aspects of the content-based semantic monitoring operations described herein in accordance aspects of the invention. In addition to computer 1202, exemplary computer system 1200 includes network 1214, which connects computer 1202 to additional systems (not depicted) and can include one or more wide area networks (WANs) and / or local area networks (LANs) such as the Internet, intranet(s), and / or wireless communication network(s). Computer 1202 and additional system are in communication via network 1214, e.g., to communicate data between them.

[0071] Exemplary computer 1202 includes processor cores 1204, main memory (“memory”) 1210, and input / output component(s) 1212, which are in communication via bus 1203. Processor cores 1204 includes cache memory (“cache”) 1206 and controls 1208, which include branch prediction structures and associated search, hit, detect and update logic, which will be described in more detail below. Cache 1206 can include multiple cache levels (not depicted) that are on or off-chip from processor 1204. Memory 1210 can include various data stored therein, e.g., instructions, software, routines, etc., which, e.g., can be transferred to / from cache 1206 by controls 1208 for execution by processor 1204. Input / output component(s) 1212 can include one or more components that facilitate local and / or remote input / output operations to / from computer 1202, such as a display, keyboard, modem, network adapter, etc. (not depicted).202318192

[0072] A cloud computing system 50A is in wired or wireless electronic communication with the computer system 1200. The cloud computing system 50A can supplement, support or replace some or all of the functionality (in any combination) of the computing system 1200. Additionally, some or all of the functionality of the computer system 1200 can be implemented as a node of the cloud computing system 50A.

[0073] For the sake of brevity, conventional techniques related to making and using the disclosed embodiments may or may not be described in detail herein. In particular, various aspects of computing systems and specific computer programs to implement the various technical features described herein are well known. Accordingly, in the interest of brevity, many conventional implementation details are only mentioned briefly or are omitted entirely without providing the well-known system and / or process details.

[0074] For convenience, some of the technical operations described herein are conveyed using informal expressions. For example, a processor that has data stored in its cache memory can be described as the processor “knowing” the data. Similarly, a user sending a load-data command to a processor can be described as the user “telling” the processor to load data. It is understood that any such informal expressions in this detailed description should be read to cover, and a person skilled in the relevant art would understand such informal expressions to cover, the formal and technical description represented by the informal expression.

[0075] Many of the functional units of the systems described in this specification have been labeled as modules. Embodiments of the invention apply to a wide variety of module implementations. For example, a module can be implemented as a hardware circuit including custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module can also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like. Modules can also be implemented in software for execution by various types of processors. An identified202318192 module of executable code can, for instance, include one or more physical or logical blocks of computer instructions which can, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but can include disparate instructions stored in different locations which, when joined logically together, function as the module and achieve the stated purpose for the module.

[0076] The various components / modules / models of the systems illustrated herein are depicted separately for ease of illustration and explanation. In embodiments of the invention, the functions performed by the various components / modules / models can be distributed differently than shown without departing from the scope of the various embodiments of the invention describe herein unless it is specifically stated otherwise.

[0077] Aspects of the invention can be embodied as a system, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

[0078] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination202318192 of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0079] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0080] The terms “about,” “substantially,” “substantial,” “approximately,” and equivalents thereof are intended to include the degree of error associated with measurement of the particular quantity based upon the equipment available at the time of filing the application. For example, “about” can include a range of ± 8% or 5%, or 2% of a given value.

[0081] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, element components, and / or groups thereof.202318192

[0082] While the present invention has been described with reference to an exemplary embodiment or embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the present invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present invention without departing from the essential scope thereof. Therefore, it is intended that the present invention not be limited to the particular embodiment disclosed as the best mode contemplated for carrying out this present invention, but that the present invention will include all embodiments falling within the scope of the claims.

Claims

202318192 CLAIMS What is claimed is:

1. A computer-implemented method comprising: detecting, using a processor system, a monitored environment (ME) event based at least in part on information of an operational technology (OT) network that monitors the ME; and generating, using the processor system, a contextualized simulation of the ME event based at least in part on the information of the OT network, information of the ME, and information of the ME event.

2. The computer-implemented method of claim 1, wherein the contextualized simulation of the ME event comprises a simulation of: the ME event; ME activity associated with the ME event; and OT network activity associated with ME event.

3. The computer-implemented method of any of claims 1 or 2, wherein the ME event comprises an anomalous ME event.

4. The computer-implemented method of any of claims 1 or 2, wherein the contextualized simulation of the ME event comprises a three-dimensional (3D) representation of the contextualized simulation of the ME event.

5. The computer-implemented method of any of claims 1 or 2, wherein the contextualized simulation has a time duration.202318192 6. The computer-implemented method of claim 5, wherein the time duration includes a time duration of the ME event.

7. The computer-implemented method of claim 6, wherein the time duration covers prior to and subsequent to the ME event.

8. The computer-implemented method of any of claims 1 or 2 further comprising generating, using the processor system, a replay of the contextualized simulation of the ME event.

9. The computer-implemented method of any of claims 1 or 2, wherein: the computer-implemented method further comprises providing to a display the contextualized simulation of the ME event; the display is part of a head mounted display; and the display comprises a three-hundred and sixty degree (360-degree) video frame.

10. The computer-implemented method of any of claims 1 or 2, wherein: the information of the OT network comprises information of a digital twin of the OT network; the information of the ME comprises information of a digital twin of the ME; the contextualized simulation of the ME event further includes natural language text annotations associated with the ME event; and the contextualized simulation of the ME event further includes one or more audible generated natural language ME event descriptions associated with the ME event.202318192 11. A computer system comprising a processor system communicatively coupled to a memory, wherein the processor system performs processor system operations comprising: detecting a monitored environment (ME) event based at least in part on information of an operational technology (OT) network that monitors the ME; and generating a contextualized simulation of the ME event based at least in part on the information of the OT network, information of the ME, and information of the ME event.

12. The computer system of claim 11, wherein the contextualized simulation of the ME event comprises a simulation of: the ME event; ME activity associated with the ME event; and OT network activity associated with ME event.

13. The computer system of any of claims 11 or 12, wherein the ME event comprises an anomalous ME event.

14. The computer system of any of claims 11 or 12, wherein the contextualized simulation of the ME event comprises a three-dimensional (3D) representation of the contextualized simulation of the ME event.

15. The computer system of any of claims 11 or 12, wherein the contextualized simulation has a time duration.

16. The computer system of claim 15, wherein the time duration includes a time duration of the ME event.202318192 17. The computer system of claim 16, wherein the time duration covers prior to and subsequent to the ME event.

18. The computer system of any of claims 11 or 12, wherein the processor system operations further comprise generating a replay of the contextualized simulation of the ME event.

19. The computer system of any of claims 11 or 12, wherein: the processor system operations further comprise providing to a display the contextualized simulation of the ME event; the display is part of a head mounted display; and the display comprises a three-hundred and sixty degree (360-degree) video frame.

20. The computer system of any of claims 11 or 12, wherein: the information of the OT network comprises information of a digital twin of the OT network; the information of the ME comprises information of a digital twin of the ME; the contextualized simulation of the ME event further includes natural language text annotations associated with the ME event; and the contextualized simulation of the ME event further includes one or more audible generated natural language ME event descriptions associated with the ME event.

Images