Method to detect and mitigate adversarial attacks impacting ai and human perception in the industrial metaverse

Abstract

An attack vector to user perception in a metaverse scenario includes calculating a probability that a user perception in the metaverse is affected based on each perceivable element in the metaverse environment and a probability that each perceivable element is likely to affect the user's perception. The probability is calculated as a ratio of an unconditional probability of a conjunction of a hypothesis and a given body of data to an unconditional probability of the data alone. The probability is further calculated by looping over each of the perceivable elements and updating a probability of alteration of the user perception. The method may further define a threshold for the calculated probability and generate an alert when the calculated probability exceeds the defined threshold. A knowledge graph defines domain knowledge and is leveraged to produce contextual information that defines the likely hood that a combination of inputs will affect the viewer's perception.

Description

METHOD TO DETECT AND MITIGATE ADVERSARIAL ATTACKS IMPACTING Al AND HUMAN PERCEPTION IN THE INDUSTRIAL METAVERSETECHNICAL FIELD

[0001] This application relates to virtual environments such as the industrial metaverse. More particularly, this application relates to cyber security in virtual environments.BACKGROUND

[0002] The emergence of the industrial metaverse and the rapid development in the field of digital realities has left several cybersecurity exposures including: a lack of support in most of the metaverses where theft of nonfungible tokens cannot be fixed by moderation, identity spoofing where users can see their accounts and their role in a factory taken over by a hacker, client vulnerabilities where virtual reality (VR) and augmented reality (AR) headsets serve as targets for malicious inadvertent hacks, trust in user-to-user communications where a bad actor may cause tremendous damage, data accuracy where the perception of a worker based on inaccurate data can compromise his / her / their judgement on the situation, and privacy in the absence of metaverse regulations where the use of the invasive data may be collected for a truly personalized immersive experience is uncertain.SUMMARY

[0003] According to aspects of embodiments described in this disclosure, a method for defining an attack vector to user perception in a metaverse scenario includes calculating a probability that a user perception in the metaverse is affected based on eachperceivable element in the metaverse environment and a probability that each perceivable element is likely to affect the user's perception, so as define a calculated probability. An alert can be generated based on the calculated probability. The calculated probability is calculated as a ratio of an unconditional probability of a conjunction of a hypothesis and a given body of data to an unconditional probability of the data alone. The calculated probability is further calculated by looping over each of the perceivable elements and updating a probability of alteration of the user perception after each perceivable element. The method may further define a threshold for the calculated probability and generate an alert when the calculated probability exceeds the defined threshold. A knowledge graph may be constructed to define domain knowledge relating a virtual environment of the metaverse scenario and the knowledge graph may be leveraged to produce contextual information that defines the likely hood that a combination of inputs will affect the viewer's perception. Looping over each perceivable element includes, for each perceivable element: interpreting semantics of the perceivable element, mapping a received input to the knowledge graph, and updating the probability based on the new input in view of existing elements in a domain of the knowledge graph. The method may further detect an attempted intrusion by performing the steps of mapping a first trusted interaction to the knowledge base, mapping a second real-world interaction to the knowledge base, and comparing the mapping of the second real-world interaction to the mapping of the first trusted interaction. The difference between the mapping of the trusted interaction and the real-world interaction may inform the probability that a user's perception will be altered. The first trusted interaction may be an audiostream transcribed by a trusted third party and the second real-world interaction may be the audio stream altered by a malicious actor.

[0004] A system for defining an attack vector to user perception in a metaverse scenario comprising a computer processor, and a non-transitory computer memory in communication with the computer processor, the non-transitory computer memory storing instructions that when executed by the computer processor cause the computer processor to perform the methods described above.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For the purpose of illustrating the invention, there is shown in the drawings embodiments that are presently preferred, it being understood, however, that the invention is not limited to the specific instrumentalities disclosed. Included in the drawings are the following Figures:

[0006] FIG. 1 a - FIG. 1 d are illustrations of possible exploitations of industrial metaverse scenarios.

[0007] FIG. 2 is a block diagram for mapping perceivable elements to a knowledge graph representing context within a metaverse scenario.

[0008] FIG. 3 is a process flow diagram for detecting a potential threat to a communication in an industrial metaverse according to aspects of embodiments of this disclosure.

[0009] FIG. 4 is a block diagram of a computing system which may be used for implementing embodiments of this disclosure.DETAILED DESCRIPTION

[0010] Embodiments described within this application include a method that solve the problem of identifying a new generation of adversarial attacks against the industrial metaverse. Embodiments focus on attacks leveraging side channels in a user-to-user communication where the real-time availability of contextual information to a malicious actor may implement Homophonic Translation or any combination of transcription that can lead to the same effect as Typo glycemia, which misleads interpretation of communications within the industrial metaverse.

[0011] It may be considered that the industrial metaverse is a virtual environment in which employees connect and interact among each other and machines. Further, the industrial metaverse may be considered a place where convergence of the physical and digital world enables new ways of interaction.

[0012] Embodiments described herein address the problem at the intersection of user- to-user communication and the security challenges of data accuracy in the metaverse. A typical problem targeted by the disclosed methods includes a situation where a hacker gains access to a virtual environment of a factory in the industrial metaverse with the objective of corrupting user-to-user communication, thereby corrupting the perception of the target user receiving information through side communication channels. For example, in the case of VR-based monitoring employees utilize a digital twin to assess the status of a production line to judge whether any need for maintenance is required. The ability to trust communications coming from the contributing experts and trust the accuracy of thedata representing the production line status is crucial for performing the proper execution of the maintenance assessment task. With the availability of contextual information in real time within the metaverse, a hacker could possibly trigger a distraction for the maintenance assessors by leveraging observation of their current focus and then projecting contradicting information at a location in the virtual environment. For example, the malicious actor may introduce a false display in a side display within the virtual environment accessible to the assessor’s perception. The probability that the assessor may follow the false information to perform an improper action is analogues to a user acting improperly in response to a phishing attack.

[0013] FIGs. 1 a - 1 d provide a concrete illustration depicting a side channel attack in the industrial, metaverse. These figures provide a simple example of attacks emerging within the metaverse. Consider an environment comprising a datacenter 100 designed for control of a power plant. The datacenter 100 includes a digital twin that allows the maintenance operators to investigate issues. The virtual environment used to project this digital twin is represented by a virtual room including a display 102.

[0014] Referring to FIG. 1 a, a datacenter 100 contains a workstation 101 that houses one or more servers or other processing equipment for a factory, or other control devices such as programmable logic controllers (PLCs) that monitor and control operations of equipment in the plant. The data center 100 may further include a user display 102 that provides various types of information to a user. As shown in FIG. 1 b, the display 102 may be configured to display a company logo 102a during times of normal operations. Display 102a may be updated with different information if a situation arises, including but notlimited to news, weather related information for the location, or status information within the plant indicating an anomalous condition.

[0015] FIG. 1 c shows a first attack profile, where one of the servers or control devices 103 goes offline and can no longer monitor or control devices in the plant. This offline condition may result in downtime which presents risk of loss to the stakeholders. The real time data in the plant will indicate the failure and provide notification to the operators during normal operations and update the virtual world in the metaverse to indicate the problem. However, if the real-time data that updates the display 102a is intercepted by a malicious actor, the update displaying the condition on display 102a may be compromised and the virtual environment may be altered to continue displaying a normal condition 102a, failing to alert a user observing the virtual environment to perceive the failure condition.

[0016] FIG. 1 d shows another variant of an attack, embodying the converse of the attack in FIG. 1 c. That is, the servers or control devices 101 are operating normally, however a malicious actor alters the display 102 to display an abnormal condition, such as "Server 3 is Offline" when in fact, Server 3 is operating normally. This false message may cause an operator perceiving the false warning to perform actions, such as a shutdown, or a restart of a server, when such action is not necessary. This may cause downtime creating risk of loss to the stakeholders. This could further create confusion and create enough of a distraction to entice an attacker to launch a legitimate attack on the infrastructure while the attention of security operations center (SOC) analysts and system administrators is diverted troubleshooting a false alert. Side channel attacks are directed to the Metaverse as the level of awareness of the surrounding environment ishigher in the metaverse compared to the real world. This is a function of the availability of data in real time such a AR applications where textual data may be overlaid over the objects in a scene.

[0017] As an emerging field, metaverse cybersecurity remains at early stages in that the state of the art does not yet contain approaches addressing such issues directly. However, work has been accomplished attempting to address identity spoofing and security access contributing to mitigation of the risk created by side channel attacks on the metaverse.

[0018] The attacks discussed herein have commonalities with adversarial attacks against artificial intelligence (Al). Considering that the interpretation of the environment inside the metaverse is driven by a visual immersive experience, it exposes the human brain to a hacker manipulating the employee's environment to achieve visual adversarial attacks on the human brain. Obviously, the human brain does not respond to the pixel level stimulus. However, the human brain responds to stimulus based on updates to the information as described in Bayes Theorem. Bayes Theorem suggests that when an observer is confronted with multiple inputs, confusion may occur. This is particularly true when there is conflicting information in the inputs. For example, a first input may not elicit a response or feeling in the observer, but if another input is received which relates to the first input and that second input provides a suggestion to the viewer, the viewer's impression of the first input may be affected. If one or more of the inputs are untrue, then confusion may result.

[0019] Several techniques and methods that are part of the current practices in terms of Al robustness preparing neural networks to mitigate adversarial attacks around the scenarios shown in FIGs. 1 a - 1 d include:

[0020] Formal verification of systems with Al components: develop tools and algorithms that can verify the correctness of machine-learned software with respect to a specification.

[0021] Analysis of adversarial robustness: Finding adversarial examples as a verification problem and use a verification tool to prove that no adversarial examples exist for given input domains and allowed amounts of perturbation.

[0022] Automatic test-case generation: Providing interesting and realistic test-cases can be a challenging problem for systems with Al-based components. The size of the dataset alone is not a predictor of how well the system performs.

[0023] Safe exploration and learning for better perception by Al systems: design systems that intelligently and safely balance learning about the uncertainties of the environment with exploitation of safety knowledge in order to develop better perception for 4 autonomous systems in a provably safe manner.

[0024] Safe control of Al agents: Controlling an agent safely requires reasoning about the uncertain effects of the agent’s decisions on operational objectives and safety constraints. The agent generally relies on imperfect sensor information, which results in uncertainty about the current state of the world.

[0025] Frameworks such as Reluplex and Marabou provide opportunities to explore formal verification to ensure safe decision for neural networks based on the operationalcontext, they may be considered as relevant efforts. However, their inability to operate in real-time makes them not practicable to provide protection against attacks geared toward the industrial metaverse.

[0026] Identifying side channel attacks on the metaverse may be accomplished by defining a validation technique that defines attack vectors alternating the user’s perception to alert both the user and the moderation authorities about this alteration as features of embodiments of this disclosure.

[0027] According to embodiments of this disclosure, a method is provided to define attack vectors for side channel corruption of the user perception in the metaverse. The attack vectors in this method are governed by Bayes’ Theorem where the probability of a hypothesis H that is conditional on a given body of data E is characterized as the ratio of the unconditional probability of the conjunction of the hypothesis with the data to the unconditional probability of the data alone. Accordingly, Bayes Theorem may be expressed as Posterior = Prior x (Likelihood over Marginal probability) via the equation:P(A|B) = P(A) x P(B|A) / P(B)

[0028] where:

[0029] P(A|B): Posterior probability (updated probability after the evidence is considered);

[0030] P(A): Prior probability (the probability before the evidence is considered);

[0031] P(B|A): Likelihood (probability of the evidence, given the belief is true); and

[0032] P(B): Marginal probability (probability of the evidence, under any circumstance).

[0033] The definition of the attack vector may be defined as a loop over the perceivable elements of the virtual environment where each perceivable element updates the probability of alteration of the user’s perception. A threshold may be predefined such that the probability exceeding this threshold may trigger the generation of an alert. Thus, the alert can be generated based on the calculated probability. Whereas MITRE defined attacks focus on lateral movement, embodiments herein use a unique and novel approach shifting the paradigm from a sequential vector where there is minimal dependence among the attack steps to a paradigm where each event happening in the user’s perceivable horizon within the metaverse is considered alone and within a system context.

[0034] Embodiments of this disclosure leverage Bayesian probabilities in such a way that it may be measured how the contextual information coming from the side channels impact the perception of human or machine intelligence. Leveraging knowledge graphs to define the domain knowledge about the virtual environment provide insight into contextual information in the system.

[0035] A correlation algorithm comprises the following conditions and steps:

[0036] Prerequisites: A knowledge graph (KG) containing domain knowledge;

[0037] List the perceivable elements that could impact the target observer.• For each perceivable element■ Interpret the semantics of the perceivable element.Map the input to the knowledge graph.■ Update probability of a change in user perception according to the intersection between the new input and existing elements part of the knowledge domain• Alert the user within the metaverse by explaining the alert.

[0038] By way of example, the correlation between an intentional mis-transcription that leads to a misleading interpretation of the contextual information by the user or machine learning module may be detected. Consider an example of retrieving the contextual information associated with an instance of an original audio stream and associated video stream and the audio's transcription. The audio stream may be transcribed by a trusted third party mapping its concepts to the knowledge base. The trusted transcription may be mapped to specific concepts within the knowledge graph. A malicious transcription may be extracted from the video stream and injected against the same knowledge graph. The key to the detection is to identify any contradicting messages received in parallel for the same concept contained in the domain knowledge. For instance, it may be a mistranscription by omitting a negation statement. This statement, once mapped to the knowledge graph from the two streams, would appear as a contradiction in such a way that it would feed a marginal probability contained in the calculation.

[0039] FIG. 2 is a block diagram illustrating the mapping of a perceived element to a knowledge base according to aspects of embodiments of this disclosure. The virtual environment within the metaverse 210 includes a plurality of perceivable elements including a first perceivable element 211 and an nthperceivable element 212. The first perceived element 211 is denoted as X 213 and provided to the knowledge graph 220.Similarly, the nthperceived element 212 is denoted Y 214 and provided to the knowledge graph 220.

[0040] Knowledge graph 220 includes a plurality of nodes 222, where each node represents an element of the real-world system being replicated in the virtual environment 210. The nodes 222 are connected by edges 221 which define relationships between two elements via their respective nodes 222. The elements and their relationships define contextual information relating to the system. The first perceived element 21 1 is mapped to the knowledge graph 220 based on the element's 21 1 placement within the knowledge graph 223. Based on the object's location within knowledge graph 220 and the corresponding context defined by the node's 223 relationships to other nodes in the graph, the expected state of the object may be determined. Similarly, the nthperceived element 212 is mapped to its location 223 in the knowledge graph 223. The properties determined for input X 213 and input Y 214 may be compared to determine if the inputs from the elements are consistent with each other. If the mapped elements are not equal, it may be considered that an anomaly exists, which may be representative of a malicious act. Based on the comparison, further action, including generation of a warning, may be performed to inform the user of the detected discrepancy.

[0041] According to aspects of embodiments in this disclosure, a method identifies patterns in the transcribed text capable of causing confusion and mislead the interpretation of the what the user perceives. This may be accomplished by leveraging information from other streams along with heuristics.

[0042] Referring to FIG. 3, a flow diagram for a method of identifying an attack on a virtual environment in the metaverse is shown. In a first step, a perceivable element ofthe environment is generated 301. A trusted instance of a communication with the perceived element is mapped to a knowledge graph 302, the knowledge graph. Meanwhile, the real-world instance of the element communication is mapped to the knowledge graph 303. In view of the mapping to the knowledge graph and taking into account the communication within the context of the overall system, a difference between the trusted instance and the real-world instance of the element is identified 304. The contribution of the difference is added to an overall probability that a malicious attack is underway 305. As the differences are collected and the probability updated with each perceivable element in the system, the computed probability is compared to a threshold 306. If the probability that a malicious action has occurred exceeds the threshold an alert is generated 307 to notify a user of the malicious action.

[0043] Whereas MITRE attacks focus on lateral movement, our method is unique and novel as it shifts the paradigm from a sequential vector where there is minimal dependence between the attack steps to a paradigm where each event happening in the user’s perceivable horizon within the metaverse. Methods of the described embodiments are an improvement to existing attempts to provide solutions to attacks in the metaverse which are sparse and cannot provide the described advantages.

[0044] Embodiments described herein provide various technical advantages or benefits as compared to current approaches. For example, embodiments enable detection of inconsistencies by comparing a semantic interpretation of the distinct streams influencing the perception of a human or machine intelligence. Compared to the current methods that tries to make machine intelligence more robust, embodiments define a non- invasive method that leverages a property unique to metaverse where all the context ofthe communication is available within the virtual environment in real time. Whereas those existing method focuses on aspects of the perception such as the vision, embodiments described herein introduce a multi-modality to the identification process in such a way that the contradiction that might be introduced by a hacker can be detected by confronting the semantics of the different perception streams.

[0045] FIG. 4 illustrates an exemplary computing environment 400 within which embodiments of the invention may be implemented. Computers and computing environments, such as computer system 410 and computing environment 400, are known to those of skill in the art and thus are described briefly here.

[0046] As shown in FIG. 4, the computer system 410 may include a communication mechanism such as a system bus 421 or other communication mechanism for communicating information within the computer system 410. The computer system 410 further includes one or more processors 420 coupled with the system bus 421 for processing the information.

[0047] The processors 420 may include one or more central processing units (CPUs), graphical processing units (GPUs), or any other processor known in the art. More generally, a processor as used herein is a device for executing machine-readable instructions stored on a computer readable medium, for performing tasks and may comprise any one or combination of, hardware and firmware. A processor may also comprise memory storing machine-readable instructions executable for performing tasks. A processor acts upon information by manipulating, analyzing, modifying, converting or transmitting information for use by an executable procedure or an information device, and / or by routing the information to an output device. A processor may use or comprisethe capabilities of a computer, controller or microprocessor, for example, and be conditioned using executable instructions to perform special purpose functions not performed by a general-purpose computer. A processor may be coupled (electrically and / or as comprising executable components) with any other processor enabling interaction and / or communication there-between. A user interface processor or generator is a known element comprising electronic circuitry or software or a combination of both for generating display images or portions thereof. A user interface comprises one or more display images enabling user interaction with a processor or other device.

[0048] Continuing with reference to FIG. 4, the computer system 410 also includes a system memory 430 coupled to the system bus 421 for storing information and instructions to be executed by processors 420. The system memory 430 may include computer readable storage media in the form of volatile and / or nonvolatile memory, such as read only memory (ROM) 431 and / or random-access memory (RAM) 432. The RAM 432 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM). The ROM 431 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 430 may be used for storing temporary variables or other intermediate information during the execution of instructions by the processors 420. A basic input / output system 433 (BIOS) containing the basic routines that help to transfer information between elements within computer system 410, such as during start-up, may be stored in the ROM 431 . RAM 432 may contain data and / or program modules that are immediately accessible to and / or presently being operated on by the processors 420.System memory 430 may additionally include, for example, operating system 434, application programs 435, other program modules 436 and program data 437.

[0049] The computer system 410 also includes a disk controller 440 coupled to the system bus 421 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 441 and a removable media drive 442 (e.g., floppy disk drive, compact disc drive, tape drive, and / or solid-state drive). Storage devices may be added to the computer system 410 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire).

[0050] The computer system 410 may also include a display controller 465 coupled to the system bus 421 to control a display or monitor 466, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. The computer system includes an input interface 460 and one or more input devices, such as a keyboard 462 and a pointing device 461 , for interacting with a computer user and providing information to the processors 420. The pointing device 461 , for example, may be a mouse, a light pen, a trackball, or a pointing stick for communicating direction information and command selections to the processors 420 and for controlling cursor movement on the display 466. The display 466 may provide a touch screen interface which allows input to supplement or replace the communication of direction information and command selections by the pointing device 461 . In some embodiments, an augmented reality device 467 that is wearable by a user, may provide input / output functionality allowing a user to interact with both a physical and virtual world. The augmented reality device 467 is in communication with the display controller 465 and theuser input interface 460 allowing a user to interact with virtual items generated in the augmented reality device 467 by the display controller 465. The user may also provide gestures that are detected by the augmented reality device 467 and transmitted to the user input interface 460 as input signals.

[0051] The computer system 410 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 420 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 430. Such instructions may be read into the system memory 430 from another computer readable medium, such as a magnetic hard disk 441 or a removable media drive 442. The magnetic hard disk 441 may contain one or more datastores and data files used by embodiments of the present invention. Datastore contents and data files may be encrypted to improve security. The processors 420 may also be employed in a multiprocessing arrangement to execute the one or more sequences of instructions contained in system memory 430. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.

[0052] As stated above, the computer system 410 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term “computer readable medium” as used herein refers to any medium that participates in providing instructions to the processors 420 for execution. A computer readable medium may take many forms including, but not limited to, non- transitory, non-volatile media, volatile media, and transmission media. Non-limitingexamples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as magnetic hard disk 441 or removable media drive 442. Non-limiting examples of volatile media include dynamic memory, such as system memory 430. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the system bus 421 . Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0053] The computing environment 400 may further include the computer system 410 operating in a networked environment using logical connections to one or more remote computers, such as remote computing device 480. Remote computing device 480 may be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer system 410. When used in a networking environment, computer system 410 may include modem 472 for establishing communications over a network 471 , such as the Internet. Modem 472 may be connected to system bus 421 via user network interface 470, or via another appropriate mechanism.

[0054] Network 471 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 410 and other computers (e.g., remote computing device 480). The network 471 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-6, or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 471 .

[0055] An executable application, as used herein, comprises code or machine- readable instructions for conditioning the processor to implement predetermined functions, such as those of an operating system, a context data acquisition system or other information processing system, for example, in response to user command or input. An executable procedure is a segment of code or machine-readable instruction, subroutine, or other distinct section of code or portion of an executable application for performing one or more particular processes. These processes may include receiving input data and / or parameters, performing operations on received input data and / or performing functions in response to received input parameters, and providing resulting output data and / or parameters.

[0056] A graphical user interface (GUI), as used herein, comprises one or more display images, generated by a display processor and enabling user interaction with a processor or other device and associated data acquisition and processing functions. The GUI also includes an executable procedure or executable application. The executable procedure or executable application conditions the display processor to generate signals representing the GUI display images. These signals are supplied to a display device which displays the image for viewing by the user. The processor, under control of an executable procedure or executable application, manipulates the GUI display images inresponse to signals received from the input devices. In this way, the user may interact with the display image using the input devices, enabling user interaction with the processor or other device.

[0057] The functions and process steps herein may be performed automatically or wholly or partially in response to user command. An activity (including a step) performed automatically is performed in response to one or more executable instructions or device operation without user direct initiation of the activity.

[0058] The system and processes of the figures are not exclusive. Other systems, processes and menus may be derived in accordance with the principles of the invention to accomplish the same objectives. Although this invention has been described with reference to particular embodiments, it is to be understood that the embodiments and variations shown and described herein are for illustration purposes only. Modifications to the current design may be implemented by those skilled in the art, without departing from the scope of the invention. As described herein, the various systems, subsystems, agents, managers and processes can be implemented using hardware components, software components, and / or combinations thereof.

Claims

CLAIMSWhat is claimed is:1 . A method for defining an attack vector to user perception in a metaverse scenario comprising: calculating a probability that a user perception in the metaverse is corrupted based on each perceivable element in the metaverse environment and a probability that each perceivable element is altered, so as to define a calculated probability; and based on the calculated probability, generating an alert.

2. The method of Claim 1 , wherein the calculated probability is calculated as a ratio of an unconditional probability of a conjunction of a hypothesis and a given body of data to an unconditional probability of the data alone.

3. The method of Claim 2 wherein the calculated probability is calculated by looping over each of the perceivable elements and; updating a probability of alteration of the user perception after each perceivable element.

4. The method of Claim 3, further comprising: defining a threshold for the calculated probability; and generating the alert on a condition that the calculated probability exceeds the defined threshold.

5. The method of Claim 3, further comprising:constructing a knowledge graph to define domain knowledge relating a virtual environment of the metaverse scenario; and leveraging the knowledge graph to produce contextual information that could impact the user's perception.

6. The method of Claim 5, wherein looping over each perceivable element comprises: for each perceivable element: interpret semantics of the perceivable element; map a received input to the knowledge graph; and update the calculated probability based on the new input in view of existing elements in a domain of the knowledge graph.

7. The method of Claim 5, further comprising: detecting an attempted intrusion by performing the steps of: mapping a first trusted interaction to the knowledge base; mapping a second real-world interaction to the knowledge base; and comparing the mapping of the second real-world interaction to the mapping of the first trusted interaction.

8. The method of Claim 7, further comprising: detecting the attempted intrusion based on a difference between the mapping of the trusted interaction and the real-world interaction.

9. The method of Claim 8, wherein the first trusted interaction comprises an audio stream transcribed by a trusted third party and the second real-world interaction is the audio stream altered by a malicious actor.

10. A system for defining an attack vector to user perception in a metaverse scenario comprising: a computer processor; and a non-transitory computer memory in communication with the computer processor, the non-transitory computer memory storing instructions that when executed by the computer processor cause the computer processor to perform the steps of: calculating a probability that a user perception in the metaverse is corrupted based on each perceivable element in the metaverse environment and a probability that each perceivable element is altered, so as to define a calculated probability; and generating an alert based on the calculated probability.1 1 . The system of Claim 10, wherein the calculated probability is calculated as a ratio of an unconditional probability of a conjunction of a hypothesis and a given body of data to an unconditional probability of the data alone.

12. The system of Claim 11 wherein the calculated probability is calculated by looping over each of the perceivable elements and; updating a probability of alteration of the user perception after each perceivable element.

13. The system of Claim 12, further comprising:defining a threshold for the calculated probability; and generating the alert on a condition that the calculated probability exceeds the defined threshold.

14. The system of Claim 12, further comprising: constructing a knowledge graph to define domain knowledge relating a virtual environment of the metaverse scenario; and leveraging the knowledge graph to produce contextual information that could impact the user's perception.

15. The system of Claim 14, wherein looping over each perceivable element comprises: for each perceivable element: interpret semantics of the perceivable element; map a received input to the knowledge graph; and update the calculated probability based on the new input in view of existing elements in a domain of the knowledge graph.

16. The system of Claim 14, further comprising: detecting an attempted intrusion by performing the steps of: mapping a first trusted interaction to the knowledge base; mapping a second real-world interaction to the knowledge base; andcomparing the mapping of the second real-world interaction to the mapping of the first trusted interaction.

17. The system of Claim 16, further comprising: detecting the attempted intrusion based on a difference between the mapping of the trusted interaction and the real-world interaction.

18. The system of Claim 17, wherein the first trusted interaction comprises an audio stream transcribed by a trusted third party and the second real-world interaction is the audio stream altered by a malicious actor.

Images