Quantifying the Human-Likeness of Artificial Intelligence Agents Using Statistical Methods and Techniques

Abstract

The apparatus includes a processor configured to determine a first distribution associated with the artificial agent based on a behavior associated with the artificial agent and a second distribution based on a behavior of a user. The processor is further configured to generate a human-likeness similarity measure by comparing the first distribution to the second distribution and to modify a behavior of the artificial agent in response to the similarity measure failing a similarity threshold.

Description

[Background technology]

[0001] One mechanism for implementing artificial intelligence (AI) is through artificial agents, which are non-human entities that have knowledge of their environment and autonomously perform actions to achieve goals. Artificial agents are becoming an integral part of many different technologies and industries. For example, vehicles implement artificial agents to perform autonomous driving operations, video games implement artificial agents as non-playable characters (NPCs), and factories implement robots as artificial agents to autonomously perform various actions. In many cases, artificial agents are trained using various techniques to emulate human behavior in their environment. Thus, assessing the human-likeness of an agent's behavior is often a critical part of the design and development of an artificial agent.

[0002] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by reference to the following drawings, in which: The use of the same reference numbers in different drawings indicates similar or identical items. [Brief description of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram of an operating environment for evaluating and quantifying similarity between artificial agent behavior and human counterpart behavior, according to some embodiments. [Diagram 2] FIG. 1 is a flow diagram illustrating a general exemplary method for modifying an artificial agent's behavior based on a similarity measure that quantifies the artificial agent's human-likeness, according to some embodiments. [Diagram 3] FIG. 2 is a flow diagram illustrating an exemplary method for evaluating and quantifying similarity between artificial agent behavior and human counterpart behavior, according to some embodiments. [Figure 4]FIG. 2 illustrates an example of a top-down projection representation of an artificial agent and a human counterpart behavior, according to some embodiments. [Diagram 5] FIG. 4 is a flow diagram illustrating an example method for performing a two-sample hypothesis test to evaluate and quantify the similarity between an artificial agent behavior and a human counterpart behavior in block 310 of FIG. 3 , according to some embodiments. [Figure 6] FIG. 2 illustrates an example of an artificial agent behavior analysis result, according to some embodiments. [Figure 7] 4 is a table illustrating how hyperparameters can be tuned to control the sensitivity of the two-sample hypothesis test performed in block 310 of FIG. 3 , in accordance with some embodiments. [Figure 8] FIG. 1 is a block diagram of a processing system, according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0004] Developing artificial agents capable of learning complex human-like behaviors is a goal of artificial intelligence research. Advances in machine learning techniques such as deep reinforcement learning have enabled the development of highly skilled artificial agents capable of complex intelligent behaviors. For example, in video games, artificial agents have been deployed as non-playable characters (NPCs) designed to mimic human behavior and enhance the experience of the human player. Although progress has been made in designing and developing highly skilled artificial agents, quantifying the human-likeness of an artificial agent remains difficult. For example, the plausibility of an artificial agent's behavior is often measured solely by the agent's proficiency in a given task, but proficiency alone is not sufficient to measure human-like behavior.

[0005] Various approaches have been attempted to quantify / measure the likelihood of artificial agent behavior, but each of these approaches has various drawbacks. For example, one approach to evaluate artificial agent behavior involves human evaluation of the agent (e.g., Turing test). However, this approach is impractical for most environments, given the speed, scalability, and cost limitations of human evaluation. Another approach performs environment (domain) specific tests that rely on rules or heuristics to detect human-like behavior of the agent. This approach requires a large amount of manual work by domain experts, which is time-consuming and expensive. Also, environment-specific tests tend to coarsely evaluate the difference in results rather than directly evaluating the agent behavior, and fall short of capturing the fine-grained details of human-likeness (e.g., detecting periodic behavior, counting collisions, or measuring steps taken to achieve a goal). Other approaches perform machine learning to evaluate the artificial agent's behavior. However, machine learning models are usually adapted to a specific environment and are not easily generalizable.

[0006] This disclosure describes embodiments of systems and methods for overcoming various problems associated with conventional techniques for quantifying the human-likeness of artificial agent behaviors, including computational cost, efficiency, and accuracy. As described in more detail below, the similarity between the artificial agent's behavior and that of a human counterpart is evaluated and quantified using a non-parametric two-sample hypothesis testing technique. The non-parametric two-sample hypothesis testing technique uses the distribution of the artificial agent's state-action pairs (e.g., trajectories) to determine whether the agent's behavior is human-like and further quantifies how human-like the behavior is. For example, the similarity of each of the artificial agents' behaviors is compared using the distribution of human behavior sequences and the distribution of artificial intelligence behavior sequences associated with the same or similar tasks / goals. The two-sample hypothesis testing technique outputs a similarity measure that quantifies the human-likeness of the artificial agent's behavior.

[0007] An advantage of the artificial agent behavior analysis techniques described herein is that they are domain / environment independent and are applicable to multiple different capabilities associated with intelligent behavior, unlike traditional analysis techniques such as machine learning-based analysis techniques. Also, the analysis techniques described herein avoid the costs associated with human judgment and do not require training / retraining or inference optimization required by techniques implementing deep neural networks (DNNs). Another advantage is that the analysis techniques described herein are easily parallelizable and accelerated because they are based on statistical distributions. Furthermore, the results of the analysis techniques (e.g., similarity measures) are used, at least in some embodiments, to improve or modify the artificial agent, improve the system or environment in which the artificial agent is implemented, combinations thereof, and the like. For example, the analysis results are used to rank or score the artificial agent, to train or retrain the artificial agent to improve the human-likeness of the agent's behavior, or to adjust the artificial agent's behavior in real time as it executes in the operating environment (e.g., video games, vehicles, robots, etc.). Other applications include optimizing curricula in curriculum-based learning paradigms, evaluating the design of reward signals, optimizing neural architectures, and early termination of agent training when sufficient similarity is achieved. It should be appreciated that the two-sample hypothesis techniques described herein are also applicable to evaluating the behaviors of multiple artificial agents to determine the behavioral similarity between the multiple artificial agents. The results of this analysis can be used, for example, to increase behavioral diversity in a multi-agent setting.

[0008] FIG. 1 illustrates an exemplary operating environment 100 for evaluating the human-likeness of artificial agent behaviors and modifying an artificial agent based on the evaluation. As shown in FIG. 1, the operating environment 100 includes one or more information processing systems 102 (depicted as systems 102-1 through 102-3) communicatively coupled by one or more networks 104. It should be understood that the number of information processing systems 102 illustrated in FIG. 1 is for illustrative purposes only. For example, a single processing system 102 or a fewer / greater number of information processing systems 102 may include the components and perform the operations described herein. Examples of information processing systems 102 include servers, desktop computers, laptop / notebook computers, mobile devices, or various other types of computing systems or devices. The network 104, in at least some embodiments, is implemented utilizing wired and / or wireless networking mechanisms. For example, the network 104, in at least some embodiments, may include wireless communication networks, non-cellular networks such as Wi-Fi networks, public networks such as the Internet, private networks, and the like. A wireless communication network may support any wireless communication standard and may include one or more networks based on such a standard.

[0009] 1, the first information processing system 102-1 includes an artificial agent behavior analysis (AABA) system 106 that evaluates one or more behaviors of an artificial agent 108 (shown as 108-1 and 108-2) and, in at least some embodiments, modifies the behavior of the artificial agent 108 based on the evaluation. An artificial agent behavior is an autonomous action taken by the artificial agent 108 during operation / performance in response to the artificial agent's 108's environment. Examples of artificial agent behaviors include autonomous driving behaviors (e.g., lane negotiation, collision avoidance, turning, etc.), autonomous navigation of a virtual gaming environment, autonomous manufacturing of a product, autonomous interaction with humans, prediction, perception, reasoning, etc.

[0010] The artificial agent 108 is implemented in at least some embodiments in the first information processing system 102-1, one or more other information processing systems 102, or a combination of both. For example, one or more agents 108 are implemented in the second information processing system 102-2 and the third information processing system 102-3. The second information processing system 102-2 includes an artificial agent training system 110 that uses training data 112 to train the artificial agent 108-1 to autonomously perform one or more actions / behaviors depending on the artificial agent's 108-1 environment. In one example, the artificial agent training system 110 is a machine learning (ML) based training system. However, different techniques such as navigation meshes and behavior trees can also be used to program or train the artificial agent 108. The third information processing system 102-3 includes one or more artificial agents 108-2 that execute within an operating environment 114, such as a virtual environment (e.g., a video game) 114-1 or a user-assisted environment 114-2 (e.g., a virtual chat). In another example, the operating environment 114 is an entity such as a vehicle 114-3 (manned or unmanned) or a robot 114-4 that implements the artificial agent 108-2. It should be understood that the first and second information processing systems 102-2, 102-3 are provided merely as examples of the types of systems in which the artificial agent 108 may be implemented. The AABA system 106 of the first information processing system 102-1 is capable of analyzing / evaluating different types of behaviors performed by different types of artificial agents across a variety of domains and systems.

[0011] 1 illustrates the AABA system 106 implemented on a first information processing system 102-1, in other embodiments, the AABA system 106 is implemented as part of or in conjunction with an artificial agent training system 110, an artificial agent operating environment 114, another system / environment capable of implementing an artificial agent 108, a combination thereof, etc. In at least some embodiments, the AABA system 106 includes an action distribution module 116, an action information conversion module 118, an action similarity determination module 120, and a dimensionality reduction module 122. One or more of these modules may be distributed across multiple information processing systems in at least some embodiments, and two or more of these modules may be combined. In at least some embodiments, the AABA system 106 includes one or more interfaces 124, such as an artificial agent training system interface 124-1 and an artificial agent operating environment interface 124-2. The AABA system 106 uses an interface 124 to interface and communicate with one or more systems that include artificial behavior information 126, such as artificial agent behavior information 126-1 and human behavior information 126-2, or that apply results 128 of the artificial agent behavior analysis performed by the AABA system 106.

[0012] As described in more detail below, the AABA system 106 uses one or more of the behavior distribution module 116, the behavior information transformation module 118, the behavior similarity determination module 120, or the dimensionality reduction module 122 to perform an artificial agent behavior analysis that quantitatively evaluates the similarity of behaviors exhibited by the artificial agent 108 to those exhibited by a human counterpart (e.g., a human user) in a domain-independent manner. Behavior analysis is a statistical method that uses the distribution of the agent's state-action pairs (e.g., trajectories) to determine whether the agent's behavior is human-like and further to quantify how human-like the behavior is. For example, the AABA system 106 analyzes the human-like behavior (HLB) of the artificial agent 108 based on a comparison of the distribution of behaviors between the distribution of human behavior sequences and the distribution of artificial intelligence behavior sequences associated with the same or similar tasks / goals. Based on the behavioral analysis, the AABA system 106 generates behavioral analysis results 128, including, for example, a similarity measure 128-1 (also referred to herein as similarity measure 128-1) for each evaluated artificial agent 108. The similarity (human-likeness) measure 128-1 quantifies / measures how similar the behavior of the artificial agent 108 is to that of its human counterpart. For example, if the behavior being evaluated is navigating a video game environment to achieve a goal (e.g., reach an endpoint), the similarity measure 128-1 quantifies the human-likeness of the artificial agent's movement in achieving this goal. In another example, the similarity measure 128-1 quantifies the performance (in terms of human-like behavior) of a robot with respect to a behavior such as locomotion or performing a task. In a further example, the similarity measure 128-1 quantifies how plausible a human-assist software agent or a non-playable character is when interacting with a human.

[0013] In at least some embodiments, the behavioral analysis results 128 are applied by one or more systems to, for example, improve or enhance one or more artificial agents 108, improve or enhance the system or environment in which the artificial agent 108 is implemented, combine them, etc. For example, the AABA system 106 or another system, such as the artificial agent training system 110 or the artificial agent operating environment 114, may use the similarity measure 128-1 to rank or score the evaluated artificial agents 108 and train or retrain the artificial agent 108 to improve the human-likeness of the agent's behavior or adjust the artificial agent's behavior in real-time as it executes in the operating environment 114 (e.g., video games, vehicles, robots, etc.). Other applications include optimizing the curriculum in a curriculum-based learning paradigm, evaluating the design of reward signals, optimizing neural architectures, and early termination of agent training if sufficient similarity is achieved.

[0014] For example, FIG. 2 outlines in flow chart form one exemplary method 200 for applying the behavior analysis results 128 as part of an artificial agent training process. In this example, p is a similarity measure 128-1 and γ is a pre-defined confidence measure 201. In block 202, the training system 110 trains the artificial agent 108 (e.g., an AI / ML agent) using one or more training techniques, such as reinforcement learning (RL). During training, agent behavior information 126-1 associated with the artificial agent 108 is generated for one or more tasks (e.g., navigating an environment in a video game). In block 204, the AABA system 106 analyzes the agent behavior information 126-1 with respect to human behavior information 126-2 associated with the same task. The human behavior information 126-2, in at least some embodiments, includes the behavior of an actual human counterpart or the behavior of the artificial agent 108 controlled by a human counterpart. Based on this analysis, the AABA system 106 generates a similarity measure (p) 128-1 that quantifies the human-likeness of the artificial agent's behavior 126-1. In block 206, the AABA system 106 (or the training system 110) determines whether the similarity measure p 128-1 satisfies a similarity threshold (γ) 201 (e.g., p≧γ). If so, in block 210, the AABA system 106 instructs the training system 110 to stop training the AABA system 106, which reduces training computation time and prevents overfitting of the artificial agent 108. The process then ends in block 210. The trained artificial agent 108 is then deployed in one or more operating environments 114. If not, in block 212, the AABA system 106 instructs the training system 110 to adjust the behavior of the artificial agent 108 by continuing to train the artificial agent 108, and the process returns to block 202. In other embodiments, the training system 110 receives the similarity measure p 128-1 and performs the actions in block 206 rather than the AABA system 106.

[0015] In at least some embodiments, one or more of the techniques implemented by the AABA system 106 to evaluate artificial agent behavior are based on, for example, deep reinforcement learning and distribution analysis and comparison. Deep reinforcement learning is one technique for training / programming an artificial agent 108 by a system such as the training system 110 of FIG. 1. In one example, a standard formulation of reinforcement learning (RL) is a Markov decision process (MDP) in which the artificial agent 108 interacts with an environment E according to a policy that aims to maximize its cumulative expected reward. π θ Let (a│s) denote the decision strategy of a given artificial agent 108, where π θ models the conditional distribution over action a given state s, parameterized by θ. In deep reinforcement learning (DRL), this conditional distribution is modeled using deep neural networks (DNNs). The space of all actions and states can be denoted as A and S, respectively, such that a∈A and s∈S. At each time step t, the artificial agent 108 calculates the conditional distribution over the current state s t Observe π θ Follow the action a t Then, the environment E is sampled with the transition value and p ε (s t+1 │s,a) is sampled from the transition dynamics of E, t+1 and a scalar reward r reflecting t Note that given a state s and an action a, the transition dynamics satisfies the Markov property such that the probability distribution for the next state is conditionally independent of all previous states and actions. The objective under this paradigm is to find optimal parameters θ that maximize the expected return as given by Eq. * The purpose of this study is to learn:

[0016]

number

[0017] Here, θ (τ) is the policy π θ represents the distribution over all possible trajectories τ when 0 ) represents the initial state distribution. 0 to end state s N A sequence of state-action pairs spanning T to N time steps is defined as an episode, and a successive subsequence of episodes spanning T ≤ N time steps is defined as a trajectory, denoted as τ. In other words, each trajectory τ is a set of trajectories τ = (s 0 ,a 0 ,s 1 ,...,a T-1 ,s τ ), over a span of T time steps.

[0018]

number

[0019] One example of a distribution analysis and comparison technique implemented by the AABA system 106 is Maximum Mean Discrepancy (MMD). MMD is a class of kernel-based divergence metrics used to calculate the distance between the projections of two high-dimensional data distributions. The MMD distance is defined as follows:

[0020]

number

[0021] The AABA system 106 empirically estimates this distance by independently drawing n samples from each distribution. If the kernel k is mapped to a reproducing kernel Hilbert space, then the MMD distance given by Equation 2 is 0 if and only if the distributions X and Y are identical, where x and y each represent a sample distribution of n elements drawn independently from X and Y, where x' indicates that the samples have been shuffled, and

[0022]

number

[0023]

number

[0024]

number

[0025]

number

[0026]

number

[0027]

number

[0028]

number

[0029] As described below, the AABA system 106 implements one or more nonparametric statistical hypothesis testing techniques for evaluating artificial agent behavior. Statistical inference is the process of extracting conclusions about population parameters or population distributions from a sampling distribution. Unlike parametric statistical inference, which estimates sample statistics to model population distributions using assumptions about the data, nonparametric statistical inference directly analyzes the sampling distribution. Within this class of tests and tools, sampling methods such as bootstrap resampling and permutation testing offer improved performance and generality over older methods without requiring distributional assumptions. With bootstrap resampling, the AABA system 106 repeatedly draws samples of size m with replacement from a sampling distribution of size n and recalculates the sample statistics. Although some bootstrap resampling methods set m=n, bootstrapping n out of m (i.e., m≦n) may yield more consistent results. Like bootstrap resampling, permutation testing does not require a priori knowledge of the data distribution. However, unlike bootstrap resampling, this class of tests resamples from the sampling distribution without replacement. In at least some embodiments, the AABA system 106 implements one or more of bootstrap resampling or permutation testing techniques for hypothesis testing, although other techniques are equally applicable.

[0030] 3 illustrates in flow chart form an exemplary method 300 for evaluating artificial agent behavior. Although the subprocesses of method 300 are shown and described in an exemplary order, method 300 is not limited to this particular order, and in some embodiments, certain processes may be performed in a different order, simultaneously rather than sequentially, or may be omitted entirely. Additionally, while method 300 is described in the exemplary context of a video game environment and movement / navigation behavior, it should be understood that the techniques described herein are not limited to such environments and behaviors.

[0031] As described below, method 300 analyzes the similarity between an artificial agent's behavior and a human counterpart's behavior (or another artificial agent) using a non-parametric two-sample hypothesis testing technique configured according to the behavioral similarity hypothesis. In at least some embodiments, the behavioral similarity hypothesis indicates that any two agent behaviors are sufficiently similar if the distributions over the agents' respective behaviors, which can be represented as episodes / trajectories, are sufficiently similar. Method 300 also analyzes a pre-trained decision strategy π that will drive the artificial agent 108 to complete a given task when the artificial agent 108 is deployed in an environment ε, such as a video game environment. θ Similarly, comparing a human counterpart in method 300 to an artificial agent 108 independently completing the same task in the same environment ε. For example, if the artificial agent 108 is an NPC in a video game environment, the human counterpart may be independently given control over the same (or a different) artificial agent 108 to complete the same task in the same environment ε. Thus, in at least some embodiments, both the artificial agent 108 and the human counterpart may be independently given control over the same (or a different) artificial agent 108 to complete the same task in the same environment ε.

[0032]

number

[0033] In block 302, the AABA system 106 receives as input agent behavior information 126-1 for one or more artificial agents 108 and human behavior information 126-2 for one or more human counterparts. The behavior information 126, in at least some embodiments, is stored locally on an information processing system including the AABA system 106. In other embodiments, the behavior information 126 is stored remotely from the information processing system including the AABA system 106. The behavior information 126, in at least some embodiments, is obtained from an artificial agent training system 110, an operating environment 114, a human tester, a human player, a combination thereof, or the like. For illustrative purposes, the behavior information 126 in this example represents navigation / movement behavior of the artificial agents 108 and human counterparts within a three-dimensional (3D) space of a video game environment. Thus, in this example, each movement identified by the behavior information 126 is defined by an x-coordinate, a y-coordinate, and a z-coordinate. It should be understood that other types of behaviors can be represented / defined by different characteristics or attributes.

[0034] Also, each of the agent behavior information 126-1 and the human behavior information 126-2 is represented by one or more episodes 301 (denoted as episodes 301-1 and episodes 301-2). For example, if the behavior being evaluated is navigation in a virtual environment such as a video game, each of the agent behavior information 126-1 and the human behavior information 126-2 includes one or more episodes 301 representing all navigation movements of the artificial agent 108 and the human counterpart for a given task. It should be understood that the behavior of the artificial agent 108 and the human counterpart is not limited to being represented by episodes and trajectories, as other techniques are equally applicable. Each episode 301-1 of the artificial agent behavior information 126-1 is composed of a trajectory τ 303-1, and each episode 301-2 of the human behavior information 126-1 is composed of a trajectory τ 303-2. As mentioned above, the trajectory τ 303 is a continuous subsequence of episodes 301 over a range of T≦N time steps. In other words, each trajectory τ 303 is a function of τ=(s 0 ,a 0 ,s 1 ,...,a T-1 ,s τ ), each trajectory τ 303 is defined by a sequence of state-action pairs over a span of T time steps, where τ 303 is a subset of the .... Thus, a trajectory τ 303 is a slice / window of an episode that contains a subsequence of behaviors / actions performed by the artificial agent 108 or a human counterpart. For example, if an episode 301 represents 1000 moves made by the artificial agent 108, then each trajectory τ 303 represents a subset of these 1000 moves.

[0035] In block 304, the distribution module 116 of the AABA system 106 determines / generates a distribution P 303 of the artificial agent behavior information 126-1 and the human behavior information 126-2. For example, the behavior distribution module 116 may determine / generate a distribution P 303 of the artificial agent behavior information 126-1 and the human behavior information 126-2. θ 303-1 and the distribution P of human behavior information 126-2 *In this example, the distribution P 305 is generated based on the space of the trajectories 303, e.g., P 303-2 determined in block 304. θ (τ)301-1 and P * (τ)301-2 covers the entirety of the text.

[0036] In blocks 306 and 308, the distribution module 116 and the behavior information transformation module 118 (referred to herein for brevity as the transformation module 118) of the AABA system 106 operate to generate a sample distribution X 309-1 of the artificial agent behavior information 126-1 and a sample distribution Y 309-2 of the human behavior information 126-2. θ 305-1 and Distribution P * 305-2. For example, let E denote the set of M episodes 301 collected independently from each of the artificial agent behavior information 126-1 and the human behavior information 126-2 in block 306. The initial state s 0 and the end state s N Let N denote the total number of time steps between |X|, |Y|, and |Y|. The distribution module 116 independently subsamples K trajectories 303 of length T from each episode 301 with reconstruction. In some cases, the length of the episode 301 can be significantly skewed. Therefore, to correct for any bias from larger episodes with more time steps, K is set to the length of the largest episode 301 in the given set. Thus, the size of each sample distribution 309 (e.g., |X|, |Y|) generated in block 308 is M·K, at least in some embodiments. This ensures that a trajectory randomly drawn from the aggregated sample distribution has uniform probability from any of the collected episodes 301. Also, there is no point-to-point correspondence between sample distribution X 309-1 and sample distribution X 309-2.

[0037] In block 308, the transformation module 118 transforms the behavior information 126-1 of the artificial agent 108 and the behavior information 126-2 of the human counterpart from a first representation to a different representation, such as vectors 307-1 and images 307-2. In at least some embodiments, the different representation is a high-dimensional representation of the behavior information 126, such as vectors 307-1, images 307-2, etc. In one example, the transformation module 118 receives as input each sampled trajectory τ 303 generated by the distribution module 116 in block 306. Then, for each sampled trajectory τ 303, the transformation module 118 represents the sampled trajectory τ 303 as a vector 307-1 that includes the absolute three-dimensional (3D) location of the artificial agent 108 at each time step. The transformation module 118 uses the vectors 307-1 to transform and visualize the sampled trajectories 303 into two-dimensional (2D) renderings (images) 307-2. The absolute 3D location of the artificial agent 108 may represent a minimal representation of the artificial agent's response to a given state. In this example, t Let c(τ) be the 3D Cartesian coordinates of the artificial agent 108 at time t, and c(τ) be defined as c(τ)={c 0 ,...,c T} and c t =(x t ,y t ,z t Let c(τ) be a sequence of coordinates for a given sampled trajectory τ 303 such that c(τ) is a vector of the trajectory of the artificial agent 108 along the z-axis. The transformation module 118 uses c(τ) to project the sampled trajectory τ 303 into a 2D rendering by viewing the navigation of the artificial agent 108 from the z-axis and applying min-max scaling across the (x,y) coordinates to project each vector in Cartesian coordinates into the range [0,1]. The transformed coordinates

[0038]

number

[0039]

number

[0040]

number

[0041] As the objective becomes more and more complex, the episodes 301 change significantly and non-uniformly, which may introduce artifacts. For example, fine details of long and complex trajectories lose their salience in the projection, and the continuity of short smooth trajectories breaks down, introducing sparse projections. Thus, in at least some embodiments, the transform module 118 introduces two modifications to the top-down trajectory projection process described above. In the first modification, the transform module 118 exploits the conditional independence of the MDP to subsample fixed-length trajectories 303 without restoration from each episode 301 using a finite time horizon represented by T. In the second modification, the transform module 118 linearly interpolates the axes along c(τ) and dynamically shifts and scales them to increase the fidelity of the resulting image 307-2. FIG. 4 shows various examples of the projection output 402 generated by the transform module 118. In this example, the projection output 402 includes one or more projections / images 404 (shown as projections 404-1 through 404-4) that represent the navigational behavior of one or more artificial agents 108 and one or more human counterparts. In at least some embodiments, each sampled trajectory τ 303 determined for the behavior information 126 is represented as a projection / image 307-2. In at least some embodiments, the output of the transformation module 118 is a sample distribution X 309-1 (distribution P ) that includes the collection of projections / images 307-2 generated for the artificial agent behavior information 126-1. θ 305-1) and a sample distribution Y309-2 (distribution P * 305-2).

[0042] 3, block 308, in another example, the transformation module 118 transforms the behavior information 126 into a different representation by vectorizing the sampled trajectory τ 303 representing the artificial agent behavior information 126-1 and the human behavior information 126-2 using the absolute 3D location of the artificial agent 108 at each time step, similar to the example above. However, in this example, the transformation module 118 does not transform the vector 307-1 representing the trajectory τ 303 into a 2D visual representation. Instead, the transformation module 118 generates the sample distribution X 309-1 and the sample distribution Y 309-2 directly from the vector 307-1 representing the sampled trajectory τ 302, which includes the x, y, and z coordinate data of the sampled trajectory τ 303. For example, the transformation module 118 transforms each episode 301-1 and 301-2, which represent the artificial agent behavior information 126-1 and the human behavior information 126-2, respectively, into a distribution of movements by subsampling a fixed-length trajectory τ 303 uniformly without replacement from each episode 301 using a finite time horizon represented by T. More formally, c t Let c(τ) be the 3D Cartesian coordinates of the artificial agent 108 at time t, and c(τ) be defined as c(τ)={c 0 ,...,c T} and c t =(x t ,y t ,z t ) for a given trajectory τ 303. Given an episode of length N, the transformation module 118 computes c(τ i ) and c(τ j ) are sampled the same

[0043]

number

[0044] 305-1 and P θ 305-2 respectively. As described above, P * 305-1 represents the distribution of the trajectories τ303-1 induced by the artificial agent 108 according to the decision-making policy π θ while P θ 305-2 represents the distribution of the trajectories τ303-2 induced by the human counterpart. In at least some embodiments, for the purpose of evaluating the action similarity between P * 305-1 and P * 305-1, as summarized below, these distributions are alternative hypotheses (H θ which are different 1) against the null hypothesis of equality (H 0 ) is configured to evaluate: H 0 :P * =P θ H 1 :P * ≠P θ .

[0045] In at least some embodiments, the behavioral similarity determination module 120 implements a two-sample hypothesis testing technique to evaluate the behaviors 126 of the artificial agent 108 and the human counterpart. Figure 5 illustrates in flowchart form an example method 500 for implementing the two-sample hypothesis testing technique. The example method 500 is implemented in block 310 of Figure 3. The two-sample hypothesis testing technique, in at least some embodiments, is motivated by the following insight: if the null hypothesis is true, then P * 305-1 and P θ 305-2 should be due to sampling error. Therefore, the behavioral similarity determination module 120 implements one or more techniques to calculate a test statistic 503 for the purpose of evaluating the difference between the sample distribution X 309-1 and the sample distribution Y 309-2. Examples of these techniques include the MMD and the n out of m bootstrap resampling technique described above. The behavioral similarity determination module 120 then evaluates and compares this distribution of distances (e.g., MMD) in two settings, the separate and pooled sample distributions, to generate a behavioral analysis result 128 that includes a similarity measure 128-1, such as a p-value 507.

[0046] For example, in block 502, the behavioral similarity determination module 120 operates in a first setting to evaluate across the separated sample distributions X 309-1 and Y 309-2. i and y i represents a subsample of size m independently drawn with replacement from the sample distribution X 309-1 and the sample distribution Y 309-2, respectively. The behavioral similarity determination module 120 calculates the MMD over S iterations for i ∈ {1, ..., S}. k[x i ,y i ] to form a distribution of first MMD distances 501-1. This distribution of first distances 501-1 is X,Y In block 506, the behavioral similarity determination module 120 calculates a test statistic 503, represented as δ, according to Equation 4. δ = quantile(δ X,Y ,α), (where α∈(0,1)) Equation 4, Here, quantile(δ X,Y ,α) is the distribution δ X,Y It is understood that other techniques for generating the test statistic 503 are equally applicable. For example, a confidence interval can be used to generate the test statistic 503. In this example, δ X,Y Given α, the mean and standard deviation are calculated to generate a test statistic 503 using the lower limit of the confidence interval defined by α.

[0047] In block 508, the behavioral similarity determination module 120 operates in a second setting and combines the sample distribution X 309-1 and the sample distribution Y 309-2 to generate a pooled sample distribution 505 denoted as Z. In block 510, the behavioral similarity determination module 120 performs an evaluation operation by forming a second MMD distance distribution 501-2. The similarity determination module 120 performs an MMD distance evaluation on the samples independently drawn S times, i∈{1,...,S}. k [x i ,y i ] to form a second MMD distance distribution 501-2. However, in this second setting, x i and y i are sampled independently from the pooled distribution Z, both with replacement. The distribution of this second distance 501-2 is ZIn block 512, the behavior similarity determination module 120 compares the distributions of distances 501 from the two settings to generate a behavior analysis result 128 that includes a similarity measure 128-1 that quantifies how similar the artificial agent's behavior 126-1 is to the human counterpart's behavior 126-2. In at least some embodiments, the similarity measure 128-1 is represented by a p-value 507. The similarity determination module 120, in at least some embodiments, compares the distributions of distances 501 from the two settings to generate a behavior analysis result 128 that includes a similarity measure 128-1 that quantifies how similar the artificial agent's behavior 126-1 is to the human counterpart's behavior 126-2. In at least some embodiments, the similarity measure 128-1 is represented by a p-value 507. 0 To evaluate p-value 507, we set the p-value 507 to an estimate δ that is greater than the test statistic δ, as shown in Equation 5 below: Z Generate as a percentage of .

[0048]

number

[0049] Referring to FIG. 3, in block 312, the behavioral similarity determination module 120 generates an artificial agent behavioral analysis result 128 including a similarity measure 128-1 (e.g., p-value 507). FIG. 6 shows an example of an artificial agent behavioral analysis result 128 in the form of a table 600. In this example, the "T" column 602 represents the time range used in the evaluation, and the "a" column 604 represents a quantile across the baseline distribution. The "human" column 606 represents a similarity measure between the behavior of two human counterparts. The "agent 1" column 608 represents a similarity measure of a first artificial agent and the combined behavior of multiple human counterparts. The "agent 2" column 610 represents a similarity measure of a second artificial agent and the combined behavior of multiple human counterparts. Agent 1 and Agent 2 are trained / programmed in this example. The percentages in brackets are the interquartile ranges (IQR) indicating the degree of dispersion of the similarity measures, whereas the percentages outside brackets are the medians across a given number of runs.

[0050] 3, in block 314, the AABA system 106 determines whether the similarity measure 128-1 indicates that the artificial agent 108 should be modified / reconfigured. For example, the AABA system 106 determines whether the similarity measure 128-1 indicates that the human-likeness of the artificial agent's behavior does not satisfy a similarity threshold. If so, the process then ends in block 318 or returns to block 302. If not, in block 316, the AABA system 106 modifies or reconfigures / programs the artificial agent 108 as described above with respect to FIG. 1 and FIG. 2. For example, the artificial agent 108 is modified such that the human-likeness of the artificial agent's behavior is improved. Alternatively, the AABA system 106 instructs another system, such as the training system 110, to perform the modification or reconfiguration of the artificial agent 108. The process then ends in block 318 or returns to block 302.

[0051] As mentioned above, the p-value 507 is used as the similarity measure 128-1 by the AABA system 106 in at least some embodiments. In these embodiments, P * and P θ is the null hypothesis (H 0 ), the distribution X 307-1 of the first MMD distance 501-1 and the distribution Y 307-2 of the sample distributions Y 307-2 as calculated over the separated sample distributions should be the same as the distribution of the second MMD distance 501-2 as calculated over the pooled sample distribution Z 505. Thus, P * =P θ , the resulting p-value 507 will converge towards 1-α as S → ∞. Furthermore, P * ≠P θ If so, the AABA system 106 may then compute the derived p-value 507 based on the distribution P * and P θ To illustrate this, let P *Consider a series of experiments using a simple example where p is distributed as a 128-dimensional standard Gaussian distribution with zero mean and unit variance, represented as N(0,1). For each experiment, S=1000 iterations are performed using a subsample size m=100. Due to the inherent stochasticity of the two-sample hypothesis testing technique implemented by the behavioral similarity determination module 120, each experiment is repeated 10 times and both the median p-value and the observed interquantile range (IQR) are reported.

[0052] First, P using α = 0.10 θ =P * Consider the case where N = N(0,1). A median p-value of 88.5% with an IQR of 0.43% is observed. Next, the p-values ​​resulting from the two-sample hypothesis tests performed by the behavioral similarity determination module 120 are distributed as θ 305-1 and Distribution P * To illustrate how p can be used as a measure of similarity between the distribution P θ It can be shown that 305-1 decreases monotonically as it is incrementally shifted by epsilon ∈ = 0.02. θ =N(∈,1) to P * =N(0,1), the observed results are shown in table 700 of FIG. 7. Furthermore, the distribution of MMD distance 501 is P θ Shift with 305-1. P θ If N(0.10,1), then δ X,Y and δ Z The distributions of are approximately separated. In at least some embodiments, the AABA system 106 controls the sensitivity of the two-sample hypothesis test by adjusting α. Intuitively, a higher value of α results in a larger test statistic δ, and therefore increases the sensitivity of the two-sample hypothesis test performed by the behavioral similarity determination module 120. As shown in table 700 of FIG. 7, a more sensitive test results in a lower p-value as the distributions diverge. Thus, when the behaviors of multiple artificial agents 108 are being compared against the behaviors of their human counterparts, α can be adjusted to control the sensitivity and produce a more informative comparison.

[0053] The computational cost of estimating the MMD distance between two samples grows quadratically with the size of each sample, but also linearly with the dimensionality of the data. Thus, in at least some embodiments, the dimensionality reduction module 122 of the AABA system 106 implements one or more dimensionality reduction techniques to reduce such computational costs by removing redundancy in high-dimensional data (e.g., images) and reducing the number of random variables (e.g., pixels) under consideration, while preserving all the information necessary to understand the original distribution. One example of a dimensionality reduction technique is principal component analysis (PCA). To illustrate the effect of PCA on the two-sample hypothesis testing performed by the behavioral similarity determination module 120, consider a sample distribution D consisting of 22,600 fixed-length trajectories subsampled from 100 different episodes observed from four human subjects. In this example, the dimensionality reduction module 122 reduces the high-dimensional (HD) data to

[0054]

number

[0055]

number

[0056] The techniques described herein may, in different embodiments, be utilized in any of a variety of processors or parallel processors (e.g., vector processors, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), non-scalar processors, highly parallel processors, artificial intelligence (AI) processors, inference engines, machine learning processors, other multi-threaded processing units, etc.). Referring now to FIG. 8, a block diagram of a processing system 800, such as systems 102-1-102-3, according to some embodiments, configured with a parallel processor is shown. Processing system 800 includes a central processing unit (CPU) 802 and a graphics processing unit (GPU) 804. In at least some embodiments, CPU 802, GPU 804, or both CPU 802 and GPU 804 are configured to implement AABA system 106. CPU 802, in at least some embodiments, includes one or more single-core or multi-core CPUs. In various embodiments, GPU804 includes any cooperating collection of hardware and / or software that performs functions and calculations associated with accelerated graphics processing tasks, data parallel tasks, nested data parallel tasks, in an accelerated manner in conjunction with resources such as a conventional CPU, a conventional graphics processing unit (GPU), and combinations thereof.

[0057] In the embodiment of FIG. 8, the CPU 802 and the GPU 804 are formed and combined on a single silicon die or package to provide an integrated programming and execution environment. The environment allows the GPU 804 to be used fluidly in the same way as the CPU 802 for some programming tasks. In other embodiments, the CPU 802 and the GPU 804 are formed separately and implemented on the same or different substrates. It should be understood that the processing system 800, in at least some embodiments, includes more or fewer components than those shown in FIG. 8. For example, the processing system 800, in at least some embodiments, additionally includes one or more input interfaces, non-volatile storage, one or more output interfaces, a network interface, and one or more displays or display interfaces.

[0058] As shown in FIG. 8, the processing system 800 includes a system memory 806, an operating system 808, a communication infrastructure 810, and one or more applications 812. Access to the system memory 806 is managed by a memory controller (not shown) coupled to the system memory 806. For example, requests from the CPU 802 or other devices to read from or write to the system memory 806 are managed by the memory controller. In some embodiments, the one or more applications 812 include various programs or commands to perform calculations that are also executed in the CPU 802. The CPU 802 sends selected commands for processing to the GPU 804. The operating system 808 and the communication infrastructure 1010 are discussed in more detail below. The processing system 800 further includes a memory management unit, such as a device driver 814 and an input / output memory management unit (IOMMU) 816. The components of the processing system 800 are implemented as hardware, firmware, software, or any combination thereof. In some embodiments, processing system 800 includes one or more software, hardware, and firmware components in addition to or different from those shown in FIG.

[0059] Within processing system 800, system memory 806 includes non-persistent memory such as dynamic random access memory (not shown). In various embodiments, system memory 806 stores processing logic instructions, constant values, variable values ​​during execution of a portion of an application or other processing logic, or other desired information. For example, in various embodiments, a portion of the control logic for performing one or more operations on CPU 802 resides in system memory 806 during execution of the respective portion of the operation by CPU 802. During execution, the respective application, operating system functions, processing logic commands, and system software reside in system memory 806. Control logic commands essential to operating system 808 typically reside in system memory 806 during execution. In some embodiments, other software commands (e.g., sets of instructions or commands used to implement device drivers 814) reside in system memory 806 during execution of processing system 800.

[0060] The IOMMU 816 is a multi-context memory management unit. As used herein, a context considers the environment in which a kernel executes and the domain in which synchronization and memory management are defined. A context includes a set of devices, memory accessible to those devices, corresponding memory properties, and one or more command queues used to schedule execution of kernels or operations on memory objects. The IOMMU 816 includes logic to perform virtual to physical address translations for memory page accesses for devices such as the GPU 804. In some embodiments, the IOMMU 816 includes or has access to a translation lookaside buffer (TLB) (not shown). The TLB is implemented in a content addressable memory (CAM) to accelerate the translation of logical (i.e., virtual) memory addresses to physical memory addresses for requests made by the GPU 804 for data in the system memory 806.

[0061] In various embodiments, communication infrastructure 810 interconnects the components of processing system 800. Communication infrastructure 810 includes one or more of a peripheral component interconnect (PCI) bus, an extended PCI (PCI-E) bus, an advanced microcontroller bus architecture (AMBA) bus, an advanced graphics port (AGP), or other such communication infrastructures and interconnects (not shown). In some embodiments, communication infrastructure 810 includes an Ethernet network or any other suitable physical communication infrastructure that meets the data transfer rate requirements of the application. Communication infrastructure 810 also includes functionality for interconnecting components, including the components of processing system 800.

[0062] Drivers such as device driver 814 communicate with devices (e.g., GPU 804) through interconnect or communication infrastructure 810. When a calling program invokes a routine in device driver 814, device driver 814 issues a command to the device. When the device sends data back to device driver 814, device driver 814 invokes the routine of the original calling program. Generally, device drivers are hardware dependent and operating system specific to provide the requisite interrupt handling for any necessary asynchronous dependent hardware interfaces. In some embodiments, compiler 818 is embedded within device driver 814. Compiler 818 compiles source code into program instructions necessary for execution by processing system 800. During such compilation, compiler 818 applies transformations to program instructions at various stages of compilation. In other embodiments, compiler 818 is a standalone application. In various embodiments, device driver 814 controls the operation of GPU 804, for example, by providing an application programming interface (API) to software executing on CPU 802 (e.g., applications 812) for accessing various functionality of GPU 804.

[0063] CPU 802 includes one or more of a control processor, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a digital signal processor (DSP) (not shown). CPU 802 executes at least a portion of the control logic that controls the operation of processing system 800. For example, in various embodiments, CPU 802 executes an operating system 808, one or more applications 812, and device drivers 814. In some embodiments, CPU 802 initiates and controls the execution of one or more applications 812 by distributing processing associated with one or more applications 812 between CPU 802 and other processing resources, such as GPU 804.

[0064] The GPU 804 executes commands and programs for selected functions, such as graphics operations and other operations that are particularly suited to parallel processing. In general, the GPU 804 is frequently used to perform graphics pipeline operations, such as pixel operations, geometric calculations, and rendering images to a display. In some embodiments, the GPU 804 performs computational operations (e.g., non-graphics related operations, such as video operations, physics simulations, computational fluid dynamics, etc.) based on commands or instructions received from the CPU 802. For example, such commands include special instructions that are typically not defined in the instruction set architecture (ISA) of the GPU 804. In some embodiments, the GPU 804 receives image geometry representing a graphics image along with one or more commands or instructions for rendering and displaying the image. In various embodiments, the image geometry corresponds to a two-dimensional (2D) or three-dimensional (3D) computerized representation of the graphics image.

[0065] In various embodiments, GPU 804 includes one or more computational units, such as one or more processing cores 820 (shown as 820-1 and 820-2) including one or more single-instruction multiple-data (SIMD) units 822 (shown as 822-1 through 822-4), each of which is configured to execute a thread concurrently with execution of other threads in a wavefront by the other SIMD units 822, e.g., according to a SIMD execution model, in which multiple processing elements share a single program control flow unit and program counter, and thus may execute the same program, but with different data. Processing cores 820 are also referred to as shader cores or streaming multi-processors (SMX). The number of processing cores 820 implemented in GPU 804 is configurable. Each processing core 820 includes one or more processing elements, such as a scalar and / or vector floating point unit, an arithmetic and logic unit (ALU), etc. In various embodiments, processing core 820 includes special purpose processing units (not shown), such as an inverse square root unit and a sine / cosine unit.

[0066] Each of the one or more processing cores 820 executes a respective instantiation of a particular work item to process input data, and the basic unit of execution in the one or more processing cores 820 is a work item (e.g., a thread). Each work item represents a single instantiation of a collection of parallel executions of kernels launched on a device, for example, by commands that are executed in parallel. Work items execute on one or more processing elements as part of a workgroup that executes in the processing core 820.

[0067] The GPU 804 issues and executes work items, such as groups of threads, that execute simultaneously as a "wavefront" on a single SIMD unit 822. A wavefront, in at least some embodiments, is referred to interchangeably as a warp, vector, or thread. In some embodiments, a wavefront includes instances of parallel execution of shader programs, but each wavefront includes multiple work items that execute simultaneously on a single SIMD unit 822 according to the SIMD paradigm (e.g., one instruction control unit executing multiple streams of data and the same instructions). The scheduler 824 is configured to perform operations related to scheduling the various wavefronts on the different processing cores 820 and SIMD units 822, as well as performing other operations to coordinate various tasks on the GPU 804.

[0068] To reduce the latency associated with off-chip memory accesses, various GPU architectures include a memory cache hierarchy (not shown) including, for example, an L1 cache and a local data share (LDS). The LDS is a high-speed, low-latency memory dedicated to each processing core 820. In some embodiments, the LDS is a full gather / scatter model such that a workgroup writes anywhere within its allocated space.

[0069] The parallel processing provided by one or more processing cores 820 is suitable for graphics-related operations such as pixel value calculations, vertex transformations, tessellation, geometry shading operations, and other graphics operations. A graphics processing pipeline 826 accepts graphics processing commands from the CPU 802 and thereby provides computational tasks to one or more processing cores 820 for execution in parallel. Some graphics pipeline operations, such as pixel processing and other parallel computational operations, require the same command stream or computational kernel to be performed on a stream or collection of input data elements. Respective instances of the same computational kernel execute simultaneously on multiple SIMD units 822 in one or more processing cores 820 to process such data elements in parallel. As referred to herein, for example, a computational kernel is declared in a program and executed on an accelerated processing device (APD) processing core 820. The function may also be referred to as a kernel, a shader, a shader program, or a program.

[0070] In at least some embodiments, processing system 800 is a computer, a laptop / notebook, a mobile device, a gaming device, a wearable computing device, a server, or any of a variety of other types of computing systems or devices. It should be noted that the number of components of processing system 800 may vary from embodiment to embodiment. In at least some embodiments, there are more or fewer components / subcomponents than are shown in FIG. 8. It should also be noted that processing system 800, in at least some embodiments, includes other components not shown in FIG. 8. Additionally, in other embodiments, processing system 800 is constructed in other ways than as shown in FIG. 8.

[0071] In some embodiments, the above apparatus and techniques are implemented in a system that includes one or more integrated circuit (IC) devices (also referred to as integrated circuit packages or microchips). In at least some embodiments, electronic design automation (EDA) and computer aided design (CAD) software tools can be used to design and manufacture these IC devices. These design tools are typically represented as one or more software programs. The one or more software programs include code executable by a computer system to operate the computer system to operate on code representing circuits of one or more IC devices to perform at least a portion of a process for designing or adapting a manufacturing system to manufacture the circuits. This code may include instructions, data, or a combination of instructions and data. The software instructions representing the design tool or manufacturing tool are typically stored in a computer readable storage medium accessible to the computing system. Similarly, code representing one or more stages of the design or manufacture of the IC device is stored in and accessed from the same computer readable storage medium or a different computer readable storage medium.

[0072] In at least some embodiments, a computer-readable storage medium includes any non-transitory storage medium or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media can include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tape, magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS) based storage media. The computer-readable storage medium (e.g., system RAM or ROM) may be internal to the computing system, the computer-readable storage medium (e.g., a magnetic hard drive) may be permanently attached to the computing system, the computer-readable storage medium (e.g., an optical disk or Universal Serial Bus (USB)-based flash memory) may be removably attached to the computing system, or the computer-readable storage medium (e.g., network-accessible storage (NAS)) may be coupled to the computer system via a wired or wireless network.

[0073] In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied in a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by the one or more processors, operate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device such as a flash memory, a cache, a random access memory (RAM), or other non-volatile memory device(s), etc. The executable instructions stored in the non-transitory computer-readable storage medium may be implemented as source code, assembly language code, object code, or other form of instructions that can be interpreted or otherwise executed by one or more processors.

[0074] In addition to the above, it should be noted that not all activities or elements described in the summary description are required, some of the specific activities or devices may not be required, one or more additional activities may be performed, and one or more additional elements may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will appreciate that various changes and modifications can be made without departing from the scope of the invention as set forth in the claims. Thus, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the invention.

[0075] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and features by which any benefit, advantage, or solution may occur or be manifested are not to be construed as critical, essential, or essential features of any or all claims. Moreover, the specific embodiments described above are illustrative only, as the disclosed invention may be modified and practiced in different but similar manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as set forth in the appended claims. It is therefore apparent that the specific embodiments described above may be altered or modified, and all such variations are considered to be within the scope of the disclosed invention. Accordingly, the protection sought herein is set forth in the appended claims.

Claims

1. 1. A method comprising: determining, at a processor, a first distribution based on a behavior associated with the artificial agent and a second distribution based on a behavior associated with the user; generating, in the processor, a similarity measure by comparing the first distribution to the second distribution, the similarity measure quantifying a human-likeness of a behavior associated with the artificial agent; and in response to the similarity measure not satisfying a similarity threshold, modifying, at the processor, a behavior of the artificial agent based on the similarity measure. method.

2. Determining the first distribution and the second distribution includes: vectorizing the first behavioral information set to generate a first plurality of vectors, each vector of the first plurality of vectors representing a different behavioral sub-sequence of the first behavioral information set; vectorizing the second behavioral information set to generate a second plurality of vectors, each vector of the second plurality of vectors representing a different behavioral sub-sequence of the second behavioral information set.

2. The method of claim 1.

3. Determining the first distribution and the second distribution includes: translating the first plurality of vectors into a first plurality of visual representations and translating the second plurality of vectors into a second plurality of visual representations. The method of claim 2.

4. the first distribution includes the first plurality of visual representations and the second distribution includes the second plurality of visual representations. The method of claim 3.

5. the first distribution includes the first plurality of vectors and the second distribution includes the second plurality of vectors. The method of claim 2.

6. generating the similarity measure by comparing the first distribution to the second distribution includes performing a two-sample hypothesis test.

2. The method of claim 1.

7. Conducting the two-sample hypothesis test includes: operating in a first mode including evaluating across the first distribution and the second distribution separately to generate a first distance distribution and generating a test statistic based on the first distance distribution; operating in a second mode, the second mode including combining the first distribution and the second distribution to generate a pooled sample distribution of behavior and evaluating across the pooled sample distribution to generate a second distance distribution; generating the similarity measure based on the test statistic and the second distance distribution. The method of claim 6.

8. Evaluating individually across the first distribution and the second distribution includes: for a predetermined number of iterations, independently drawing a first subsample of a predetermined size from the first distribution with replacement and independently drawing a second subsample of the predetermined size from the second distribution with replacement; determining, for each iteration, a statistical distance between the first subsample and the second subsample; generating the first distance distribution based on each statistical distance determined for each iteration; The method of claim 7.

9. The test statistic is given by δ=quantile(δ Ｘ，Ｙ , α), where α∈(0,1), where δ is the test statistic, and δ Ｘ，Ｙ is the first distance distribution, α is a hyperparameter for controlling the sensitivity of the two-sample hypothesis test, and quantile (δ Ｘ，Ｙ , α) returns the α quantiles across the first distance distribution. The method of claim 8.

10. Evaluating across the pooled sample distribution to generate a second distance distribution includes: for the predetermined number of iterations, independently drawing, with replacement, a third subsample of the predetermined size from the pooled sample distribution and independently drawing a fourth subsample of the predetermined size from the pooled sample distribution; determining, for each iteration, a statistical distance between the third subsample and the fourth subsample; generating the second distance distribution based on each statistical distance determined for each iteration. The method of claim 8.

11. the similarity measure is generated as the proportion of distances in the second distance distribution that have a distance greater than the test statistic. The method of claim 10.

12. modifying the artificial agent based on the similarity measure comprises modifying one or more behaviors of the artificial agent.

2. The method of claim 1.

13. 1. A method comprising: determining a first distribution based on behavior associated with a first artificial agent and a second distribution based on behavior associated with a second artificial agent; generating a similarity measure by comparing the first distribution to the second distribution, the similarity measure quantifying a similarity between a behavior of the first artificial agent and a behavior of the second artificial agent; and modifying at least one of a behavior of the first artificial agent or a behavior of the second artificial agent based on the similarity measure. method.

14. the behavior of the first artificial agent is represented by a first episode defined as a first behavior sequence extending from an initial state to a final state of the first artificial agent, and the behavior of the second artificial agent is represented by a second episode defined as a second behavior sequence extending from an initial state to a final state of the second artificial agent; 14. The method of claim 13.

15. Determining the first distribution and the second distribution includes: sampling a first plurality of trajectories from the first episode, each trajectory of the first plurality of trajectories being a subsequence of the first behavioral sequence; sampling a second plurality of trajectories from the second episode, each trajectory of the second plurality of trajectories being a subsequence of the second behavioral sequence; vectorizing each sample trajectory of the first plurality of trajectories to generate a first plurality of vectors; vectorizing each sample trajectory of the second plurality of trajectories to generate a second plurality of vectors; the first distribution is determined based on the first plurality of vectors, and the second distribution is determined based on the second plurality of vectors.

15. The method of claim 14.

16. 1. An apparatus comprising: A processor configured to perform the method of any one of claims 1 to 15. Device.

Images