System and procedures for understanding automated decisions

Abstract

A method for analyzing a decision for an automated system. The method involves receiving a vector of length . A plurality of distribution vectors is generated, each with a length corresponding to the length of the vector. Divergence calculations are performed on the vector and each of the plurality of distribution vectors. A minimum output value, determined by the divergence calculations, is computed. From the plurality of distribution vectors, a minimizing distribution vector is identified that corresponds to the minimum output value from the divergence calculations. The validity of a decision is determined based on the vector with respect to the minimizing distribution vector.

Description

introduction

[0001] The subject matter of the disclosure relates to automated decision making, and more particularly to a system and method for determining a better understanding of automated decisions.

[0002] Artificial intelligence (AI) can be used in an autonomous or semi-autonomous vehicle to determine an action the vehicle should perform. Automated systems routinely make decisions based on a vector reflecting the system's options for choosing the behavior, with entries containing values ​​representing the desirability of choosing that behavior. Automated systems can make a final decision based on the vector by selecting an entry with the largest value or a value above a predetermined threshold. However, these approaches do not consider alternative courses of action. Description

[0003] Disclosed herein is a method for analyzing a decision for an automated system. The method includes receiving a vector having a length. Generating a plurality of distribution vectors, each having a length consistent with the length of the vector. Divergence calculations are performed on the vector and each of the plurality of distribution vectors. A minimum output value determined by the divergence calculations is calculated. From the plurality of distribution vectors, a minimizing distribution vector corresponding to the minimum output value from the divergence calculations is identified. The conclusiveness of a decision is determined based on the vector with respect to the minimizing distribution vector.

[0004] According to another aspect of the disclosure, the divergence calculations may include Jensen-Shannon divergence calculations and the vector is received by an automated system.

[0005] According to another aspect of the disclosure, sorting the vector may be included to generate a sorted vector and performing the divergence calculations based on the sorted vector and each of the plurality of distribution vectors.

[0006] According to another aspect of the disclosure, sorting the vector to generate the sorted vector may include generating a permutation vector to determine an initial ordering of the entries in the vector.

[0007] According to another aspect of the disclosure, generating a conclusiveness vector from the minimizing distribution vector may be included by applying the permutation vector to the minimizing distribution vector to associate the conclusiveness of the decision with at least one corresponding value in the vector.

[0008] According to another aspect of the disclosure, creating a conclusiveness vector from the minimizing distribution vector may be included to associate the conclusiveness of the decision with at least one corresponding value in the vector.

[0009] According to another aspect of the disclosure, determining the conclusiveness of the decision based on the minimizing distribution vector may include identifying at least one non-zero value in the minimizing distribution vector.

[0010] According to a further aspect of the disclosure, the conclusiveness of the decision may be low if the at least one non-zero value contains a number of non-zero values ​​that exceeds a predetermined threshold.

[0011] According to a further aspect of the disclosure, the conclusiveness of the decision may be high if the at least one non-zero value includes a number of non-zero values ​​below a predetermined threshold.

[0012] According to another aspect of the disclosure, each of the plurality of distribution vectors may include at least one non-zero value.

[0013] According to another aspect of the disclosure, the at least one non-zero value in each of the plurality of distribution vectors may be equal to the multiplicative inverse of a set of non-zero values ​​in a corresponding one of the plurality of distribution vectors.

[0014] According to another aspect of the disclosure, a number of vectors in the plurality of distribution vectors may be less than or equal to the number of entries in the vector.

[0015] According to another aspect of the disclosure, the divergence calculations may include Jensen-Shannon divergence calculations and the vector is received by an automated system.

[0016] According to another aspect of the disclosure, each of the plurality of distribution vectors may be filled with at least one non-zero distribution value in a left-to-right order.

[0017] Disclosed herein is a non-transitory computer-readable medium containing programmed instructions that, when executed by a processor, are suitable for performing a method. The method includes receiving a vector having a length. Generating a plurality of distribution vectors, each having a length that matches the length of the vector. Performing divergence calculations on the vector and each of the plurality of distribution vectors. A minimum output value determined by the divergence calculations is calculated. From the plurality of distribution vectors, a minimizing distribution vector is identified that corresponds to the minimum output value from the divergence calculations. The conclusiveness of a decision is determined based on the vector with respect to the minimizing distribution vector.

[0018] A vehicle system is disclosed herein. The system includes a plurality of sensors and a controller in communication with the plurality of sensors. The controller is configured to receive a vector having a length, generate a plurality of distribution vectors each having a length consistent with the length of the vector, and perform divergence calculations with the vector and each of the plurality of distribution vectors. The controller is also configured to identify a minimum output value determined by the divergence calculations and to identify a minimizing distribution vector from the plurality of distribution vectors that corresponds to the minimum output value from the divergence calculations. The controller is further configured to determine a conclusiveness of a decision based on the vector, taking into account the minimizing distribution vector. Brief description of the drawings Fig. 1 shows a vehicle that can be operated in an autonomous or automated mode, according to an exemplary embodiment. Fig. 2 shows a method for determining an action for an automated system. Fig. Figure 3 shows a method for analyzing a decision based on a vector generated by the automated system.

[0019] Some embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings. The same reference numeral designates the same element or type of element throughout the drawings. Detailed description

[0020] In accordance with an exemplary embodiment, Fig. 1 shows a vehicle 100 that can be operated in an autonomous mode or automated mode. The vehicle 100 can be a fully autonomous vehicle or a semi-autonomous vehicle. The vehicle 100 includes a driving system 102 that controls the autonomous operation of the vehicle. The driving system 102 includes a sensor system 104 for sensing information about the environment or surroundings of the vehicle 100 and a controller 106 for calculating possible actions for the autonomous vehicle based on the sensed information and for executing one or more of the possible actions, as well as a human-machine interface 108 for explaining the actions performed by the vehicle to an occupant of the vehicle, such as a driver or passenger. The sensor system 104 can include devices such as cameras, lidar, radar, GPS, etc.

[0021] The human-machine interface 108 need not be specifically designed to explain actions and may be an interface used in the vehicle for purposes other than explaining actions. The human-machine interface 108 may include one or more interfaces, such as a speaker, a display, a haptic device, etc. Each interface is suitable for a particular modality of a message. For example, the speaker may be used to play an audible message. The display may be used to present a written message, a graphic, or an animation. The haptic device may be used to convey a tactile message, such as a vibration, that is perceived by the user.In another embodiment, the human-machine interface 108 may be a mobile device that can be used by the occupant either inside the vehicle or outside the vehicle.

[0022] The controller 106 may include processing circuitry, which may include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (collectively, dedicated, or as a group), and memory executing one or more software or firmware programs, combinational logic circuitry, and / or other suitable components that provide the described functionality. The controller 106 may include a non-transitory computer-readable medium storing instructions that, when processed by one or more processors of the controller 106, implement a method for determining soft understanding of automated decisions in accordance with one or more embodiments described herein. The controller 106 may execute programs that operate artificial intelligence. However, this disclosure relates to decision-making systems that use vectors, such asProbability or confidence vectors, use or generate in the decision-making process.

[0023] Fig. 2 illustrates a method 200 for operating an automated system, such as driving system 102 for vehicle 100, to make a final decision based on a vector P1. According to an exemplary embodiment, system 102 generates (block 202) vector P1, such as a probability vector or a confidence vector, that reflects a current understanding of system 102 in a decision-making process. Vector P1 contains a plurality of entries and arrives at the final decision by automatically selecting one of the plurality of entries from vector P1. These decisions are often made as a result of hard thresholds applied to a soft inference (e.g., a probability vector or a confidence vector).Decisions resulting from the application of hard thresholds, commonly referred to as hard decisions, typically select a maximum argument in the vector P1 and return it as the decision for the automated system.

[0024] The method 200 analyzes the vector P1 as part of the decision analysis performed at block 206 in connection with the solution process of the system 102 to determine a soft understanding of the system's reasoning process for selecting the final decision, as opposed to a hard decision. For example, in automated driving algorithms, such as a Monte Carlo Tree Search (MCTS), the vector P1 is analyzed to determine the dominance of one of the decisions in the vector. Using the method 200, the system 102 can determine whether the vector P1 contains a single dominant action that it is considering taking, two competing actions, or whether the method 200 has not determined a clear decision from the various possibilities in the vector P1.

[0025] Fig.3 shows an example of a method 300 for performing a decision analysis of the vector P1 in block 206 of the method 200. The decision analysis performed in block 206 can be used in different ways depending on the context of the system 102 and the end recipient of that decision (e.g., user or engineer).

[0026] The decision analysis performed by method 300 includes divergence calculations based on vector P1 and a plurality of distribution vectors. Method 300 begins by receiving vector P1 from system 102 (block 302). Vector P1 includes a plurality of entries, each having a value representing a decision to be made by the automated system. Equation 1 below represents an example vector P1 received in block 302. Each value in vector P1 corresponds to a possible decision to be made by the automated system. In the example shown, the values ​​in vector P1 could correspond to the system 102's decision that vehicle 100 should turn right, turn left, accelerate, decelerate, or make no change in action.With a hard decision-making approach, the system might choose the action that corresponds to the highest value, which in this example would be “slow down.” P1=(0.05, 0.35, 0.05, 0.45, 0.1)

[0027] In one example, the method 300 sorts the values ​​in the vector P1 in descending order to produce a sorted vector P 1S as shown in equation 2 below. P1S=(0.45, 0.35, 0.1, 0.05, 0.05)

[0028] The method 300 may also include a permutation vector P 1P to preserve the original order of the values ​​in the vector P1 after sorting. An example of a permutation vector P 1P , which contains the sorted vector P 1S to the original vector P1 is shown below in equation 3. P1P=(4, 2, 5, 1, 3)

[0029] The method 300 then generates a plurality of distribution vectors in block 304. In one example, the plurality of distribution vectors includes a first set of distribution vectors for performing the divergence calculations with the sorted vector P 1S . Each of the first set of distribution vectors Q j (Equation 4) has a length that matches a length of the vector P1 to be analyzed. Qj=(1j,…,1j,0,…0)

[0030] In Equation 4 above, j is a variable corresponding to a number of nonzero values ​​in the distribution vector. For each nonzero value in a distribution vector, the nonzero value is equal to the multiplicative inverse of the number of nonzero values ​​in the distribution vector. For example, if j is equal to "1," the equation for Q1 would be represented as shown in Equation 5 below. Also, the nonzero values ​​fill the distribution vectors from left to right in the illustrated example to perform the divergence calculation. Q1=(11,0,0,0,0)

[0031] The same approach is used to generate the distribution vectors Q2, Q3, Q4, and Q5. For the distribution vector Q5, the variable j is equal to 5, and Q5 would be represented as in Equation 6 below. Q5=(15,15,15,15,15)

[0032] For the illustrated example with P1 having five entries, the distribution vectors would be created for integer values ​​of j from "1" to "5," such that the number of distribution vectors corresponds to the number of entries in the vector P1. The distribution vector Q5, as shown in Equation 6, also represents a uniform distribution vector for the actions, and the distribution vector Q1 represents the greatest possible degree of conclusiveness for a decision based on the vector P1, as explained in more detail below.

[0033] After the vectors have been generated, the method 300 performs a divergence calculation in block 306. The divergence calculation may be performed between the sorted vector P 1S and the distribution vectors Q1 -Q NIn one example, the divergence calculation performed is a Jensen-Shannon divergence calculation, as shown in Equation 7 below. The divergence calculation provides an output value that represents a measurement of the distance between two data distributions, showing how different the two distributions are from each other. JSD(P‖Q)=12D(P‖M)+12D(Q‖M)

[0034] The variable M in equation 7 above is shown in equation 8 below. M=12(P+Q)

[0035] While the method 300 performs the divergence calculations with the first set of distribution vectors and the sorted vector P 1S , the method 300 identifies a minimum output value from these calculations (block 308). The method 300 may then identify the distribution vector corresponding to the minimum output value (block 310).

[0036] Alternatively, the distribution vectors Q1, ..., Q N be analyzed in an order corresponding to an increasing value of j. The method 300 may identify which of the distribution vectors Q1, Q2, Q3, Q4, or Q5 corresponds to a minimum output value from the divergence calculations based on the comparison with the vector P 1S. In addition, a local minimum output value can be determined by selecting the previous distribution vector when an output value of the convergence calculations has increased compared to a previously calculated output value. This leads to a reduction in the computational resources required to perform the divergence calculations in some cases if the distribution vector associated with the local minimum output value is determined before performing the divergence calculations on the remaining distribution vectors. In addition, the distribution vector associated with the minimum output value can be identified as the minimizing distribution vector because it results in a minimum output value for the divergence calculations.

[0037] Method 300 uses the minimizing distribution vector to determine the conclusiveness of the final decision based on vector P1 for system 102. A feature of identifying the minimum output value from the divergence calculation is that it identifies the distribution vector with the most similar distribution. Therefore, method 300 allows system 102 to consider more possible actions from vector P1 that would have been eliminated using a hard decision-making approach.

[0038] The conclusiveness of an action based on vector P1 can be determined by identifying a set of non-zero values ​​in the minimizing distribution vector (block 312). If the number of non-zero values ​​exceeds a predetermined threshold, then the conclusiveness of a decision based on vector P1 is low. For example, if the number of non-zero values ​​in the minimizing distribution vector is equal to the total number of entries in the distribution vector, then each of the possible actions based on vector P1 has a similar degree of conclusiveness. This is because the minimizing distribution vector corresponds to a distribution of a uniform probability vector.

[0039] However, if the number of non-zero values ​​in the minimizing distribution vector is below a predetermined threshold, then the conclusiveness of a decision based on vector P1 is high. For example, if the number of non-zero values ​​in the minimizing distribution vector is equal to one, then the automated system has the highest possible degree of conclusiveness to select this entry as the final decision for system 102.

[0040] Furthermore, if the minimizing distribution vector contains multiple non-zero values ​​below the specified threshold, the automated system can know with a high degree of certainty that the decisions corresponding to these values ​​should be considered, while the remaining decisions can be ignored. The system 102 can also communicate the considered decisions via the human-machine interface 108 to explain the actions taken by the vehicle 100 to an occupant, e.g., a driver or passenger, or even request the driver to intervene.

[0041] The method 300 may determine which values ​​of the vector P1 should be considered with a soft understanding by mapping the at least one non-zero value in the minimizing distribution vector to at least one corresponding value in the vector P1. These values ​​correspond to the soft decision values ​​that should be considered by the automated system, which could be disregarded in a hard decision process. If the sorted vector P 1S used in the divergence calculations, the method 300 uses the permutation vector P 1P to map the non-zero values ​​in the minimizing divergence vector to a corresponding entry in the vector P1.

[0042] The method 300 may also create a conclusiveness vector from the minimizing distribution vector. The conclusiveness vector indicates significant decisions from the vector P1 that should be considered beyond a hard decision for the vector P1. The conclusiveness vector corresponds to a length of the vector P1 and returns a value of "1" for each entry point that corresponds to a non-zero value in the minimizing distribution vector. If the sorted vector P 1S used in the divergence calculations, the method 300 uses the permutation vector P 1P to map the values ​​in the minimizing divergence vector to a corresponding entry in the vector P1.

[0043] Using the conclusiveness vector or the minimizing distribution vector generated as in method 300 described above, method 200 determines in block 208 whether a decision based on vector P1 is sufficiently conclusive with respect to the conclusiveness vector or the minimizing distribution vector. If either of these vectors is sufficiently conclusive, method 200 proceeds to block 210 to make a final decision. If these vectors are not sufficiently conclusive, method 200 may perform further analysis in block 212 before returning to system 102 to generate a new vector in block 202. System 102 may then generate another vector, which is analyzed by method 200 to determine whether a decision based on this vector would be sufficiently conclusive to make a final decision.In this way, the method 200 may include an additional level of consideration, request additional information, or re-execute the decision algorithm on the system 102 during the next action of the planning algorithm.

[0044] Although various embodiments have been described, the description is intended to be exemplary rather than restrictive. It will be apparent to those skilled in the art that many other embodiments and implementations are possible that fall within the scope of the embodiments. Any feature of one embodiment may be used in combination with, or in place of, another feature or element in another embodiment, unless expressly limited. Accordingly, the embodiments are not to be limited except in accordance with the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

Claims

[1] A method for analyzing a decision for an automated system, the method comprising: Receive a vector with length; generating a plurality of distribution vectors each having a length equal to the length of the vector; performing divergence calculations on the vector and each of the plurality of distribution vectors; Identifying a minimum output value determined by the divergence calculations; Identifying a minimizing distribution vector from the plurality of distribution vectors that corresponds to the minimum output value from the divergence calculations; and Determining the conclusiveness of a decision based on the vector with respect to the minimizing distribution vector. [2] The method of claim 1, wherein the divergence calculations comprise Jensen-Shannon divergence calculations, and wherein the vector is received from an automated system. [3] The method of claim 1, including sorting the vector to generate a sorted vector, and performing the divergence calculations based on the sorted vector and each of the plurality of distribution vectors. [4] The method of claim 3, wherein sorting the vector to generate the sorted vector comprises generating a permutation vector to determine an initial order of the entries in the vector. [5] The method of claim 4, including generating a conclusiveness vector from the minimizing distribution vector by applying the permutation vector to the minimizing distribution vector to associate the conclusiveness of the decision with at least one corresponding value in the vector. [6] The method of claim 1, including generating a conclusiveness vector from the minimizing distribution vector to associate the conclusiveness of the decision with at least one corresponding value in the vector. [7] The method of claim 1, wherein determining the conclusiveness of the decision based on the minimizing distribution vector comprises identifying at least one non-zero value in the minimizing distribution vector. [8] The method of claim 7, wherein the conclusiveness of the decision is low if the at least one non-zero value contains a number of non-zero values ​​that exceeds a predetermined threshold. [9] The method of claim 7, wherein the conclusiveness of the decision is high if the at least one non-zero value contains a number of non-zero values ​​below a predetermined threshold. [10] The method of claim 1, wherein each of the plurality of distribution vectors includes at least one non-zero value.

Images