System and method for soft understanding of automated decisions

Abstract

A method of analyzing decisions of an automation system. The method includes receiving a vector having a length. A plurality of distribution vectors are created, each distribution vector having a length that matches the length of the vector. A divergence calculation is performed using the vector and each of a plurality of distribution vectors. A minimum output value determined by the divergence calculation is calculated. A minimum distribution vector is identified from the plurality of distribution vectors, the minimum distribution vector corresponding to a minimum output value from the divergence calculation. In view of minimizing the distribution vector, the conclusion of the decision is determined based on the vector.

Description

Background Art

[0001] The subject disclosure relates to automated decision making and, more particularly, to systems and methods for determining a better understanding of automated decisions.

[0002] Artificial intelligence (AI) can be used in autonomous or self-driving vehicles to determine the actions that the vehicle takes. Automated systems routinely make decisions based on a vector of system options that reflect the selection of an action, with the vector having entries that include values ​​that represent the desirability of selecting the action. The automated system can make a final decision based on the vector by selecting the entry with the maximum value or a value above a predetermined threshold. However, these approaches fail to consider alternative actions. Summary of the invention

[0003] A method for analyzing a decision for an automated system is disclosed herein. The method includes receiving a vector having a length. Creating a plurality of distribution vectors, each distribution vector having a length matching the length of the vector. Performing a divergence calculation with the vector and each of the plurality of distribution vectors. Calculating a minimum output value determined by the divergence calculation. Identifying a minimized distribution vector from the plurality of distribution vectors that corresponds to the minimum output value from the divergence calculation. In view of the minimized distribution vector, determining the conclusiveness of the decision based on the vector.

[0004] Another aspect of the disclosure may be wherein the divergence calculation comprises a Jensen-Shannon divergence calculation and the vector is received from an automated system.

[0005] Another aspect of the present disclosure may include sorting the vector to create a sorted vector, and performing a divergence calculation based on the sorted vector and each of the plurality of distribution vectors.

[0006] Another aspect of the disclosure may be where sorting the vector to create the sorted vector includes creating a permutation vector to determine the original order of entries in the vector.

[0007] Another aspect of the present disclosure may include creating a conclusiveness vector from the minimized distribution vector by applying a permutation vector to the minimized distribution vector to associate the conclusiveness of a decision with at least one corresponding value in the vector.

[0008] Another aspect of the present disclosure may include creating a conclusiveness vector from the minimized distribution vector to associate a conclusiveness of a decision with at least one corresponding value in the vector.

[0009] Another aspect of the disclosure may be where determining the conclusiveness of the decision based on minimizing the distribution vector includes identifying at least one non-zero value in the minimizing the distribution vector.

[0010] Another aspect of the disclosure may be wherein the decision is less conclusive if the at least one non-zero value includes a number of non-zero values ​​exceeding a predetermined threshold.

[0011] Another aspect of the present disclosure may be where the decision is highly conclusive if the at least one non-zero value includes a number of non-zero values ​​below a predetermined threshold.

[0012] Another aspect of the present disclosure may be wherein each of the plurality of distribution vectors includes at least one non-zero value.

[0013] Another aspect of the disclosure may be, wherein at least one non-zero value in each of the plurality of distribution vectors is equal to the multiplicative inverse of the number of non-zero values ​​in a corresponding one of the plurality of distribution vectors.

[0014] Another aspect of the disclosure may be where the number of vectors in the plurality of distribution vectors is less than or equal to the number of entries in the vector.

[0015] Another aspect of the disclosure may be wherein the divergence calculation comprises a Jensen-Shannon divergence calculation and the vector is received from an automated system.

[0016] Another aspect of the present disclosure may be, wherein each of the plurality of distribution vectors is filled with at least one non-zero distribution value in order from left to right.

[0017] A non-transitory computer readable medium containing programming instructions is disclosed herein, and when executed by a processor, the programming instructions are operable to perform a method. The method includes receiving a vector having a length. Creating a plurality of distribution vectors, each distribution vector having a length matching the length of the vector. Performing a divergence calculation with the vector and each of the plurality of distribution vectors. Calculating a minimum output value determined by the divergence calculation. Identifying a minimized distribution vector from the plurality of distribution vectors that corresponds to the minimum output value from the divergence calculation. In view of the minimized distribution vector, determining the conclusiveness of a decision based on the vector.

[0018] A vehicle system is disclosed herein. The system includes a plurality of sensors and a controller in communication with the plurality of sensors. The controller is configured to receive a vector having a length, create a plurality of distribution vectors, each distribution vector having a length matching the length of the vector, and perform a divergence calculation using the vector and each of the plurality of distribution vectors. The controller is also configured to identify a minimum output value determined by the divergence calculation, and to identify a minimized distribution vector corresponding to the minimum output value from the divergence calculation from the plurality of distribution vectors. The controller is also configured to determine the conclusiveness of a decision based on the vector in view of the minimized distribution vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1A vehicle capable of operating in an autonomous mode or an automated mode is shown according to an exemplary embodiment.

[0020] Figure 2 A method of determining an action for an automation system is shown.

[0021] Figure 3 A method of analyzing decisions based on vectors generated from an automated system is shown.

[0022] Some embodiments of the present disclosure will now be described by way of example only and with reference to the accompanying drawings.On all drawings, the same reference number represents the same element or the same type of element. DETAILED DESCRIPTION

[0023] According to an exemplary embodiment, Figure 1 A vehicle 100 is shown that can operate in an autonomous mode or an automated mode. The vehicle 100 can be a fully autonomous vehicle or a semi-autonomous vehicle. The vehicle 100 includes a driving system 102 that controls the autonomous operation of the vehicle. The driving system 102 includes: a sensor system 104 for obtaining information about the surroundings or environment of the vehicle 100; and a controller 106 for calculating possible actions of the autonomous vehicle based on the information obtained, and for implementing one or more possible actions; and a human-machine interface 108 for explaining the actions taken by the vehicle to an occupant of the vehicle (such as a driver or passenger). The sensor system 104 may include devices such as cameras, lidar, radar, GPS, etc.

[0024] The human-machine interface 108 need not be dedicated to interpreting actions, and may be an interface used in the vehicle for purposes other than interpreting actions. The human-machine interface 108 may include one or more interfaces, such as a speaker, a display, a tactile device, etc. Each interface is applicable to a given message modality. For example, a speaker may be used to present an auditory message. A display may be used to present a written message, a chart, or an animation. A tactile device may be used to present a tactile message, such as a vibration, felt by a passenger. In another embodiment, the human-machine interface 108 may be a mobile device that a passenger may use inside or outside the vehicle.

[0025] The controller 106 may include processing circuitry that may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) executing one or more software or firmware programs and memory, combinational logic circuits, and / or other suitable components that provide the functionality described. According to one or more embodiments detailed herein, the controller 106 may include a non-transitory computer-readable medium storing instructions that, when processed by one or more processors of the controller 106, implement a method for determining a soft understanding of an automated decision. The controller 106 may operate a program that executes artificial intelligence. However, the present disclosure is applicable to decision making systems that utilize or generate vectors (such as probability or confidence vectors) in the decision making process.

[0026] Figure 2 A method 200 of operating an automated system, such as the driving system 102 of the vehicle 100, is shown to determine a final decision based on a vector P1. According to an exemplary embodiment, the system 102 generates (block 202) a vector P1, such as a probability vector or a confidence vector, which reflects the current understanding of the system 102 in the decision making process. The vector P1 includes a plurality of entries, and the final decision is reached by automatically selecting one of the plurality of entries from the vector P1. These decisions are typically reached as a result of applying a hard threshold to a soft conclusion (e.g., a probability vector or a confidence vector). The decision resulting from applying the hard threshold, which is typically identified as a hard decision, typically selects the maximum independent variable in the vector P1 and returns it as the decision for the automated system.

[0027] The method 200 analyzes the vector P1 as part of the decision analysis performed at box 206 in the context of the solution process of the system 102 to determine a soft understanding of the system's reasoning process for selecting a final decision, rather than making a hard decision. For example, using an automated driving algorithm, such as Monte Carlo Tree Search (MCTS), the vector P1 is analyzed to determine the dominance of one of the decisions in the vector. The method 200 allows the system 102 to determine whether the vector P1 includes a single dominant action that it is considering performing, two competing actions, or the method 200 may not have determined a clear decision to make from each of the different possibilities in the vector P1.

[0028] Figure 3 An example method 300 is shown that performs decision analysis on vector P1 in block 206 of method 200. The decision analysis performed in block 206 may be utilized in different ways depending on the context of system 102 and the ultimate recipient of the decision (eg, a user or engineer).

[0029] The decision analysis performed by method 300 includes a divergence calculation based on vector P1 and multiple distribution vectors. Method 300 begins by receiving vector P1 from system 102 (box 302). Vector P1 includes multiple entries, each entry has a value representing a decision to be made by the automated system. The following equation 1 represents an example vector P1 received at box 302. Each value in vector P1 corresponds to a possible decision taken by the automated system. In the example shown, the values ​​in vector P1 can correspond to the system 102 determining that the vehicle 100 should turn right, turn left, speed up, slow down, or do not change the action. Under the hard decision making method, the system may select the action corresponding to the highest value, which is "slow down" in this example.

[0030] P1 = (0.05, 0.35, 0.05, 0.45, 0.1) Equation 1

[0031] In one example, method 300 sorts the values ​​in vector P1 in descending order to generate a sorted vector P as shown in Equation 2 below: 1S .

[0032] P 1S =(0.45, 0.35, 0.1, 0.05, 0.05) Equation 2

[0033] The method 300 may also generate a permutation vector P 1P , to maintain a record of the original order of the values ​​in vector P1 after sorting. The following equation 3 shows the order of the values ​​in the sorted vector P1. 1S Example permutation vector P mapped onto the original vector P1 1P .

[0034] P 1P =(4,2,5,1,3) Equation 3

[0035] The method 300 then generates a plurality of distribution vectors at block 304. In one example, the plurality of distribution vectors includes a first set of distribution vectors for using the ordering vector P 1S Perform divergence calculation. The first set of distribution vectors Q j Each distribution vector in (Equation 4) has a length that matches the length of the vector P1 being analyzed.

[0036]

[0037] In Equation 4 above, j is a variable equal to the number of non-zero values ​​in the distribution vector. For each non-zero value in the distribution vector, the non-zero value is equal to the multiplicative inverse of the number of non-zero values ​​in the distribution vector. For example, in the case where j is equal to "1", the equation for Q1 will be expressed as shown in Equation 5 below. In addition, in the example shown for performing the divergence calculation, the non-zero values ​​fill the distribution vector from left to right.

[0038]

[0039] This same method is used to generate distribution vectors Q2, Q3, Q4 and Q5. For distribution vector Q5, the variable j is equal to 5, and Q5 will be expressed as shown in Equation 6 below.

[0040]

[0041] For the illustrated example with P1 having five entries, distribution vectors will be created for integer values ​​of j from "1" to "5" such that the number of distribution vectors equals the number of entries in vector P1. The distribution vector Q5 as shown in Equation 6 also represents a uniform distribution vector for actions, and the distribution vector Q1 represents the maximum possible level of conclusiveness for a decision based on vector P1, as will be discussed in more detail below.

[0042] Once the vector has been generated, the method 300 performs a divergence calculation at block 306. 1S and distribution vector Q1-Q N A divergence calculation is performed between . In one example, the divergence calculation performed is a Jensen-Shannon divergence calculation as shown below in Equation 7. The divergence calculation provides an output value representing a measure of the distance between two data distributions, which shows how different the two distributions are from each other.

[0043]

[0044] The variable M in Equation 7 above is shown in Equation 8 below.

[0045]

[0046] When the method 300 uses the first set of distribution vectors and the sorting vector P 1S When performing divergence calculations, the method 300 identifies a minimum output value from those calculations (block 308). The method 300 may then identify a distribution vector corresponding to the minimum output value (block 310).

[0047] Alternatively, the distribution vectors Q1, ..., Q may be analyzed in order corresponding to increasing values ​​of j. N The method 300 can be based on the vector P 1SBy comparing the distribution vectors Q1, Q2, Q3, Q4 or Q5, it is identified which one of the distribution vectors Q1, Q2, Q3, Q4 or Q5 corresponds to the minimum output value from the divergence calculation. In addition, when the output value of the convergence calculation has increased from the output value of the previous calculation, the local minimum output value can be determined by selecting the previous distribution vector. If the distribution vector associated with the local minimum output value is identified before performing the divergence calculation on the remaining distribution vectors, this will provide a reduction in the computational resources required to perform the divergence calculation in some cases. In addition, the distribution vector associated with the minimum output value can be identified as the minimizing distribution vector because it results in the minimum output value for the divergence calculation.

[0048] Method 300 utilizes minimizing the distribution vector to determine the conclusiveness of the final decision based on vector P1 of system 102. One feature of identifying the minimum output value from the divergence calculation is that it identifies the distribution vector with the most similar distribution. Therefore, method 300 allows system 102 to consider more possible actions from vector P1 that would have been eliminated under the hard decision making method.

[0049] The conclusiveness of an action based on vector P1 can be determined by identifying the number of non-zero values ​​in the minimized distribution vector (block 312). When the number of non-zero values ​​exceeds a predetermined threshold, then the conclusiveness of the decision based on vector P1 is low. For example, if the number of non-zero numbers in the minimized distribution vector is equal to the total number of entries in the distribution vector, then each possible action based on vector P1 will have a similar level of conclusiveness. This is because the minimized distribution vector matches the distribution of the uniform probability vector.

[0050] However, when the number of non-zero values ​​in the minimized distribution vector is below a predetermined threshold, then the conclusiveness of the decision based on vector P1 is high. For example, if the number of non-zero numbers in the minimized distribution vector is equal to one, then the automated system has the highest level of conclusiveness in possibly selecting that entry as the final decision of the system 102.

[0051] Furthermore, if the minimized distribution vector includes multiple non-zero values ​​below a predetermined threshold, the automated system can know with a high degree of certainty that the decisions corresponding to those values ​​should be considered, while the remaining decisions can be ignored. The system 102 can also communicate the decision being considered via the human-machine interface 108 for explaining the action taken by the vehicle 100 to an occupant (such as a driver or passenger), or even as a request for driver intervention.

[0052] Method 300 can determine which values ​​of vector P1 should be considered using soft decision making by matching at least one non-zero value in the minimized distribution vector with at least one corresponding value in vector P1. Those values ​​correspond to soft decision values ​​that the automated system should consider, which may be ignored under hard decision making. 1SWhen the method 300 uses the permutation vector P 1P Map the non-zero values ​​in the minimized divergence vector to corresponding entries in vector P1.

[0053] The method 300 can also create a conclusive vector from the minimized distribution vector. The conclusive vector indicates important choices from vector P1 that should be considered in addition to making hard decisions on vector P1. The conclusive vector matches the length of vector P1 and returns a value of "1" for each entry position corresponding to a non-zero value in the minimized distribution vector. When using the sorted vector P1 in the divergence calculation 1S When the method 300 uses the permutation vector P 1P Map the values ​​in the minimized divergence vector to the corresponding entries in vector P1.

[0054] Using the conclusive vector or minimized distribution vector generated as described in method 300 above, at box 208, method 200 determines whether the decision based on vector P1 is conclusive enough in terms of the conclusive vector or minimized distribution vector. If any of these vectors are conclusive enough, method 200 will proceed to box 210 to provide a final decision. If these vectors are not conclusive enough, method 200 can perform further analysis at box 212 before returning to system 102 to generate new vectors at box 202. System 102 can then generate another vector to be analyzed by method 200 to determine whether the decision based on the vector is conclusive enough to present a final decision. This allows method 200 to incorporate an additional layer of reconsideration into the next action of the planning algorithm, request additional information, or rerun the decision-making algorithm on system 102.

[0055] Although various embodiments have been described, the description is intended to be exemplary and not restrictive. It will be apparent to those of ordinary skill in the art that more embodiments and implementations are possible within the scope of the embodiments. Unless otherwise specifically limited, any feature of any embodiment may be used in combination with or in place of any other feature or element in any other embodiment. Therefore, the embodiments are not limited except in accordance with the appended claims and their equivalents. In addition, various modifications and variations may be made within the scope of the appended claims.

Claims

1. A method for analyzing a decision of an automated system, the method comprising: Receives a vector with length; Create a plurality of distribution vectors, each distribution vector having a length matching the length of the vector; performing a divergence calculation using the vector and each of a plurality of distribution vectors; identifying the minimum output value determined by the divergence calculation; identifying a minimized distribution vector from the plurality of distribution vectors, the minimized distribution vector corresponding to a minimum output value from the divergence calculation; and In view of the minimized distribution vector, a conclusiveness of a decision is determined based on the vector.

2. The method of claim 1, wherein the divergence calculation comprises a Jensen-Shannon divergence calculation and the vector is received from an automated system. 3 . The method of claim 1 , comprising sorting the vector to create a sorted vector, and performing a divergence calculation based on the sorted vector and each of the plurality of distribution vectors.

4. The method according to claim 3, wherein: Sorting the vector to create a sorted vector includes creating a permutation vector to determine the original order of the entries in the vector.

5. The method of claim 4, comprising creating a conclusiveness vector from the minimized distribution vector by applying a permutation vector to the minimized distribution vector to associate the conclusiveness of a decision with at least one corresponding value in the vector.

6. The method of claim 1, comprising creating a conclusiveness vector from the minimized distribution vector to associate the conclusiveness of a decision with at least one corresponding value in the vector.

7. The method of claim 1, wherein determining the conclusiveness of a decision based on the minimized distribution vector comprises identifying at least one non-zero value in the minimized distribution vector.

8. The method according to claim 7, wherein: If the at least one non-zero value comprises a number of non-zero values ​​exceeding a predetermined threshold, the decision is less conclusive.

9. The method according to claim 7, wherein: If the at least one non-zero value comprises a number of non-zero values ​​below a predetermined threshold, the decision is highly conclusive.

10. The method of claim 1, wherein each of the plurality of distribution vectors comprises at least one non-zero value.

Images