Method and system for blending feature values ​​- Patents.com

Abstract

To harmonize a value of a feature.SOLUTION: With a computer-implemented method, each value is provided by a different artificial intelligence (AI) system. The method includes the steps of: collecting a log from a different AI system, each log including a value of the feature; identifying discrepancy between the values if any; generating global information from the value when there is discrepancy, the global information taking into account part or all of the values; and transmitting the global information to the AI system when the global information is different from a value of the AI system of one of the AI systems.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the use of artificial intelligence (AI) or autonomous systems. The present invention has many applications in everyday life and industry. [Background technology]

[0002] Situations involving different AI systems making autonomous decisions are becoming more and more frequent in society. These pose new challenges, problems and therefore the need for new solutions to protect and promote the well-being of people and society in general.

[0003] The increasing use and adoption of AI systems has prompted society to consider ethical considerations for AI. To date, several governments, industries, and academic institutions have developed their own principles, guidelines, and frameworks for ethics in AI.

[0004] Furthermore, several technical aspects to address ethics in AI systems have been explored and published, such as mechanisms to ensure responsibility and accountability for AI systems and their outcomes before and after their development, deployment and use. This includes auditability, minimization and reporting of negative impacts, trade-offs, and remediation. These are explicitly linked to the EU's "Ethics Guidelines for Trustworthy AI", which include four main principles (respect for human autonomy, prevention of harm, fairness, and explainability).

[0005] Closely linked to the principle of fairness is the requirement of accountability, which can be seen as a set of actions enabling developers to: prepare for the unexpected (actions are taken to prevent or control unexpected situations), prepare for error scenarios (actions to prevent or control error scenarios), deal with errors (handle errors in the software), and ensure data security (ensuring the cybersecurity of the system and the safe handling of data). Summary of the Invention [Problem to be solved by the invention]

[0006] It is desirable to mitigate potential harms that may result from the increased use of multiple AI systems. Also, identifying information discrepancies coming from AI systems is not limited to preventing "harm", but also includes discrepancies, confusion, and errors in the sets of data (information) coming from disparate AI systems. Errors, confusion, and discrepancies can range very widely from errors in information or design or manufacturing, but should not be limited to these examples. [Means for solving the problem]

[0007] According to an embodiment of an aspect of the present invention, there is provided a computer-implemented method of reconciling values ​​of features, each value being provided by a different artificial intelligence (AI) system, the method including collecting logs from the different AI systems, each log including a value of the feature, identifying any discrepancies between the values, and if there is a discrepancy, generating global information from the values, the global information taking into account some or all of the values, and sending the global information to one of the AI ​​systems when the global information differs from the value of that AI system.

[0008] Putting the principles and ethical requirements into practice is a challenging task. The embodiments of the invention provide a first step towards allowing accountability of AI systems. In particular, the embodiments focus on preventing harm caused by errors / faults in an AI system(s) that are detected using the collective knowledge of other AI systems presented in the same context (physical setting). For this, traceability of the system and reconciliation of conflicting information are essential.

[0009] Prior art is known to focus on accountability and to detect errors to avoid negative consequences of AI systems, and AI or deep learning techniques have been applied to create defect detection systems and calibrate sensors.

[0010] To avoid negative effects, other prior art techniques prevent AI actors from getting out of control in order to reduce harm. This solution is based on analyzing information from the AI ​​system to detect when the AI ​​is out of control and avoid negative outputs and harm to humans.

[0011] Unlike embodiments of the present invention, the prior art does not have access to information from other AI systems to augment knowledge from other perspectives and improve detection of contradictory information.

[0012] The embodiments address the analysis and reconciliation of a context (situation or environment) where there are multiple AI systems with different perceptions / information of one (or a set of) characteristics or features in the context, and these perceptions may cause negative effects.

[0013] Values ​​herein may include numerical values ​​or classifications, or any other form of value. One example of a context in which embodiments may be applied is a road (or a scenario defined by a set of coordinates) where an accident between two or more vehicles or other objects may occur because one of the two or more vehicles or other objects perceives an incorrect location or classification of the object (such as one of the vehicles, or a person, or an inanimate object). Another example is a smart factory or other industrial space where various robots make autonomous decisions to generate products, but defective products may be generated because the robots incorrectly classify materials.

[0014] To solve the problems in the prior art, an embodiment of the present invention provides a resolution system that resolves conflicting contexts based on traceable context information of an AI system presented in a context, identifies discrepancies in received information, and calculates new global information (GI), which can be used to resolve discrepancies and avoid negative events in the context.

[0015] In particular, embodiments address the identification of discrepancies regarding conflicting context information (i.e., location, distance, classification, etc.) perceived by the AI ​​systems; the calculation of new values ​​in the form of Global Information (GI) for features where there is a discrepancy between the AI ​​systems; and the use of this new global information to resolve this discrepancy in real time, thus avoiding negative effects in the context through technical measures. Examples of technical measures may include deactivating an AI component. In the context of an autonomous vehicle, other technical measures may change the behavior of the AI-controlled vehicle or alert the operator / manager. For example, alerts or actions such as "brake", "stop in 100 meters", "move forward 100 meters", etc. All of these have the effect of ignoring / ignoring said information / device.

[0016] The global information can be presented to an administrator of the AI ​​system with any discrepancies, used to adjust values ​​provided by the system, or used in any other manner. In some embodiments, the global information replaces values ​​provided by the AI ​​system with values ​​from the global information that are more trusted values. In other embodiments, values ​​from the global information may be used to adjust "local" values ​​provided by the system.

[0017] To provide a more sophisticated calculation, each value used to create the global information may be modified, for example, by a confidence score for the AI ​​system that provided that value. Thus, when there is no clear single discrepancy, i.e. when there are many AI systems and different types of conflicting information are received, the method results in utilizing the confidence scores to provide a clearer discrepancy. That is, if system 1 and system 2 see different values, but 1 has a higher confidence, 2 should use the information closer to 1.

[0018] The confidence score may be formed by adding together a number of different partial confidence scores, which are preferably weighted individually.

[0019] The trust score may be based on AI ethics guidelines. For example, if an AI system is known not to follow such guidelines, the AI ​​system may receive a lower score. Other factors, such as the AI ​​system's past discrepancies, may be taken into account. Additionally or alternatively, certain values ​​may be given greater weight. For example, values ​​that indicate potentially dangerous situations (e.g., high speeds or anomalous values) or classifications of vulnerable objects (people rather than vehicles) may be weighted with a factor greater than 1 or given greater weight than other values.

[0020] The method may further include deactivating the portion of the AI ​​system that provides the value that is replaced by the global information.

[0021] The log may provide a stream of data values, which may typically or occasionally be time-separated values ​​for the same feature. The method may further include a first step of checking whether there are changes in the values ​​in the log before identifying discrepancies between the values.

[0022] The method may further include generating a report based on the log and the global information and providing the report to an auditor.

[0023] In one embodiment, the global information of a feature F is:

number

[0024] As previously mentioned, the method can be used with any AI system, whether financial or related to other non-physical data, that provides a value for the same feature, hi one embodiment, the AI ​​system determines the location of an object or identifies an object, and the feature is the determined location of the object or the identity of the object.

[0025] An AI system that receives the global information may control the movement and / or position of an entity, and receiving the global information may result in a change in the movement and / or position of the entity. For example, the AI ​​system may be part of a vehicle, and the global information may be translated into an action, such as changing lanes or braking the vehicle.

[0026] In this context, and as mentioned above, a feature may be the location of an object. The simplest reconciliation is the addition of any numerical attribute and division by the number of AI systems. For example, the X, Y (and Z) coordinates may be added separately. In an alternative embodiment, the reconciliation between values ​​may use triangles that may be formed between the locations of any three AI systems, all triangles that contain the object within the triangle, partial global information for each triangle is calculated from the three AI systems within that triangle, and then an overall global information is calculated by combining the partial global information for each triangle.

[0027] According to an embodiment of a further aspect of the invention, there is provided a system for reconciling values ​​of features, each value being provided by a different artificial intelligence (AI) system, the system having a memory and a processor configured to provide: an information collector for collecting logs from the different AI systems, each log including a value of the feature; a discrepancy auditor for identifying any discrepancies between the values ​​and, in the event of a discrepancy, generating global information from the values, the global information taking into account the values; and a global information activator for transmitting the global information to an AI system to replace the value provided by the AI ​​system when the global information differs from the value of the AI ​​system.

[0028] A (trusted) AI network may include a group of autonomous AI systems linked to a resolution system as defined above.

[0029] According to an embodiment of a further aspect of the present invention there is provided a computer program comprising instructions which, when executed by a computer, cause the computer to carry out the method as set out above.

[0030] An apparatus or computer program according to preferred embodiments of the present invention may include any combination of method aspects. Methods or computer programs according to further embodiments may be described as computer-implemented in that they require processing and memory functionality.

[0031] Apparatus according to the preferred embodiments is described as being configured or arranged to perform, or simply "perform" certain functions. This configuring or arranging may be done using hardware, middleware, or any other suitable system. In the preferred embodiment, the configuring or arranging is by software.

[0032] Thus, according to one aspect, there is provided a program comprising instructions which, when loaded into at least one computer, configures the computer to become a device according to any of the above device definitions or any combination thereof.

[0033] According to a further aspect, there is provided a program comprising instructions which, when loaded into at least one computer, configures said at least one computer to perform method steps according to any of the above method definitions or any combination thereof.

[0034] In general, a computer may include the recited elements as configured or arranged to provide the defined functionality. For example, the computer may include memory, processing, and network interfaces.

[0035] The invention may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them.

[0036] The present invention may be implemented as a computer program or computer program product, i.e. a computer program tangibly embodied in a non-transitory information carrier, for example a machine-readable storage device, or in a propagating signal, for execution by or to control the operation of one or more hardware modules.

[0037] The computer program may be in the form of a stand-alone program, computer program parts or multiple computer programs, may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, whether as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a data processing environment. A computer program may be deployed to be executed on one module or on multiple modules at one site, or distributed across multiple sites and interconnected by a communication network.

[0038] The method steps of the present invention may be performed by one or more programmable processors executing a computer program to perform the functions of the present invention by operating on input data and generating output. Apparatus of the present invention may be implemented as programmed hardware or as special purpose logic circuitry including, for example, an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0039] Processors suitable for the execution of a computer program include, by way of example, both general purpose and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions coupled to one or more memory devices for storing instructions and data.

[0040] The invention is described with respect to specific embodiments. Other embodiments are within the scope of the following claims. For example, the steps of the invention may be performed in a different order and still achieve desirable results. Multiple test script versions may be compiled and invoked as a unit without using object-oriented programming techniques; for example, the elements of a script object may be organized in a structured database or file system, and the operations described as being performed by a script object may be performed by a test control program.

[0041] Elements of the invention have been described using terms such as "information collectors," "conflict auditors," etc. Those skilled in the art will understand that such functional terms and their equivalents may refer to spatially separate but combined parts of the system that perform the defined functions. Similarly, the same physical part of the system may provide two or more of the defined functions.

[0042] For example, separately defined means may be implemented using the same memory and / or processor, where appropriate.

[0043] Preferred features of the present invention will now be described, purely by way of example, with reference to the accompanying drawings, in which: [Brief description of the drawings]

[0044] [Figure 1] 1 is a flowchart of a method in one general embodiment. [Diagram 2] FIG. 1 is a block diagram of major system components in a typical embodiment of the present invention. [Diagram 3] FIG. 1 is a block diagram of a particular invention embodiment. [Figure 4] FIG. 1 is an overview sequence diagram of a solution system. [Diagram 5] 13 is a flowchart illustrating the workflow of the Discrepancy Auditor. [Figure 6] 1 is a table of collected information. [Figure 7] 11 is a table showing an exemplary output. [Figure 8] 1 is a flowchart illustrating the workflow of a Global Information (GI) Activator. [Figure 9] 1 is a flowchart showing the workflow of the confidence calculator. [Figure 10] FIG. 1 is an overhead view of a use case scenario. [Figure 11] 1 is a graph of a use case scenario. [Figure 12] Here is a JSON example of the information sent by each object. [Figure 13] 1 is a table of information received by the discrepancy auditor. [Figure 14] 1 is a graph showing the barycentric coordinates of the use case scenario. [Figure 15] FIG. 2 is a diagram of suitable hardware for implementation of invention embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0045] AI systems can make autonomous decisions in various situations, and these decisions may evoke negative consequences (i.e., accidents, discrimination, etc.) on people and society. AI systems are considered as black boxes or opaque software tools whose behavior is difficult to interpret, explain, and modify.

[0046] Independently analyzing and correcting the behavior of AI systems is a major technical challenge that needs to be solved to provide for the safe, beneficial, and fair use of AI.

[0047] The inventors have come to realize that collective knowledge from different systems can help determine why certain actions were taken, identify possible errors or unexpected outcomes, and avoid negative impacts of AI in specific situations or contexts.

[0048] Thus, one improvement provided by an embodiment of the present invention is a way to analyze a set of AI systems in a given context where negative impacts may occur. This analysis is based on using data from different systems to identify discrepancies between the systems / data, and can detect which systems have errors or are responsible for these discrepancies, and even responsibly correct this discrepancy to avoid negative impacts. Furthermore, the penalization and possibility of correction of AI systems allows the minimization of risks in future contexts with similar information. In this sense, correction may be the deactivation of the defective component and the use of the GI transmitted by the system. Penalization is more related to the confidence in the defective AI component. Deactivating an AI component may result in a decrease in the confidence score.

[0049] In an embodiment, accountability focuses on analyzing a particular context (situation or perception) in which two or more AI systems exist to understand or explore who or what is responsible for the occurrence of a potential event (generally with negative impact) or incident in this context, and preferably applying mechanisms to prevent harm, reduce negative impact, and minimize the risk of repeating the event. Thus, for example, an erroneous location or other characteristic determined by an AI system can be corrected.

[0050] In other words, the embodiments disclose methods to analyze and correct the behavior of different AI systems in a given context where negative effects may occur. They are based on tracking data from different systems to identify discrepancies between them, detecting the sources (blame) of these discrepancies, and remediating the situation to resolve such discrepancies for self-control or auditability.

[0051] Remediation mechanisms are technical measures that enable the responsible correction of AI components / systems, which may thus eliminate or minimize the risk of negative impacts in that context.

[0052] One major benefit is that by considering the output of an AI system, it is possible to trace back erroneous or inconsistent inputs that may produce negative effects in context. In this way, by tracing collective knowledge from different AI systems, it is possible to clarify why an AI system behaved in a certain way, identify possible errors, correct them, and eliminate or minimize future unexpected outputs due to the auditability of the AI ​​system.

[0053] Embodiments focus on the analysis and reconciliation of conflicting information that arise from contexts or situations (e.g., physical conditions or settings) where two or more AI systems exist and negative effects may occur because one of the systems may misinterpret certain features of the context.

[0054] Figure 1 shows a general embodiment in which values ​​from different AI systems are reconciled. In step S10, logs are collected from those AI systems, where each AI system may repeatedly send logs. Each log is a set of values ​​for the same set of features (such as physical coordinates or other data points). The resolution system may align these logs in time and by features.

[0055] In S20, the resolution system identifies any discrepancies between values ​​of the same feature (and at the same point in time) from these different logs. For efficiency, the resolution system may first check whether there has been a change in value, and if not, the method may be aborted.

[0056] In S30, the systems generate Global Information (GI) from the values ​​(if there are discrepancies). For example, if four systems give a value and a fifth system gives a different value for the same frequency, a GI is generated. The GI may be a kind of average or weighted average of those values, or may be calculated in other ways from some or all of those values. For example, each AI system may be given a confidence score, which gives it a weight (e.g., between 0 and 1).

[0057] In S40, the GI is sent to any discrepant AI systems. In fact, the GI may be sent to all AI systems, and each AI system may contain detectors to identify differences between its perception and the received GI. These detectors may simply prioritize the GI (over its perception) if there is any discrepancy. Thus, the AI ​​system adopts the GI.

[0058] Figure 3 shows an overview of the solution system used together with the AI ​​system and its interaction with external elements. On the left side, the different AI systems and the auditor are shown in context. On the right side is the system itself.

[0059] In more detail, the elements of the figure are as follows:

[0060] ● A system for resolving conflicting contexts arising from AI systems : This may be an independent computer system to which the AI ​​systems send information in real time, or it may be part of an existing server that is already in communication with the AI ​​systems. In essence, the resolution system receives information (logs) from a collection of two or more AI systems, detects discrepancies between the received information, and calculates global information (GI). The global information may be used to activate or deactivate specific components of those AI systems to avoid negative effects in the context (e.g., the physical setting in which those AI systems operate). Additionally, the system can generate dynamic reports that include a summary of the exchanged information and the technical instructions sent to the AI ​​systems to avoid negative events in the context. This report can be analyzed to allow an auditor or an external entity to take action on the AI ​​systems (e.g., redesign an AI component, replace a technical device (i.e., a sensor), etc.).

[0061] ● AI system An AI system is an autonomous system that makes autonomous decisions in a defined context. An AI system is a smart object, an object equipped with different types of sensors that allow a virtual awareness of its surroundings, a processor and connectivity that improves interaction with other smart objects and external servers. These AI systems send different logs (information about a set of features) to a resolution system in order to resolve contradictory perceptions of the context arising from the AI ​​systems, and receive instructions in return that can be translated into technical actions to avoid negative effects in the context. Examples of possible AI systems are (or are part of) robots, autonomous vehicles (whether on land, water or in the air), drones, industrial bots, virtual assistance, home automation components, etc.

[0062] ● AuditorAn optional external entity may analyze (manually or automatically) the reports or information generated by the resolution system in order to resolve conflicting perceptions of context arising from AI systems and to correct or prevent certain behaviors in next versions or future AI systems.

[0063] Detailed Description of Invention Embodiments Three major components within the resolution system 10 are described in more detail below and include:

[0064] ● Inconsistency Auditor 40 : This component is fed by an information collector that collects logs from the AI ​​systems 20 and also receives historical data. It has three modules for identifying discrepancies, estimating perceptions, and calculating Global Information (GI) using the information perceived by the AI ​​systems.

[0065] ● Confidence Score Generator 60 This component may use a multi-method approach to calculate a trust score for each AI system using heuristics, majority voting, and nearest neighbor modules, along with historical data, and may perform trust calculations based on stored standards or guidelines, such as ethically-harmonized trust calculations based on published AI ethics guidelines. These may also be trust calculations based on other standards, e.g., health and safety / critical systems, where the standards / guidelines apply. For example, the entire system may be made ethically compliant (not just the trust score calculation) by following appropriate guidelines and incorporating such guidelines into the inherent functionality / mechanisms of the system. For example, traceability (i.e. tracking where data originates and logging a trail) may be one of the key requirements. This also leads to accountability of the system with regard to flawed data (source) and remediation capabilities. The system may include a knowledge database of ethical guidelines that includes definitions of the principles to be adhered to.

[0066] ● GI Activator 50: This component sends technical measures or commands to individual AI systems, attempting to resolve conflicts in the context.

[0067] In addition, the solution system includes other elements for the support of the overall system, the detailed description of which is beyond the scope of this document: ● Information Collector 30 : It can receive inputs (logs) from different AI systems and perform various processes such as formatting and rounding to normalize and standardize the information received. ● Report Generator 70 : Allows external systems or users to check information received, generated and sent by the resolution system. ● Historical Data 80 This local or remote storage component stores and keeps track of all information received, generated, and transmitted by the various components of the system.

[0068] The main sequence diagram of the solution system according to an embodiment is shown in FIG.

[0069] The sequence diagram begins at step S100, where a set of AI systems in the context send logs. In step S110, the information collector processes the logs and sends new information to the inconsistency auditor. In step S120, the inconsistency auditor requests a historical confidence score for each AI system based on certain characteristics (e.g., taking into account previous deactivations) and uses this information to calculate a global information (GI). In step S130, the GI is calculated for use and storage as historical data. The inconsistency auditor then activates the GI Activator (S140) and Inconsistency Score Generator (S150) processes.

[0070] The GI activator, in step S160, sends instructions to the AI ​​system, if necessary, to deactivate AI components of the AI ​​system that have erroneous information (different from the GI).

[0071] The trust score generator requests heuristic data (S170) and updates the system's trust scores, potentially using a database of standards / published AI ethics guidelines and ensuring they are ethically aligned based on a multi-method approach (S180). This information is then stored in a history database.

[0072] The processes and methods for each component are then detailed.

[0073] Information Collector 30 This component may be required to perform any of the following functions: - Receive information from different AI systems, · Normalize and standardize, Store raw and processed information; Resubmit the processed information of a particular event to the Discrepancy Auditor. An event can be defined as something that occurs (or is perceived) by an AI system within a given time. It is a set of perceptions that the AI ​​system observes in a context.

[0074] Historical Data 80 This component may be a database that stores all information and data received, transmitted, and processed by the system. For example, historical data may include: Logs or raw data sent from the AI ​​system GI information calculated by the system to resolve conflicting contexts arising from AI systems An ethically-aligned trust score for each AI system for each feature etc.

[0075] Inconsistency Auditor 40 The discrepancy auditor is programmed to detect discrepancies between the received information and to compute a Global Information (GI) for each discrepant feature, which is used to avoid negative effects in the context.

[0076] The workflow of the Discrepancy Auditor is shown in Figure 5. At the top there is a table with events defined by n features and recognized by m systems.

[0077] The process begins when the discrepancy auditor receives information about AI systems that may be being processed by the information collector.

[0078] The received information is a matrix of size m×n, where m is the number of AI systems in the context and n is the number of features detected by each AI system. For example, the matrix may have the form shown in Figure 6, where the first column is the ID of the AI ​​system and the value A_B is the Bth feature of the Ath AI system.

[0079] Next, the workflow steps compare the received log with previous logs stored by the historical data component to identify changes (S200). If there are changes (S210), the discrepancy auditor performs the following tasks for each feature to be reviewed in the received information (S220). First, the discrepancy auditor compares the values ​​of that feature in all received logs (S230). If a different value is detected (S240), global information is calculated using the confidence scores from the historical data (S250). This information is stored and retained in memory (S260) to be sent to the confidence score generator and GI activator. Once all features have been examined (no features to review S220), the process ends. Two outputs can be generated: an empty list when no changes from the previous received log are detected or there are no discrepancies in the recognition, or a list of discrepant features and new global information calculated by the component.

[0080] An example output when feature 1 is detected by AI systems and there is a discrepancy between the values ​​of this feature is shown in Figure 7.

[0081] The calculation of the global information (GI_1 for feature 1) may be calculated as follows:

number

[0082] So in this example, more trustworthy AI systems (higher ethically-aligned trust scores and / or fewer deactivations) will be weighted more heavily for a particular feature than less trustworthy AI systems. R,1 If ≠ 1, the denominator is m. Otherwise, CS R,1 For each AI system with ≠ 1, CS R,1 are subtracted, the results are added together and subtracted from m to form the denominator.

[0083] GI Activator The GI Activator receives the Global Information (GI) for each feature calculated by the Conflict Auditor and sends instructions to different AI systems, thereby initiating a workflow (shown in FIG. 8) to avoid negative effects in the context. Once the GI Activator receives the data from the Conflict Auditor, the component checks in step S300 whether there is an item in the received list. An item in the list indicates that there is a discrepancy for the particular feature. If a new GI is calculated, this indicates that a discrepancy has been detected.

[0084] If there is a discrepancy, for each feature and associated GI, the GI activator compares the GI in S310 with the information recognized by each AI system (for which historical data is examined).

[0085] If a difference is found between the GI and the recognized information in step S320, the GI activator sends an instruction to the specific AI system in S330 to deactivate that AI component that recognizes features that differ from the global information.

[0086] for example: There are three AI systems A, B, and C. A and B detect an object as a car. C detects the same object as a pedestrian. The global information computed by the inconsistency auditor, taking into account the reliability of each AI system, determines that the object is a car. The GI activator sends a command to AI system C to stop its classification component and take into account the computed GI:car as a feature for classification.

[0087] The command to stop the classification component (or any other component that generates bad values ​​in an AI system) can take any suitable form. For example, the solving system can atomically produce technical commands that the AI ​​system must interpret. Technical commands can have the format "subject + command + arguments". · Target: The component (component of the system) to which it should be applied. · Order: The technical outcome for the problematic AI component (stop, update, continue, etc.). Argument: the value to apply. This means for example the GI (or correction based on GI) to apply.

[0088] GI may be applied in technical instructions that can be translated into actions in each AI system (e.g., change lanes, brake a car, etc.) These actions can avoid negative consequences in the context (i.e., accidents).

[0089] The AI ​​component in question may include a mechanism to update its functionality based on the GI, in the form of recalibration or "auto-learning."

[0090] Confidence Score Generator The trust score generator provides an ethically-harmonized trust score for each AI system for each feature by using a database of standard / published AI ethics guidelines based on a multi-method approach.

[0091] The Confidence Score Generator workflow begins when the Discrepancy Auditor finishes its workflow and sends global information and processed AI system information (the received processed log with details as perceived by each AI system). Using this information, the Confidence Score Generator uses various techniques to calculate a set of partial confidence scores.

[0092] Here, three parallel processes may be run to calculate the three partial confidence scores, although other techniques may be added or only one or two techniques may be used. ● Heuristic Methods : Historical data is taken into account to calculate a partial confidence score for each feature of each AI system. Various techniques can be used to implement this method. For example, a machine learning or deep learning model can be created using historical data to predict the reliability of an AI system, and a set of AI models can be used to calculate the reliability of each feature. ● Majority voting: It takes into account what other AI systems detect and perceive for a particular feature. So, if the majority of AI systems detect a particular value for a feature, the partial score associated with that majority vote is increased. Otherwise, the confidence score is decreased. ● nearest neighbor : A scale or summation is applied on features that have a collection of infinite norms. Scales can be applied to numerical features (i.e., distance) and categorical features (e.g., in classifying objects, pedestrians are more important than trees or animals), with a scale related to relevance.

[0093] After computing the three partial confidence scores, the confidence score generator computes a final confidence score for each AI system for each feature. The formula for system K might be: cs k =a*cs k,heuristic +b*cs k,majority +c*cs k,neighbour Where: cs k is the final score of AI system k on a particular feature, cs k,heuristic is the partial confidence score using the heuristic method, cs k,majority is the partial confidence score using the majority voting method, cs k,neighbour is the partial confidence score using the nearest neighbor method, a, b, c are scores that give greater or lesser importance to previous partial scores.

[0094] In summary, the workflow for computing the confidence score is shown in Figure 9.

[0095] The heuristic method S400, majority voting method S410 and nearest neighbor method S420 are run in parallel using GI and system information. The heuristic methods also use historical data. The results for these methods are combined in S430 to give a final confidence score that is stored in the historical data.

[0096] Working example This example relates to a transportation use case.

[0097] Specifically, consider a road in which there are different autonomous vehicles (AI systems with a set of AI components for detecting objects, object location and classification, etc.).

[0098] Some common AI systems for detecting location are based on radar, GPS, LIDAR, and inertial measurement units. Examples can be found in Non-Patent Document 1. [Non-Patent Document 1] https: / / en.wikipedia.org / wiki / Vehicle_tracking_system

[0099] Accidents can occur because one or more vehicles mislocate an object (e.g., a pedestrian) or the AI ​​system misclassifies a pedestrian. An exemplary scenario is shown in Figure 10. Assume north is shown at the top. Car 1 is heading north onto an east-west highway, exiting a side road. Car 2 is heading east on the highway towards the side road. Car 3 is heading west on the highway towards the side road. Car 4 is heading west on the highway and is turning south off the highway onto the side road. Person 1 is on the highway just east of the side road and car 4.

[0100] For this working example, consider the scenario graph shown in Figure 11.

[0101] In Figure 11, the four cars and the person are objects shown as points. The solid lines with arrows represent the direction of the objects. For example, car 1 moves north and car 2 moves east, as explained above.

[0102] The dashed lines are the cars' perception (vision) of Person 1. For example, cars 1 and 2 see Person 1 in the correct position (the dashed lines end in the correct place), but cars 3 and 4 have a false perception of Person 1 and believe him to be at the end of the dotted line.

[0103] Taking into account the above information, the scenario is as follows: Car 1 and Car 2 detect Person 1 in the correct position. Car 3 and Car 4 detect Person 1 in the wrong position.

[0104] First, to resolve conflicting information coming from AI systems, vehicles (or AI systems) send the information they perceive to an information collector. The information collector component normalizes and standardizes the received logs into the same format and ensures that the received logs are of the same context and time. For example, Car 1 sends the information shown in Figure 12 in JSON format (only the relevant information is shown in Figure 12).

[0105] The information indicates that car 1 at position (428,400) sees or detects person 1 at position (600,200).

[0106] The discrepancy auditor receives this information from all the cars. For each feature it detects if there is a discrepancy. The component detects a discrepancy in positions for person 1 since car 3 detects person position (400,450) and car 4 detects position (600,400).

[0107] The x-direction is the direction the car is moving in relative to the x-axis. For example, in this table, car_1 is not moving on the x-axis and is heading north (this is included in the y-direction feature, not shown).

[0108] The information received by the discrepancy auditor is shown in FIG.

[0109] To compute the global information for Person 1's location, the discrepancy auditor takes into account the information of each vehicle (in this example, a car) and its previous confidence score.

[0110] This working example is based on a method inspired by the coordinates of the center of gravity of a person in relation to the different triangles that the vehicle can define. The reliability and accuracy of this method is higher than simply combining all the inputs. Creating a triangle is a way of checking the position perceived by each vertex.

[0111] First, the conflict auditor detects which triangles defined by the car could contain Person 1 in or near it (considering a margin such as 40% of the area). In this example, four triangles are detected, as shown in Figure 14: ·car_1car_3car_4→Triangle 1 ·car_1car_3car_2→Triangle 2 ·car_1car_4car_2→Triangle 3 ·car_3car_4car_2→Triangle 4

[0112] For each triangle, a partial global information (GI) is computed. The partial global information (GI) for features x, y and triangle t is:

number

[0113] Considering that all cars have a confidence of 1 for each feature, except car 3, which has a confidence of 0.4 for feature x, the partial global information for each triangle is: ·Triangle 1(car_1car_3car_4)

number

number

number

number

[0114] Finally, the global information about the person's position is the median of all the triangles.

number

[0115] The GI activator receives the calculated GI and compares it with the information the cars received from the AI ​​system. Since the GI differs from the received log of the AI ​​system, the GI activation may deactivate the AI ​​components of all cars to avoid an accident. If the last calculation is used, only car 3 will deactivate its AI components for features x and y.

[0116] 15 is a block diagram of a computing device, such as a data storage server, that may be used to implement the method of resolving value conflicts between AI systems embodying the present invention. The computing device has a processor 993 and memory 994. Optionally, the computing device also includes a network interface 997 for communicating with other computing devices, such as other computing devices of the invention embodiments.

[0117] For example, an embodiment may be comprised of a network of such computing devices. Optionally, the computing devices also include one or more input mechanisms, such as a keyboard and mouse 996, and one or more display units, such as a monitor or monitors 995. These components may be connected to each other via a bus 992.

[0118] The memory 994 may include a computer-readable medium, which term may refer to a single medium or multiple media (e.g., centralized or distributed databases and / or associated caches and servers) configured to carry computer-executable instructions or store data structures. Computer-executable instructions may include, for example, instructions and data accessible by a general purpose computer, special purpose computer, or special purpose processing device (e.g., one or more processors) to cause them to perform one or more functions or operations. Thus, the term "computer-readable storage medium" may include any medium capable of storing, encoding, or carrying a set of instructions for execution by a machine and causing the machine to perform any one or more of the methods of the present disclosure. Thus, the term "computer-readable storage medium" may be interpreted to include, but is not limited to, solid-state memory, optical media, and magnetic media. For example, and without limitation, such computer readable media may include non-transitory computer readable storage media including random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), compact disc read only memory (CD-ROM) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, and flash memory devices.

[0119] The processor 993 is configured to control the computing device and perform processing operations, for example, executing code stored in memory to implement various different functions of modules such as the information collector, the discrepancy auditor, and the global information activator described herein. The memory 994 stores data that is read and written by the processor 993. As referred to herein, the processor may include one or more general-purpose processing devices, such as a microprocessor, a central processing unit, etc. The processor may include a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or a combination of instruction sets. The processor may also include one or more special-purpose processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. In one or more embodiments, the processor is configured to execute instructions to perform the operations and steps discussed herein.

[0120] The display unit 997 may display representations of data stored by the computing device (e.g., values ​​from an AI system), and may also display cursors and dialog boxes and screens that allow interaction between a user and programs and data stored on the computing device. The input mechanism 996 may allow a user to input data and instructions into the computing device.

[0121] The network interface (network I / F) 997 may be connected to a network such as the Internet and may be connected to other computing devices (e.g., the AI ​​system itself) via the network. The network I / F 997 may control data input / output from / to other devices via the network. Other peripheral devices such as microphones, speakers, printers, power units, fans, cases, scanners, tracker balls, etc. may be included in the computing device.

[0122] The information collector 30 may include processing instructions stored in a portion of the memory 994, the processor 993 executes the processing instructions, and the portion of the memory 994 stores values ​​during execution of the processing instructions. The output of the information collector may be stored in the memory 994 and / or a connected storage unit, or may be transmitted / transferred / communicated to a historical data store / report generator.

[0123] The discrepancy auditor 40 may include processing instructions stored in a portion of the memory 994, which the processor 993 executes and which portion of the memory 994 stores data relating to discrepancies identified during execution of the processing instructions. The output of the discrepancy auditor may be stored in the memory 994 and / or a connected storage unit, or may be transmitted / transferred / communicated to a historical data store / report generator.

[0124] The global information activator 50 may include processing instructions stored in a portion of the memory 994, which the processor 993 executes and which portion of the memory 994 stores the results of the calculations and the GI during execution of the processing instructions. The output of the global information activator may be stored in the memory 994 and / or a connected storage unit, or may be transmitted / transferred / communicated to a historical data store / report generator.

[0125] A method embodying the invention may be performed on a computing device such as that shown in Figure 15. Such a computing device need not have all the components shown in Figure 15, but may consist of a subset of these components. A method embodying the invention may be performed by a single computing device in communication with one or more data storage servers over a network. The computing device may be a data storage device that stores said values ​​and GIs and any intermediate results.

[0126] Methods embodying the present invention may be performed by a plurality of computing devices operating in cooperation with one another, one or more of which may be data storage servers that store at least a portion of the AI ​​system / solution system data.

[0127] summary The embodiments propose a system and method for integrating accountability into existing AI systems. The above description is domain independent, so it can be applied to different business domains such as finance, transportation, robotics, etc., where two or more AI systems participate.

[0128] AI systems can be considered as black boxes even for their developers. Understanding and exploring errors and erroneous information is a major technical challenge and must be accompanied by avoidance of negative consequences. New mechanisms must be defined to achieve this.

[0129] Embodiments disclose mechanisms for retroactively tracing erroneous or inconsistent inputs by considering the outputs of an AI system; systems and methods for detecting and correcting errors in AI systems based on knowledge of other systems; and mechanisms for preventing harm, reducing negative impacts, and minimizing the risk of repeating adverse events.

Claims

1. 1. A computer-implemented method for blending feature values, each value provided by a different artificial intelligence (AI) system, the method comprising: collecting logs from the different AI systems, each log including values ​​of the features; identifying any discrepancies between said values; and if there is a discrepancy, generating global information from the values, the global information taking into account some or all of the values, each value used to generate the global information being modified by a confidence score for the AI ​​system that provided the value, the confidence score being formed from a number of different partial confidence scores that are added together, the partial confidence scores including two or more of a partial confidence score using a heuristic method, a partial confidence score using a majority voting method, and a partial confidence score using a nearest neighbor method; and if the global information differs from the value of one of the AI ​​systems, and transmitting the global information to the AI ​​system. method.

2. The method of claim 1 , wherein the global information replaces values ​​provided by the AI ​​system with values ​​from the global information.

3. The heuristic method calculates a partial confidence score for a feature of the AI ​​system taking into account historical data; The majority voting method calculates a partial confidence score of an AI system for a feature by taking into account whether a majority of the AI ​​systems detect a particular value for the feature; Nearest neighbor methods compute partial confidence scores for features with scale or features with summation-infinite norm collections. The method of claim 1.

4. The method of claim 1, wherein the partial confidence scores are individually weighted.

5. The method of claim 3 or 4, wherein the confidence score is based on AI ethics guidelines.

6. 6. The method of claim 1, further comprising deactivating parts of the AI ​​system that provide the values ​​that are replaced by the global information.

7. 7. The method of claim 1, further comprising a first step of checking whether there is a change in a value in the log before identifying discrepancies between the values.

8. The method of claim 1 , further comprising: generating a report based on the log and the global information; and providing the report to an auditor.

9. The global information about a feature F is [0097] It is calculated as follows: m is the number of AI systems that have information about feature F, cs k,F is the stored confidence score of the AI ​​system with ID k for feature F, valuek_F is the value of AI system k for feature F, 9. The method according to any one of claims 1 to 8.

10. 10. The method of claim 1, wherein the AI ​​system determines a location of an object or identifies an object, and the feature is the determined location of the object or an identity of the object.

11. The method of claim 10 , wherein the AI ​​system that receives the global information controls movement and / or position of an entity, and receipt of the global information results in a change in movement and / or position of the entity.

12. 12. A method according to any one of claims 1 to 11, wherein the feature is the position of an object and a convergence between values ​​can be formed between the positions of any three AI systems, using all triangles that contain the object within that triangle, partial global information for each triangle is calculated from the three AI systems within that triangle, and then an overall global information is calculated by combining the partial global information for each triangle.

13. 1. A system for blending feature values, each value being provided by a different artificial intelligence (AI) system, the system having a memory and a processor, the processor comprising: an information collector that collects logs from different AI systems, each log including values ​​of the features; a discrepancy auditor that identifies discrepancies, if any, between the values ​​and, if any discrepancies exist, generates a global information from the values, the global information taking into account the values, each value used to generate the global information being modified by a confidence score for the AI ​​system that provided the value, the confidence score being formed from a number of different partial confidence scores that are added together, the partial confidence scores including two or more of a partial confidence score using a heuristic method, a partial confidence score using a majority voting method, and a partial confidence score using a nearest neighbor method; a global information activator that transmits the global information to an AI system when the global information differs from the value of the AI ​​system, replacing the value provided by the AI ​​system; configured to provide system.

14. A trusted AI network comprising the system of claim 13 and a plurality of autonomous AI systems linked to the system.

15. A computer program comprising instructions which, when said program is executed by a computer, cause said computer to carry out a method according to any of claims 1 to 12.

Images