Method and system for monitoring changes in ai models

Abstract

Methods and systems for monitoring drift in AI models (i.e., changes in the properties of an AI model, including changes in both internal model characteristics and in the model's use). A current status of the model is received and compared with a previous status of the model. The difference between the current status and the previous status is computed and evaluated against a threshold. If the difference is below the threshold, in this embodiment, no action is taken. If the difference is at or above the threshold, however, at least one action is taken. The action(s) may include sending notification(s) to user(s), displaying one or more warnings, and / or disabling all or portions of the AI model. In some embodiments, the status is determined using a separate AI model. In some embodiments, the threshold is determined using another separate AI model.

Description

METHOD AND SYSTEM FOR MONITORING CHANGES IN Al MODELSTECHNICAL FIELD

[0001] The present invention relates to artificial intelligence (Al) models. More specifically, the present invention relates to monitoring changes in Al models over time.BACKGROUND

[0002] Many Al models “learn” as they are used. That is, as the model is provided with new data by users, that new data is incorporated into the model and affects subsequent outputs from the model. This can be extremely useful. For example, a voice-to-text model can learn to recognize a specific user’s speech patterns and vocal characteristics, thereby producing more accurate transcripts of that user’s speech. However, as is well-documented, this feature of Al models can also raise concerns, as the model may learn based on harmful and / or inaccurate input data, thus contaminating future outputs.

[0003] An example of such contamination is the chatbot Tay, which was released on Twitter™ in 2017. Tay’s earliest communication was fairly neutral, but it was soon the target of large numbers of racist and offensive comments. Within a matter of days, the chatbot was reproducing the racist and offensive material with very little or no prompting and was taken offline.

[0004] Unfortunately, not all changes in models are as easy to spot as the changes that occurred in Tay. Moreover, as Al is increasingly common in many contexts, including as examples immigration review and law enforcement, minor changes in the model overtime may easily manifest in real-world harms. Further, a related phenomenon known as “usage drift” may occur when a model trained for use in one situation or for one type of data is used for another situation or another type of datafor which it has not been trained. The intemal / inherent properties of the model may not have changed in such situations, but as the usage is different, the performance of the model will nevertheless suffer.

[0005] As such, there is a need for automated systems and methods that monitor changes in the model’s properties, including both the use(s) of the model and the intemal / inherent characteristics of the model. Such changes may be generally referred to as “drift” or “model drift”.SUMMARY

[0006] This document discloses methods and systems for monitoring drift in Al models (j.e., changes in the properties of an Al model over time, including internal characteristics and the model’s use). A current status of the model is received and compared with a previous status of the model. The difference between the current status and the previous status is computed and evaluated against a threshold. If the difference is below the threshold, in this embodiment, no action is taken. If the difference is at or above the threshold, however, at least one action is taken. The action(s) may include sending notification(s) to interested user(s), displaying one or more warnings, and / or disabling all or portions of the Al model. In some embodiments, the status is determined using a separate Al model. In some embodiments, the threshold is determined using another separate Al model.

[0007] In a first aspect, this document discloses a method for monitoring changes in properties of an Al model, the method comprising: (a) receiving, at a server, a status of said Al model, said status being representative of a state of the Al model at a specific time; (b) at said server, comparing said status to a previous status of said Al model, to thereby determine a difference between said status and said previous status; (c) at said server, evaluating said difference against a threshold; (d) when said difference is below said threshold, returning to step (a); and (e) when said difference is at or above said threshold, taking at least one action.

[0008] In another embodiment, this document discloses a method wherein said at least one action comprises at least one of: automatically sending a notification to at least one interested user, said interested user being a person with an interest in said properties of said Al model; automatically displaying a warning on an interface, said warning relating to said changes; automatically disabling at least a part of said Al model; and automatically disabling a specific interface for said Al model, said specific interface being for a specific use case or industry.

[0009] In another embodiment, this document discloses a method wherein said status is a numerical score.

[0010] In another embodiment, this document discloses a method wherein said status is a binary encoding.

[0011] In another embodiment, this document discloses a method wherein said status is determined by: providing a specific input to said Al model; and using a separate Al model to assess an output produced by said Al model responsive to said input; and generating, with said separate Al model, a status based on said output, wherein said specific input is used each time said status is determined to thereby track differences in said output over time.

[0012] In another embodiment, this document discloses a method wherein said threshold is a predetermined change tolerance.

[0013] In another embodiment, this document discloses a method wherein said threshold is based on an external regulation or legal requirement.

[0014] In another embodiment, this document discloses a method wherein said threshold is determined by a separate Al model, said separate Al model being trained on previous change data.

[0015] In another embodiment, this document discloses a method wherein step (d) further comprises sending a notification to at least one interested user, said interested userbeing a person with an interest in said properties of said Al model and said notification relating to a level of said difference relative to said threshold.

[0016] In another embodiment, this document discloses a method wherein said properties correlate with at least one of: a reliability of said Al model; an accuracy of said Al model; a bias of said Al model; a fairness of said Al model; a transparency of said Al model; an accountability of said Al model; a performance of said Al model; compliance of said Al model with a predetermined standard; an explainability of said Al model; an interpretability of said Al model; a privacy level of said Al model; ethics of said Al model; a safety of said Al model; a security of said Al model; and a use of said Al model.

[0017] In another embodiment, this document discloses a method further comprising generating a human-readable display of a plurality of differences of over time.

[0018] In another embodiment, this document discloses a method wherein the Al model has been operationally implemented.

[0019] In a second aspect, this document discloses a system for monitoring changes in properties of an Al model, said system comprising: a database for storing statuses of said Al model and for storing a threshold; and a server for: receiving a status of said Al model, wherein said status is representative of a state of the Al model at a specific time; comparing said status to a previous status of said Al model retrieved from said database, to thereby determine a difference between said status and said previous status; evaluating said difference against said threshold; and implementing at least one action to be taken when said difference is at or above said threshold.

[0020] In another embodiment, this document discloses a system wherein said at least one action comprises at least one of: automatically sending a notification to at least one interested user, said interested user being a person with an interest in said properties of said Al model; automatically displaying a warning on an interface, said warning relating to said changes; automatically disabling at least a part of said Al model; andautomatically disabling a specific interface for said Al model, said specific interface being for a specific use case or industry.

[0021] In another embodiment, this document discloses a system wherein said status is a numerical score.

[0022] In another embodiment, this document discloses a system wherein said status is a binary encoding.

[0023] In another embodiment, this document discloses a system wherein said server further comprises an assessment module, said assessment module comprising a separate Al model, and wherein said assessment module determines said status of said Al model by: providing a specific input to said Al model; assessing an output produced by said Al model responsive to said specific input; and generating a status based on said output, wherein said input is used each time said status is determined, to thereby track differences in said output over time.

[0024] In another embodiment, this document discloses a system wherein said threshold is a predetermined change tolerance.

[0025] In another embodiment, this document discloses a system wherein said threshold is based on an external regulation or legal requirement.

[0026] In another embodiment, this document discloses a system wherein said server further comprises a threshold module, said threshold module comprising a separate Al model for determining said threshold and said separate Al model being trained on previous change data.

[0027] In another embodiment, this document discloses a system wherein said processor sends a notification to at least one interested user when said difference is below said threshold, said interested user being a person with an interest in said properties of said Al model and said notification relating to a level of said difference relative to said threshold.

[0028] In another embodiment, this document discloses a system wherein said status correlates to at least one of: a reliability of said Al model; an accuracy of said Al model; a bias of said Al model; a fairness of said Al model; a transparency of said Al model; an accountability of said Al model; a performance of said Al model; compliance of said Al model with a predetermined standard; an explainability of said Al model; an interpretability of said Al model; a privacy level of said Al model; ethics of said Al model; a safety of said Al model; a security of said Al model; and a use of said Al model.

[0029] In another embodiment, this document discloses a system further comprising a display module for displaying results based on said statuses to at least one user of said Al model.

[0030] In a third aspect, this document discloses non-transitory computer-readable media having encoded thereon computer-readable and computer-executable instructions that, when executed, implement a method for monitoring changes in properties of an Al model, the method comprising: (a) receiving, at a server, a status of said Al model; (b) at said server, comparing said status to a previous status of said Al model, to thereby determine a difference between said status and said previous status; (c) at said server, evaluating said difference against a threshold; (d) when said difference is below said threshold, returning to step (a); and (e) when said difference is at or above said threshold, taking at least one action.BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present invention will now be described by reference to the following figures, in which identical reference numerals refer to identical elements and in which:Figure 1 is a flowchart detailing a method according to one aspect of the invention;Figure 2 is a block diagram of a system according to another aspect of the invention;Figure 3A is a block diagram of another embodiment of the system of Figure 2;Figure 3B is a block diagram of another embodiment of the system of Figure 2;Figure 3C is a block diagram of another embodiment of the system of Figure 2;Figure 4 is an exemplary visual display generated by the invention;Figures 5A and 5B are other exemplary visual displays generated by the invention; andFigure 6 is a block diagram of another embodiment of the system of Figure 2.DETAILED DESCRIPTION

[0032] The present invention comprises methods and systems for monitoring drift in Al models (i.e., changes in the properties of an Al model over time, including both internal properties / characteristics, such as internal weights, and external properties, such as the model’s use), as shown in Figure 1. At step 100, a current status of the model is received. At step 110, the current status is compared with a previous status of the model, and a difference between the current status and the previous status is computed at step 120. (The previous status, as should be understood, is preferably retrieved from a database associated with the monitoring system.) At decision 130, the difference between the current status and the previous status is evaluated against a threshold. If the difference is below the threshold, in this embodiment, no action is taken. If the difference is at or above the threshold, however, at least one action is taken at step 140.

[0033] The at least one action comprises at least one of: automatically sending a notification to at least one user of said Al model; automatically displaying a warning on an interface; automatically disabling at least a part of said Al model; and / or automatically disabling a specific interface for said Al model, said specific interface being for a specific use case or industry. For example, if the evaluating step identifies significant and / or significantly concerning model drift when a particular model is used in the law enforcement context, interfaces designed to allow forspecific law enforcement uses of the model may be automatically disabled. However, other uses of the model, such as for tracking time that an anonymized ‘average shopper’ spends in a retail establishment, may be permitted to continue. As well, many other actions may be taken depending on the embodiment. As one example, an enterprise system of an organization implementing the assessed Al model may receive commands from the assessment system (via one or more suitable application programming interface(s), API(s)). For example, if severe drift is detected, such commands may instruct the enterprise system to ‘quarantine’ or ‘freeze’ data received from the Al model since the last drift assessment. This ‘freezing’ / ‘quarantine’ would prevent automated decision-making from accounting for such data until the data and the drift can be further investigated. In some embodiments, such data may be immediately discarded in response to such commands to the enterprise system from the assessment system. Many other possible actions can be taken by way of suitable APIs, and the examples herein should not be construed as limiting the capabilities of the system.

[0034] In some embodiments, the warning displayed on an interface is a warning on an interface used for accessing the Al model itself. In such embodiments, the warning would be visible to end users / users of the Al model being assessed. In other embodiments, the warning displayed on an interface is a warning on an interface used for assessing the Al model — i.e., an interface used to access the present assessment system. In such embodiments, the warning would be visible to users of the assessment tools, such as Al governance personnel, but not to end users / users of the Al model being assessed. Further, in some embodiments, warnings may be displayed on both types of interfaces (i.e., interfaces for accessing the Al model and interfaces for assessing the Al model). In such embodiments, the waming(s) may be displayed at different times or under different circumstances. For example, the interface for an assessment tool may be set to display a warning when the drift of a particular is higher than 3%, while the interface used to access and use the Al model may only display a warning when higher levels of drift, for example 30%, are reached. As should be understood, these levels are purely exemplary and not intended to limit the conditions under which an interface may display a warning.Additionally, warnings displayed on different interfaces / to different users may be different, both in form and content, depending on their intended recipients and purpose.

[0035] The at least one interested user to whom the notification is sent is a person who has an interest in the properties of the Al model. In some embodiments, this interested user is an administrative user responsible for the assessed Al model, such as an Al governance administrator of the organization implementing the assessed Al model. Upon receiving the notification, that administrative user may make further determinations regarding how to proceed. The interested user may also be, depending on the embodiment, a third-party assessor of the Al model, including an academic, a professional assessor, a civil society member such as a journalist, or any other assessing party. The at least one user to whom the notification is sent, depending on the embodiment, may also include end users of the model. For example, logged-in users of systems that incorporate or access the assessed Al model, or users who have accounts on such systems, may receive a notification indicating that the model has drifted from its original configuration and / or may not be suitable for some uses or users. It should be understood that many persons may have an interest in the model’s properties, and the term ‘interested user’ is not intended to limit the scope of parties to whom notifications may be sent.

[0036] The notification may comprise an alarm, a report, or any suitable / desired notification with any suitable / desired level of urgency. Further, the notification may be sent through any suitable communication means, including without limitation, email, SMS, push notifications delivered on a desktop or mobile communications device, communications through third-party messaging applications, telephone calls, etc. Different levels of drift may result in different types or frequency of notifications, as well as different notification content and format. Further, in some embodiments, as mentioned above, different levels of drift may result in notifications being sent to one interested user or type of interested user but not to other interested users.

[0037] The status, in some embodiments, is a numerical representation of a state of the system at a specific time, as represented by the model’s properties. In some embodiments, the numerical representation is a single numerical “score” representing an overall quality of the system. In other embodiments, the numerical representation is a plurality of numerical scores corresponding to various criteria and / or categories. These criteria can include, without limitation, reliability; reproducibility; accuracy; bias; fairness; transparency; accountability; performance; compliance with a predetermined standard (including, for example, a legal or regulatory standard); explainability; interpretability; a privacy level / degree to which the Al model enables data privacy; ethics; safety; security; and the use of the Al model. (For example, a medical triage model may be suited for use in emergency situations but not as part of a non-emergency practice.) As should be understood, in such category-based embodiments, the monitored properties of the Al model may be correlated with these categories and each of the plurality of numerical scores may correspond to a specific one of the categories.

[0038] In other embodiments, the status of the system is a binary encoding / binary vector or tensor that represents various properties of the system.

[0039] In still other embodiments, the status is non -numerical. For example, the status may take the form of qualitative assessments of the model properties or categories of properties, such as “excellent”, “good”, “poor”, or with a colour-coded categorization scheme such as “green”, “yellow”, and “red”, with green corresponding to an acceptable status, yellow corresponding to a warning or potentially unacceptable status, and red corresponding to an unacceptable status. Of course, any suitable categorization scheme can be used, depending on the implementation, including any desired number of categories and any desired naming / labelling convention forthose categories. As should be understood, such qualitative assessments may ultimately depend on quantitative / numerical assessments and / or may be provided / displayed to a user along with quantitative / numerical assessments. Further, separate qualitative assessments may be providedfor individual criteria / assessment categories. For example, a specific model may be assessed as being “excellent” in terms of privacy but “poor” in terms of accuracy.

[0040] In some embodiments, a single overall status may be determined for an assessed model based on a plurality of category-specific scores. The overall status / score may be computed as a weighted average of various scores or in any other suitable manner.

[0041] The status of the model, in some embodiments, is determined using an assessment module. The assessment module provides a specific input to the Al model and receives an output from the Al model based on that specific input. The assessment module then uses a separate Al model to assess the output and generate a current status of the first Al model. The same specific input is provided each time the first Al model is assessed, so that differences in output can be tracked without confounding factors and changes in the model itself may be observed. As should be understood, the separate Al model of the assessment module may determine the properties of the assessed model (and thus determine the status of the assessed model) using numerous Al methods and approaches. These may include, without limitation, natural language processing (NLP), including processing of the assessed model’s output as well as, in some embodiments, processing of metadata and of user feedback related to the assessed model. These methods may also include classification and image processing methods, as applicable, as well as predictive Al methods.

[0042] In other embodiments, the assessments may be performed using non-AI methods or methods that do not incorporate machine learning, including statistical analysis methods.

[0043] In some embodiments where the status is based on assessments of the model output, the status can be obtained without requiring any deeper knowledge of the model itself. That is, obtaining the model status does not necessarily require knowledge of or access to the model’s configuration or code.

[0044] Note that the assessments may be as nuanced as desired for any implementation. For example, a facial recognition system used in law enforcement may be assessed as being ‘excellent’ at recognizing and differentiating young Caucasian men from each other, while simultaneously being assessed as ‘very poor’ at recognizing and differentiating elderly Black women from each other. The degree of nuance / sensitivity of the assessment may depend on the embodiment and on the desired implementation.

[0045] The threshold may be a predetermined value / a predetermined change tolerance level. For example, if the status of the model is a single numerical score, the threshold to take a certain action may be a 5% change in that score. More complex statuses may have more complex thresholds. For example, with multiple numerical scores corresponding to different categories, multiple thresholds may be set. As one example, a ‘bias’ category may have a threshold of 3%, while a ‘reproducibility / reliability’ category has a threshold of 8%. Further, one or more thresholds may be based on external regulations or legal requirements, such as a requirement that observable drift in racial bias not exceed 10% at any time.

[0046] Additionally, the action to be taken may depend, in some embodiments, on the threshold. For example, if the difference is between 2% and 5%, that drift may simply trigger an automatic notification being sent to one or more users, or displayed on an interface of the model. However, a higher difference, such as 20% or higher, may result in the immediate automatic disabling of the model. It should be understood that different actions may be taken based on different thresholds, and that the nature of the thresholds and actions may depend on the implementation and the Al model itself.

[0047] In some embodiments, further, the thresholds and / or actions are determined by a threshold module. The threshold module implements a separate Al model that determines thresholds and / or actions. This Al model is trained on previous drift / change data, which preferably includes information about acceptable levels of drift in different models and use cases. Such a threshold module allows the threshold(s)to be dynamically determined and potentially adapt over time as the model analysis improves.

[0048] In some embodiments, an absolute value of the difference between the current status and the previous status is compared to the threshold. Such embodiments allow for differences to be monitored even when the differences are considered to be “positive”. For example, if the overall bias displayed by a specific model decreases, a numerical score associated with that specific model’s bias category may decrease as well. However, while lower bias is generally preferred, rapid shifts in observed bias may be indicative of other issues in the model. For example, if the bias in a model drops by 75% in one day, this may require further investigation. Accordingly, both “positive” and “negative” drift is monitored in some embodiments.

[0049] That said, depending on the embodiment, differences considered to be “positive” may be ignored, or notifications may only be sent in certain cases. Further, as should be understood, the directionality of “positive” vs. “negative” drift depends on the implementation. As one example, a high score may indicate a high-performing model with few issues around bias and a high level of accuracy. In another example, however, a high score may indicate a high degree of bias and / or a high level of inaccurate data. Many such implementations fall within the scope of the invention, and none of the examples herein should be considered to limit the invention in any way.

[0050] It should be understood that the ‘previous status’, in some embodiments, is a predetermined threshold status, baseline status, or idealized status. For example, if no assessments have yet been performed, the current model status may be compared to a predetermined status.

[0051] Further, changes in the model properties may be monitored in either or both the model’s training data and the model’s operational data. It should be understood that, depending on the model, operational data may be recycled into new training data, and thus monitoring the changes in the model properties may be required to account for such new data on an ongoing basis. The specific configuration of theassessments depends on the embodiment. Further, any type of data (e.g., text, numerical, image, video, audio, etc.) may be used in assessments. As well, the data sources may include synthetic data generated by the assessed Al model, synthetic data generated by other Al models, synthetic data generated by humans, and real- world (“wild”) data collected by means of sensors and / or from human input.

[0052] The changes in the model properties (z. e. , the model drift) may be assessed at any suitable interval, including continuous monitoring or assessments that are performed daily, weekly, quarterly, etc. As well, for specific Al models or specific aspects of the Al models, new statuses may be obtained , compared, and evaluated against the relevant thresholds on model-specific or aspect-specific schedules. For example, facial recognition used in law enforcement may be assessed daily for racial bias and assessed on a weekly basis for accuracy. Many other combinations of assessments and assessment schedules fall within the scope of the invention.

[0053] Figure 2 shows a system 200 for monitoring changes in model properties according to an aspect of the invention. A server 210 is in communication with a database 220. Data related to Al models is stored in the database 220. For example, previous statuses of the Al model(s) to be assessed is stored in the database 220 and retrieved by the server 210 as necessary. The server 210 interfaces with a plurality of monitoring agents 230a, 230b, 230c, within Al models to be assessed 240a, 240b, 240c. As should be understood, any number of Al models may be assessed by the system at one time. Three are depicted for illustrative convenience, but should not be construed as limiting the invention. The server 210 may also interface with the Al models 240a, 240b, 240c directly or using any suitable interface, including purpose-built application interfaces, third-party application programming interfaces (APIs), additional servers or server images, etc. The interfacing may employ any suitable data transfer method, depending on the embodiment.

[0054] The server 210 may be configured in any suitable way. In some embodiments, the server 210 comprises one or more on-premises server units (z.e., physical server units located at a physical location overseen by the system administrators or by users ofthe system 200). In other embodiments, the server 210 comprises one or more cloud-based or remote server units and / or distributed server units. In some embodiments, the server 210 comprises both on-premises and remote server units.

[0055] In some embodiments, as mentioned above, the system 200 further comprises an assessment module for assessing the Al models. A block diagram of such an embodiment is shown in Figure 3A. The server 210 comprises an assessment module 211 which assesses each of the Al models 240a, 240b, 240c, as described above (for example, through monitoring agents 230a, 230b, 230c), to thereby generate a status of each model.

[0056] In some embodiments, the monitoring agents 230a, 230b, 230c collect data from the respective Al models 240a, 240b, 240c and report the data to the assessment module 211, which performs the assessments based on that data and generates statuses for the Al models. In such embodiments, the monitoring agents may comprise comparatively small programs or fdes. In other embodiments, the monitoring agents 230a, 230b, 230c both collect and assess the data. Depending on the embodiment and the assessment techniques used, the monitoring agents may incorporate Al and machine learning methods and may comprise programs of any size.

[0057] In some embodiments, and as shown in the block diagram in Figure 3B, the system 200 further comprises a threshold module 212 for determining one or more suitable thresholds for taking action in response to observed changes in properties (drift). The threshold module 212 may employ Al methods as described above. (Note that, in embodiments without such a threshold module, threshold data may be stored in the database 220 and / or may be transmitted to the server 210 at the time of each assessment.)

[0058] In some embodiments, as shown in the block diagram in Figure 3C, the server 210 comprises both an assessment module 211 and a threshold module 212.

[0059] Further, in some embodiments, a visual display of the drift is provided to one or more interested users. Such a visual display may comprise tabulated or writtenscores / differences in status. Such a visual display may also comprise a plot / chart detailing the changes over time. Some exemplary charts are shown in Figures 4, 5A, and 5B. As can be seen, Figure 4 depicts changes in a plurality of numerical “trust scores”, each trust score being associated with a specific assessment category / aspect. In Figure 4, these assessment categories / aspects are “Bias / Faimess”, “Compliance”, “Data Governance”, “Ethics”, “Explainability & Interpretability”, “Privacy”, and “Technical Performance”. As should be understood, these categories / aspects are purely exemplary and should not be construed as limiting the invention. The central (thicker, red) line is an overall score computed based on all the category scores.

[0060] Figures 5A and 5B show certain aspects in more detail. Figure 5A shows scores for the following categories: “Performance”, “Technical Evaluation”, and “Survey Evaluation”. Figure 5B shows scores for: “Bias / Faimess”, “Technical Evaluation”, and “Survey Evaluation”. That is, these figures show overall performance, a technical score for the model, and a “survey” score for the model. The survey score, in some embodiments, is based on qualitative criteria such as answers to survey questions related to compliance, data privacy, and applicable legal regimes. As well, the survey score may be based on user feedback, survey results, and / or metadata about the model. This feedback and / or qualitative data may be provided by end users, model developers, and any other persons.

[0061] Again, it should be clear that any suitable categories or aspects may be evaluated / monitored by the system and may be displayed to one or more users. These suitable categories or aspects may depend on the specific implementation and / or the specific model to be assessed. Additionally, in some embodiments, the visual display may be user-configurable, thereby allowing users to focus on elements of particular interest to them. Nothing in these exemplary data plots should be considered to limit the scope of the invention in any way.

[0062] As depicted in Figure 6, in some embodiments, the visual displays are provided to user(s) through the user’s personal computing device(s) 250. As should beunderstood, personal computing devices may include without limitation desktop or laptop computers, mobile devices such as smartphones and tablets, wearable devices, and any other device(s) capable of receiving and displaying the visual display data. The visual display may be provided via an Internet browser or within a specific program or application, including a dedicated program or application or a third-party program or application integrated with the system 200. In other embodiments, the visual display is provided / displayed to the user through a separate, dedicated display device.

[0063] As well, the collected statuses and relevant data may be used, in some embodiments, to determine trends in the properties of the Al model. Such trends may indicate that a specific model is becoming more accurate (or less accurate) with prolonged use, or that a specific model is less suited to a certain use case or industry after a period of use than it was at its launch, or any other trend that may be of interest to users.

[0064] As used herein, the expression “at least one of [x] and [y]” means and should be construed as meaning “[x], [y], or both [x] and [y]”.

[0065] It should be clear that various aspects of the present invention may be implemented as software modules in an overall software system. As such, the present invention may thus take the form of computer-executable instructions that, when executed, implements various software modules with predefined functions.

[0066] Additionally, it should be clear that, unless otherwise specified, any references herein to ‘image’ or to ‘images’ refer to a digital image or to digital images, comprising pixels or picture cells. Likewise, any references to an ‘audio file’ or to ‘audio files’ refer to digital audio files, unless otherwise specified. ‘Video’, ‘video files’, ‘data objects’, ‘data files’ and all other such terms should be taken to mean digital files and / or data objects, unless otherwise specified.

[0067] Embodiments of the invention may be executed by a computer processor or similar device programmed in the manner of method steps, or may be executed by an electronic system which is provided with means for executing these steps. Similarly,an electronic memory means such as computer diskettes, CD-ROMs, Random Access Memory (RAM), Read Only Memory (ROM) or similar computer software storage media known in the art, may be programmed to execute such method steps. As well, electronic signals representing these method steps may also be transmitted via a communication network.

[0068] Embodiments of the invention may be implemented in any conventional computer programming language. For example, preferred embodiments may be implemented in a procedural programming language (e.g., “C” or “Go”) or an object-oriented language (e.g., “C++”, “java”, “PHP”, “PYTHON” or “C#”). Alternative embodiments of the invention may be implemented as pre-programmed hardware elements, other related components, or as a combination of hardware and software components.

[0069] Embodiments can be implemented as a computer program product for use with a computer system. Such implementations may include a series of computer instructions fixed either on a tangible medium, such as a computer readable medium (e.g. , a diskette, CD-ROM, ROM, or fixed disk) or transmittable to a computer system, via a modem or other interface device, such as a communications adapter connected to a network over a medium. The medium may be either a tangible medium (e.g., optical or electrical communications lines) or a medium implemented with wireless techniques (e.g, microwave, infrared or other transmission techniques). The series of computer instructions embodies all or part of the functionality previously described herein. Those skilled in the art should appreciate that such computer instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Furthermore, such instructions may be stored in any memory device, such as semiconductor, magnetic, optical or other memory devices, and may be transmitted using any communications technology, such as optical, infrared, microwave, or other transmission technologies. It is expected that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation (e.g., shrink-wrapped software), preloaded with a computer system(e.g. , on system ROM or fixed disk), or distributed from a server over a network (e.g. , the Internet or World Wide Web). Of course, some embodiments of the invention may be implemented as a combination of both software (e.g., a computer program product) and hardware. Still other embodiments of the invention may be implemented as entirely hardware, or entirely software (e.g., a computer program product).

[0070] A person understanding this invention may now conceive of alternative structures and embodiments or variations of the above all of which are intended to fall within the scope of the invention as defined in the claims that follow.

Claims

We claim:

1. A method for monitoring changes in properties of an Al model, the method comprising:(a) receiving, at a server, a status of said Al model, said status being representative of a state of the Al model at a specific time;(b) at said server, comparing said status to a previous status of said Al model, to thereby determine a difference between said status and said previous status;(c) at said server, evaluating said difference against a threshold;(d) when said difference is below said threshold, returning to step (a); and(e) when said difference is at or above said threshold, taking at least one action.

2. The method according to claim 1, wherein said at least one action comprises at least one of: automatically sending a notification to at least one interested user, said interested user being a person with an interest in said properties of said Al model; automatically displaying a warning on an interface, said warning relating to said changes; automatically disabling at least a part of said Al model; and automatically disabling a specific interface for said Al model, said specific interface being for a specific use case or industry.

3. The method according to claim 1, wherein said status is a numerical score.

4. The method according to claim 1, wherein said status is a binary encoding.

5. The method according to claim 1, wherein said status is determined by: providing a specific input to said Al model; using a separate Al model to assess an output produced by said Al model responsive to said input; and generating, with said separate Al model, a status based on said output,wherein said specific input is used each time said status is determined to thereby track differences in said output over time.

6. The method according to claim 1, wherein said threshold is a predetermined change tolerance.

7. The method according to claim 1, wherein said threshold is based on an external regulation or legal requirement.

8. The method according to claim 1, wherein said threshold is determined by a separate Al model, said separate Al model being trained on previous change data.

9. The method according to claim 1, wherein step (d) further comprises sending a notification to at least one interested user, said interested user being a person with an interest in said properties of said Al model and said notification relating to a level of said difference relative to said threshold.

10. The method according to claim 1, wherein said properties correlate with at least one of: a reliability of said Al model; an accuracy of said Al model; a bias of said Al model; a fairness of said Al model; a transparency of said Al model; an accountability of said Al model; a performance of said Al model; compliance of said Al model with a predetermined standard; an explainability of said Al model; an interpretability of said Al model; a privacy level of said Al model; ethics of said Al model; a safety of said Al model;a security of said Al model; and a use of said Al model.

11. The method according to claim 1, further comprising generating a human-readable display of a plurality of differences of over time.

12. The method according to claim 1, wherein the Al model has been operationally implemented.

13. A system for monitoring changes in properties of an Al model, said system comprising: a database for storing statuses of said Al model and for storing a threshold; and a server for: receiving a status of said Al model, wherein said status is representative of a state of the Al model at a specific time; comparing said status to a previous status of said Al model retrieved from said database, to thereby determine a difference between said status and said previous status; evaluating said difference against said threshold; and implementing at least one action to be taken when said difference is at or above said threshold.

14. The system according to claim 13, wherein said at least one action comprises at least one of: automatically sending a notification to at least one interested user; automatically displaying a warning on an interface, said warning relating to said changes; automatically disabling at least a part of said Al model; and automatically disabling a specific interface for said Al model, said specific interface being for a specific use case or industry.

15. The system according to claim 13, wherein said status is a numerical score.

16. The system according to claim 13, wherein said status is a binary encoding.

17. The system according to claim 13, wherein said server further comprises an assessment module, said assessment module comprising a separate Al model, and wherein said assessment module determines said status of said Al model by: providing a specific input to said Al model; assessing an output produced by said Al model responsive to said specific input; and generating a status based on said output, wherein said input is used each time said status is determined, to thereby track differences in said output over time.

18. The system according to claim 13, wherein said threshold is a predetermined change tolerance.

19. The system according to claim 13, wherein said threshold is based on an external regulation or legal requirement.

20. The system according to claim 13, wherein said server further comprises a threshold module, said threshold module comprising a separate Al model for determining said threshold and said separate Al model being trained on previous change data.

21. The system according to claim 13, wherein said processor sends a notification to at least one interested user when said difference is below said threshold, said interested user being a person with an interest in said properties of said Al model and said notification relating to a level of said difference relative to said threshold.

22. The system according to claim 13, wherein said status correlates to at least one of: a reliability of said Al model; an accuracy of said Al model; a bias of said Al model;a fairness of said Al model; a transparency of said Al model; an accountability of said Al model; a performance of said Al model; compliance of said Al model with a predetermined standard; an explainability of said Al model; an interpretability of said Al model; a privacy level of said Al model; ethics of said Al model; a safety of said Al model; a security of said Al model; and a use of said Al model.

23. The system according to claim 13, further comprising a display module for displaying results based on said statuses to at least one interested user.

24. Non-transitory computer-readable media having encoded thereon computer-readable and computer-executable instructions that, when executed, implement a method for monitoring changes in properties of an Al model, the method comprising:(a) receiving, at a server, a status of said Al model;(b) at said server, comparing said status to a previous status of said Al model, to thereby determine a difference between said status and said previous status;(c) at said server, evaluating said difference against a threshold;(d) when said difference is below said threshold, returning to step (a); and(e) when said difference is at or above said threshold, taking at least one action.

Images