Methods and systems for assessing artificial intelligence models

Abstract

Computer-implemented methods of assessing performance accuracy of an artificial intelligence, AI, model are provided. Methods comprise: providing, by an assessment server, an assessment dataset to a cloud-based secure enclave, wherein a copy of the AI model is stored on the cloud-based secure enclave, and wherein the cloud-based secure enclave defines a communicatively isolated processing environment within the network. The method further comprises: receiving output data from the secure enclave, wherein the output data has been generated by applying the AI model to the assessment dataset within the secure enclave. The method further comprises: determining, based on the output data, an accuracy score for the AI model by comparing the generated output data to ground truth data associated with the assessment dataset.

Description

[0001] METHODS AND SYSTEMS FOR ASSESSING ARTIFICIAL INTELLIGENCE MODELS

[0002] Field of the Invention

[0003] The present invention relates to systems and methods for providing a secure service for assessing Al- based models and particularly, although not exclusively, to systems and methods adapted to provide a secure benchmarking service for Al-based models. In one example, the invention may be used to assess and / or benchmark Al models that have been trained to analyse facial images, for example, for the purpose of behavioural / emotional / attentiveness response determination, or face verification / identification.

[0004] Background

[0005] Algorithms and models leveraging the capabilities of artificial intelligence (Al) have become increasingly more prevalent and deployed in increasingly wide and varied applications as the immense potential of Al has been realised. Together with this surge in the number and variety of Al models and algorithms available to be used across a broad range of applications, there has been a correspondingly rapid increase in the number of providers and developers of such Al models.

[0006] In other words, not only are there more models available to be deployed in a wider range of contexts, but these models can be obtained from a much larger pool of developers - i.e., the variety of sources from which Al models and algorithms can be obtained has surged along with the number and variety of available models.

[0007] With such a wide range of algorithms from such disparate sources, it is becoming increasingly difficult (and important) for a user of an algorithm to trust that a model that they have selected to use meets their requirements. Each developer (i.e., model source) may provide validation information indicative of the performance and / or other properties associated with the model, but this is a largely unregulated process, meaning that the user of the model must take the developer’s assertions at face value. Additionally, because there are no universally adopted standard metrics or tests, it is difficult, and sometimes impossible, to compare models from different sources. This can make it prohibitively difficult and / or expensive for a user to compare a large number of models from a large number of different sources, thereby restricting the user’s choice to just a few models with which they are familiar. This is clearly unsatisfactory because it reduces the user’s confidence that they are using the model best suited or adapted to their needs.

[0008] Standard benchmarking procedures to date have typically relied upon centrally curated processing centres to which models are submitted. For example, a processing centre at the U.S. National Institute of Standards and Technology (NIST) rigorously tests a submitted model against a predefined set of guaranteed-quality benchmarking data and, after some period of time, publishes the results of their analysis (see e.g., the NIST Face Recognition Vendor Test (FRVT) databases). For example, NIST benchmarks submitted face-recognition algorithms against government-issued and government-collected images depicting a variety ef faces in the traditional face forward, uniform background, “passport-photo” format, such as that illustrated schematically in Figure 1 .

[0009] The present invention has been devised in light of the above considerations.

[0010] Summary of the Invention

[0011] In a general sense, the present invention provides a framework and processes to securely receive and assess Al models, and to make the results of that assessment publicly accessible to potential users in such a way that the potential users can compare different models and select the model that best suits their needs. The assessment may be used a way to benchmark a model with respect to previous models that have been assessed.

[0012] Current benchmarking services typically use so-called “perfect” data to benchmark Al models. For example, in the context of face recognition, the NIST database uses government-collected images that are all (or nearly all) in the traditional “passport-photo” format, as shown in Figure 1 . Models are therefore benchmarked against data of uniform type and quality. As such, the results of any benchmarking cannot capture the robustness of a model to variations in the quality of the input data. For example, in the context of face recognition and facial image processing, current databases fail to quantify how models perform when the data input to the model is less-than-perfect - e.g., in cases where the lighting of the facial images or the angle at which the image is captured means that a face depicted in the image is fully or partially occluded.

[0013] The present invention may therefore provide an Al model assessment system that is configured to analyse and report on the robustness of models submitted thereto.

[0014] Further, current benchmarking services require the developer of a model to trust the processing centre with the detailed workings of the model - including its source code. This is clearly undesirable as the developer is no longer able to control the security of their model and a malicious third party could illicitly obtain their model if the benchmarking service suffers from a data breach or other lapse in security. The present invention therefore provides an Al model assessment system where the provider of the assessment service never has access to the full details (e.g., the source code) of the submitted model.

[0015] Conversely, current benchmarking services do not permit model developers to review the operations and processes (including, for example, the sourced code implemented to assess the Al model submitted for assessment). As a consequence of this, model developers have to blindly trust the processing centre assessing their model without being able to verify that the assessment itself is fit for purpose. The present invention therefore may provide an Al model assessment system where model developers have access to (e.g., read-only access to) the operational details, such as the source code, of the computerexecutable script used to assess the Al model.

[0016] In some examples, it may be preferable for the model developer to review and agree to the details of code used to assess the Al model. For example, the Al model assessment system may provide the model developer with a copy of a computer-executable script (referred to herein as a cloud formation script) so that the model developer and review and - importantly - approve the details of the model assessment process. In this way, for example, the model developer can be assured that the Al model assessment system will not, for example, be able to gain access to the source code illegitimately (i.e., steal proprietary information belonging to the model developer).

[0017] In this way, the data security of the assessment system is significantly enhanced. Simultaneously, model developers are made party to the assessment process, thereby facilitating a community- and trust-based approach to model assessment.

[0018] In a first aspect there is provided a computer-implemented method of assessing performance accuracy of an artificial intelligence, Al, model. The method comprises: providing, by an assessment server, an assessment dataset to a cloud-based secure enclave. A copy of the Al model is stored one the cloudbased secure enclave. The cloud-based secure enclave defines a communicatively isolated processing environment within the network. The method further comprises receiving output data from the secure enclave. The output data is generated by applying the Al model to the assessment dataset within the secure enclave. The method further comprises determining, based on the output data, an accuracy score for the Al model by comparing the generated output data to ground truth data associated with the assessment dataset.

[0019] The ground truth data may be data that is indicative of a model output that corresponds to an output that would be generated by an Al model that is perfectly (i.e., 100%) accurate.

[0020] A secure enclave is to be understood to mean a communicatively isolated processing environment within the overall networked environment that the methods disclosed herein are carried out within. In particular, the secure enclave (interchangeably referred to as a network enclave) may be understood as being a part of a network that is subdivided and isolated from the rest of the network. For example, the enclave may consist of standalone resources that do not interact with systems, networks, or resources outside of the enclave. The enclave may comprise a firewall that cannot be traversed by any communication protocols. Alternatively, in some examples, the enclave may effectively take the form of a so-called demilitarized zone (DMZ) that permits a preselected and predefined number of communication protocols to traverse the firewall.

[0021] As discussed above, the establishment of a secure enclave and the assessment of the Al model within said enclave is particularly advantageous as it provides the model owner with full security such that neither the the assessment provider is not able to access the full details (e.g., the respective source codes) of the model owner’s programming / software, but the model owner is able to review and verify a cloud formation script that is executable to establish the cloud-based secure enclave. In this way, the secure enclave enhances the security of the assessment system and methods described herein.

[0022] In some examples, the determining of the accuracy score may be carried out by the cloud-based secure enclave. In other examples, the determining of the accuracy score may be carried out by a separate assessment server maintained by the provider of the Al model assessment system. In some embodiments, the cloud-based secure enclave may be established on a cloud server associated with the client device.

[0023] In other words, the methods disclosed herein may be implemented (e.g., by the assessment provider) using computing resources provided or otherwise arranged (e.g., through a trusted third party) by the client device. This may be particularly beneficial because it allows for the methods disclosed herein to be implemented simultaneously (or in quick succession - i.e., consecutively) across multiple client devices. Each Al model may be assessed within its own bespoke cloud-based secure enclave meaning that each client device may determine the amount of computing resources to be dedicated to the assessment of the corresponding Al model. For example, the client device, having its own cloud-based server (e.g., the Amazon Web Service, AWS; Microsoft’s Azure; the Google Cloud Platform; the IBM Cloud; and similar other cloud-based systems), may determine how much of its own cloud resources to allocate to an assessment operation - i.e., the methods disclosed herein. In this way, the client device (or the user thereof) is provided with the flexibility to allocate any number of resources to control the amount of time that the assessment process takes. For example, allocating comparatively more resources may result in a comparatively quicker processing time for the methods disclosed herein.

[0024] In some examples, it may be possible for the client device to request that the assessment of the Al model is carried out based on only a subset of the assessment dataset. For example, it may be desirable to run a testing assessment to check that the overall assessment is correctly executable (e.g., a testing assessment could be implemented as part of a debugging process).

[0025] In some examples, the assessment server may be able to limit a model owner’s access to the assessment protocols described herein. In other words, the assessment server may be configured to prevent the methods described herein from being applied to the same Al model more than a predetermined number of times. For example, applying the same assessment to the same Al model repeatedly may cause the Al model to train itself to overfit to the assessment dataset. Therefore, the assessment server, or the cloud-based secure enclave may be configured to safeguard against this overfitting. For example, the cloud formation script may include instructions to prevent the assessment protocols being applied to the Al model if those assessment protocols (using the same assessment dataset) have previously been applied to that Al model more than a predetermined number of times (e.g., 1 time or more, 2 times or more, 5 times or more, 10 times or more, or 20 times or more).

[0026] Further, by distributing each iteration of the method to the cloud servers of each of any number of client devices, the processing burden of the methods disclosed herein is removed from the assessment provider and handed out to each of the client devices on demand. In this way, it is made significantly less costly and more feasible to provide an assessment service because the requirements placed on assessment providers in the state of the art to provide sufficient computing resources (historically in the form of vast swathes of computing banks storing several processors) is removed. Further, the model owner benefits from a greater sense of security as the assessment protocols are implemented within the model owner’s own cloud environment such that the assessment provider never receives the model owner’s actual model. In some embodiments, the method may further comprise: establishing the secure enclave in response to the received request to assess the Al model.

[0027] In some examples, the client device may establish the secure enclave on, for example, its own cloudbased server.

[0028] In some examples, it may instead be the assessment server that establishes the secure enclave. For example, the client device may provide the assessment server with the necessary permissions and instructions (e.g., in the form of an executable script) to be able to establish the secure enclave on a cloud-based server associated with the client device.

[0029] In some examples, it may instead be a trusted intermediary (or other third party) that establishes the secure enclave. For example, the secure enclave may be provided by a trusted cloud-service provider able to provide confidential computing resources - e.g., the Amazon Web Services, or similar.

[0030] The cloud-based secure enclave may be established my executing a cloud formation script. The cloud formation script may be an executable script reviewed, agreed upon, and locked (against post-agreement editing) by both the model owner and the assessment system provider. Additionally or alternatively, the cloud formation script may comprise instructions that, when executed within the secure enclave, cause the Al model stored within the secure enclave to be assessed / bench marked against the assessment dataset, as is set out in relation to the methods described herein. For example, the cloud formation script may include instructions that provide processors within the secure enclave the directory addresses to access both the Al model and the assessment dataset, and instructions that set out the operations that the Al model is to be configured to apply to the assessment dataset for the purposes of assessing / benchmarking the performance of the Al model.

[0031] In some embodiments, the method may further comprise: providing, by the client device, a copy of the Al model to the cloud-based secure enclave.

[0032] In some examples, the cloud-based secure enclave may download the copy of the Al model from the client device. In some examples, the client device may upload the copy of the Al model to the cloudbased secure enclave.

[0033] By providing the enclave with a copy of the Al model and isolating the remaining parts of the methods disclosed herein within the cloud-based enclave it is possible to ensure the data security of the assessment process as discussed above.

[0034] In some examples, the cloud-based secure enclave may be effectively formed on one or more clusters of a cloud-based server to which a copy of the Al model has been allocated for storage. In effect, the enclave may be formed around the copy of the Al model, enclosing the copy and communicatively isolating it from the assessment server so that the assessment server cannot access the full details (e.g., the source code) of the model.

[0035] In some embodiments, establishing the secure enclave may comprise: providing, by the assessment server, a first encryption key; providing, by the client device, a second encryption key; and establishing the secure enclave as an isolated computing environment having first and second encryption keys stored therein. Providing the assessment dataset to the cloud-based secure enclave may comprise providing a copy of the assessment dataset encrypted with the first encryption key. Providing the copy of the Al model may comprise providing a copy of the Al model encrypted with the second encryption key. The encrypted copies of the assessment dataset and Al model may be decryptable within the cloud-based secure enclave.

[0036] For example, the enclave may effectively be established around the first and second encryption keys, thereby communicatively isolating both of the encryption keys. In this way, the only party external to the enclave in possession of the first encryption key may be the assessment server, while the only party external to the enclave in possession of the second encryption key may be the client device. In this way, it is possible for the client device to transmit an encrypted copy of the Al model across the firewall of the enclave safe in the knowledge that the encrypted copy of the Al model is only decryptable (i.e., retrievable) from within the enclave. Similarly, it is possible for the assessment server to transmit an encrypted copy of the assessment dataset to the enclave safe in the knowledge that the encrypted copy of the assessment dataset is only decryptable (i.e., retrievable) from within the enclave.

[0037] In some examples, the assessment server may be provided with the necessary permissions and instructions (e.g., in the form of one or more executable scripts) to be able to apply the copy of the Al model to the assessment dataset within the cloud-based secure enclave. This may also be understood as the assessment server being equipped to test the Al model against the assessment dataset within a so- called confidential computing environment.

[0038] In some embodiments, the method may further comprise retrieving the assessment dataset from a data repository.

[0039] The data repository may be a database configured to store multiple examples of raw data useable in the methods disclosed herein to assess the performance accuracy of Al models submitted by client devices.

[0040] In some examples, the data repository may comprise one or more partitions configured to divide the data stored therein into categories. For example, in the context of assessment methods adapted to assess the performance accuracy of Al models configured for the analysis of facial images and / or videos, the data repository may be partitioned such that the data stored therein may be categorised according to the standard of data privacy / security satisfied by the corresponding piece of data.

[0041] In some embodiments, the data repository may be remote from the assessment server.

[0042] For example, the data repository may be stored on a cloud-based server that is communicatively connected with the assessment server. In other examples, the data repository may be stored at a same physical site as the assessment server.

[0043] Cloud-based servers may be particularly beneficial because they facilitate the distribution of the data repository and the resources required to implement the assessment server across a cloud-based system to reduce the costs and resource requirements associated with maintaining the assessment server and / or the data repository. In some embodiments, retrieving the assessment dataset from the data repository may comprise: selectively combining one or more raw datasets from amongst a plurality of raw datasets stored in the data repository to define the assessment dataset. Each raw dataset may have associated therewith: ground truth data for comparison with output data generated by the Al model when applied to the raw dataset for determining the accuracy score of the Al model.

[0044] As discussed above, the ground truth data may be data that is indicative of a model output that corresponds to an output that would be generated by an Al model that is perfectly (i.e., 100%) accurate.

[0045] The accuracy score of the Al model may, in some examples, be determined as an aggregate or average of a series of accuracy scores determined for each of the raw datasets that constitute the assessment dataset.

[0046] In some embodiments, each raw dataset may further have associated therewith a quality score indicative of a quality of the raw dataset. The trigger may be further configured to cause the processor to: determine, based on the output data, a robustness score that is indicative of variation in the accuracy score across a range of quality scores associated with the one or more raw datasets constituting the assessment dataset.

[0047] In this way, it may be possible to determine how well the accuracy score of the benchmarked Al model is maintained as the quality of the input data input into the Al model is varied.

[0048] The robustness score (equivalently referred to as the variation score) may be indicative of the robustness of the model - i.e., how well the precision of the model is maintained as the quality of the input data decreases. This may be expressed, for example, as the precision of the model when the input data has a quality score of, e.g., 0.8 (or any other selected score) divided by the precision of the model when the input data has a quality score of e.g., 1 . The robustness score may be expressed in any other form as long as it provides information indicative of how well the performance of the model is maintained as the quality of the input data deteriorates. The robustness score may, in some examples, be expressed as multiple scores so that the rate of decay in performance of the model vs. the rate of decay in the quality of the input data can be inferred from the robustness score.

[0049] In some embodiments, each raw dataset may further have associated there with a context label indicative of a category of data of the raw dataset. The trigger may be further configured to cause the processor to: determine, based on the output data, an equity score that is indicative of variation in the accuracy score across a range of context labels associated with the one or more raw datasets constituting the assessment dataset.

[0050] The equity score (equivalently referred to as the fairness score) may be indicative of the fairness, or equitable accuracy, of the model - i.e., how well the accuracy / precision of the model is maintained as the qualitative nature of the input data is varied. For example, in the context of facial image analysis models, the equity score may be indicative of how well the accuracy of the Al model is maintained for different demographics of subjects depicted in facial images. The equity score may be expressed in any form that provides the user with information indicative of how the accuracy of the Al model varies for different context labels. For example, the equity score may - in some examples - be expressed as multiple scores, or a function that defines a relationship between the accuracy of the model and changes in the context labels of the raw datasets. In some examples, the equity score may be based on the absolute values of the determined accuracy scores for each raw dataset while in other examples, the equity score may be based on a ratio of the determined accuracy scores for each raw dataset with a reference (e.g., a maximum) accuracy score.

[0051] In another aspect, there is provided a computer-implemented method of assessing performance accuracy of an artificial intelligence, Al, model. The method comprises: receiving, at an assessment server from a client device over a network, a request to assess an Al model held by the client device; and selectively combining, from a data repository, one or more raw datasets from amongst a plurality of raw datasets stored in the data repository to define an assessment dataset. Each raw dataset has associated therewith: (i) ground truth data for comparison with output data generated by the Al model when applied to the raw dataset for determining an accuracy score of the Al model, and (ii) a quality score indicative of a quality of the raw dataset. The method further comprises: applying the Al model to the assessment dataset to generate output data; determining, based on the output data, an accuracy score for the Al model by comparing the generated output data to ground truth data associated with the assessment dataset; and determining, based on the output data, a robustness score that is indicative of variation in the accuracy score across a range of quality scores associated with the one or more raw datasets constituting the assessment dataset.

[0052] In this way, this aspect of the invention provides a means for determining not only the accuracy of an Al model but also how robust said accuracy / precision is against variations in the quality of input data input into the Al model.

[0053] It may be beneficial to selectively combine multiple datasets to form the assessment dataset so that the assessment dataset comprises data that matches one or more assessment criteria set by the model owner. For example, the model owner may request that their Al model be assessed for accuracy in a specific use-context (e.g., analysing facial images from a particular type of camera, such as a smartphone’s camera, a webcam, a CCTC camera, or other suitable example). In such a case, the assessment dataset may be formed by selectively combining data from a plurality of different datasets such that the assessment dataset only comprises data that matches the specific use-context required by the model owner.

[0054] The robustness score (equivalently referred to as the variation score) may be indicative of the robustness of the model - i.e. , how well the precision of the model is maintained as the quality of the input data decreases. This may be expressed, for example, as the precision of the model when the input data has a quality score of, e.g., 0.8 (or any other selected score) divided by the precision of the model when the input data has a quality score of e.g., 1 . The robustness score may be expressed in any other form as long as it provides information indicative of how well the performance of the model is maintained as the quality of the input data deteriorates. The robustness score may, in some examples, be expressed as multiple scores so that the rate of decay in performance of the model vs. the rate of decay in the quality of the input data can be inferred from the robustness score.

[0055] In some embodiments of said aspect, each raw dataset may further have associated therewith a context label indicative of a category of data of the raw dataset. The method may further comprise: determining, based on the output data, an equity score that is indicative of variation in the accuracy score across a range of context labels associated with the one or more raw datasets constituting the assessment dataset.

[0056] The equity score (equivalently referred to as the fairness score) may be indicative of the fairness, or equitable accuracy, of the model - i.e., how well the accuracy / precision of the model is maintained as the qualitative nature of the input data is varied. For example, in the context of facial image analysis models, the equity score may be indicative of how well the accuracy of the Al model is maintained for different demographics of subjects depicted in facial images. The equity score may be expressed in any form that provides the user with information indicative of how the accuracy of the Al model varies for different context labels. For example, the equity score may - in some examples - be expressed as multiple scores, or a function that defines a relationship between the accuracy of the model and changes in the context labels of the raw datasets. In some examples, the equity score may be based on the absolute values of the determined accuracy scores for each raw dataset while in other examples, the equity score may be based on a ratio of the determined accuracy scores for each raw dataset with a reference (e.g., a maximum) accuracy score.

[0057] In a further aspect, there is provided a computer-implemented method of assessing performance accuracy of an artificial intelligence, Al, model. The method comprises: receiving, at an assessment server from a client device over a network, a request to assess an Al model held by the client device; and selectively combining, from a data repository, one or more raw datasets from amongst a plurality of raw datasets stored in the data repository to define an assessment dataset. Each raw dataset has associated therewith: (i) ground truth data for comparison with output data generated by the Al model when applied to the raw dataset to determine an accuracy score of the Al model, and (ii) a context label indicative of a category of date of the raw dataset. The method further comprises: applying the Al model to the assessment dataset to generate output data; determining, based on the output data, an accuracy score for the Al model by comparing the generated output data to ground truth data associated with the assessment dataset; and determining, based on the output data, an equity score that is indicative of variation in the accuracy score across a range of context labels associated with the one or more raw datasets constituting the assessment dataset.

[0058] As discussed above, the equity score (equivalently referred to as the fairness score) may be indicative of the fairness, or equitable accuracy, of the model - i.e., how well the accuracy / precision of the model is maintained as the qualitative nature of the input data is varied. For example, in the context of facial image analysis models, the equity score may be indicative of how well the accuracy of the Al model is maintained for different demographics of subjects depicted in facial images. The equity score may be expressed in any form that provides the user with information indicative of how the accuracy of the Al model varies for different context labels. For example, the equity score may - in some examples - be expressed as multiple scores, or a function that defines a relationship between the accuracy of the model and changes in the context labels of the raw datasets. In some examples, the equity score may be based on the absolute values of the determined accuracy scores for each raw dataset while in other examples, the equity score may be based on a ratio of the determined accuracy scores for each raw dataset with a reference (e.g., a maximum) accuracy score.

[0059] In some embodiments, the data repository may be remote from the assessment server.

[0060] For example, as discussed above, the data repository may be stored on a cloud-based server that is communicatively connected with the assessment server. In other examples, the data repository may be stored at a same physical site as the assessment server.

[0061] Cloud-based servers may be particularly beneficial because they facilitate the distribution of the data repository and the resources required to implement the assessment server across a cloud-based system to reduce the costs and resource requirements associated with maintaining the assessment server and / or the data repository.

[0062] The embodiments and examples set out below may be combinable with any of the aspects, embodiments or examples set out above.

[0063] In some embodiments, the one or more raw datasets selectively combined to define the assessment dataset may be selected based on one or more criteria set by the client device.

[0064] For example, the client device may select a range of context labels and / or quality scores for the raw datasets that constitute the assessment dataset. In other words, the client device may set criteria against which the Al model should be benchmarked.

[0065] In some embodiments, the methods may further comprise: communicating one or more of the determined accuracy score, the determined robustness score, and / or the determined equity score to the client device.

[0066] In other words, the results of the assessment of the Al model may be communicated to the client device so that they are aware of the accuracy performance of their model.

[0067] In some embodiments, the methods may further comprise: communicating one or more of the determined accuracy score, the determined robustness core, and / or the determined equity score to a marketplace server. The marketplace server may include a model store that stores a plurality of model datasets, each model dataset including information indicative of at least one of a respectively determined accuracy score, robustness score, and / or equity score associated with a corresponding Al model.

[0068] The marketplace server may, in effect, store a database of bench marked / assessed models. The database may be accessible by potential model users to review, compare and contrast different benchmarked / assessed models so that they may select the Al model that best suits their needs. This determination may be based on one or more of the accuracy score, robustness score, and / or equity score associated with the each of the stored models. In some embodiments, the methods may further comprise: communicating metadata associated with the Al model to the marketplace server; and generating, by the marketplace server, a model dataset for the Al model. Said model dataset may include the communicated metadata and the one or more communicated scores.

[0069] The metadata stored in the model store may include additional information useable by potential model users to compare, contrast and select an Al model that best suits their needs.

[0070] In some embodiments, the metadata may include one or more data items indicative of: a name of the Al model; information associated with training data used to train the Al model; information associated with the assessment dataset; information indicative of how a potential model user may access the Al model; and / or one or more context labels indicative of a preferred use context of the Al model.

[0071] The information associated with the training data may include information indicative of the quality, variation in quality, and / or context labels associated with the data used to train the Al model.

[0072] The information associated with the assessment dataset may include information indicative of the quality, variation in quality, and / or context labels associated with the assessment dataset against which the Al model is assessed.

[0073] The information indicative of how a potential model use may access the Al model may include information about whether, for example, the corresponding Al model is accessible under a subscription model, a licensing model, a free model, a pay-per-use model, and / or any other form of access model.

[0074] The one or more context labels indicative of a preferred used context of the Al model may include recommended operational parameters for the corresponding model. For example, they may include, recommended operating systems, input data structures and contents and datatypes, preferred processing requirements, and any other information that may inform the potential model user how the model is intended to be used, and in which contexts said model is intended to be used.

[0075] In some embodiments, each of one or more data items constituting the metadata may comprise its own privacy tag, each privacy tag may be independently modifiable by the client device such that each data item may be independently configured to be publicly or privately accessible by a potential model user in accordance with the corresponding privacy tag.

[0076] In effect, therefore, the model store may be partitioned into a public section and a private section wherein any potential model user may be able to access the data items stored in the public section of the model store, while only those potential model users with the required permissions (e.g., a password-protected user account, compliant IP address, and / or required payment status) are able to access data items stored in the private section of the model store for which said use has the requisite permissions.

[0077] In this way, the client device (or model owner) can choose which information, in the form of data items, to make public, and which data items should benefit from an enhanced level of data security.

[0078] In some embodiments, the methods may further comprise: in response to receiving a request from a potential model user, retrieving one or more of the stored model datasets from the model store; and displaying, by the marketplace server, benchmarking information related to each of the models associated with the one or more retrieved model datasets to the potential model user.

[0079] In some embodiments, the display of the benchmarking information may be an interactive display that is modifiable on request such that the displayed information related to each of the models is sortable and / or filterable to arrange and / or convey selected information to the potential model user.

[0080] In some embodiments, the benchmarking information related to each of the models may be sortable and / or filterable according to one or more of a determined accuracy score, a determined robustness score, and / or a determined equity score associated with each of the models.

[0081] In some examples, the benchmarking information related to each of the models may be sortable and / or filterable according to the contents of one or more of the data items that constitute the metadata associated with each of the Al models stored in the model store.

[0082] In some embodiments, the marketplace server and the assessment server may be the same server.

[0083] For example, the marketplace server and the assessment server may be provided and / or administered by the same entity - the assessment provider, with the necessary computing resources for both servers administered and / or executed in the same cloud-based server under the control of said assessment provider. In this way, it may be possible to provide a complete overall benchmarking-and-marketplace service to both model owners and potential model users, thereby providing an end-to-end service with reduced latency and redundancy in the network so that Al models may be efficiently and effectively assessed and compared.

[0084] In some embodiments, the method of the first aspect may further comprise: dismantling the secure enclave and deleting its contents after communicating the one or more determined scores. In this way, the data security of the overall assessment / benchmarking methods disclosed herein may be ensured because, after the assessment has taken place, there is no trace of the scripts used to execute the assessment thereby ensuring that the client device never gets access to the full details (e.g., the source code) used to assess the Al model, and the assessment server never gets access to the full details of the Al model itself.

[0085] In some embodiments of any of the methods disclosed herein, the request to assess the Al model may include a request for a particular level of assessment.

[0086] In some embodiments a comparatively higher level of assessment may test the performance accuracy of the Al model: for distinguishing between comparatively more similar input data, and / or against a comparatively larger assessment dataset, and / or against a comparatively more varied assessment dataset.

[0087] In other words, a comparatively easier assessment may test an Al model’s capability of distinguishing between very different input data, while a comparatively harder assessment may test an Al model’s capability of distinguishing between very similar input data. In this way, it may be possible to determine the accuracy of an Al model at distinguishing between input data of varying levels of similarity, and presenting this to potential model users so that they can determine which is the most suitable model for their needs.

[0088] Additionally or alternatively, a comparatively easier assessment may test an Al model’s capability of distinguishing between input data based on a relatively small amount of assessment data (i.e. , a relatively small assessment dataset) and / or a dataset with relatively less varied data (i.e., an assessment dataset wherein the data making up the assessment dataset has comparatively fewer differences therebetween). In contrast, a comparatively harder assessment may test an Al model’s capability of distinguishing between input data based on a relatively large amount of assessment data (i.e., a relatively big assessment dataset) and / or a dataset with relatively more varied data (i.e., an assessment dataset wherein the data making up the assessment dataset has comparatively more differences therebetween).

[0089] In some embodiments of any of the methods disclosed herein, the Al model may be an Al model configured to analyse images and / or videos depicting the faces of one or more subjects. The methods and systems described herein may be particularly applicable to such contexts, where accuracy, robustness and equity scores are all vitally important metrics to understand - as will be discussed in greater detail below.

[0090] In some embodiments, the output generated by the Al model may include one or more of: an identification of a subject depicted in an image; a verification that the same subject is depicted in two or more images; an attentiveness output indicative of a level of attentiveness of a subject depicted in an image (for example a level of attentiveness of a subject to consumed media content); an emotional output indicative of an emotional response of a subject depicted in an image (for example an emotional response to consumed media content); and / or a behavioural output indicative of a behavioural response of a subject depicted in an image (for example a behavioural response to consumed media content).

[0091] In some embodiments, the quality score of each raw dataset may be indicative of one or more of: a degree of occlusion of the one or more faces depicted in the corresponding image; a lighting level associated with the corresponding image; an imaging angle from which the corresponding image is capture; and / or a degree of obscuration of the one or more faces depicted in the corresponding image.

[0092] The degree of occlusion may be expressed, for example, for each face depicted in the image, as a proportion of said face that is occluded (i.e., blocked) from view in the image.

[0093] The lighting level may be expressed, for example, numerically as an effective brightness of the image.

[0094] The imaging angle may be expressed, for example, as an angle relative to a “face-on” angle in which the face of the imaged subject is looking directly towards the device that captured the image.

[0095] The degree of obscuration may be expressed, for example, for each face depicted in the image, as a proportion of said face that is obscured e.g., by glasses, hair, a headdress or similar.

[0096] In some embodiments, the context label of each raw dataset may be indicative of one or more of: a demographic group to which each of the one or more imaged subjects in the corresponding image belongs; and / or a use context in which the corresponding image was captured. The demographic group may, for example, be an ethnicity-based demographic (e.g., skin-tone), an agebased demographic, a sex-based demographic (e.g., the apparent sex of the subject), or any other demographic suitable for categorising image data.

[0097] The use context may, for example, be an indication of what type of device captured the image (e.g., a webcam, a phone camera, CCTV, etc.), what type of activity the subject was engaged in when the image was captured (e.g., watching media on a personal device, walking around an exhibition or similar, commuting, or any other activity), or any other contextual information that may be beneficial for potential model users to be aware of.

[0098] In a further aspect, there is provided a computer-readable medium comprising instructions that, when executed by one or more networked computers, cause the one or more networked computers to carry out the methods disclosed herein.

[0099] In another aspect, there is provided a computer program product comprising logic that, when executed by one or more processors, causes the one or more processors to carry out the methods disclosed herein.

[0100] In a further aspect, there is provided a networked system comprising: an assessment server and a client device communicatively linked together and configured to carry out the methods disclosed herein.

[0101] In some examples, the networked system may comprise a plurality of client devices.

[0102] In some embodiments, the networked system may further comprise a marketplace server that may include a model store that stores a plurality of model datasets, each model dataset may include information indicative of at least one of a respectively determined accuracy score, robustness score, and / or equity score associated with a corresponding Al model.

[0103] In some examples, as discussed above, corresponding communicated metadata associated with each Al model may also be stored in the model store.

[0104] Each of the embodiments and examples set out above is expressly combinable with one another except where such a combination would clearly be logically or technically impossible.

[0105] Summary of the Figures

[0106] Embodiments and experiments illustrating the principles of the invention will now be discussed with reference to the accompanying figures in which:

[0107] Figure 1 is a schematic illustration of a facial image in a format used in the state of the art to benchmark face recognition models.

[0108] Figure 2 is a schematic of an assessment system that is an embodiment of the invention.

[0109] Figure 3 is a more detailed schematic of the assessment data database depicted in Figure 2.

[0110] Figure 4 is a schematic of an exemplary data structure of assessment data as stored in the database of Figure 3. Figure 5 is a more detailed schematic of the assessment engine depicted in Figure 2.

[0111] Figure 6 is a more detailed schematic of the model database of assessed models depicted in Figure 2.

[0112] Figure 7 is an illustration of an exemplary user interface by which a model can be submitted to the system of Figure 2.

[0113] Figure 8 is an illustration of an exemplary user interface by which a user can access a benchmarked list of assessed models.

[0114] Figure 9 shows a method of assessing the performance accuracy of an Al model that is an embodiment of the invention.

[0115] Detailed Description of the Invention

[0116] Aspects and embodiments of the present invention will now be discussed with reference to the accompanying figures. Further aspects and embodiments will be apparent to those skilled in the art. All documents mentioned in this text are incorporated herein by reference.

[0117] As discussed above, Figure 1 is a schematic illustration of the type of facial image used in the state of the art to benchmark face recognition models. These facial images take the form of traditional “passportstyle” photos. In other words, there is little to no variance in the image capture angle and / or the lighting conditions meaning that all the data used to benchmark models suffers from no occlusion whatsoever. In this way, state-of-the-art benchmarking models use “perfect” or near-perfect benchmarking data. As such, while it may be possible to determine a very reliable metric for how precise a given model is, there is no way to determine how robust a model is against deterioration in the quality of the input data that said model is configured to process and analyse.

[0118] Figure 2 shows a schematic of an assessment system 100 that is configured to assess Al models against assessment data of variable quality so as to provide a metric for the robustness of the models.

[0119] Additionally, the assessment system 100 of Figure 2 is communicatively secure in the sense that the provider of the assessment system never has access to the full details (e.g., the source code) of the submitted models, while the model submitter never has access to the full details of the assessment system 100.

[0120] While the assessment system 100 disclosed herein has been devised first in relation to face-recognition and facial-image-analysis applications, the system 100 can be deployed and used to assess any Al models against assessment data where the quality of data that may be input to the Al model in practice is likely to be of variable quality. In other words, the assessment system 100 of Figure 2 is not restricted to facial-image analysis contexts but rather is applicable to determine the robustness of any submitted model provided a sufficiently varied and populous assessment dataset is available.

[0121] Assessment system 100 comprises an assessment data database 200 (hereinafter referred to as ‘the database’), an assessment engine 300 (hereinafter referred to as ‘the engine’), and an assessment model store 400 (hereinafter referred to as ‘the model store’). The database 200 is preferably maintained by the assessment provider on an assessment server maintained by said provider. The engine 300 is preferably a temporary computing environment established on a cloud-based server to carry out the assessment of an Al model. As will be discussed in greater detail below, the engine 300 may be configured with confidential computing capabilities. It may be preferable for the engine to be established primarily on a cloud server maintained by a model owner (i.e., a party submitting their Al model for assessment) so that said model owner can control the resources allocated to the assessment, and so that the burden of maintaining suitable computing resources is not placed solely on the assessment provider. The model store 400 is preferably maintained on a marketplace server maintained by a market provider. The market provider may be the assessment provider, in which case, it may be preferable for the marketplace server and the assessment server to be a single common server. In other cases, the market provider may be a trusted third party that maintains their own, separate, marketplace server.

[0122] Assessment data may be added to the database 200 via one or more private data portals 102, 104. Each of the private data portals 102, 104 may be securely connected to the database 200 so that the data provided to the database 200 is not publicly accessible. For example, the respective communication channels linking the private data portals 102, 104 to the database 200 may be encrypted (e.g., with an AES256 encryption protocol or similar). Keeping the data secure may be particularly important in the context of data that includes private and / or personal data - e.g., facial images or video streams depicting facial images of one or more subjects.

[0123] Private data portals 102, 104 may only be accessible to entities with the correct access rights. The access rights may be verifiable, for example, by confirmation that a providinglP address corresponds to an entity with the necessary permissions to add data to the database 200. Additionally or alternatively, private data portals 102, 104 may be password protected or benefit from another form of authentication protection - e.g., two-factor authentication and / or biometric authentication. In this way it is possible to limit access permissions for adding assessment data to the database 200 to trusted sources to ensure that the content of the database 200 has a guaranteed level of quality, and / or complies with all necessary data privacy regulations, or similar other requirements imposed upon the assessment system 100.

[0124] In some examples, assessment data may also be provided to the database 200 via one or more public data portals 106. The communication channel linking the public data portals 106 and the database 200 may nonetheless be a secure communication link so that the data to the database 200 is not publicly accessible (e.g., the communication channels may be encrypted with a suitable encryption protocol such as the AES256 encryption protocol). The public data portals 106 may be public in the sense that it is possible for anyone to provide assessment data to the database without having to authenticate and / or verify their identity to prove that they have access rights. To ensure the suitability of any assessment data provided via a public data portal 106, any assessment data provided through such a portal 106 may be subject to one or more quality control checks to ensure that the data is suitable (e.g., of the right quality, in the right format, and compliant with the necessary regulations) for use in assessment. Meanwhile, models to be assessed may be provided to the engine 300 via one or more private model portals 108, 110, 112. Each of the private model portals may be securely connected to the engine 300 so that the models provided to the engine 300 are not publicly accessible. As will be discussed in greater detail below, the assessment engine 300 may be logically divided into various compartments, each of which is communicatively isolated from one another to ensure that the provided of the assessment system 100 never has full access to the details (e.g., the source code) of the submitted model, and that the entity providing the model for assessment never has full access to the details (e.g., the source code) underlying the assessment processes. Further, the private model portals 108, 110, 112 may only be accessible to entities with the correct access rights. The access rights may be verifiable, for example, by confirmation that a providing IP address corresponds to an entity with the necessary permissions to submit a model to the engine 300 for assessment. Additionally or alternatively, private model portals 108, 110, 112 may be password protected or benefit from another form of authentication protection - e.g., two- factor authentication and / or biometric authentication. In this way it is possible to limit access permissions for submitting models for assessment to entities that have been given the necessary permissions (e.g., in response to passing any number of requisite security and / or identify verification checks). This may guarantee that only entities with bona fide credentials are able to submit models for assessment.

[0125] Data indicative of one or more of the assessment models (e.g., an arranged or otherwise benchmarked list of assessed models) may be downloaded from the model store 400 via one or more public model download portals 114, 116. It is preferable for at least some of the information assigned to each model in the model store 400 to be publicly available, for example the data in the model store 400 could be accessible in the form of a marketplace-style format that allows users to compare and contrast the performances of different models according to various (preferably modifiable) criteria. The comparison of different models will be discussed below in relation to Figure 8.

[0126] The communication channel linking the public model download portals 114, 116 to the model store 400 may be (but need not necessarily be) a secure communication link so that the data downloaded from the model store 400 is not publicly accessible. However, the communication channel may also be an unsecure communication link as only information that the model owner (i.e., the entity that submitted the model for assessment by the engine 300) consents to being made public may be downloaded to the public model download portals 114, 116. The public model download portals 114, 116 may be public in the sense that it is possible for anyone to download the information stored in the model store 400 without having to authenticate and / or verify their identity to prove that they have access rights. For example, the model store may present the data contained therein on request to a dedicated webpage or similar.

[0127] In some examples, data indicative of one or more of the assessed models (e.g., an arranged or otherwise benchmarked list of models) may be downloaded from the model store 400 via one or more private model download portals 118. In some examples, the model owner that submitted a particular model for assessment may wish to restrict access to the some or all of the assessment results determined by the assessment engine 300 to only those users to whom they have given permission. For example, as will be discussed in greater detail below in relation to Figure 6, the model owner may decide which information is to be made publicly available from the model store 400, and which information is to be made secure - accessible only through private authentication or another similar access route.

[0128] Figure 3 is a more detailed schematic of the assessment data database 200 depicted in Figure 2.

[0129] The database 200 is configured to receive and store assessment data that may be used to assess models provided to the engine. As such, the database 200 must be communicatively linked with the engine 300 to facilitate the transmission of assessment data to the engine 300 so that a provided model can be assessed.

[0130] In the context of facial-image recognition / analysis, the data used to train and benchmark the Al models configured to carry out the analysis may be divided, broadly speaking into two categories: “Clean” and “Dirty”. So-called clean data may be considered to be data that includes image data and / or video streams depicting facial images that have been obtained with the explicit consent of the individuals depicted in said images. Preferably, this explicit consent includes consent for the image data and / or video streams to be used for the purpose of assessing models, although this consent may be qualified to use in certain contexts (e.g., models that have been adapted for specific purposes and / or provided by a select list of model owners). Conversely, so-called dirty data is data that may have been collected without the explicit consent of the individuals depicted therein. For example, dirty data may include surveillance camera footage and / or “scraped” data that has, for example, been scraped from publicly accessible sources of image and video data (e.g., social media websites, news websites, blogs, and the like).

[0131] It may be desirable to categorise assessment data according to whether it is clean or dirty because different users may have different requirements based, for example, on their personal / company ethics and / or local data processing and management requirements imposed in their jurisdiction (e.g., the General Data Protection Regulation, GDPR, in force throughout much of Europe).

[0132] In such contexts, therefore, the database 200 comprises a partition 202 that divides the database 200 into two distinct and communicatively unlinked (or even communicatively blocked) segments: a first “clean” segment 210, and a second “dirty” segment 220.

[0133] The clean database segment 210 comprises a plurality of pieces of clean assessment data 212a-m. Correspondingly, the dirty database segment 220 comprises a plurality of pieces of dirty assessment data 222a-n. The clean and dirty segments 210, 220 of the database 200 are preferably independently addressable by the engine 300 to reduce the risk of cross-contamination - i.e., to reduce the risk that dirty assessment data 222a-n is inadvertently stored in the clean database segment 210.

[0134] Figure 4 is a schematic of an exemplary data structure of assessment data as stored in the database 200 of Figure 3.

[0135] Assessment data may be stored in the database 200 as a database entry 230 comprising a plurality of tags. Each tag is an item of information associated with the assessment data. The database entry 230 comprises a datatype tag 240, metadata 250, and the assessment data itself 260. The assessment data 260 may also be referred to as the content, data content, or assessment content. The assessment data 260 may be in the form of image data representing one or more still images, or video data representing one or more video streams. The datatype tag provides an indication of whether the assessment data 260 is clean data or dirty data. Upon provision to the database 200, a processor of the database 200 may be configured to allocate the database entry 230 to either the clean database segment 210 or the dirty database segment 220 based on the indication provided by the datatype tag 240.

[0136] In some examples, the datatype tag 240 may indicate that the assessment data 260 comprises a mix of clean data and dirty data (hereinafter referred to as mixed data). In the case of mixed data, the processor of the database 200 may be configured to parse the assessment data 260 into separate sections, each section being comprising either solely clean or dirty data. These parsed sections may then be stored separately in the clean and dirty database segments 210, 220 according to the datatype within each parsed section. In other examples, the database 200 may further comprise a mixed database segment (not shown). In yet other examples, where there is no mixed database segment, assessment data 260 having a datatype tag 240 indicating that the data is of mixed type may all be stored in the dirty database segment 220.

[0137] The metadata 250 of the database entry 230 may comprise information useable by the engine 300 to selectively filter and use the correct data for the assessment of models provided thereto. For example, the metadata 250 may comprise an image quality tag 252 that is indicative of the quality of the assessment data 260. For example, the image quality tag 252 may include a score that is indicative of how close to “perfect” the assessment data 260 is - i.e., how similar the quality of the assessment data 260 is (in terms of quality) to the quality of the standard data used in contemporary benchmarking systems, such as depicted in Figure 1 . For example, a score of ‘1 ’ may indicate that the assessment data includes an image of a face that is perfectly and consistently lit, at a full face-on angle with no occlusion of the face whatsoever, while a score of ‘0’ may indicate that the assessment data includes an image of a face that is unlit and / or almost fully or fully occluded.

[0138] The metadata 250 may further comprise an image variation tag 254 that is indicative of the extent of variation in the quality of the assessment data 260. This variation may, for example, be a variation in the quality of a facial image that is depicted in a video stream over the course of the video stream.

[0139] Additionally or alternatively, the variation may be a variation in the quality across multiple facial images of different people all depicted within the same assessment data 260.

[0140] The metadata 250 may further comprise a use context tag 256 indicative of a context or a use case of the assessment data 260. For example, the use context tag 256 may be indicative of a format in which the assessment data 260 was collected - e.g., still images and / or video streams. Additionally or alternatively, the use context tag 256 may be indicative of the type of device by which the assessment data 260 was collected - e.g., a webcam, a mobile phone camera, a surveillance camera, a social media post, a newsarticle image, a blog-article image, a camera, or any other type of device suitable for collecting data. Additionally or alternatively, the use context tag 256 may be indicative of the type of activity depicted in the assessment data 260 - e.g., consuming media content (for example by watching videos), walking through an environment, attending an online meeting or conference call, taking part in a physical meeting, attending an event (for example a lecture or class or another content delivery event in an educational setting). Additionally or alternatively, the context use tag 256 may be indicative of one or more demographic groups depicted in the assessment data 260 - e.g., one or more age demographic groups, one or more race / ethnicity demographic groups, and / or one or more gender-presentation demographic groups. The context use tag 256 may also include any other information indicative of a context or a use case of the assessment data 260.

[0141] Each of the contexts / use cases set out above may encoded in a separate and distinct context use tag 256. Alternatively, each of the indicia discussed above may be combined into a single context use tag 256, for example by hashing all of the indicative information together to generate an overall context use tag 256.

[0142] The metadata 250 may further comprise an accessibility tag 258 indicative of the access permissions required to be able to use the corresponding assessment data 260 in an assessment process carried out by the engine 300. For example, the accessibility tag 258 may indicate that the corresponding assessment data 260 is suitable for use with any model to be assessed, submitted by any entity. In other examples, the accessibility tag 258 may indicate that the corresponding assessment data 260 may only be used to assess models submitted to the engine 300 by one or more permitted model owners. The identity of the model owners may be verifiable, for example, by requiring the entry of a specific password to enable the engine 300 to access the corresponding assessment data 260. In other examples, as will be discussed in more detail below, the engine 300 may be deployed to a model owner’s own cloud server (e.g., Amazon Web Service, AWS, cloud or similar), and the accessibility tag 258 may be indicative that the corresponding behavioural data may only be used for assessment in one or more preselected permitted cloud servers. In other examples, the accessibility tag 258 may indicate that the data is locked (i.e. , prevented) from being used in the assessment process until the accessibility tag 258 is “unlocked”. The unlocking may be performed, for example, by successfully completing an authentication protocol, such as two-factor and / or biometric authentication, or by completing a transaction - i.e., by the model owner purchasing, licensing, or renting access rights to the corresponding assessment data 260.

[0143] The metadata 250 may comprise any one or more of the tags 252, 254, 256, 258 discussed above in any number or combination. In some examples, any of the tags 252, 254, 256, 258 set out above may or may not be included in the metadata 250 in any combination.

[0144] The assessment data 260, or content, may comprise raw data 262 and labels 264. In other words, the assessment data 260 is labelled data. The nature of the labelling is based on the type of models that can be assessed by the engine 300. For example, to assess models directed towards facial recognition or verification, the assessment data 260 must include one or more labels 264 that annotate the raw data 262 with an identifier for the face. In another example, to assess model used for determining a behavioural and / or emotional response to content, the assessment data 260 must include one or more labels 264 that annotate the raw data 262 to indicate verified markers for specific behavioural and / or emotional responses to content. Similarly, for the engine 300 to be able to assess models / algorithms that determine a level of attentiveness of a person depicted in a facial image, the assessment data 260 must include one or more labels 264 that annotate the raw data 262 to indicate verified markers for attentiveness levels in the facial images. The labels 264, in other words, may associate so-called ground truth data with the assessment data 260. The ground truth data may be data that is indicative of a model output that corresponds to an output that would be generated by an Al model that is perfectly (i.e., 100%) accurate.

[0145] The labels 264 may also include a quality score indicative of a quality of the assessment data 260 so that a robustness of the model may be determined (as introduced above and discussed below). Additionally or alternatively, the quality score may be associated with the metadata 250 as image quality tag 252.

[0146] For example, the quality score of each raw dataset may be indicative of one or more of: a degree of occlusion of the one or more faces depicted in the corresponding image; a lighting level associated with the corresponding image; an imaging angle from which the corresponding image is capture; and / or a degree of obscuration of the one or more faces depicted in the corresponding image.

[0147] The degree of occlusion may be expressed, for example, for each face depicted in the image, as a proportion of said face that is occluded (i.e., blocked) from view in the image.

[0148] The lighting level may be expressed, for example, numerically as an effective brightness of the image.

[0149] The imaging angle may be expressed, for example, as an angle relative to a “face-on” angle in which the face of the imaged subject is looking directly towards the device that captured the image.

[0150] The degree of obscuration may be expressed, for example, for each face depicted in the image, as a proportion of said face that is obscured e.g., by glasses, hair, a headdress or similar.

[0151] The labels may also (or alternatively include) a context label indicative of a category of data of the Al model so that an equity score of the model may be determined (as introduced above and discussed below). Additionally or alternatively, the context label may be associated with the metadata 250 as use context tag 256.

[0152] For example, the context label of each raw dataset may be indicative of one or more of: a demographic group to which each of the one or more imaged subjects in the corresponding image belongs; and / or a use context in which the corresponding image was captured.

[0153] The demographic group may, for example, be an ethnicity-based demographic (e.g., skin-tone), an agebased demographic, a sex-based demographic (e.g., the apparent sex of the subject), or any other demographic suitable for categorising image data.

[0154] The use context may, for example, be an indication of what type of device captured the image (e.g., a webcam, a phone camera, CCTV, etc.), what type of activity the subject was engaged in when the image was captured (e.g., watching media on a personal device, walking around an exhibition or similar, commuting, or any other activity), or any other contextual information that may be beneficial for potential model users to be aware of.

[0155] Figure 5 is a more detailed schematic of the assessment engine 300 depicted in Figure 2.

[0156] The engine 300 is configured to receive information from a model owner via the one or more private model portals 108, 110, 112, and to receive assessment data from the database 200 via a secure communication link therebetween. In order to preserve the data security and privacy of all contributors to the assessment system 100, the engine 300 is implemented as a confidential computing server taking the form of a secure enclave environment.

[0157] The engine 300 is preferably established as a secure enclave on a cloud server maintained by the model owner. In some examples, the secure enclave is established on the model owner’s cloud resources by the assessment provider. For example, the model owner may provide the assessment provider with the necessary permissions and instructions (e.g., in the form of an executable script) to be able to establish the secure enclave on a cloud-based server associated with / maintained by the model owner (e.g., by a client device of the model owner).

[0158] By establishing the engine 300 as a secure enclave, the model owner never has access to the models and / or algorithms that are actually used to assess a submitted model. Instead, the model owner provides one or more tags 302a-k that contain information indicative of what assessment the model owner requires from the engine. For example, the one or more tags 302a-k may include information indicative of whether the model owner would like the model to be assessed using clean, dirty, or mixed data. Additionally or alternatively, the one or more tags 302a-k may include information indicative of the purpose of the model - i.e. , what performance needs to be assessed (for example, behavioural / emotional response to consumed content, or attentiveness during content consumption). Additionally or alternatively, the one or more tags 302a-k may include information about how much variation in the quality of the assessment data is desired (e.g., based on the image quality tag 252), or how much variation within each piece of assessment data (e.g., based on the image variation tag 254). Additionally or alternatively, the one or more tags 302a-k may include information that enables the model to be identified (e.g., a name of the model and / or a name of the developer / provider / licensor of the model). Additionally or alternatively, the one or more tags 302a-k may include information indicative of the accessibility of the model - for example, whether the model is an open-source, freely accessible model, or whether access is subject to purchasing, renting or licensing conditions. Additionally or alternatively, the one or more tags 302a-k may include information indicative of which parts of the assessment results that are to be stored in the model store are to be made publicly available and which are to be stored securely and only provided to private model download portals 118 subject to a user meeting certain subject access requirements (e.g., two- factor and / or biometric authentication, correct password entry, access request from a permitted IP address, etc.).

[0159] Based on the one or more tags 302a-k, the engine selects and retrieves suitable raw datasets 304a-j to constitute an assessment dataset for use in benchmarking the submitted model. The assessment dataset preferably has the form and structure discussed above in relation to Figure 4. Importantly, the model owner never has access to the raw datasets 304a-j or the engine processor 306 because the assessment dataset is provided directly to the engine 300 in a secure manner. For example, the secure enclave may be established on the model owner’s cloud server resources with a first encryption key in-built into the enclave. The only party in possession of this first encryption key may be the assessment provider. More particularly, the processor responsible for communicating the raw datasets 304a-j to the engine 300 may be configured to encrypt the raw datasets 304a-j with the first encryption key to define a first encrypted item 306. The engine 300 is configured to receive the first encrypted item 306 from the database 200.

[0160] In some examples, the model owner may also provide the training data used to train the model 310 that is submitted for assessment. In some examples (and sometimes only in cases where express consent is provided), this training data may be assessed for suitability for use in assessing other models. Subject to a satisfactory assessment result, the submitted training data may be transmitted from the engine 300 to the database 200 and stored as assessment data, in the same way as discussed above for all other stored assessment data (as discussed in relation to Figures 3 and 4 above).

[0161] Meanwhile, the model 308 that is submitted by the model owner for assessment is provided directly to the engine in a secure manner. For example, the secure enclave may be established on the model owner’s cloud server resources with a second encryption key in-built into the enclave. The only party in possession of this second encryption key may be the model owner. This may be achieved by the model owner storing the second encryption key on a computing core of their cloud server resource, and by subsequently providing the assessment provider with the necessary permissions and instructions (e.g., in the form of an executable script) to form the secure enclave engine 300 on a set of cores including the core on which the second encryption key is stored. The client device responsible for communicating the model 308 to the engine 300 may be configured to encrypt the model 308 with the second encryption okey to define a second encrypted item310. The engine 300 is configured to receive the second encrypted item 310 from the client device. The assessment provider is therefore unable to access the contents of the second encrypted item 310. In this way, the data privacy and security of the model 310 itself is ensured.

[0162] The engine 300 further comprises an engine processor 312 that is securely located within a secure enclave environment 314 defined by the engine 300. No parties, processors, or devices external to the secure enclave environment 314 is able to access the engine processor 312. In other words, the secure enclave environment 314 defines a firewall that is impenetrable to external third parties.

[0163] The engine processor 306 preferably is configured to interact with storage that stores both the first and second encryption keys. The engine processor is consequently configured to obtain the first and second encrypted items 306, 310 and decrypt them both with the first and second encryption keys respectively. The engine processor, having successfully decrypted the first and second encrypted items 306, 310 can then assess the copy of the Al model 308 against the raw datasets 304a-j that define the benchmarking dataset.

[0164] Figure 6 is a more detailed schematic of the model store 400 of Fig. 2, which shows a benchmarked display of assessed models.

[0165] The model store 400 is configured to receive and store assessed models from the engine 300 depicted in Figure 5. As such, the model store 400 must be communicatively linked with the engine to facilitate the transmission of assessed models’ data to the model store 400 so that information can be downloaded from the model store 400 via the one or more public and / or private model download portals 114, 116, 118.

[0166] The model store 400 may comprise a partition 402 that divides the model store 400 into two distinct and communicatively separated segments: a first “public” segment 404, and a second “private” segment 406.

[0167] The public segment 404 may be publicly accessible, e.g., via the one or more public model download portals 114, 116 to access information about the assessed models that the corresponding model owner has approved as being suitable for public access. This public access may be achieved, for example, by displaying the contents of the public segment 404 on a free-to-access public website or similar.

[0168] The private segment 406 may only be accessible to one or more approved users via the one or more private model download portals 118. Accessing data stored in the private segment 406 may only be available by satisfactorily passing one or more authentication processes. For example, access to data in the private segment may require two-factor and / or biometric authentication, correct password entry, and / or an access request from a permitted IP address, or any other authentication process(es).

[0169] In some examples, there may be one or more designated private “superusers” who have global access to the entire contents of the private segment 406, for example for auditing or regulatory purposes. Preferably, the majority of users able to access data in the private segment 406 via the one or more private model download portals 118 are only able to access data in the private segment 406 that specifically relates to one or more benchmarked models for which they have been given permission to access. In other words, in some examples, the access of each private user to the private segment 406 is content-limited such that data stored in the private segment 406 is more selectively secure, according to the needs of the model owner.

[0170] Each assessed model may be stored in the model store 400 in the form of a benchmarked data entry 410, 420, 430, 440, 450. Each benchmarked data entry 410, 420, 430, 440, 450 may be stored in a bespoke manner between the public and private segments 404, 406 with a single data entry crossing the partition 402 as necessary depending on the tags 302a-k provided to the engine 300 by the model owner.

[0171] Each benchmarked data entry 410, 420, 430, 440, 450 may comprise one or more data items including one or more of: (i) a ‘name’, (ii) a precision or accuracy score, (iii) a variation or robustness score, (iv) a fairness or equity score, (v) a training datatype, (vi) a benchmarking datatype, (vii) an access type, (viii) an access link, and (ix) a use context, amongst any other information that may be considered useful or suitable for distribution to public and / or private users via the public and / or private model download portals 114, 116, 118.

[0172] The name item may be a name useable to identify the model, for example it may be the name of the model given by the model owner and may also include a name of a developer, owner and / or distributor of the model. This name may or may not be the same as the name of the model owner.

[0173] The precision score (equivalently referred to as the accuracy score) may be a measure of the accuracy (i.e., the precision) of the model. This may be expressed, for example, as the number of “true” positive determinations divided by the total number of positive determinations. For example, a precision score of “1” may indicate that all the positive determinations made by a model are correct. Meanwhile, a precision score of “0.8” may indicate that 80% of all the positive determinations made by a model are correct.

[0174] The robustness score (equivalently referred to as the variation score) may be indicative of the robustness of the model - i.e. , how well the precision of the model is maintained as the quality of the input data decreases. This may be expressed, for example, as the precision of the model when the input data has a quality score of, e.g., 0.8 (or any other selected score) divided by the precision of the model when the input data has a quality score of e.g., 1 . The robustness score may be expressed in any other form as long as it provides information indicative of how well the performance of the model is maintained as the quality of the input data deteriorates. The robustness score may, in some examples, be expressed as multiple scores so that the rate of decay in performance of the model vs. the rate of decay in the quality of the input data can be inferred from the robustness score.

[0175] The equity score (equivalently referred to as the fairness score) may be indicative of the fairness, or equitable accuracy, of the model - i.e., how well the accuracy / precision of the model is maintained as the qualitative nature of the input data is varied. For example, in the context of facial image analysis models, the equity score may be indicative of how well the accuracy of the Al model is maintained for different demographics of subjects depicted in facial images. The equity score may be expressed in any form that provides the user with information indicative of how the accuracy of the Al model varies for different context labels. For example, the equity score may - in some examples - be expressed as multiple scores, or a function that defines a relationship between the accuracy of the model and changes in the context labels of the raw datasets. In some examples, the equity score may be based on the absolute values of the determined accuracy scores for each raw dataset while in other examples, the equity score may be based on a ratio of the determined accuracy scores for each raw dataset with a reference (e.g., a maximum) accuracy score.

[0176] The training datatype item may include information indicative of whether the training data is clean, dirty, or of mixed type.

[0177] Similarly, the benchmarking datatype item may include information indicative of whether the benchmarking data is clean, dirty, or of mixed type.

[0178] The access type item may include information indicative of how a downloading user can access the model referred to in the benchmarked data entry 410, 420, 430, 440, 450. For example, the access type could be “Free” meaning that access is free and publicly available. Alternatively, the access type could be “license”, or similar, meaning that access is subject to application for and granting of a licence. The licence could include, for example, a subscription. Alternatively, the access type could be “PPU”, or similar, meaning that access is subject to a one-off payment fee per download of the model. Alternatively, the access type could be “time-limited”, or similar, meaning that access is available for only a limited amount of time (e.g., 30 minutes) before payment (or authentication) is required to fully access the model. Alternatively, the access type could be “service-limited”, or similar, meaning that access is available for only a limited consumption of data or complexity of model deployment before payment (or authentication) is required to fully access the model. In other words, the time-limited and service-limited access types may effectively correspond to free “Demo” access. Alternatively, the access type could be “securitylimited”, or similar, meaning that access is only available subject to satisfactory authentication - for example, two-factor and / or biometric authentication, correct password entry, and / or access request originating from a permitted IP address. Any other suitable access type may be indicated by the access type item.

[0179] The access link item may include a link to a third-party server (e.g., an external website) from which the corresponding model can be accessed. Additionally or alternatively, the access link item may include a direct download link enabling the model to be directly downloaded to the user via the public and / or private model download portal 114, 116, 118. In the case of direct download, the model is stored within the second black box 312 in the engine 300 and is transmitted to the model store 400 within the second black box 312 (keeping said black box intact). The model is only extracted from the second black box - i.e., made available - after it has been successfully downloaded to the user via the public and / or private model download portal 114, 116, 118. In this way, the data security and privacy of the model 310 is maintained throughout the duration of the benchmarking and storage by the benchmarking system 100.

[0180] The use context item may include information of the context in which the model is intended to be deployed, and / or for what purpose the model is intended. For example, the context use item may include information indicative of a format in which input data should be submitted to the mode - e.g., still images and / or video streams. Additionally or alternatively, the context use item may include information indicative of a type of device by which the input data should be collected - e.g., a webcam, a mobile phone camera, a surveillance camera, a social media post, a news-article image, a blog-article image, a camera, or any other type of device suitable for collecting data. Additionally or alternatively, the use context item may include information indicative of the type of activity depicted in the input data for which the model is adapted - e.g., a reaction to the consumption of media content (emotional / behavioural response and / or attentiveness)., attentiveness or emotional / behavioural response when walking through an environment, taking part in a conference call, or participating in an educational setting, and / or face recognition, or any other use case. Additionally or alternatively, the use context item may include information indicative of one or more demographic groups to which the model is particularly well adapted - e.g., one or more age demographic groups, one or more race / ethnicity demographic groups, and / or one or more genderpresentation demographic groups. The use context item may also include any other information indicative of a context or a use case to which the model is adapted.

[0181] Five exemplary benchmarked data entries 410, 420, 430, 440, 450 are depicted in Figure 6. As can be seen from Figure 6, the first and fifth exemplary entries 410, 450 are both wholly stored in the public segment 404 of the model store 400 meaning that all information transmitted from the engine 300 to the model store 400 pertinent to the models “ABC” and “1 -2-1 ” is made publicly available and is accessible via the one or more public model download portals 114, 116.

[0182] Meanwhile, the second benchmarked data entry 420 is wholly stored in the private segment 406 of the model store 400 meaning that all information transmitted form the engine 300 to the model store 400 pertinent to the model “DEF” is only accessible to users via the one or more private model download portals 118, provided they satisfy the access / authentication requirements prescribed by the model owner of model “DEF”.

[0183] Similarly, the third and fourth benchmarked data entries 430, 440 are both stored (to different degrees) partially in the public segment 404 of the model store 400 and partially in the private segment 406 of the model store 400. This means that some of the information (that stored in the public segment 404) transmitted from the engine 300 to the model store 400 that is pertinent to the models “AaBbCc” and “Alpha-Q” is made publicly available and is accessible via the one or more public model download portals 114, 116. The remainder of the information, however, is only accessible to users vie the one or more private model download portals 118, provided that the users satisfy the access / authentication (or, possibly, payment) requirements prescribed by the model owners of models “AaBbCc” and “Alpha-Q” respectively.

[0184] Figure 7 is an illustration of an exemplary user interface 500 by which a model 310 can be submitted to the system 100 of Figure 2.

[0185] The user interface 500 provides the model owner with the ability to input information relating to the one or more tags 302a-k shown in Figure 5. For example, the user interface 500 provides functionality for the model owner to provide the name 502 of the model, information related to the use context of the model 504 (as discussed above), information indicative of whether the training data is clean, dirty, or of mixed type 506, and / or a copy of the training data 508, together with any other useful contextual information (not shown). Additionally, the user interface 500 may further comprise the functionality to enable the model owner to designate any of the information provided to the engine as either ‘public’ or ‘private’ and specify any access / authentication restrictions or requirements.

[0186] The user interface 500 further comprises functionality enabling the model owner to select the extent of source and context variation 510, 512 in the raw datasets 304a-j to be used by the engine 300, and an option 514 to select whether the raw datasets 304a-j should be clean, dirty, or of mixed type. The user interface 500 may further comprise the functionality to enable the model owner to designate any of these selected options as being public or private.

[0187] Information / options designated as public will ultimately be stored in the public segment 404 of the model store 400, while information / options designated as private will ultimately be stored in the private segment 406 of the model store.

[0188] Figure 8 is an illustration of an exemplary user interface 600 by which a user can access a list of benchmarked models.

[0189] The downloading user interface 600 provides the model downloader with the ability to arrange, sort and filter database entries 410, 420, 430, 440, 450 from the model store according to their particular requirements.

[0190] The downloading user interface 600 comprises a precision priority selection tool 602, a robustness priority selection tool 604, and a fairness priority selection tool 606 that enables the user to select which of precision, robustness, and fairness (equivalently: accuracy, robustness, and equity) of a model they are prioritising for their requirements, and to what extent they prioritise each of these performance benchmark parameters.

[0191] The downloading user interface 600 further comprises functionality that enables the user to filter 608 models so that only those that are suitable, e.g., for their particular use context, are displayed, and functionality that enables the user to sort 610 those displayed models according to a criterion of particular interest to the user.

[0192] The downloading user interface 600 may display the benchmarked data entries 410, 420, 430, 440, 450 in any suitable presentation format 612 - e.g., a tabular format that is suitable for easy consumption by a user.

[0193] The downloading user interface 600 may further comprise functionality 614 that enables the user to switch between public and privately accessible information subject to passing the required security / authentication checks (e.g., correct password entry, logging on from a permitted IP address, two- factor and / or biometric authentication, payment, etc.). This may effectively be a security lock / unlock functionality 614. In this way the user may be able to switch between accessing data stored only in the public segment 404 of the model store 400 and accessing data stored in both the public and private segments 404, 406 of the model store 400.

[0194] In a particular example of the assessment system 100 discussed above in relation to Figures 2 to 8, the assessment system 100 may deployed across one or more cloud servers. In such an example, the public and / or private model portals 108, 110, 112 may be located within each respective model owner’s own cloud server. By requesting an assessment of their model, the model owner may provide the assessment provider with access to their cloud server. The engine 300 is then activated on the model owner’s own cloud server by activating a necessary number of cores using the model owner’s cloud capacity. As discussed above, the engine 300 implements a secure enclave structure such that, even though the engine 300 is implemented on the model owner’s cloud server, the model owner nonetheless does not have, and never has, access to the engine processor 306. Meanwhile, the database 200 and model store 400 are maintained on the assessment provider’s own cloud servers to maintain the data security and privacy of their contents.

[0195] At the end of the assessment procedure, the data on the cores on which the engine’s 300 processing was implemented is erased to maintain the data security and / or privacy of the engine 300.

[0196] Figure 9 shows a method of assessing the performance accuracy of an Al model.

[0197] The method 700 commences with an operation 710 whereby the assessment provider (preferably via the assessment server) receives a request to assess an Al model by a model owner (preferably via a client device associated therewith).

[0198] In response, to receiving the request, the method 700 may include operation 720 which comprises establishing a secure enclave (e.g., the assessment engine 300 described above in relation to Figure 5). A secure enclave is to be understood to mean a communicatively isolated environment within the overall networked environment that the methods disclosed herein are carried out within. In some examples, it may be preferred for the secure enclave to be established on a cloud server associated with the client device (i.e. , the model owner).

[0199] It may be preferable for the assessment server to establish the secure enclave. For example, the client device may provide the assessment server with the necessary permissions and instructions (e.g., in the form of an executable script) to be able to establish the secure enclave on a cloud-based server associated with the client device.

[0200] Independently of any establishment of the secure enclave, the method 700 may further include operation 730 which comprises selectively retrieving and combining raw datasets 304a-j to constitute an assessment dataset against which the Al model will be assessed.

[0201] Each raw dataset may have associated therewith ground truth data for comparison with output data generated by the Al model when applied to the raw dataset for determining the accuracy score of the Al model. As discussed above, the ground truth data may be data that is indicative of a model output that corresponds to an output that would be generated by an Al model that is perfectly (i.e., 100%) accurate.

[0202] Each raw dataset may further have associated therewith a quality score indicative of a quality of the raw dataset.

[0203] In the context of facial image analysis, the quality score of each raw dataset may be indicative of one or more of: a degree of occlusion of the one or more faces depicted in the corresponding image; a lighting level associated with the corresponding image; an imaging angle from which the corresponding image is capture; and / or a degree of obscuration of the one or more faces depicted in the corresponding image. The degree of occlusion may be expressed, for example, for each face depicted in the image, as a proportion of said face that is occluded (i.e., blocked) from view in the image. The lighting level may be expressed, for example, numerically as an effective brightness of the image. The imaging angle may be expressed, for example, as an angle relative to a “face-on” angle in which the face of the imaged subject is looking directly towards the device that captured the image. The degree of obscuration may be expressed, for example, for each face depicted in the image, as a proportion of said face that is obscured e.g., by glasses, hair, a headdress or similar.

[0204] Each raw dataset may further have associated therewith a context label indicative of a category of date of the raw dataset.

[0205] In the context of facial image analysis, the context label of each raw dataset may be indicative of one or more of: a demographic group to which each of the one or more imaged subjects in the corresponding image belongs; and / or a use context in which the corresponding image was captured. The demographic group may, for example, be an ethnicity-based demographic (e.g., skin-tone), an age-based demographic, a sex-based demographic (e.g., the apparent sex of the subject), or any other demographic suitable for categorising image data. The use context may, for example, be an indication of what type of device captured the image (e.g., a webcam, a phone camera, CCTV, etc.), what type of activity the subject was engaged in when the image was captured (e.g., watching media on a personal device, walking around an exhibition or similar, commuting, or any other activity), or any other contextual information that may be beneficial for potential model users to be aware of.

[0206] The raw datasets 304a-j selectively and retrieved (e.g., from a data repository such as the benchmarking data database 200) to constitute the assessment dataset may be selected based on assessment criteria set by the model owner via the client device, as discussed above.

[0207] Once the assessment dataset has been properly constituted, for example through operation 730, the method 700 includes operation 740 which comprises providing the assessment dataset to the assessment engine 300. As discussed above, the assessment engine 300 may take the form of a secure enclave (e.g., as established in operation 720).

[0208] The assessment provider, via the assessment server, may encrypt the assessment dataset with a first encryption key that is inbuilt into the secure enclave so that no other third party can access the assessment dataset.

[0209] In some cases, the assessment engine 300 may already have a copy of the Al model to be assessed. For example, the assessment engine 300 may be a secure enclave that has been established on the model owner’s own cloud server resources, with the Al model to be assessed pre-loaded.

[0210] In other examples, the method 700 may include operation 750 which comprises providing a copy of the model 308 to the assessment engine 300.

[0211] The model owner, via an appropriate client device, may encrypt the copy of the model 308 with a second encryption key that is inbuilt into the secure enclave so that no other third party can access the model 308.

[0212] Once the assessment engine 300 is in possession of both the assessment dataset and the copy of the model to be assessed, the method 700 may then include one or more of operations 760, 762 and 764.

[0213] Operation 760 comprises determining an accuracy score of the model. Determining the accuracy score may include comparing the output data generated by the ground truth data associated with the assessment dataset to determine how accurate (or precise) the model is. The accuracy score of the Al model may, in some examples, be determined as an aggregate or average of a series of accuracy scores determined for each of the raw datasets that constitute the assessment dataset.

[0214] Operation 762 comprises determining a robustness score of the model. The robustness score (equivalently referred to as the variation score) may be indicative of the robustness of the model - i.e., how well the precision of the model is maintained as the quality of the input data decreases. This may be expressed, for example, as the precision of the model when the input data has a quality score of, e.g., 0.8 (or any other selected score) divided by the precision of the model when the input data has a quality score of e.g., 1 . The robustness score may be expressed in any other form as long as it provides information indicative of how well the performance of the model is maintained as the quality of the input data deteriorates. The robustness score may, in some examples, be expressed as multiple scores so that the rate of decay in performance of the model vs. the rate of decay in the quality of the input data can be inferred from the robustness score.

[0215] Operation 764 comprises determining an equity score of the model. The equity score (equivalently referred to as the fairness score) may be indicative of the fairness, or equitable accuracy, of the model - i.e. , how well the accuracy / precision of the model is maintained as the qualitative nature of the input data is varied. For example, in the context of facial image analysis models, the equity score may be indicative of how well the accuracy of the Al model is maintained for different demographics of subjects depicted in facial images. The equity score may be expressed in any form that provides the user with information indicative of how the accuracy of the Al model varies for different context labels. For example, the equity score may - in some examples - be expressed as multiple scores, or a function that defines a relationship between the accuracy of the model and changes in the context labels of the raw datasets. In some examples, the equity score may be based on the absolute values of the determined accuracy scores for each raw dataset while in other examples, the equity score may be based on a ratio of the determined accuracy scores for each raw dataset with a reference (e.g., a maximum) accuracy score.

[0216] Finally, the method 700 may then include one or more of operations 770 and 780. Operation 770 comprises communicating the results of the assessment to the model owner (e.g., to the client device). Meanwhile operation 780 comprises communicating the results of the assessment to the marketplace server (e.g., to the model store 400) for storage and access by potential model users. The manner in which the results of the assessment are stored in the model store 400 may be based on criteria set by the model owner (e.g., via the client device) and take the form of public / private partitioning as described above in relation to Figure 6.

[0217] After the results of the assessment have been communicated to the relevant parties, operation 790 may comprise dismantling the secure enclave and erasing its contents to preserve the data security and confidentiality of the benchmarking assessment process.

[0218] The features disclosed in the foregoing description, or in the following claims, or in the accompanying drawings, expressed in their specific forms or in terms of a means for performing the disclosed function, or a method or process for obtaining the disclosed results, as appropriate, may, separately, or in any combination of such features, be utilised for realising the invention in diverse forms thereof.

[0219] While the invention has been described in conjunction with the exemplary embodiments described above, many equivalent modifications and variations will be apparent to those skilled in the art when given this disclosure. Accordingly, the exemplary embodiments of the invention set forth above are considered to be illustrative and not limiting. Various changes to the described embodiments may be made without departing from the spirit and scope of the invention.

[0220] For the avoidance of any doubt, any theoretical explanations provided herein are provided for the purposes of improving the understanding of a reader. The inventors do not wish to be bound by any of these theoretical explanations. Any section headings used herein are for organizational purposes only and are not to be construed as limiting the subject matter described.

[0221] Throughout this specification, including the claims which follow, unless the context requires otherwise, the word “comprise” and “include”, and variations such as “comprises”, “comprising”, and “including” will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps.

[0222] It must be noted that, as used in the specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” one particular value, and / or to “about” another particular value. When such a range is expressed, another embodiment includes from the one particular value and / or to the other particular value. Similarly, when values are expressed as approximations, by the use of the antecedent “about,” it will be understood that the particular value forms another embodiment. The term “about” in relation to a numerical value is optional and means for example + / - 10%.

[0223] References

[0224] A number of publications are cited above in order to more fully describe and disclose the invention and the state of the art to which the invention pertains. Full citations for these references are provided below. The entirety of each of these references is incorporated herein.

[0225] NIST (6 April 2023) Face Recognition Vendor Test (FRVT) Ongoing, https: / / www.nist.gov / programs- projects / face-recognition-vendor-test-frvt-ongoing

[0226] Reference Numerals

[0227] 100 Assessment system

[0228] 102 Private data portal

[0229] 104 Private data portal

[0230] 106 Public data portal

[0231] 108 Private model portal

[0232] 1 10 Private model portal

[0233] 1 12 Private model portal

[0234] 1 14 Public model download portal

[0235] 1 16 Public model download portals

[0236] 1 18 Private model download portals

[0237] 200 Assessment data database

[0238] 202 Database partition

[0239] 210 Clean database segment

[0240] 212a-m Clean assessment data

[0241] 220 Dirty database segment

[0242] 222a-n Dirty assessment data Assessment data database entry Datatype tag Metadata

[0243] Image quality tag Image variation tag Use context tag Accessibility tag Assessment data content Raw data

[0244] Assessment labels

[0245] Assessment engine a-k Tags a-j Raw datasets

[0246] First encrypted item Model for assessment

[0247] Second encrypted item Engine processor

[0248] Secure enclave environment

[0249] Model store

[0250] Model store partition

[0251] Public model store segment Private model store segment First benchmarked data entry Second benchmarked data entry Third benchmarked data entry Fourth benchmarked data entry Fifth benchmarked data entry User interface

[0252] Name input

[0253] Use context input

[0254] Training datatype input Training data portal

[0255] Assessment data source variation selector Assessment data context variation selector Assessment datatype selector Downloading user interface Precision priority selection tool Robustness priority selection tool Fairness priority selection tool

[0256] Filtering tool Sorting tool Database presentation

[0257] Security lock / unlock tool

[0258] Method of assessing Al model

[0259] Receive request to assess

[0260] Establish secure enclave

[0261] Receive first encryption key

[0262] Receive second encryption key

[0263] Retrieve and combine raw datasets

[0264] Provide assessment dataset

[0265] Provide copy of model to be assessed

[0266] Determine accuracy score

[0267] Determine robustness score

[0268] Determine equity score

[0269] Communicate to client device

[0270] Communicate to marketplace server

Claims

Claims:1 . A computer-implemented method of assessing performance accuracy of an artificial intelligence, Al, model, the method comprising: providing, by an assessment server, an assessment dataset to a cloud-based secure enclave, wherein a copy of the Al model is stored on the cloud-based secure enclave, and wherein the cloud-based secure enclave defines a communicatively isolated processing environment within the network; receiving output data from the secure enclave, wherein the output data is generated by applying the Al model to the assessment dataset within the secure enclave; and determining, based on the output data, an accuracy score for the Al model by comparing the generated output data to ground truth data associated with the assessment dataset.

2. The computer-implemented method according to claim 1 , wherein the cloud-based secure enclave is established on a cloud server associated with the client device.

3. The computer-implemented method according to claim 1 or 2 further comprising: establishing the secure enclave in response to a received request to assess the Al model.

4. The computer-implemented method according to any preceding claim further comprising: providing, by the client device, a copy of the Al model to the cloud-based secure enclave.

5. The computer-implemented method according to claim 4, wherein establishing the secure enclave comprises: providing, by the assessment server, a first encryption key; providing, by the client device, a second encryption key; and establishing the secure enclave as an isolated processing environment having first and second encryption keys stored therein, and wherein providing the assessment dataset to the cloud-based secure enclave comprises providing a copy of the assessment dataset encrypted with the first encryption key, wherein providing the copy of the Al model to the cloud-based secure enclave comprises providing a copy of the Al model encrypted with the second encryption key, and wherein the encrypted copies of the assessment dataset and Al model are decryptable within the cloud-based secure enclave.

6. The computer-implemented method according to any preceding claim, further comprising retrieving the assessment dataset from a data repository.

7. The computer-implemented method according to claim 6, wherein the data repository is remote from the assessment server.

8. The computer-implemented method according to claim 6 or 7, wherein retrieving the assessment dataset from the data repository comprises: selectively combining one or more raw datasets from amongst a plurality of raw datasets stored in the data repository to define the assessment dataset, wherein each raw dataset has associated therewith: ground truth data for comparison with output data generated by the Al model when applied to the raw dataset for determining the accuracy score of the Al model.

9. The computer-implemented method according to claim 8, wherein each raw dataset further has associated therewith a quality score indicative of a quality of the raw dataset, and the method further comprises: determining, based on the output data, a robustness score that is indicative of variation in the accuracy score across a range of quality scores associated with the one or more raw datasets constituting the assessment dataset.

10. The computer-implemented method according to claim 8 or 9, wherein each raw dataset further has associated therewith a context label indicative of a category of data of the raw dataset, and the method further comprises: determining, based on the output data, an equity score that is indicative of variation in the accuracy score across a range of context labels associated with the one or more raw datasets constituting the assessment dataset.

11. A computer-implemented method of assessing performance accuracy of an artificial intelligence, Al, model, the method comprising: selectively combining, from a data repository, one or more raw datasets from amongst a plurality of raw datasets stored in the data repository to define an assessment dataset, wherein each raw dataset has associated therewith:(i) ground truth data for comparison with output data generated by the Al model when applied to the raw dataset for determining an accuracy score of the Al model, and(ii) a quality score indicative of a quality of the raw dataset; applying the Al model to the assessment dataset to generate output data; determining, based on the output data, an accuracy score for the Al model by comparing the generated output data to ground truth data associated with the assessment dataset; and determining, based on the output data, a robustness score that is indicative of variation in the accuracy score across a range of quality scores associated with the one or more raw datasets constituting the assessment dataset.

12. The computer-implemented method according to claim 11 , wherein each raw dataset further has associated therewith a context label indicative of a category of data of the raw dataset, and the method further comprises:determining, based on the output data, an equity score that is indicative of variation in the accuracy score across a range of context labels associated with the one or more raw datasets constituting the assessment dataset.

13. A computer-implemented method of assessing performance accuracy of an artificial intelligence, Al, model, the method comprising: selectively combining, from a data repository, one or more raw datasets from amongst a plurality of raw datasets stored in the data repository to define an assessment dataset, wherein each raw dataset has associated therewith:(i) ground truth data for comparison with output data generated by the Al model when applied to the raw dataset to determine an accuracy score of the Al model, and(ii) a context label indicative of a category of data of the raw dataset; applying the Al model to the assessment dataset to generate output data; determining, based on the output data, an accuracy score for the Al model by comparing the generated output data to ground truth data associated with the assessment dataset; and determining, based on the output data, an equity score that is indicative of variation in the accuracy score across a range of context labels associated with the one or more raw datasets constituting the assessment dataset.

14. The computer-implemented method according to any of claims 11 to 13, wherein the data repository is remote from the assessment server.

15. The computer-implemented method according to any of claims 8 to 14, further comprising receiving a request to assess an Al model held by the client device, said request comprising assessment criteria parameters, wherein the one or more raw datasets selectively combined to define the assessment dataset are selected based on the received assessment criteria parameters.

16. The computer-implemented method according to any preceding claim, further comprising: communicating one or more of the determined accuracy score, the determined robustness score, and / or the determined equity score to the client device.

17. The computer-implemented method according to any preceding claim, further comprising: communicating one or more of the determined accuracy score, the determined robustness score, and / or the determined equity score to a marketplace server, wherein the marketplace server includes a model store that stores a plurality of model datasets, each model dataset including information indicative of at least one of a respectively determined accuracy score, robustness score, and / or equity score associated with a corresponding Al model.

18. The computer-implemented method according to claim 17, further comprising: communicating metadata associated with the Al model to the marketplace server; andgenerating, by the marketplace server, a model dataset for the Al model, said model dataset including the communicated metadata and the one or more communicated scores.

19. The computer-implemented method according to claim 18, wherein the metadata includes one or more data items indicative of: a name of the Al model; information associated with training data used to train the Al model; information associated with the assessment dataset; information indicative of how a potential model user may access the Al model; and / or one or more context labels indicative of a preferred use context of the Al model.

20. The computer-implemented method according to claim 18 or 19, wherein each of one or more data items constituting the metadata comprises its own privacy tag, each privacy tag being independently modifiable by the client device such that each data item is independently configured to be publicly or privately accessible by a potential model user in accordance with the corresponding privacy tag.21 . The computer-implemented method according to any of claims 17 to 20, further comprising: in response to receiving a request from a potential model user, retrieving one or more of the stored model datasets from the model store; and displaying, by the marketplace server, benchmarking information related to each of the models associated with the one or more retrieved model datasets to the potential model user.

22. The computer-implemented method according to claim 21 , wherein the display of the benchmarking information is an interactive display that is modifiable on request such that the displayed information related to each of the models is sortable and / or filterable to arrange and / or convey selected information to the potential model user.

23. The computer-implemented method according to claim 22, wherein the benchmarking information is sortable and / or filterable according to one or more of a determined accuracy score, a determined robustness score, and / or a determined equity score associated with each of the models.

24. The computer-implemented method according to any of claims 17 to 23, wherein the marketplace server and the assessment server are the same server.

25. The computer-implemented method according to any of claims 17 to 24 as dependent on any of claims 1 to 10, further comprising: dismantling the secure enclave and deleting its contents after communicating the one or more determined scores.

26. The computer-implemented method according to any preceding claim, wherein the request to assess the Al model includes a request for a particular level of assessment.

27. The computer-implemented method according to claim 26, wherein a comparatively higher level of assessment tests the performance accuracy of the Al model: for distinguishing between comparatively more similar input data, and / or against a comparatively larger assessment dataset, and / or against a comparatively more varied assessment dataset.

28. The computer-implemented method according to any preceding claim, wherein the Al model is an Al model configured to analyse images and / or videos depicting the faces of one or more subjects.

29. The computer-implemented method according to claim 28, wherein the output generated by the Al model includes one or more of: an identification of a subject depicted in an image; a verification that the same subject is depicted in two or more images; an attentiveness output indicative of a level of attentiveness of a subject depicted in an image; an emotional output indicative of an emotional response of a subject depicted in an image; and / or a behavioural output indicative of a behavioural response of a subject depicted in an image.

30. The computer-implemented method according to claim 28 or 29 as dependent on any of claims 9 or 11 , or any claim dependent thereon, wherein the quality score of each raw dataset is indicative of one or more of: a degree of occlusion of the one or more faces depicted in the corresponding image; a lighting level associated with the corresponding image; an imaging angle from which the corresponding image is captured; and / or a degree of obscuration of the one or more faces depicted in the corresponding image.31 . The computer-implemented method according to any of claim 28 to 30 as dependent on any of claims 10, 12 or 13, or any claim dependent thereon, wherein the context label of each raw dataset is indicative of one or more of: a demographic group to which each of the one or more imaged subjects in the corresponding image belongs; and / or a use context in which the corresponding image was captured.

32. A computer-readable medium comprising instructions that, when executed by one or more networked computers, cause the one or more networked computers to carry out the method of any preceding claim.

33. A computer program product comprising logic that, when executed by one or more processors, causes the one or more processors to carry out the method of any of claims 1 to 31 .

34. A networked system comprising: an assessment server and a client device communicatively linked together and configured to carry out the method of any of claims 1 to 31 .

35. The networked system of claim 34 further comprising a marketplace server that includes a model store that stores a plurality of model datasets, each model dataset including information indicative of at least one of a respectively determined accuracy score, robustness score, and / or equity score associated with a corresponding Al model.

Images