Combining model outputs with combined model outputs

Abstract

The present invention relates to a prediction system 100 for applying multiple trained models to an input instance, for example, for detecting or segmenting an object in a medical image. The multiple trained models form a combined model. The trained models determine a model output for the input instance by determining a representation of the input instance in a common latent space and determining the respective model outputs therefrom. The combined model further includes dataset fingerprints for the multiple trained models that characterize the representation of that training instance in the latent space. To determine the combined model output for the input instance, correspondence scores are determined between the input instance and the multiple trained models, indicating the correspondence between the input instance and each training dataset in the latent space. The combined model output is determined by combining the respective models according to the correspondence scores.

Description

[Technical Field]

[0001] The present invention relates to a prediction system for applying multiple trained models to an input instance, for example for detecting or segmenting objects in medical images. The present invention further relates to a training system for fingerprinting the trained models, and a corresponding computer-implemented method. The present invention also relates to a combination system for determining a combined model from multiple trained models, and a corresponding computer-implemented method. The present invention further relates to a computer-readable medium. [Background technology]

[0002] Epilepsy is a chronic brain disorder and one of the most common neurological disorders worldwide. According to the World Health Organization, approximately 50 million people worldwide suffer from epilepsy. One of the most common methods for diagnosing and monitoring epilepsy is based on the noninvasive measurement of the brain's electrical activity, called electroencephalography (EEG). One common task associated with clinical practice is offline seizure labeling ("segmentation"). For example, to monitor disease progression and plan treatment, a patient's EEG is continuously monitored, e.g., for an hour or more. After signal acquisition is complete, a physician can review the EEG signal and manually label the period during which epileptic EEG activity was observed for subsequent analysis, such as localization of the source of the epileptic activity.

[0003] Segmenting EEG signals is an example of a task in (medical) signal processing or (medical) image processing where hard-coded and / or statically constructed knowledge representations are increasingly being replaced with solutions created using artificial intelligence (AI) such as machine learning (ML) and / or deep learning (DL) methods. Machine learning is based on formulating a solution in a parametric form and determining model parameters through training, e.g., in an iterative process. For example, in supervised learning, a system may be presented with a set of input instances and their corresponding desired model outputs. The collection of input / output pairs may be referred to as a training dataset. The training system typically analyzes the training data and generates a trained model that can be used to map new, previously unseen input instances (called "inference"). An important property of machine learning models is their ability to generalize, e.g., to perform well on new, previously unseen input instances after training.

[0004] In the setting of epileptic seizure detection, it is known that epileptic patterns in EEG signals typically vary significantly from patient to patient. For example, epileptic behavior in some patients may be very similar to the normal behavior of other patients, and vice versa. To provide good seizure segmentation quality, models need to be trained on numerous or even vast numbers of examples of epileptic behavior recorded from a variety of patients. More generally, to build accurate machine learning models with good generalization capabilities, it is usually beneficial to use as much training data as possible from as many different sources as possible. For example, in the case of medical imaging data, it is important to collect as much data as possible to ensure that the machine learning model performs well on different populations, data from different medical device vendors, etc. For highly heterogeneous data, the size of the training dataset is particularly important. For example, it is important to avoid bias toward rare diseases or conditions with low prevalence, for which machine learning models built with a relatively small number of subjects suffer from generalization difficulties.

[0005] In practice, building a sufficiently large dataset often means collecting data from multiple organizations, which may represent, for example, different countries and / or different cohorts in terms of gender, age, ethnicity, etc. For example, in the case of epileptic seizure detection, relatively small databases of labeled epileptic seizure EEG data are available in clinics, but combining this data into a single dataset is extremely difficult in practice due to technical, legal, and ethical concerns. Like other medical data, EEG data is highly sensitive, so its sharing is highly regulated, and confidentiality, consent, privacy, and geographic / residency requirements make it virtually impossible to collect the data in a single location. Furthermore, the exchange of non-personal information can be practically difficult due to confidentiality concerns. For example, systems with trained models deployed at customer sites improve as the system learns from the data used, but customers are typically hesitant to provide such information. In general, data sharing restrictions are common throughout the healthcare domain and in many other domains, such as finance and commerce, where trade secrets and confidential information cannot be shared.

[0006] In the paper "Multi-center machine learning in imaging psychiatry: A meta-model approach" by Petr Dluhos et al. (incorporated herein by reference), they propose creating a meta-model by combining support vector machine (SVM) classifiers trained on respective local datasets without sharing underlying training datasets, such as medical images or other personal data. An SVM model is constructed to separate patients from controls based on three different imaging features obtained from structural MRI scans. The meta-model is an average model calculated as the geometric mean of the SVM weights of the local models. In the presented experiments, the distributions of confounding parameters, such as age, gender, and handedness, between patients and controls were balanced, and the individual datasets were normalized before calculation. Summary of the Invention [Problem to be solved by the invention]

[0007] Improved techniques are needed for applying multiple trained models to input instances, where multiple trained models are trained on respective training datasets, without the need for each training dataset to be collected and processed in a single location, as in, for example, a medical sensor data classification or regression setting. For example, it would be desirable to further increase the accuracy of such techniques. It would be particularly desirable to have high accuracy for models combined from less homogeneous datasets, such as datasets where the patient age / gender / ... distributions vary significantly between datasets, and / or where different types of sensors, such as different or configured medical imaging scanners, are used between datasets. [Means for solving the problem]

[0008] According to a first aspect of the present invention, there is provided a prediction system for applying a plurality of trained models to an input instance, as defined by claim 1. According to a further aspect of the present invention, there is provided a training system for fingerprinting trained models, as defined by claim 8. According to a further aspect of the present invention, there is provided a combination system for determining a combined model from a plurality of trained models, as defined by claim 10. According to a further aspect of the present invention, there is provided a computer-implemented method for applying a plurality of trained models to an input instance, as defined by claim 12. According to a further aspect of the present invention, there is provided a computer-implemented method for fingerprinting trained models, as defined by claim 13. According to a further aspect of the present invention, there is provided a computer-implemented method for determining a combined model from a plurality of trained models, as defined by claim 14. According to a further aspect of the present invention, there is provided a computer-readable medium, as defined by claim 15.

[0009] Various functions include determining a combined model and using such a combined model to make predictions. Here, the term "prediction" is used in the general machine learning sense of the word, i.e., performing inference on previously unseen input instances based on a training dataset, such as, for example, regression or classification tasks. Thus, the term "prediction" does not imply a temporal relationship and also includes object detection and delineation, classification, and labeling. Combined models typically operate on sensor data. As an example, the input instance includes an image or a stack of multiple images. As another example, the input instance includes single-channel or multi-channel time-series sensor data. For example, such time-series sensor data includes spatiotemporal physiological measurements, such as vital parameters from an intensive care unit (ICU), e.g., electroencephalography (EEG) data, electrocardiogram (ECG) data, and / or 2D or 3D medical images.

[0010] The model used for prediction is a "combined model" in the sense that it includes multiple trained models. A combined model is also referred to in the art as a metamodel or ensemble model, but these terms are more specifically used to refer to the combination of multiple trained models each trained on the same dataset. However, in various embodiments, the multiple trained models are trained on respective training datasets of different organizations, for example. For example, these organizations cannot share their training datasets with each other, but would like to be able to create and use combined models based on their respective training datasets. The multiple trained models have a common architecture, for example, such that each trained model can determine model output from input instances using the same procedure, but uses a respective set of parameters obtained by training on the respective datasets. However, this is not necessary, as different types of trained models can be obtained for different training datasets, for example, as a result of manual or automatic model selection.

[0011] Interestingly, multiple trained models typically share a common latent space defined by the combined model. Generally, a latent space is an intermediate representation of input instances that machine learning models use to determine model outputs. Typically, the latent space representation represents low-level features of image / sensor data. For example, the latent space may include the values ​​of neurons in the inner layers of a neural network or a deep Gaussian process, the output of any type of embedding model, such as the encoder portion of an autoencoder, or the values ​​of manually constructed features extracted from the input data. Thus, the trained model is configured to determine the model output for an input instance by determining a representation of the input instance in this latent space and determining the model output therefrom. The portion of the trained model that determines the latent space representation is referred to throughout as the "feature extractor," and the portion of the trained model that determines the model output from the latent space representation is referred to as the "predictive model," e.g., a classifier or regression model.

[0012] As described in more detail elsewhere, multiple trained models can each use the same feature extractor, but they can also use different feature extractors, so long as the results of comparing the latent spatial representations of the input instances with the latent spatial representations of the training instances are measurable and consistent across the trained models. For example, as described below, the trained models include improved versions of pre-trained feature extractors that have been improved to improve the performance of the trained models on their respective training datasets. In this and other similar cases, each trained model uses a different feature extractor, e.g., differing only in parameters, but the differences between the latent spatial representations defined by each feature extractor are still comparable.

[0013] Interestingly, the various features include not only trained models but also combined models that include respective dataset fingerprints for each trained model. Such dataset fingerprints characterize at least the latent spatial representations of the training instances of the training dataset for the trained model. For example, the dataset fingerprints include statistical properties of the latent spatial representations of the training instances, such as parameters of a probability distribution that fits some or all features of the latent spatial representations. Other examples of dataset fingerprints are provided throughout.

[0014] By characterizing the latent representations of training instances, the dataset fingerprint makes it possible to determine, given a latent representation of an instance, e.g., a non-training instance, the deviation of the latent representation of the instance from the latent representation of the training instance, thereby establishing whether the instance corresponds to the instance on which the model was trained. In other words, based on the dataset fingerprint, it is possible to determine whether an instance is an outlier with respect to a particular training dataset, e.g., as unlikely or unlikely to occur in the training population or in the population from which the training dataset was drawn.

[0015] Preferably, dataset fingerprints are aggregated to represent the entire training dataset, rather than including, for example, the latent spatial representation of a specific training instance. Therefore, information about any single training instance of the training dataset is typically not obtainable from the dataset fingerprint, thereby enabling dataset fingerprints to be shared without affecting privacy, e.g., in the case of medical data, or confidentiality, e.g., of transactions, e.g., in financial applications. This is aided by the use of latent representations in dataset fingerprints that include other information about the input features, even in aggregated approaches, which generally leak more information about a specific training instance than including information about the latent spatial representation. Dataset fingerprints can also be configured to satisfy anonymity properties for the training dataset, e.g., k-anonymity, by applying techniques known per se, such as adding noise or removing data calculated from an insufficient number of training instances. Therefore, because dataset fingerprints are aggregated, they can be shared between organizations, e.g., by being included in a shared combined model, even in situations where training datasets cannot be shared, e.g., due to privacy constraints.

[0016] While privacy is an important aspect, it is not the only reason to prefer data fingerprinting based on latent representations; for example, latent representations can be smaller than input instances, sometimes much smaller. As a result, characterizing training datasets in terms of latent spatial representations can save a lot of storage and communication. For example, training datasets for medical images can easily be gigabytes, terabytes, or even petabytes in size. As a result, when transferring some or all of the training instances, for example, to determine which instances are best represented in which training datasets, a lot of communication and / or computation can be saved.

[0017] As the inventors have recognized, the use of dataset fingerprints allows the combined model to be effectively applied as an ensemble that is dynamically adapted to a particular input instance, rather than as a fixed ensemble of the trained models it contains, by taking into account the relevance of each trained model for that input instance. Essentially, as described below, the combined model and the information about the relevance of the trained models, captured as correspondence scores, are considered together as a dynamically constructed ensemble that is optimized for that input instance.

[0018] For example, by taking into account the correspondence scores when selecting or weighting each trained model, a more accurate overall model output can be obtained. For example, consider an ensemble including a first and a second machine learning model. Applying a fixed ensemble model to a first and a second instance that are more similar to the first and second machine learning models, respectively, can result in suboptimal results by assigning less importance to the output of the first model for the first instance, by assigning less importance to the output of the second model for the second instance, or both. For example, the first instance may be an outlier for the second model, resulting in unpredictable results that may adversely affect the accuracy of the overall output. However, by dynamically adapting the ensemble based on the correspondence scores as described herein, the overall model output for the first instance can be based more heavily on the model output of the first model, and the overall model output for the second instance can be based more heavily on the model output of the second model. Thus, the overall accuracy of the combined model is improved while using only aggregate information about the training dataset, e.g., the trained model and the dataset fingerprint. For example, the improvement in accuracy can be measured in terms of various objective measures, such as, for example, sensitivity and / or specificity and / or DICE coefficient.

[0019] Specifically, when applying the combined model to an input instance, a correspondence score between the input instance and multiple trained models is determined based on the representation of the input instance in latent space and the dataset fingerprints of the trained models. The manner in which the latent space representation is compared to the fingerprints depends on the exact fingerprints used. For example, if the fingerprint includes the mean or median of one or more features, Euclidean distance or weighted Minkowski distance can be used. Various other examples are described throughout. The same latent space representation can be used to determine each correspondence score; for example, this is appropriate when the trained models have a common feature extractor. However, for example, each latent feature representation determined by each feature extractor of each trained model is also used to determine each correspondence score.

[0020] The correspondence score between an input instance and a trained model indicates the correspondence between the input instance and the training dataset of the trained model. Therefore, the accuracy of the model output can be improved by assigning a relatively high importance to the model output of a model with a high correspondence. Thus, the model output of the input instance is determined for one or more of the multiple trained models, for example, all trained models that sufficiently correspond to the training instance or a strict subset of the trained models. The determined model outputs are then combined into the combined model output according to the determined correspondence scores of each trained model, for example, as an average, a weighted average, a majority score, a weighted majority score of the subset of sufficiently corresponding models, or any other manner in which the model outputs of trained models with a high correspondence have a greater influence on the combined model output. For example, the model outputs of a strict subset of the trained models can be combined into the combined model output.

[0021] Advantageously, a dataset fingerprint for a trained model is determined based on a training dataset. This is typically done when the model is being trained; e.g., the training and determination of the dataset fingerprint typically occur at the same organization. The trained model and corresponding dataset fingerprint can then be provided to another party, who can combine multiple such trained models and corresponding dataset fingerprints into a combined model and upload it to a repository of trained models with associated fingerprints, etc. The combining can be performed, for example, by the organization applying the model, e.g., a hospital. For example, an organization can train a model to generate accompanying dataset fingerprints, share the model and fingerprints with one or more other organizations, and receive respective trained models and fingerprints from the other organizations in return.

[0022] However, trained models and fingerprints can also be collected centrally by an organization that does not necessarily need to apply the models to new instances itself. For example, provisioning of combined models can be offered as a service. Interestingly, such combining parties that perform the combining methods or operate the combining system as described herein also perform various other functions related to the combined model. For example, the combining parties coordinate the overall training and fingerprint determination process, for example, by providing training parameters or hyperparameters, or by specifying the format of the trained models and / or fingerprints, such as the machine learning model type and / or architecture. Specifically, when combining models and fingerprints into a combined model, validation of the combined model can optionally be performed on a validation dataset to also provide quality assurance of the combined model. Validation can include determining the contribution of individual models to the overall accuracy, for example, by comparing the accuracy of the combined model with the accuracy of a combined model in which one of the trained models has been removed. Validation results, such as the contributions of individual models, can be provided to users as feedback on the inner workings of the combined model and / or can be used automatically, for example, by removing models that negatively impact accuracy. The impact of each model on the combined model, evaluated on actual word sequences in the input data, is also used to reward, for example financially, entities that contributed to individual models.

[0023] Interestingly, the use of the techniques described herein allows for the use of knowledge from multiple models contained in a combined model, e.g., at least three, at least five, or at least ten such models. The respective training datasets for each model do not necessarily need to be very large. In contrast, transfer learning techniques known in the art typically use a single training dataset to obtain a pre-trained model, allowing this pre-trained model to be improved at each educational institution where it is deployed, but do not allow for knowledge sharing between these institutions. Furthermore, in transfer learning, accuracy is highly dependent on the quality of the initial dataset used to obtain the pre-trained model, e.g., requiring a large initial dataset. Furthermore, the dataset used to improve the pre-trained model must be relatively large and available to a single institution. For example, in image classification, transfer learning has produced results based on very large datasets with ground truth labeling provided by large-scale crowdsourcing efforts. In various cases, for example in medical image classification, it is difficult to collect such large datasets in one place, and even more difficult to obtain labels using crowdsourcing, so such labeling must be performed by doctors, and the images usually cannot be shared with the outside world, for example, for privacy reasons. Another advantage compared to transfer learning is that it avoids so-called "catastrophic forgetting," i.e., the tendency in training neural networks to completely and suddenly forget previously learned information.

[0024] For example, obtaining a single large labeled dataset for detecting epileptic seizures in electroencephalography (EEG) signals is extremely challenging. However, for this application, relatively small but high-quality datasets exist from several clinics with significantly different populations and environments, to which the techniques described herein can be successfully applied.

[0025] Another advantage of the techniques provided herein is that they work with a variety of machine learning models. For example, each trained model included in a combined model must share a common latent space, but there are many different types of machine learning models that meet this property, such as many neural networks or deep Gaussian process architectures. As mentioned above, the types of models used by various organizations may also differ from each other, or at least the parameters or hyperparameters used across training datasets may differ. For example, for a relatively small or relatively coherent dataset, different hyperparameters for the predictive model may be appropriate compared to a larger or less coherent dataset. Yet another advantage is that the training of each trained model can be largely independent of each other; for example, synchronization may not be required while each trained model is being trained on its respective dataset. The combined models described herein also allow for the addition, removal, and / or replacement of trained models from the combined model, thereby allowing for continuous improvement of the combined model as more or better data becomes available.

[0026] Optionally, the determined model outputs can be combined into a combined model output by applying a trainable combined model to the determined correspondence scores and model outputs. The trained combined model is typically included in the combined model and is therefore distributed along with the trained models and their fingerprints for applying the combined model. As described, various untrainable techniques can be used to combine the determined model outputs into the combined model, such as averaging or majority voting. However, as the inventors have recognized, the accuracy of the determined model outputs can be further improved by training the combined model. For example, the combined model can be a decision tree for selecting one or more specific model outputs to combine and use, a linear regression model for calculating the combined model output from the determined model outputs, etc. The combined model can be trained, for example, by the party building the combined model, but it is also possible to train the combined model using distributed learning techniques on the training datasets of the respective trained models, which is more feasible than an overall distributed learning approach because the dataset used to train the combined model is relatively small and allows for smaller inputs.

[0027] Optionally, a confidence score for the combined model output is determined based on the determined correspondence scores. The confidence score indicates the correspondence between the input instance and the combination of the training datasets of the multiple trained models. As the inventors have recognized, the correspondence score indicates the correspondence between the input instance and the respective training datasets, and is therefore used not only as a measure of the relative fitness of the trained models, but also as a measure of the expected reliability of the combined model that can be compared between the input instances. For example, the number of model outputs combined in the combined model output is used as the correspondence score, or an appropriate combination of the correspondence scores, such as the average. If the confidence indicated by the confidence score does not correspond to a predefined threshold, for example, an error may be logged and / or flagged to the user in a perceptible manner.

[0028] Optionally, the dataset fingerprint of the trained model includes multiple cluster centroids in the latent space. The cluster centroids represent clusters of the training input instances. For example, a feature extractor of the trained model is used to obtain a representation of the training instances of the training dataset in the latent space, and a clustering algorithm known in the art is applied to the latent space representation to obtain multiple cluster centroids. A correspondence score between the input instance and the trained model is determined based on the similarity values ​​between the input instance and the multiple cluster centroids. By using clustering techniques known in the art, an appropriate aggregate summary of the various training instances of the training dataset can be obtained. The latent feature vectors of the input instances can be compared to the centroids in various ways, for example, by calculating cosine similarity, Euclidean distance, or weighted Minkowski distance. Depending on the application, using the minimum, maximum, or average distance to the centroid may be appropriate as the correspondence score. For example, the closest centroid distance may be preferred for accuracy, while the average or maximum centroid distance may be more appropriate for reliability.

[0029] Optionally, the dataset fingerprint of the trained model includes a generative model. The correspondence score between the input instance and the trained model is determined based on the likelihood of a latent spatial representation generated by the generative model. For example, as known in the art, various known generative models, such as a generative adversarial network (GAN) or an autoencoder or variational autoencoder (VAE) generator, can be trained on the dataset, and then, for an instance, the probability that the instance is generated by the generative model can be determined. The autoencoder model is trained to generate latent spatial representations of instances in the training dataset. In this case, the correspondence score of the input instance can be determined by determining a latent representation of the input instance and calculating the likelihood that the latent representation is generated by the autoencoder model, and similarly for other types of generative models. The autoencoder model can also correspond to the feature extractor of the trained model, and the correspondence score is determined by determining the likelihood that the input instance itself is generated by the autoencoder model. Interestingly, the trained generative model in the training dataset typically does not reveal information about the specific training instance, which may be privacy-sensitive.

[0030] Optionally, the determined model outputs are combined into a combined model output by calculating a weighted sum of the determined model outputs based on their respective correspondence scores. For example, using a weighted sum of the determined model outputs instead of selecting a subset of model outputs to use allows for a more granular use of correspondence scores, thereby improving accuracy. The weights in the weighted sum typically increase with the correspondence indicated by the correspondence score; for example, the weights may be proportional to the correspondence score, which represents the degree of correspondence, or inversely proportional to the correspondence score, which represents the difference between the dataset fingerprint and the input instance.

[0031] Optionally, the correspondence score between the input instance and the trained model is further based on the input instance and / or the model output of the trained model for the input instance. The dataset fingerprint of the trained model further characterizes the training instance and / or the training output, typically the label of the training dataset. For example, some or all features of the input instance or the trained model output are processed separately, such as by adding them to the latent spatial representation to determine the dataset fingerprint, or by including statistics on the input or output features in the fingerprint separately. In this way, accuracy can be further improved by determining a more accurate correspondence score. Note that if the model output is used to determine the correspondence score, this does not mean that the model output is also used in the combined model, e.g., only a subset of the determined model outputs is used.

[0032] Optionally, the input instances include time-series sensor data, e.g., of physiological measurements. This type of data, e.g., temporal vital sign segment data, occurs frequently in medical data analysis. Examples of physiological measurements include EEG data, ECG data, ICU monitoring data, etc. Optionally, the input instances include images, e.g., medical images. These types of data are typically relatively large and detailed, and therefore may contain a lot of sensitive information that is difficult or simply too large to exchange between organizations. Nevertheless, the techniques herein can be used to obtain a combined model.

[0033] Various types of trained models can be used, such as SVM classifiers, gradient boosting trees, neural networks, etc., as appropriate depending on the data to which they are applied. Various types of neural network architectures can be used, such as deep neural networks, convolutional neural networks, etc. Neural networks are also known as artificial neural networks. The number of parameters of the trained model can be very large, such as at least 10,000, or at least 1 million, or at least 100 million.

[0034] Optionally, a pre-trained feature extractor for determining the representation of the input instances in the latent space can be used. For example, multiple combined models share the same pre-trained feature extractor, which has been pre-trained on an initial dataset by the party determining the combined model, e.g., a publicly available dataset. Each trained model of the combined model is trained based on the pre-trained feature extractor. Optionally, this training involves improving the pre-trained feature extractor to increase accuracy, as known per se from the field of transfer learning. However, regardless of whether the feature extractor is improved or not, using a common pre-trained feature extractor among the trained models helps to ensure that the latent representations, or at least the correspondence scores resulting from the latent representations, are comparable among the trained models of the combined model.

[0035] Optionally, each trained model is trained based on a pre-trained predictive model for determining model output from representations of input instances in the latent space. Similar to the pre-trained feature extractor, this pre-trained predictive model is trained on an initial dataset, such as a publicly available dataset, by the party generating the combined model. When training each trained model, the pre-trained predictive model is refined based on the respective training dataset, for example, using techniques from transfer learning known per se. Basing the predictive models on a common pre-trained model not only improves the accuracy of the resulting models, but also improves the comparability of the outputs, for example, by providing similarly scaled outputs that can be more easily combined with the combined model output. Optionally, the party generating the combined model trains a feature extractor for determining representations of input instances in the latent space. For example, the feature extractor can be an initial feature that is refined in each trained model or a common feature extractor used in each of the trained models. It is also possible to train the feature extractor at least in part using distributed learning techniques on the training datasets of the respective trained models. This allows for at least improved accuracy while training the predictive models individually. Optionally, the parties generating the combined model train an initial predictive model to determine model outputs from representations of input instances in latent space, e.g., for refinement in each trained model.

[0036] Optionally, the party generating the combined model trains a combined model to combine model outputs into the combined model output to improve the accuracy of the combining operation. For example, training the combined model includes one or more iterations of determining combined model outputs for training input instances based on the outputs and corresponding scores of each trained model, deriving training signals from the differences between the determined combined model outputs and the training outputs corresponding to the training input instances, and adjusting the combined model according to the training signals. The combined model is included in the combined model.

[0037] Optionally, the party generating the combined model is configured to update the combined model by adding, deleting, and / or replacing fingerprints from the trained model and the respective datasets. Interestingly, this is more feasible than in the case of a single model trained on the combined datasets, since the respective trained models are relatively distant from each other. Thus, models based on the respective training datasets are obtained that are more flexible in adapting to changes in the available training data.

[0038] Optionally, a single entity fingerprints multiple trained models, and optionally, the multiple trained models are combined into a combined model as described herein. For example, a combined dataset is obtained and then split into multiple training datasets, such as multiple separate training datasets. Then, trained models for each training dataset are obtained, their fingerprints are determined as described herein, and the multiple trained models are combined into a combined model. Interestingly, because dynamic adaptive combination via latent features is better at adapting to different sub-cohorts combined with the original datasets, this leads to a more accurate model of the combined dataset compared to traditional training of a model of the combined dataset.

[0039] As will be appreciated, combined systems are also contemplated that include multiple training systems as described herein, a combined system as described herein, and one or more prediction systems as described herein. Some or all of the systems serve dual roles, e.g., the set of training systems overlaps with the set of prediction systems. In such combined systems, the training system trains a model and determines the dataset fingerprints that are provided to the combined system, thereby determining the combined model that is provided to the prediction system for application to input instances.

[0040] It will be understood by those skilled in the art that two or more of the above-described embodiments, implementations, and / or optional aspects of the invention may be combined in any manner deemed useful.

[0041] Modifications and variations of the systems described herein can be implemented by those skilled in the art based on this specification. Modifications and variations of any computer-implemented method and / or any computer program product corresponding to the described modifications and variations of the corresponding systems can be implemented by those skilled in the art based on this description.

[0042] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described by way of example in the following description and with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0043] [Figure 1] FIG. 1 illustrates a prediction system using a combined model. [Figure 2] FIG. 1 illustrates a training system for fingerprinting a trained model. [Figure 3] FIG. 1 illustrates a coupling system for determining a coupled model. [Figure 4]FIG. 10 illustrates a detailed example of how the combined model may be used to determine the combined model output. [Figure 5] FIG. 10 shows a detailed example of how to obtain a combined model. [Figure 6] FIG. 1 illustrates a computer-implemented method for applying multiple trained models to an input instance. [Figure 7] FIG. 1 illustrates a computer-implemented method for fingerprinting a trained model. [Figure 8] FIG. 1 illustrates a computer-implemented method for determining a combined model from multiple trained models. [Figure 9] FIG. 1 illustrates a computer-readable medium containing data.

[0044] It should be noted that the figures are purely schematic and are not drawn to scale. In the figures, elements corresponding to elements already described have the same reference numbers.

[0045] The following list of reference numbers is provided to facilitate interpretation of the drawings and should not be construed as limiting the scope of the claims. [Explanation of symbols]

[0046] 021, 022, 023 Data Storage 030 training dataset 040 Combined Model 041 trained models 042 Dataset Fingerprint 043 Bonding Model 044 Feature Extraction Model 050 validation dataset 071 Sensor 100 Prediction System 200, 201, 202 Training Systems 300 Combined System 120, 220, 320 data interface 121~124, 221~222, 321~324 Data communication 140, 240, 340 Processor Subsystem 160 Sensor Interface 380 Communication Interface 410 Input Instance 420, 520 Feature Extraction 430, 530 Latent spatial representation 440, 540 fingerprints 450 compatible score 460 model applicable 470 Model Output 480 Join operations 490 Combined Output 510 First Training Operation 511 Initial Prediction Model 515 Second Training Operation 555 training dataset 560 Training 590 Verification 600, 700, 800 Computer-Implemented Methods 900 Computer-Readable Medium 910 Non-transient Data DETAILED DESCRIPTION OF THE INVENTION

[0047] FIG. 1 illustrates a prediction system 100 for applying multiple trained models to input instances. The multiple trained models are trained with respective training datasets. The system 100 includes a data interface 120 and a processor subsystem 140 that communicate internally via data communication 121. The data interface 120 is for accessing the multiple trained models in the form of combined models 040. The combined models 040 define a latent space. The trained models are configured to determine a model output for an input instance by determining a representation of the input instance in the latent space and determining the model output therefrom. The combined models 040 further include a dataset fingerprint for each of the multiple trained models. The dataset fingerprint for the trained model characterizes the latent space representation of the training instance in the training dataset for the trained model. For example, the combined models may be received from another system, such as the combined system 300 of FIG. 3. System 100 may also be coupled with training system 200 and / or combination system 300 described herein, e.g., the system may train a model, provide the trained model to the combination system, and receive the combined model.

[0048] The processor subsystem 140 is configured to access the combined model 040 during operation of the system 100 using the data interface 120. For example, as shown in FIG. 1, the data interface 120 provides access 122 to an external data storage 021 containing said data 040. Alternatively, the data 040 may be accessed from internal data storage that is part of the system 100. Alternatively, the data 040 may be received from another entity over a network. In general, the data interface 120 may take a variety of forms, such as a network interface to a local or wide area network, e.g., the Internet, or a storage interface to internal or external data storage. The data storage 021 may take any known suitable form.

[0049] The processor subsystem 140 is further configured, during operation of the system 100, to obtain an input instance. The processor subsystem 140 is further configured to provide a combined model output to the plurality of trained models. To provide the combined model output, the processor subsystem 140 is configured, during operation of the system 100, to determine a correspondence score between the input instance and the plurality of trained models. The correspondence score between the input instance and the trained models indicates a correspondence between the input instance and a training dataset of the trained model. The correspondence score is based on a representation of the input instance in the latent space and a dataset fingerprint of the trained model. To provide the combined model output, the processor subsystem 140 is further configured, during operation of the system 100, to determine one or more model outputs of the plurality of trained models for the input instance. To provide the combined model output, the processor subsystem 140 is further configured, during operation of the system 100, to combine the model outputs into the combined model output according to the determined correspondence scores of the respective trained models.

[0050] Typically, the contents of the trained models and data fingerprints included in the combined model 040 are stored together with the combined model, e.g., in storage 021, e.g., in the same file and / or directory / folder. However, it is also possible to store the trained models and / or data fingerprints separately. For example, in some embodiments, the trained model includes links to the contents of one or more trained models and / or data fingerprints, e.g., by including a URL where the model or fingerprint can be accessed. Various other means of association are also contemplated and within the understanding of those skilled in the art.

[0051] As an optional component, the system 100 includes a sensor interface 160 or any other type of input interface for acquiring sensor data 124 from one or more sensors. The figure shows an electroencephalograph 071. The processor subsystem 140 is configured to acquire input instances based at least in part on the sensor data 124 via data communication 123, e.g., by converting the sensor data into input features included in the input instances. For example, a camera is configured to capture the electroencephalograph 124, and the processor subsystem 140 is configured to determine the input instances from the measurement data from a given time window, e.g., one or two seconds. Typically, the sensor interface is configured for various types of sensor signals, e.g., video signals, radar / LiDAR signals, ultrasound signals, etc. Alternatively, or in addition, the sensor data may be read from storage, e.g., from a recording system, e.g., a PAC (Picture Archiving and Communication) system, a Vendor Neutral Archive (VNA) system, and / or an Electronic Medical Record (EMR) system.

[0052] As an optional component, system 100 includes a display output interface or any other type of output interface (not shown) for outputting data to a rendering device such as a display. For example, the display output interface can generate display data for a display, whereby the display renders the data in a sensory-perceptible manner, e.g., as an on-screen visualization. For example, the output data may include determined combined model outputs, their confidence scores, etc.

[0053] The system 100 also comprises a communication interface (not shown) arranged for communication with other systems, such as a combined system for obtaining a combined model. The communication interface is explained in more detail in FIG. 3.

[0054] Various details and aspects of the operation of system 100, including optional aspects thereof, are further elucidated with reference to FIG. 4 and others.

[0055] Generally, system 100 is embodied as or in a single device or apparatus, such as a laptop or desktop-based workstation, or a server, or a mobile device, such as a smartphone. The device or apparatus includes one or more microprocessors that execute appropriate software. For example, the processor subsystem may be embodied by a single central processing unit (CPU), but may also be embodied by a combination or system of other types of processing units, such as a CPU and / or a graphics processing unit (GPU). The software may be downloaded and / or stored in corresponding memory, e.g., volatile memory such as RAM or non-volatile memory such as flash. Alternatively, the functional units of the system, e.g., the data interface and processor subsystem, may be implemented in the device or apparatus in the form of programmable logic, e.g., as a field programmable gate array (FPGA). Generally, each functional unit of the system is implemented in the form of a circuit. It should be noted that system 300 may also be implemented in a distributed manner, including different devices or apparatuses, such as distributed servers, e.g., in the form of cloud computing.

[0056] 2 shows a training system 200 for fingerprinting a trained model. The system 200 comprises a data interface 220 and a processor subsystem 240 that communicate internally via data communication 221. The data interface 220 is for accessing a training dataset 030. The data interface 220 is also for accessing a trained model 041 and / or a dataset fingerprint 042 determined by the system 200. The system 200 may be coupled to systems 100 or 300 as further described elsewhere.

[0057] The processor subsystem 240 is configured to access the data 030, 041, 042 during operation of the system 200 and using the data interface 220. For example, as shown in FIG. 2, the data interface 220 provides access 222 to an external data storage 022 containing the data 030, 041, 042. Alternatively, the data 030, 041, 042 may be accessed from internal data storage that is part of the system 200. Alternatively, the data 030, 041, 042 may be received from another entity over a network. In general, the data interface 220 may take a variety of forms, such as a network interface to a local or wide area network, e.g., the Internet, or a storage interface to internal or external data storage. The data storage 022 may take any known suitable form.

[0058] The processor subsystem 240 is further configured, during operation of the system 200, to train a model on the training dataset 030 to obtain a trained model 041. The trained model 041 is configured to determine a model output for an input instance by determining a representation of the input instance in a latent space and determining the model output therefrom. The processor subsystem 240 is further configured, during operation of the system 200, to determine a dataset fingerprint 042 for the trained model based on the training dataset 041. The dataset fingerprint 042 characterizes the latent space representation of the training dataset 041 on which the trained model was trained.

[0059] As an optional component, system 200 includes a sensor interface (not shown) for acquiring sensor data from one or more sensors. The training dataset is determined based at least in part on the acquired sensor data. The sensor interface is described in more detail with reference to FIG. 1. System 200 also includes a communication interface (not shown) arranged for communication with other systems, such as a combined system for providing a trained model and a dataset fingerprint. The communication interface is described in more detail in FIG. 3.

[0060] Various details and aspects of the operation of system 200, including optional aspects thereof, are further elucidated with reference to FIG. 5 and others.

[0061] Generally, system 200 is embodied as or within a single device or apparatus, such as a laptop or desktop-based workstation or server. The device or apparatus includes one or more microprocessors that execute appropriate software. For example, the processor subsystem may be embodied by a single central processing unit (CPU), but may also be embodied by a combination or system of other types of processing units, such as a CPU and / or a graphics processing unit (GPU). The software may be downloaded and / or stored in corresponding memory, e.g., volatile memory such as RAM or non-volatile memory such as flash. Alternatively, the functional units of the system, e.g., the data interface and processor subsystem, may be implemented in the device or apparatus in the form of programmable logic, e.g., as a field programmable gate array (FPGA). Generally, each functional unit of the system is implemented in the form of a circuit. It should be noted that system 200 may also be implemented in a distributed manner, including different devices or apparatus, e.g., distributed servers, e.g., in the form of cloud computing.

[0062] 3 illustrates a combining system 300 for determining a combined model from multiple trained models. The system 300 includes a data interface 320 and a processor subsystem 340 that communicate internally via data communication 321. The data interface 320 is for accessing an optional validation data set 050 that includes multiple validation input instances and corresponding validation outputs. The data interface 320 is also for accessing a combined model 040 determined by the system 300. The system 300 can also apply the determined combined model to input instances as described herein, e.g., the system 300 is combined with the system 100. The system 300 can also train a model to be included in the trained models as described herein, e.g., the system 300 is combined with the system 200.

[0063] The processor subsystem 340 is configured to access the data 040, 050 during operation of the system 300 using the data interface 320. For example, as shown in FIG. 3, the data interface 320 provides access 322 to an external data storage 023 containing said data 040, 050. Alternatively, the data 040, 050 may be accessed from internal data storage that is part of the system 300. Alternatively, the data 040, 050 may be received from another entity over a network. In general, the data interface 320 may take a variety of forms, such as a network interface to a local or wide area network, e.g., the Internet, or a storage interface to internal or external data storage. The data storage 023 may take any known suitable form.

[0064] The processor subsystem 340 receives multiple trained models and corresponding dataset fingerprints from multiple training systems during operation of the system 300, as described herein. Shown are two training systems 201, 202. Generally, the number of training systems can be at least two, at least 10, at least 50, etc. The trained models are configured to determine model outputs for input instances by determining a representation of the input instances in a latent space common to the multiple trained models and determining the model outputs therefrom. The dataset fingerprints for the trained models characterize the latent space representation of the training instances in the training dataset for the trained models. The processor subsystem 340 is further configured to combine the multiple trained models and corresponding dataset fingerprints into a combined model 040 during operation of the system 300 to determine a combined model output. Optionally, the processor subsystem 340 is further configured to validate the combined model on a validation dataset 050 during operation of the system 300. Validation need not be performed; instead, or in addition, various other operations performed by the combined system are described herein. Optionally, the processor subsystem 340 is further configured to provide the combined model 040 to one or more forecasting systems 100 during operation of the system 300, as described herein. This example shows a single forecasting system 100. The combined model may be provided in various ways, such as by direct transmission or by providing access via an external repository.

[0065] The system 300 also includes a communication interface 380 arranged for digital communication 324 with other systems, such as the plurality of training systems 201, 202 and / or one or more prediction systems 100. The communication interface 380 communicates internally with the processor subsystem 340 via data communication 323. The communication interface 360 ​​is arranged for direct communication with the other systems 100, 201, 202, using, for example, a USB, IEEE 1394, or similar interface. The communication interface 380 also communicates over a computer network, such as a wireless personal area network, the Internet, an intranet, a LAN, or a WLAN. For example, the communication interface 380 includes a connector suitable for the computer network, such as a wireless connector, an Ethernet connector, a Wi-Fi, a 4G or 5G antenna, a ZigBee chip, or the like. The communication interface 380 may also be an internal communication interface, such as a bus, an API, a storage interface, or the like.

[0066] Various details and aspects of the operation of system 300, including optional aspects thereof, are further elucidated with reference to FIG. 5 and others.

[0067] Typically, system 300 is embodied as or within a single device or apparatus, such as a laptop or desktop-based workstation or server. The device or apparatus includes one or more microprocessors that execute appropriate software. For example, the processor subsystem may be embodied by a single central processing unit (CPU), but may also be embodied by a combination or system of other types of processing units, such as a CPU and / or a graphics processing unit (GPU). The software may be downloaded and / or stored in corresponding memory, e.g., volatile memory such as RAM or non-volatile memory such as flash. Alternatively, the functional units of the system, e.g., the data interface and processor subsystem, may be implemented in the device or apparatus in the form of programmable logic, e.g., as a field programmable gate array (FPGA). Typically, each functional unit of the system is implemented in the form of a circuit. It should be noted that system 300 may also be implemented in a distributed manner, including different devices or apparatuses, e.g., distributed servers, e.g., in the form of cloud computing.

[0068] 4 shows a detailed, but non-limiting, example of the use of a combined model in classifying electroencephalographic waveform signals, for example, for epileptic seizure detection. Shown is a combined model C,040 used in determining a combined model output C,490 from an input instance II,410.

[0069] The combined model CM includes multiple trained models, e.g., at least three, at least 10, or at least 50 models, trained on respective training datasets. Each trained model is configured to determine a model output for an input instance by determining a representation of the input instance in a common latent space and determining the model output therefrom. In this particular example, each of the multiple trained models uses a common feature extraction model FEM, 044 to map the input instance to the latent space, and each uses a separate predictive model PMi, 041 to map the latent space representation to the model output. This is a good choice to ensure interchangeability of the latent space representations of each model, although, as described elsewhere, it is also possible for each trained model to have its own separate feature extractor. Because this example uses a common feature extractor between the trained models, the terms "trained model" and "predictive model" will be used interchangeably hereinafter.

[0070] A specific example of performing predictions based on extracted features of inputs in a latent space can be the analysis of EEG signals. As mentioned above, to diagnose and monitor epilepsy, the brain's electrical activity is measured using electroencephalography (EEG). Therefore, the combined model is used for various applications, including the detection of epileptic seizures in EEG signals. In such cases, the input instance is a segment, also known as a window, of the EEG signal, e.g., one or two seconds long. The task performed by the combined model can be, for example, a binary classification task of such a window, classifying whether seizure activity is present or not. Such classification is performed in a trained model by first applying a feature extraction model (FEM) that converts multi-channel EEG signals into latent vectors in a multidimensional feature space. In the trained model, a classification decision boundary is fitted using a classification model, e.g., an SVM classifier, a gradient boosting tree, or a neural network. Examples of such approaches are described in "Application of Machine Learning To Epileptic Seizure Detection" by A. Shoeb and J. Guttag, 2010, and "A robust unsupervised epileptic seizure detection methodology to accelerate large EEG database evaluation" by K. Tsiouris, S. Markoula, S. Konitsiotis, D.D. Koutsouris, and D.I. Fotiadis, "A robust unsupervised epileptic seizure detection methodology to accelerate large EEG database evaluation," Biomed. Signal Process. Control, Vol. 40, pp. 275-285, 2018 (both of which are incorporated herein by reference).

[0071] Another example of a latent spatial representation used, the trained model, includes, for example, a convolutional neural network for face recognition in images. In such a convolutional neural network, each layer is typically trained to recognize low-level image features, e.g., first at various levels of abstraction, such as contours and lines, then to recognize low-level structures such as ears, eyes, etc., before finally recognizing complete faces. In such an example, for example, either the contours / lines or the low-level structures can be used as the latent spatial representation.

[0072] However, representations of input instances in latent space commonly occur for various types of input instances II. In particular, instead of EEG signals, various types of time-series sensor data, particularly physiological measurements such as vital sign readings, are used. Such time-series data are analyzed at specific points in time by classifying segments, but their evolution over time can also be analyzed, for example, using a recurrent neural network. The output of such a recurrent neural network or other recurrent model is then further analyzed with a predictive model to reach a desired conclusion. Such recurrent model representations of time-series signals can also be used as latent space representations, as described herein, and the recurrent model can be considered a feature extraction model (FEM). As another example, trained models for analyzing images, such as convolutional neural networks, deep neural networks, etc., typically have intermediate latent space representations, e.g., the output of intermediate neural network layers. The portion of the neural network up to the intermediate layer can be considered a feature extraction model (FEM). Various other applications will become apparent.

[0073] Note that the latent spatial representation LSR is typically much smaller than the input instance II, e.g., contains many fewer elements. For example, a typical input instance contains at least 1,000, at least 10,000, or at least 1,000,000 features. The latent spatial representation may contain, for example, at most 100 or at most 1,000 features, or have at most one-tenth or one-hundredth the number of features of the input instance.

[0074] As shown, the combined model CM further includes dataset fingerprints DFi, 042 for each of the multiple trained models. The dataset fingerprints DFi of a trained model PMi characterize the latent spatial representation of the training instances of the training dataset on which the trained model PMi was trained. In general, the dataset fingerprints DFi are characteristics of the dataset that preferably capture important features of the dataset without compromising the privacy of the data. The exact type of dataset fingerprint used will usually depend strongly on the nature of the data; for example, different types of fingerprints will be defined for particular domains and / or problems. Some specific examples of dataset fingerprints are provided below.

[0075] Using the dataset fingerprints DFi, in a feature correspondence step FC, 440, correspondence scores CSi, 450 are determined between the input instance II and the multiple trained models PMi, indicating the correspondence between the input instance II and the training dataset on which the trained models were trained. Because the dataset fingerprints characterize the latent spatial representation of the training instance, the input instance II is also typically transformed into latent space. In this example, because the multiple trained models share the feature extraction mode FEM, a single latent spatial representation LSR, 430 is obtained for the input instance II by the feature extraction FE, 420, using the feature extraction model FEM. However, if each trained model has its own feature extractor, then each latent spatial representation of the input instance is also determined for comparison with its respective dataset fingerprint DFi.

[0076] In general, the correspondence score CSi for a trained model PMi is determined by comparing the distribution of the latent feature vectors of the input instance II to the distribution of the latent feature vectors of the trained model, as contained in its dataset fingerprint DFi. The distribution can be captured in various ways, for example, by storing a histogram of feature values ​​or a combination of feature values, by means of means or medians and / or parameters of probability distributions, such as univariate or multivariate Gaussian distributions, fitted to the training dataset.

[0077] One particularly effective way to capture the distribution of latent feature vectors is to use cluster centroids. In such a case, the dataset fingerprint DFi includes multiple cluster centroids in the latent space, where the cluster centroids represent clusters of training input instances. Such clusters represent multiple training instances that have the same label and / or similar representation in the latent space. Correspondence scores CSi between the input instance II and the trained model PMi are then determined based on similarity values, such as distances between the input instance II and multiple cluster centroids. Apart from the cluster centroids, the variances of the respective clusters are also included. The correspondence scores CSi are then further based on these variances, for example, by normalizing the distances according to the variances or by other techniques known per se for clustering algorithms.

[0078] When comparing latent representations or subsets of features of latent representations, e.g., comparing input instances II with cluster centroids or feature means or medians, various distance metrics can generally be used. Specific examples include Euclidean distance or weighted Minkowski distance. In particular, Minkowski distance can be used to adjust the relative contributions of various features, e.g., to emphasize the importance of important latent features and reduce sensitivity to features with high variance. Thus, for example, it is possible to select or assign higher importance to models trained on similar data whose feature vector distribution is closer to that of the input instances.

[0079] For example, in the case of EEG, the variance of the seizure and non-seizure feature vector distributions as well as the combination of centroid feature vectors corresponding to typical seizure and non-seizure windows can be used as the dataset fingerprint DFi, e.g., for the problem of epileptic seizure detection. For example, the dataset fingerprint DFi is constructed by applying a clustering algorithm to the latent space, and the dataset fingerprint DFi contains information about the detected clusters.

[0080] In various embodiments, the correspondence score CSi between the input instance II and the trained model PMi is further based on the input instance II and / or the model output of the trained model PMi for the input instance. In such cases, the data fingerprint characterizes not only the latent spatial representation of the training dataset, but also the model output of the training instance and / or the trained model for the training instance. Here, the training instance and the input instance include metadata not used by the trained model PMi, e.g., PMi is an image classifier and the metadata includes information about the patient captured in the image. If the output is used, the trained model PMi can be run on the input instance II to determine the correspondence score CSi; however, it should be noted that, for example, if the trained model does not correspond to the input instance II in terms of inputs, latent features, and / or outputs, this does not necessarily mean that this output will be used to determine the combined model output CMO.

[0081] For example, in a medical context, features or metadata of input instances that can be included in a dataset fingerprint include, e.g., - the minimum, median, and / or maximum age of a patient cohort, or the distribution of patient ages, e.g., allowing appropriate selection of models for infants or elderly patients while maintaining patient privacy; - Sex ratio of the patient cohort, - the common type of treatment in the patient cohort, - Most likely location of epilepsy focus, symptoms, medications, etc. Alternatively or additionally, the dataset fingerprint DFi may include a generative model. In this case, the correspondence score CSi is determined based on the likelihood of the latent spatial representation generated by the generative model. Various generative models, such as variational autoencoders (VAEs) or generative adversarial networks (GANs), particularly for images, are known in the art. Image datasets are very common in various applications and domains, from image classification to medical image segmentation. Such generative models, such as deep convolutional generative models, effectively learn the probability distribution of the training dataset. Therefore, the trained generative model is included in the dataset fingerprint DFi. The definition of the distance between the training dataset fingerprint DFi and the input instance II is based on the estimated probability of the input instance II belonging to the distribution of training instances in terms of the latent spatial representation. For example, in the case of a VAE, the reconstruction loss can be used as the correspondence score CSi. In the case of a GAN, the discriminative part of the trained GAN is used to distinguish between instances that are similar to the training set and those that are dissimilar. Note that generative models such as VAEs and GANs can also be used on non-image data, e.g., EEG signals, as described elsewhere.

[0082] Based on the correspondence scores CSi, the combining operation CB, 480 determines the combined model output CMO by combining each model output MOi, 470 into the combined model output CMO according to the determined correspondence scores CSi of the respective trained models PMi. For example, depending on how the outputs are combined, a strict subset of the model outputs MOi of the trained models that sufficiently correspond is used. In this case, the model application operation MA, 460 determines the model outputs of that subset of the trained models PMi. It is also possible for the combining operation CB to use the outputs MOi of all the trained models, in which case the model application operation MA determines all the model outputs. In effect, by selecting or assigning higher importance to most models, as indicated by training the dataset fingerprints DFi, the combining operation CB can be considered to build a dynamically adaptive, optimized meta-model composed of individual models trained on similar data.

[0083] Given the suitable models identified by determining the correspondence scores CSi, various approaches can be used to construct this meta-model from the individual models PMi. For example, one possibility is to perform model blending to determine the combined model output CMO as the average of the model outputs of a subset of models that correspond sufficiently, e.g., CSi>T for some predefined threshold T applied independently to the input instances. MetaModel(New data)=Σ N i=1 Model i (New data)

[0084] As another example, a weighted sum of some or all of the determined model outputs MOi can also be calculated based on the respective correspondence scores CSi. For example, the weights in the weighted sum can be, for example: MetaModel(New data)=Σ N i=1 w i Model i(New data) As shown, it is proportional to the model correspondence.

[0085] As a specific example, the weight w i is the fingerprint of new data F(New data) and the trained model F(Data i ) may be inversely proportional to the distance D between the data used to train w i ∝D -1 (F(New data),F(Data i ))

[0086] As shown, the determined model outputs MOi can also be combined into a combined model output CMO by applying a trainable combined model COM, 043, to the determined correspondence scores CSi and the model outputs MOi. The combined model COM is typically included in the combined model CM. For example, the combined model can be a decision tree or a linear regression model for calculating the combined model output from the determined model outputs, etc. The combined model COM obtains the correspondence scores as input and output weights or the selection of the model to use in the combination function, but the combined model obtains the model outputs MOi themselves as inputs in addition to the correspondence scores CSi, and directly outputs the combined model output CMO.

[0087] Alternatively or in addition to determining the combined model output CMO, a confidence score for the combined model output CMO is also determined based on the determined correspondence score CSi. The confidence score indicates the correspondence between the input instance II and the combined training dataset of the multiple trained models DFi. For example, when model blending is used as described above, a confidence score appropriate to the method of combining the model outputs MOi with the combined model output CMO is determined, such as the number of trained models used or the average correspondence score if a weighted sum is used. The combined model COM is a probabilistic model, such as a Bayesian neural network, configured to provide both the combined model output CMO and a confidence score indicating the certainty of the combined model output.

[0088] 5 shows a detailed, but non-limiting, example of fingerprinting a trained model and combining the trained model and dataset fingerprints into a combined model. For example, the fingerprinting is performed by a training system or method described herein, and the combining is performed by a combining system of the method described herein. Some elements of this figure correspond to elements of FIG. 4, particularly the combined model CM, 040, and the instances to which it applies, and any of the many options described with respect to that figure also apply here.

[0089] Specifically, shown in this figure is a training dataset TDi, 030 for training a trained model for use in the combined model CM. The training dataset typically includes data from one or several collaborating organizations that cannot be shared outside of those organizations. For example, a particular hospital dataset may include data for up to 100,000 or up to 1 million patients, e.g., for common conditions, or up to 100, up to 1,000, or up to 10,000 patients, e.g., for rare conditions.

[0090] Based on the training data set TDi, the models are trained in a training operation TR, 560 to obtain trained models PMi, 041. The trained models are configured to determine model outputs for input instances by determining representations LSRi, 530 of the input instances in a latent space and determining the model outputs therefrom, as also described with respect to FIG. 4. In this example, as in FIG. 4, each trained model includes a common feature extraction model FEM, 044, and a respective predictive model PMi. For example, the feature extraction model FEM can be used in the feature extraction operation FE, 520 to obtain latent space representations LSR, 530 of the training instances in the training data set TDi. As indicated by the dashed line from the feature extraction model FEM to the feature extractor FE, in this particular example, the feature extraction model is not trained as part of the training of the trained model, but is obtained as a pre-trained feature extraction model from another party, e.g., a coupled system. However, as part of the training, it is possible to train a feature extractor from scratch or to improve a pre-trained feature extraction model FEM based on a training dataset, for example by performing one or more gradient descent runs or by using techniques for improving trained models known per se in the field of transfer learning. In such cases, the trained model includes not only the predictive model but also the improved feature extraction model.

[0091] The predictive model PMi can be trained using techniques known in the art, such as, for example, stochastic gradient descent or any other method suitable for the type of model at hand, using the latent representation. As shown in this figure, the predictive model PMi is based on a pre-trained predictive model PMO, 511. The pre-trained predictive model PM0 is obtained and then improved, for example, using techniques known per se, as part of training the trained model PMi. The feature extractor and the predictive model can be trained in a combined training operation.

[0092] Apart from training the model PMi, also in the fingerprinting operation FP, 540, a dataset fingerprint DFi, 042 of the trained model is determined based on the training dataset TDi. The dataset fingerprint DFi characterizes the latent spatial representation LSRi of the training dataset TDi on which the trained model was trained. Various data that can be included in the dataset fingerprint are described throughout, e.g., with respect to FIG. 4, and can be determined accordingly. For example, a clustering algorithm can be used to determine cluster centroids, a generative adversarial network or a variational autoencoder can be trained to obtain a generative model, and latent features or statistical parameters of the training input or output, e.g., metadata, are determined.

[0093] For example, the trained model, such as the predictive model PMi and possibly an improved feature extraction model, is provided to another party for determining the combined model CM, e.g., uploaded to a repository or transmitted. Interestingly, because the trained model and dataset fingerprints typically represent aggregated information, this can be done with minimal impact on the privacy and confidentiality of the training dataset TDi. To ensure this, optional privacy checks can be performed, e.g., checking whether the trained model and / or dataset fingerprints satisfy privacy properties such as k-anonymity. The trained model and / or dataset fingerprints can also be improved to satisfy privacy properties, e.g., by adding noise or removing elements based on too few training records. Thus, an organization holding the training dataset TDi can make the dataset available for use in the combined model CM without having to disclose sensitive data.

[0094] We now turn our attention to methods for determining a combined model CM. A party compiling a combined model CM receives or otherwise obtains multiple trained models PMi and corresponding dataset fingerprints DFi from multiple training systems, as described herein. These multiple trained models PMi and corresponding dataset fingerprints DFi are then combined into the combined model CM to determine the combined model output. For example, the combined model CM may be stored as a single file or database containing the multiple trained models and dataset fingerprints, or the combined model may include references to the multiple trained models and dataset fingerprints, e.g., URLs from which the trained models are obtained or accessed, e.g., using API calls. In this example, the trained models have a common feature extraction model FEM, so the multiple trained models are included in the combined model CM in the form of the common feature extraction model FEM and individual predictive models PMi, such as classification models and regression models, that use the latent spatial representation as input. In addition to combining the obtained trained models PMi and the dataset fingerprints DFi, determining the combined model can include various other optional steps, some of which are shown in the figure, many of which can be combined in an arbitrary manner.

[0095] In particular, as shown, determining the combined model optionally includes training a feature extraction model FEM, 044 in a first training operation TR1, 510. This training can be performed on a training dataset TDC, 555. This can be, for example, a publicly available dataset, or, for example, a dataset that is also used to determine the trained model if the determination of the combined model CM is combined with the training of a trained model included in the combined model CM. As part of the initial training, typically on the same dataset, an initial prediction model PMO, 511 for determining model outputs from representations of input instances in latent space is also trained. The initial prediction model PMO can be, for example, an individual predictive model trained on a public dataset, or one of the predictive models PM of the combined model.

[0096] The trained feature extraction model FEM and / or the initial prediction model PMO are typically provided to a training system for performing training on the respective data sets, as indicated by the dashed lines from these models to the feature extraction operation FE and the training operation TR that are performed as part of the training. In this example, a common feature extraction model is used, so the model FEM is also included in the combined model CM, but as described elsewhere, improved respective feature extraction models can also be obtained and included in the combined model.

[0097] Another operation optionally performed as part of determining the combined model CM is training in a second training operation TR2,515 of the combined model COM,043 to combine model outputs into a combined model output. For example, based on the training dataset TDC described above or another dataset containing training instances and desired training outputs, the model outputs of the trained models TMi for each of the training instances and their corresponding scores for each of the training instances with respect to each trained model are determined. Based on this, the combined model COM is trained by calculating model weights or selections that can be used to directly calculate the desired training output or in combination with the outputs of each trained model to determine the combined model output. The trained combined model COM is then included in the combined model CM.

[0098] Another operation optionally performed as part of the determination of the combined model is a validation operation VAL,590, which validates the combined model CM with the validation dataset VDC,050. For example, the accuracy of the combined model on the validation dataset can be determined, and an error can be flagged, for example, if the accuracy does not meet a predefined threshold. Thus, the overall performance of the combined model is ensured. Also, the contribution of individual trained models to the overall model can be determined, for example, by determining the accuracy of the combined model form with one of the trained models removed and determining its impact on the overall accuracy. As another example, the contribution of individual trained models to the combined model CM can be validated, for example, by calculating their average weight in a weighted combination, by the number of times the model output of the trained model is included in the combined model output, etc. Thus, various possibilities are provided for the model user to obtain feedback on the internals of the combined model, in other words, to allow the trained model to be “debugged” to further improve its accuracy.

[0099] Specifically, determining the combined model includes two phases: a first phase in which data to provide to the training system, such as the feature extraction model FEM and the initial prediction model PMO, is determined; and a second phase in which the combined model CM is determined using information received from the training system, for example by training a combined model COM, combining the received models and fingerprints into the combined model, and / or performing validation.

[0100] The determined combined model can also be updated, for example, by adding additional trained models PMi and corresponding dataset fingerprints DFi to the combined model CM, by removing trained models and their fingerprints if they are found to have a negative impact on accuracy, or by updating them, for example, when new training data becomes available. Updating also includes, for example, retraining or improving the combined model COM. Interestingly, since the trained models are treated separately in the combined model, such update operations are performed relatively efficiently.

[0101] 6 shows a block diagram of a computer-implemented method 600 for applying multiple trained models to an input instance. The multiple trained models are trained with respective training datasets. Method 600 corresponds to the operation of system 100 of FIG. 1. However, this is not a limitation in that method 600 may be performed using another system, apparatus, or device.

[0102] The method 600 includes accessing (610) a plurality of trained models in the form of a combined model in an operation entitled "Accessing Trained Models as a Combined Model." The combined model defines a latent space. The trained model is configured to determine a model output for an input instance by determining a representation of the input instance in the latent space and determining the model output therefrom. The combined model further includes a dataset fingerprint for each of the plurality of trained models. The dataset fingerprint for the trained model characterizes a latent space representation of a training instance in the training dataset for the trained model.

[0103] The method 600 further includes obtaining an input instance (620) in an operation entitled "Obtaining an Input Instance."

[0104] The method 600 further includes determining (630) correspondence scores between the input instance and the plurality of trained models in an operation entitled "Determining Correspondence Scores." The correspondence scores between the input instance and the trained models indicate correspondence between the input instance and the training datasets of the trained models. The correspondence scores are based on a representation of the input instance in latent space and dataset fingerprints of the trained models.

[0105] The method 600 further includes determining (640) a model output of one or more of the plurality of trained models for the input instance in an operation titled "Determining Model Outputs of the Trained Models."

[0106] The method 600 further includes combining (650) the model outputs into the combined model output according to the determined correspondence scores of each trained model in an operation titled "Combining into Combined Model Output."

[0107] Figure 7 shows a block diagram of a computer-implemented method 700 for fingerprinting a trained model. Method 700 corresponds to the operations of system 200 of Figure 2. However, this is not a limitation in that method 700 may be performed using another system, apparatus, or device.

[0108] Method 700 includes accessing (710) a training data set in an operation entitled "Accessing a Training Data Set."

[0109] The method 700 further includes, in an operation entitled “Training Model,” training 720 a model on the training dataset to obtain a trained model. The trained model is configured to determine model outputs for input instances by determining representations of the input instances in a latent space and determining the model outputs therefrom.

[0110] The method 700 further includes determining 730 a dataset fingerprint for the trained model based on the training dataset in an operation entitled "Determining a Dataset Fingerprint." The dataset fingerprint characterizes a latent spatial representation of the training dataset on which the trained model was trained.

[0111] Figure 8 shows a block diagram of a computer-implemented method 800 for determining a combined model from multiple trained models. Method 800 corresponds to the operations of system 300 of Figure 3. However, this is not a limitation in that method 800 may be performed using another system, apparatus, or device.

[0112] The method 800 includes, in an operation entitled "Accessing a Validation Data Set," accessing 810 a validation data set including a plurality of validation input instances and corresponding validation outputs.

[0113] Method 800 further includes arranging digital communications with the plurality of training systems (820) in an operation entitled "Arranging Digital Communications."

[0114] Method 800 further includes receiving (830) a plurality of trained models and corresponding dataset fingerprints from a plurality of training systems in an operation titled "Receiving Model, Fingerprints." The trained models are configured to determine model outputs for input instances by determining representations of the input instances in a latent space common to the plurality of trained models and determining the model outputs therefrom. The dataset fingerprints of the trained models characterize latent space representations of training instances in the training dataset for the trained models.

[0115] The method 800 further includes combining (840) the multiple trained models and corresponding dataset fingerprints into a combined model to determine a combined model output, in an operation titled “Combining into a Combined Model.”

[0116] Method 800 further includes validating (850) the combined model on the validation dataset in an operation entitled "Validating the Combined Model."

[0117] In general, the operations of method 600 of Figure 6, method 700 of Figure 7, and / or method 800 of Figure 8 may be performed in any suitable order, e.g., sequentially, simultaneously, or a combination thereof, according to a particular order dictated, e.g., by input / output relationships, where applicable. The methods may be combined in a single method, e.g., by applying a previously determined combined model to input instances, or by determining and / or applying a combined model that includes an already trained model.

[0118] The method may be implemented on a computer as a computer-implemented method, as dedicated hardware, or as a combination of both. As shown in FIG. 9 , computer instructions, e.g., executable code, are stored on a computer-readable medium 900, e.g., in the form of a series of machine-readable physical marks 910 and / or a series of elements having different electrical, e.g., magnetic or optical, properties or values. The executable code may be stored in a temporary or non-temporary manner. Alternatively, or in addition, the medium 900 may store data representing a combined model, e.g., for use in the methods described herein. The combined model defines a latent space. The combined model comprises multiple trained models trained with respective training datasets. Such trained models are configured to determine a model output for an input instance by determining a representation of the input instance in the latent space and determining the model output therefrom. The combined model further includes a dataset fingerprint for each of the multiple trained models. The dataset fingerprint for the trained model characterizes the latent space representation of the training instance in the training dataset for the trained model. Examples of computer readable media include memory devices, optical storage devices, integrated circuits, servers, online software, etc. FIG.

[0119] Examples, embodiments, or optional features, whether or not indicated as non-limiting, should not be understood as limiting the claimed invention.

[0120] The above-described embodiments illustrate, rather than limit, the present invention, and it should be noted that those skilled in the art can design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the scope of the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in the claim. The singular form "a" or "an" does not exclude the presence of a plurality of such elements. The use of expressions such as "at least one" before a list or group of elements denotes selection of all or any subset of the elements from the list or group. For example, the expression "at least one of A, B, and C" is to be understood as including A only, B only, C only, both A and B, both A and C, both B and C, or all of A, B, and C. The present invention can be implemented by means of hardware comprising several distinct elements and by a suitably programmed computer. In a device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. 1. A prediction system for applying a plurality of trained models to input instances, the plurality of trained models being trained on respective training datasets, the prediction system comprising: a data interface for accessing the plurality of trained models in the form of a combined model, the combined model defining a latent space, the trained model determining the model output for the input instance by determining a representation of the input instance in the latent space and determining a model output therefrom, the combined model further including a dataset fingerprint for each of the plurality of trained models and a dataset fingerprint for the trained model characterizing a latent space representation of a training instance of the training dataset for the trained model; a processor subsystem, the processor subsystem comprising: Get the input instance, determining correspondence scores between the input instance and the plurality of trained models, the correspondence scores between the input instance and the trained models indicating correspondence between the input instance and the training dataset of the trained models, the correspondence scores being based on a representation of the input instance in the latent space and the dataset fingerprints of the trained models; determining a model output of one or more of the plurality of trained models for the input instance; providing the combined model output for the plurality of trained models by combining the model outputs into a combined model output according to the determined correspondence scores of each of the trained models. Prediction system.

2. The prediction system of claim 1 , wherein the input instance comprises one or more of an image, a stack of images, and time-series sensor data of a physiological measurement.

3. 3. The prediction system of claim 1, wherein the dataset fingerprint of the trained model includes a plurality of cluster centroids in the latent space, the cluster centroids representing clusters of training input instances, and the processor subsystem determines a correspondence score between the input instance and the trained model based on similarity values ​​between the input instance and the plurality of cluster centroids.

4. 3. The prediction system of claim 1, wherein the dataset fingerprint of the trained model includes a generative model, and the processor subsystem determines the correspondence score between the input instance and the trained model based on a likelihood of the latent spatial representation generated by the generative model.

5. 5. The prediction system of claim 1, wherein the correspondence score between the input instance and the trained model is further based on the input instance and / or a model output of the trained model for the input instance, and the dataset fingerprint of the trained model further characterizes a training instance and / or a training output.

6. 6. The prediction system of claim 1, wherein the processor subsystem combines the determined model outputs with the combined model outputs by applying a trainable combination model to the determined correspondence scores and model outputs.

7. 7. The prediction system of claim 1, wherein the processor subsystem determines a confidence score for the combined model output based on the determined correspondence score, the confidence score indicating a correspondence between the input instance and a combined training data set of the plurality of trained models.

8. 1. A training system for fingerprinting a trained model, comprising: a data interface for accessing the training dataset; a processor subsystem, the processor subsystem comprising: training a model on the training dataset to obtain the trained model, the trained model determining the model output for the input instance by determining a representation of the input instance in a latent space and determining a model output therefrom; determining a dataset fingerprint for the trained model based on the training dataset, the dataset fingerprint characterizing a latent spatial representation of the training dataset on which the trained model was trained; the processor subsystem obtains a pre-trained feature extractor for determining a representation of an input instance in the latent space, and trains the model based on the pre-trained feature extractor. Training system.

9. 1. A combination system for determining a combined model from a plurality of trained models, comprising: a data interface for accessing a validation data set including a plurality of validation input instances and corresponding validation outputs; a communications interface for digitally communicating with a plurality of training systems; a processor subsystem, the processor subsystem comprising: receiving a plurality of trained models and corresponding dataset fingerprints from the plurality of training systems, the trained models determining model outputs for the input instances by determining representations of the input instances in a latent space common to the plurality of trained models and determining model outputs therefrom, the dataset fingerprints of the trained models characterizing latent space representations of training instances in training datasets for the trained models; combining the plurality of trained models and corresponding dataset fingerprints into a combined model to determine a combined model output; validating the combined model on the validation dataset; Combined system.

10. 10. The combination system of claim 9, further training one or more of: a feature extractor for determining representations of input instances in the latent space; an initial predictive model for determining model outputs from the representations of input instances in the latent space; and a combination model for combining model outputs into combined model outputs.

11. 1. A computer-implemented method for applying a plurality of trained models to an input instance, the plurality of trained models being trained on respective training datasets, the computer-implemented method comprising: accessing the plurality of trained models in the form of a combined model, the combined model defining a latent space, the trained model determining the model output for the input instance by determining a representation of the input instance in the latent space and determining a model output therefrom, the combined model further including a dataset fingerprint for each of the plurality of trained models, the dataset fingerprint for a trained model characterizing a latent space representation of training instances of the training dataset for the trained model; obtaining an input instance; determining a correspondence score between the input instance and the plurality of trained models, the correspondence score between the input instance and the trained models indicating a correspondence between the input instance and the training dataset of the trained models, the correspondence score being based on a representation of the input instance in the latent space and the dataset fingerprint of the trained models; determining a model output of one or more of the plurality of trained models for the input instance; combining the model outputs into a combined model output according to the determined correspondence scores of each of the trained models. Computer-implemented method.

12. 1. A computer-implemented method for fingerprinting a trained model, comprising: accessing a training dataset; training a model on the training dataset to obtain the trained model, the trained model determining the model output for the input instance by determining a representation of the input instance in a latent space and determining a model output therefrom; determining a dataset fingerprint for the trained model based on the training dataset, the dataset fingerprint characterizing a latent spatial representation of the training dataset on which the trained model was trained; obtaining a pre-trained feature extractor for determining a representation of an input instance in the latent space, and training the model based on the pre-trained feature extractor. Computer-implemented method.

13. 1. A computer-implemented method for determining a combined model from a plurality of trained models, comprising: accessing a validation data set comprising a plurality of validation input instances and corresponding validation outputs; arranging digital communication with a plurality of training systems; receiving, from the plurality of training systems, a plurality of trained models and corresponding dataset fingerprints, wherein the trained models determine model outputs for the input instances by determining representations of the input instances in a latent space common to the plurality of trained models and determining model outputs therefrom, and the dataset fingerprints of the trained models characterize latent space representations of training instances in a training dataset for the trained models; combining the plurality of trained models and corresponding dataset fingerprints into a combined model to determine a combined model output; and validating the combined model on the validation dataset.

14. 14. A computer-readable medium containing instructions for causing a processor system to perform one or more of the computer-implemented methods of claims 11, 12, and 13, and non-transitory data representing a combined model, comprising: the combined model defines a latent space, the combined model includes a plurality of trained models for respective training datasets, the trained models determine the model output for the input instances by determining a representation of the input instances in the latent space and determining a model output therefrom, the combined model further includes a dataset fingerprint for each of the plurality of trained models, the dataset fingerprint for a trained model characterizing a latent space representation of a training instance of the training dataset for the trained model; Computer-readable medium.

Images