Federated learning with hard examples

Abstract

The invention relates to a federated learning system (300) for federated learning of a machine-learnable model. A local system (LS) obtains a hard instance (HEDi) for a machine-learnable model (MLM) and determines a model update (MU) from the hard instance for updating model parameters of the machine-learnable model. A local system (LS) provides a hard example descriptor to a coordinator system (CS), the hard example descriptor including a model update (MU) and metadata (HEDi) describing the hard example. A coordinator system collects and clusters hard example descriptors (HEDi). Based on a model update (MU) and metadata included in a hard example descriptor (HEDi) of a cluster, a coordinator system (CS) accepts the cluster for use in federated learning and provides an acceptance notification (AN) to a local system (LS) which then uses a hard example in federated learning of a machine-learnable model (MLM).

Description

Technical Field

[0001] The present invention relates to a federated learning system, a local system and a coordinator system for use in a federated learning system, and a computer-implemented method of operating the local system and the coordinator system, respectively. The present invention further relates to a computer-readable medium. Background Art

[0002] The performance of machine learning (ML) models in general and deep learning (DL) models in particular depends on the quality and content of the data used for training and validation. It has been reported in the literature that DL models may overfit subtle institutional data biases and may perform poorly on data from institutions whose data were not seen during training, see, for example, MJSheller et al., "Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data", Nature Sci Rep 10, 12598 (2020). Even when the training and test data sets are large, diverse, representative, and perfectly annotated, once the ML / DL algorithm faces rare or unprecedented cases (in the training data set), it may still fail after deployment.

[0003] The concept of post-product learning, also known as active learning or lifelong learning (LL), aims to address this issue. According to the concept of lifelong learning (LL), rare or complex (and potentially valuable) examples can be identified, mined, and added to the reference (training and validation) datasets for future retraining. In particular, in the medical context, where the model is applied to healthcare data about patients, the United States Food and Drug Administration recognized the need for post-product learning in the discussion paper "Proposed Regulatory Framework for Modifications to Artificial Intelligence / Machine Learning-Based Software as a Medical Device", which discusses adaptive AI solutions in healthcare that can learn from real-world data and post-deployment performance monitoring. Privacy, guardianship, and data governance are important concerns when applying post-product learning, especially in healthcare settings. Although healthcare providers may be interested in contributing to and then using improved models, in most cases they are unwilling or legally prohibited from sharing clinical data with each other or with model developers / vendors.

[0004] Federated learning is a known technique that can be used to alleviate some of the privacy, security, and management concerns in training models using data from different parties. In federated learning, a machine-learnable model, such as a deep neural network, can be updated with new local data samples from corresponding decentralized systems without exchanging them. This is sometimes referred to as the "model-to-data" paradigm. For example, in a healthcare environment, a machine-learnable model can be deployed at a corresponding healthcare provider (e.g., locally, or in multiple isolated cloud instances), where it can be used to infer patient data to provide care. Federated learning allows machine-learnable models to be updated based on patient data from these corresponding healthcare providers without moving that data out of the institution's administrative domain. Summary of the invention

[0005] A problem that arises when updating a machine learnable model with additional examples after deployment is that it is unclear whether using that additional example will improve the model. Training and validation datasets are typically composed of carefully compiled, annotated, and curated cases. On the other hand, real-world cases such as those from routine clinical care may be more error-prone compared to academic research settings, for example, due to the lack of final case results against ground truth, intense work environments with high workload and time pressures, and lower levels of training and experience of healthcare providers. In addition, there are significant differences between institutions in data collection, protocols, techniques, and ground truth collected, which may cause problems when all such data are collected in one large dataset.

[0006] It is therefore desirable to provide federated learning techniques that provide additional assurance that additional training examples improve the performance of the trained model.

[0007] According to a first aspect of the present invention, there is provided a federated learning system as defined in claim 1. According to a further aspect of the present invention, there is provided a local system and a coordinator system for use in a federated learning system as defined in claims 11 and 12, respectively. According to yet a further aspect of the present invention, there is provided a computer-implemented method of operating a local system and a coordinator system as defined in claims 13 and 14, respectively. According to an aspect of the present invention, there is provided a computer-readable medium as defined in claim 15.

[0008] Various aspects relate to using federated learning to train machine learnable models, and in particular to updating trained machine learnable models based on additional training examples. Federated learning refers to training machine learnable models using a distributed protocol without exchanging training examples. Using federated learning, machine learnable models can be updated based on additional training examples from multiple corresponding systems without the need for these systems to explicitly exchange these additional training examples with other systems. These systems are referred to as "local systems" in this article. Depending on the specific environment, such local systems are sometimes also referred to as "local deployments", "local nodes", "nodes" or "edge devices". In general, federated learning can operate across silos and across devices. The corresponding local systems are typically operated by the corresponding organizations.

[0009] To update the model, the local systems can determine corresponding model updates to the machine learnable model based on their local training examples, such as updated model parameters, and these model updates can be exchanged within the federated learning system. The updated model can be derived from these model updates. For example, the federated learning system can be a centralized federated learning system in which a central server coordinates the federated learning, or can be a decentralized federated learning system in which there is no such central server.

[0010] In various aspects, a local system can obtain hard examples for a machine learnable model. Typically, a hard example is a model input that has been identified at the local system as a candidate input for use in federated learning of a machine learnable model and / or for use in a reference dataset for a machine learnable model as described elsewhere. For example, a model input can be identified as a hard example because the model input is challenging for the model, e.g., the model provides an incorrect result for the model input, and / or the model provides a low confidence output for the model output.

[0011] In some cases, the use of hard examples may lead to improved machine-learnable models (e.g., in terms of robustness and / or generalizability) and / or reference datasets, while in other cases it may lead to degradation of the utility of the model. For example, hard examples may represent interesting and rare cases, also referred to herein as rare and interesting diseases (fascinomas). The use of rare and interesting diseases is often beneficial to improving the coverage of models and / or reference datasets. On the other hand, hard examples may represent ambiguous data, e.g., may contain artifacts and / or noise, and / or may be incorrectly annotated. In this case, the use of hard examples in federated learning and / or reference datasets may lead to degraded performance.

[0012] Therefore, the inventors envision using a coordinator system in a federated learning system that decides whether to accept a given hard example, for example to use the corresponding local update, or to reject the given hard example. In this way, the risk of possibly contaminating the update of the machine-learnable model is minimized. For example, the coordinator system can be the central system of a centralized federated learning system, but can also be one of the local systems or another system.

[0013] The coordinator system can decide which hard examples from the local systems to accept or reject, but interestingly, it may make this decision not based on the model inputs to the hard examples themselves (since the coordinator system may not wish to examine them), but rather based on hard example descriptors. The hard example descriptors may include model updates from the hard examples used to update model parameters of the machine learnable model, as well as metadata describing the hard examples, but not the model inputs themselves. Thus, the hard example descriptors can be provided by the local systems to the coordinator system without the privacy concerns that would arise if the model inputs themselves were shared. Nonetheless, a relatively good assessment of whether an example should be used can be determined.

[0014] In particular, model updates can be used to estimate the impact of an example on the accuracy of a machine learnable model based on a current reference dataset, while metadata can be used to determine whether an example represents a rare case that is underrepresented in the data used to train the model and / or in the current reference dataset. Model updates can be accepted based on whether the model update has a positive impact and / or is underrepresented. In particular, the inventors recognized that metadata can allow for distinguishing rare cases from low-quality and particularly mislabeled data: rare cases typically also have rare metadata, while low-quality data may have similar metadata to similar but higher-quality records.

[0015] Therefore, it can be guaranteed that by training with large and diverse datasets, including the addition of rare cases, the performance of the model can be improved without being affected by performance drift due to, for example, low-quality labels or data used during retraining. In other words, using the provided technique, rare and interesting diseases can be determined based on metadata and historical records, and relevant rare and erroneous cases can be identified based on similar events in different locations using hard case similarity assessments regarding the impact on the model and the sources of deviations from standard cases.

[0016] Specifically, to decide which hard examples to accept or reject, the inventors contemplate that the coordinator system selects clusters of similar hard example descriptors from a set of hard example descriptors that are collected from a local system of the federated learning system. The coordinator system can then decide to accept or reject a cluster based on the metadata and model updates of the hard examples in the cluster.

[0017] There are many advantages to using clusters like hard example descriptors. One advantage is that clustering itself provides valuable information to decide whether to accept a hard example in a cluster. For example, the size of the cluster and / or the number of local systems contributing to the cluster can indicate whether accepting the cluster is likely to have a positive impact, e.g., hard examples that represent isolated measurement errors or misclassifications are unlikely to appear in clusters that are large enough and / or span enough different local systems. In addition, other features that indicate whether a hard example is beneficial, such as the estimated impact on accuracy, can be more reliably estimated for clusters than for individual hard examples.

[0018] After accepting multiple hard examples for use in federated learning, the coordinator system can notify the corresponding local systems from which the hard example descriptors were collected. These local systems can update the machine learnable model by performing a federated learning update of the model using the accepted hard examples from one or more clusters. To this end, techniques known from federated learning itself can be used. Interestingly, these techniques are applied in a different way from traditional federated learning, for example, in that: training can be performed over an extended period of time; training can be initiated not by the coordinator selecting the next local system to train, but by the local system that obtains one or more hard examples and provides their descriptors to the coordinator; and hard examples can be not immediately applied to the machine learnable model, but instead processed directly or at a later point in time by a decision procedure that accepts or rejects them.

[0019] In some cases, all hard examples of a cluster can be used in federated learning. In other cases, only a subset of hard examples can be used, for example, to balance the training dataset and / or to use additional hard examples of the cluster in a reference dataset, as described herein.

[0020] Optionally, the machine learnable model can be applied to a model input comprising healthcare data about a patient. In particular, the machine learnable model can be, for example, an image classifier for radiology images; or another type of image processing model. In another example, the machine learnable model can be used to classify electronic medical records (EMRs). In such examples, the corresponding local system can be operated by a healthcare provider such as a hospital. The coordinator system can be operated by a third party. Interestingly, according to the provided technology, the coordinator system may not need to see hard examples to decide whether to accept or reject them, and therefore, despite the sensitivity of medical data, it may be feasible to use the provided technology on medical data.

[0021] In the healthcare industry, there is a growing interest in adopting solutions including AI models to improve performance and increase automation, including regulatory agencies such as the FDA. In this environment, avoiding performance drift after the model is deployed is an important issue. In addition, in the healthcare industry, it may be necessary to prove that the performance of AI products is as high in different production environments as in the validation phase (post-marketing) and does not deteriorate over time. This makes these technologies particularly useful in the medical field.

[0022] Optionally, the coordinator system may accept clusters based on one or more of: the size of the cluster, the number of local systems contributing to the cluster, the estimated impact of the cluster on the accuracy of the machine learnable model, and the degree to which the cluster represents a reference data set. These aspects may all indicate whether it is beneficial to use the cluster. Preferably, at least the size or number of systems, the accuracy estimate, and the degree of representativeness are used. Larger clusters or a larger number of system contributions may make the cluster more relevant and less likely to be an artifact. Additionally for non-rare cases, the impact on accuracy may generally indicate whether it is beneficial to train on them. The degree of representativeness may indicate the likelihood that the cluster represents a rare case. In particular, clusters may be accepted based on the fact that the cluster is sufficiently unrepresentative of the reference data set, in other words, the degree of representativeness does not exceed a threshold, for example, in combination with additional checks that the cluster is of sufficient quality, as indicated by the size of the cluster and / or the number of systems, and / or whether the metadata is abnormal.

[0023] Optionally, one or more hard examples of the accepted cluster can be used for federated learning, and one or more additional hard examples can be included in the reference dataset of the coordinator system. Interestingly, since both types are from the same cluster, they may be similar to each other, so a better trained model and a reference dataset that is more suitable for evaluating the better trained model can be obtained. The coordinator system can provide an acceptance notification of the hard example to the corresponding local system, indicating the intended use: for training or included in the reference dataset. Based on the indication, the local system can use the hard example in federated learning, or provide the hard example to the coordinator system. Although privacy-sensitive data is shared with the coordinator system in this case, the amount of records that need to be shared is still greatly introduced, and in specific cases where it is found that it is a hard example required for the reference dataset, it can also be easier to obtain permission for the shared examples.

[0024] Optionally, the local system can be configured to obtain a model input for a machine learnable model; calculate a confidence score for the model input; and select the model input as a hard example based on the confidence score being below a threshold, e.g., if the confidence score is below the threshold, the input can be selected in combination with one or more other conditions. Low confidence may indicate that the model input is potentially useful for training or validating the model, e.g., low confidence may be expected for rare cases that are underrepresented in the training data set. For example, the confidence score can be determined by the machine learnable model itself or by a separate model. For example, during routine use of the model, calculating the confidence score can be combined with applying the model to the model input. For example, if the model input is selected as a hard example, the hard examples can be manually inspected or annotated.

[0025] Instead of, or in addition to, using low-confidence model inputs, one or more hard examples can be obtained, for example, by a user marking an output of a machine learnable model as incorrect, and / or by randomly selecting and / or manually selecting by a user among the model inputs to which the model is applied.

[0026] Optionally, the metadata may include data that is neither derived from the model input nor from the labels of the hard examples (e.g., the ground truth model output). For example, the model input may be an image, and the metadata may include metadata for the image, such as information about the scanning protocol, the equipment used, etc. Such metadata may be combined with metadata based on the model input and / or the labels. For example, the metadata may include one or more of the following: a timestamp, a descriptor of the local system, a descriptor of the labeler of the hard examples, a degree of consistency in the labeling of the hard examples, information about the healthcare professional associated with the hard examples, information about the patient associated with the hard examples, and information about the image acquisition settings of the hard examples.

[0027] Optionally, the coordinator system is configured to obtain feedback about accepted clusters, and based on the feedback, update the criteria for accepting clusters. For example, the decision about whether a cluster is accepted or rejected may be manually reviewed and / or may be validated with data that later becomes available. In this case, the criteria for acceptance, in other words, parameters, may be updated. This may include, for example, adjusting thresholds, but may also involve retraining a trainable model used to make acceptance / rejection decisions.

[0028] Optionally, the coordinator system can be configured to retain hard example descriptors that are not included in the selected cluster, or that are included in an unaccepted cluster. The retained hard example descriptors can be used in further cluster selection and acceptance. For example, a hard example that initially seems abnormal may later become a valuable hard example if more cases similar to it later appear. For example, a cluster containing a hard example may initially be too small to be accepted, but may later reach a sufficient size and / or number of contributing parties. In this case, retaining the hard example can allow it to be used later in such a larger cluster, and potentially accepted and used, as described herein. As another example, the metadata of the hard example descriptor can be updated based on newly available information, and the hard example descriptor can be accepted based on the updated metadata.

[0029] Optionally, after federated learning based on the accepted hard examples, the local system and / or other local systems can be configured to apply the machine-learnable model updated based on the accepted hard examples to the model input. Thus, an output of the machine-learnable model can be obtained that benefits from improved training using hard examples.

[0030] It will be appreciated by those skilled in the art that two or more of the above-mentioned embodiments, implementations, and / or optional aspects of the invention may be combined in any way deemed useful.

[0031] Those skilled in the art may make modifications and variations to any system and / or any computer-readable medium based on the present specification, and these modifications and variations correspond to the described modifications and variations of the corresponding computer-implemented method. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] These and other aspects of the invention will become apparent and be further elucidated with reference to the embodiments described by way of example in the following description and with reference to the accompanying drawings, in which:

[0033] Figure 1 showing a coordinator system for use in federated learning of models;

[0034] Figure 2 showing a local system for use in federated learning of a model;

[0035] Figure 3 shows a federated learning system;

[0036] Figure 4 Detailed examples showing how to train machine learnable models via federated learning;

[0037] Figure 5 Showing how to accept a set of hard example descriptors for use in federated learning and / or for detailed examples included in a reference dataset;

[0038] Figure 6 A computer-implemented method of operating a local system is shown;

[0039] Figure 7 A computer-implemented method of operating a coordinator system is shown;

[0040] Figure 8 A computer readable medium including data is shown.

[0041] It should be noted that the drawings are diagrammatic only and are not drawn to scale. In the drawings, elements corresponding to elements already described may have the same reference numerals. DETAILED DESCRIPTION

[0042] Figure 1 A coordinator system 100 is shown for use in a federated learning system. The federated learning system can be used for federated learning of machine learnable models. The federated learning system can include multiple local systems. For example, the federated learning system can be used for Figure 3 The coordinator system 100 may be combined with a local system.

[0043] The system 100 may include a data interface 120 for accessing a set 030 of hard example descriptors. The hard example descriptors may include model updates and metadata about the hard examples. The system 100 may collect hard examples from local systems of the federated learning system. For example, the system 100 may be able to collect at least 100, at least 1000, or at least 10000 hard example descriptors. The system 100 may be able to collect hard example descriptors from multiple local systems, for example, the set 030 may contain hard example descriptors from multiple local systems.

[0044] The data interface 120 can further be used to access model data 040, which represents model parameters of the machine-learnable model. In addition, as described elsewhere, the coordinator system 100 can participate in the learning of the machine-learnable model, for example as a central system of a centralized federated learning system, and can be configured accordingly to aggregate model updates from corresponding local parties to obtain an updated version of the machine-learnable model and distribute the updated model to the local parties.

[0045] In general, the machine learnable model can be any machine learnable model for which federated learning techniques are available. For example, the model can be a neural network. Neural networks are also known as artificial neural networks. Examples include deep neural networks and convolutional neural networks. In this case, the set 040 of parameters can include weights of the nodes of the neural network. For example, the number of layers of the model can be at least 5 or at least 10, and the number of nodes and / or weights can be at least 1000 or at least 10000. Depending on the specific application, various known neural network architectures and other types of machine learnable models can be used. As a specific example, the machine learnable model can be an image classifier, such as for use in classifying medical images.

[0046] The data interface 120 can be further used to access a reference data set (not shown). A reference data set is a collection of typically labeled model inputs to a machine learnable model. The reference data set can be used by the coordinator system 100 to process hard example descriptors, and hard examples corresponding to hard example descriptors can be added to the reference data set, as described herein. The reference data set can be used for additional purposes, for example, some or all of the inputs in the reference data set can be used as a validation data set for testing a machine learnable model, for example, in federated learning, and / or some or all of the inputs in the reference data set can be used as training data for training a machine learnable model by the coordinator system 100. For example, the reference data set can include at least 500, at least 1000, or at least 10,000 inputs. The reference data set is preferably a high-quality data set. For example, the reference data set can be manually annotated, for example, containing only labels provided or at least checked by a human user.

[0047] In addition, Figure 1 As shown, the data interface 120 can be composed of a data storage interface that can access data from the data storage device 021, such as data 030 and / or 040. For example, the data storage interface 120 can be a memory interface or a persistent storage interface, such as a hard disk or SSD interface, but can also be a personal, local area network or wide area network interface, such as a Bluetooth, Zigbee or Wi-Fi interface or an Ethernet or fiber optic interface. The data storage device 021 can be an internal data storage device of the system 100, such as a hard disk drive or SSD, but can also be an external data storage device, such as a network accessible data storage device. In some embodiments, the data 030, 040 can each be accessed from a different data storage device, for example, via different subsystems of the data storage interface 120. Each subsystem can be of the type described above for the data storage interface 120.

[0048] System 100 may further include a processor subsystem 140 that may be configured to collect a set 030 of hard example descriptors from a plurality of local systems during operation of system 100. Processor subsystem 140 may be further configured to select a cluster of similar hard example descriptors from the set 030 of hard example descriptors. Processor subsystem 140 may be further configured to accept a cluster for use in federated learning based on model updates and metadata included in the hard example descriptors of the cluster. Processor subsystem 140 may be further configured to provide an acceptance notification for the hard example descriptors of the cluster to the local system from which the hard example descriptors were collected.

[0049] The system 100 may also include a communication interface 180 configured to communicate with a local system 162 of the federated learning system. The communication interface 180 may communicate internally with the processor subsystem 140 via data communication 126. The communication interface 180 may be arranged to communicate directly with the local system, for example, using a USB, IEEE 1394, or similar interface. As shown, the communication interface 180 may also communicate through a computer network 080, such as a wireless personal area network, the Internet, an intranet, a LAN, a WLAN, etc. For example, the communication interface 180 may include a connector, such as a wireless connector, an Ethernet connector, a Wi-Fi, a 4G or 4G antenna, a ZigBee chip, etc., suitable for a computer network. The communication interface 180 may also be an internal communication interface, such as a bus, an API, a storage interface, etc.

[0050] The system 100 may further include an output interface for outputting trained data representing the learning model. Figure 1As shown, the output interface can be composed of a data interface 120, where in these embodiments the interface is an input / output ("IO") interface, via which the trained model data can be stored in the data storage device 021. For example, during or after training, the model data defining the "untrained" model can be at least partially replaced by the model data of the trained model, because the parameters of the dynamic model, such as the weights of the neural network and other types of parameters, can be adapted to reflect the federated learning of the model. In other embodiments, the trained model data can be stored separately from the model data defining the "untrained" model. In some embodiments, the output interface can be separate from the data storage interface 120, but in general can be of the type described above for the data storage interface 120.

[0051] Figure 2 A local system 200 for use in a federated learning system is shown. The federated learning system can be used for federated learning of machine learnable models. The federated learning system can further include a coordinator system that is different from the local system 200 or can be combined with the local system 200. For example, the federated learning system can be used as a coordinator system for Figure 3 as described.

[0052] The local system 200 may include a data interface 220 for accessing model data 040, which represents model parameters of the machine-learnable model. The model data may be for example Figure 1 As described by model data 040.

[0053] The system 200 may further include a processor subsystem 240. The processor subsystem 240 may be configured to apply the machine learnable model to one or more model inputs 050, for example, to output a result to a user or use the result to control a computer controlled system.

[0054] The processor subsystem 240 may be further configured to obtain hard examples for the machine-learnable model during operation of the system 200. For example, one of the model inputs 050 to which the machine-learnable model is applied may be determined to be a hard example. Obtaining the hard example may include obtaining a manually determined or at least manually verified label of the hard example, for example, the model input may be automatically determined to be a hard example, after which a verified label may be manually provided; or the model input may be determined to be a hard example based on the mismatch between the manual label and the model output.

[0055] The processor subsystem 240 may be further configured to determine a model update from the hard example for updating model parameters of the machine learnable model. This typically uses manually verified labels. The processor subsystem 240 may be further configured to provide a hard example descriptor to the coordinator system, the hard example descriptor including the model update and metadata describing the hard example. The processor subsystem 240 may be further configured to use the hard example in federated learning of the machine learnable model after obtaining an acceptance notification for the hard example descriptor from the coordinator system.

[0056] After the machine learnable model has been updated using hard examples, the processor subsystem 240 can be configured to continue applying the now updated machine learnable model to one or more additional model inputs 050, thereby benefiting from the improved training of the model.

[0057] The system 100 may also include a communication interface 280 configured to communicate 226 with the coordinator system and / or other local systems of the federated learning system, for example, via a computer network 080. Figure 1 The same considerations and implementation options apply to the data interface 220, the processor subsystem 240, and the communication interface 280. It should be further understood that unless otherwise stated, the same Figure 1 The same considerations and implementation options of system 100 generally apply to system 200 .

[0058] As shown in the figure, the system 200 may optionally include a sensor interface 260 for obtaining sensor data 224 acquired by the sensor 072. For example, the sensor data 224 may be a medical image 224 captured by a medical imaging device 072 (e.g., a CT scanner or an MRI scanner). The system 200 may be configured to apply a machine learnable model to a model input that includes or is otherwise based on the sensor data 224. The sensor 072 may, but need not, be part of the system 200. The sensor 072 may have any suitable form, such as an image sensor, a lidar sensor, a radar sensor, a pressure sensor, a container temperature sensor, etc. The sensor data interface 260 may have any suitable form that corresponds in type to the type of sensor, including but not limited to a low-level communication interface, such as based on I2C or SPI data communication, or a data storage interface of the type described above for the data interface 220.

[0059] In some embodiments, the system 200 may include an actuator interface 280 for providing control data to an actuator (not shown). Such control data may be generated by the processor subsystem 240 to control the actuator based on the model output of the machine-learnable model. The actuator may be part of the system 200. For example, the actuator may be an electric actuator, a hydraulic actuator, a pneumatic actuator, a thermal actuator, a magnetic actuator, and / or a mechanical actuator. Specific but non-limiting examples include electric motors, electroactive polymers, hydraulic cylinders, piezoelectric actuators, pneumatic actuators, servo mechanisms, solenoids, stepper motors, and the like. In other embodiments, the system 200 may include an output interface to a rendering device, such as a display, a light source, a speaker, a vibration motor, and the like, which may be used to generate a sensory perceptible output signal based on the model output of the machine-learnable model.

[0060] Generally speaking, each system described in this specification, including but not limited to Figure 1 System 100 and Figure 2 The system 200 of can be embodied as a single device or apparatus, or in a single device or apparatus, such as a workstation or server. The device can be an embedded device. The device or apparatus may include one or more microprocessors that execute appropriate software. For example, the processor subsystem of the corresponding system can be embodied by a single central processing unit (CPU), but can also be embodied by a combination or system of such CPUs and / or other types of processing units. The software may have been downloaded and / or stored in a corresponding memory, such as a volatile memory such as RAM, or a non-volatile memory such as Flash. Alternatively, the processor subsystem of the corresponding system can be implemented in a device or apparatus in the form of programmable logic (for example, as a field programmable gate array (FPGA)). In general, each functional unit of the corresponding system can be implemented in the form of a circuit. The corresponding system can also be implemented in a distributed manner, for example, involving different devices or apparatuses, such as distributed local or cloud-based servers. In some embodiments, the system 200 can be part of a medical imaging system and can be used to improve a machine-learnable output output or used by the medical imaging system.

[0061] Figure 3 300 is shown. The federated learning system can be used for federated learning of machine learnable models MLM 340. The machine learnable model can be used as for Figure 1 or as described elsewhere.

[0062] The federated learning system 300 may include multiple local systems. For illustration purposes, three local systems LS1 201, LS2 202, and LS3 203 are shown. For example, the number of local systems may be at most or at least 10, at most or at least 20, or at most or at least 50. The local systems LS1, LS2, LS3 may be as described for Figure 2 The local system 200 is described.

[0063] The respective local system LSi may be operated by a respective organization deploying the machine learnable model MLM, for example at a respective healthcare organization, where the machine learnable model MLM may be applied to healthcare data about patients, in other words, clinical data, such as medical images. The model may be deployed at other sites that do not act as local systems. The model MLM may be deployed, for example, by an organization on a cloud or on a local site of the organization.

[0064] In addition to being used as a local system of a federated learning system, the local system LSi may be configured to apply a machine learnable model MLM, for example, to apply an image classifier MLM to a medical image to obtain a classification of the medical image. Instead of or in addition to the local system LSi applying the machine learnable model, the model may be applied by other systems of the organization operating the local system LSi.

[0065] The model MLM can be used to make inferences from patient data, e.g., to process it for the purpose of providing care, such that even when sent to the cloud (typically encrypted), it remains privacy-sensitive data. The model can be deployed at multiple sites and / or on multiple isolated cloud instances, in which case it can be used for corresponding data during care delivery in, for example, a fully identified clinical context. While healthcare organizations may be interested in contributing to and then using an improved model, in many cases they may be unwilling or legally prohibited from sharing clinical data with each other or with model developers / vendors. In particular, it may not be possible or permitted to relocate large numbers of examples from a local system LSi to a central storage device for model retraining.

[0066] In order to repeatedly enhance the machine learnable model MLM in a post-deployment scenario without providing all training examples to a centralized third party, a federated learning system 300 may be used. The federated learning system 300 may include a coordinator system CS101, for example, Figure 1The coordinator system CS may be a central system (also referred to as a central server) of the federated learning system, but this is not required. The coordinator system CS may be one of the local systems of the federated learning system. The coordinator system CS and the local systems LSi may communicate via a computer network 080 (e.g., a local network and / or the Internet) using their respective communication interfaces, as also described for Figure 1 discussed.

[0067] The local system LS1 can be configured to: obtain hard examples, typically labeled, of a machine learnable model; determine a model update from the hard examples for updating model parameters MLM 340 of the machine learnable model; and provide a hard example descriptor HEDi 360 to the coordinator system CS, which includes the model update and metadata describing the hard example.

[0068] The coordinator system CS may be configured to collect a set of hard example descriptors HEDi from a plurality of local systems LSi and accept one or more collected hard example descriptors HEDi for use in federated learning. Figure 5 A detailed example of accepting or rejecting a hard sample descriptor is given in . After acceptance, the coordinator system CS may provide an acceptance notification AN 350 for the accepted hard sample descriptor to the local system LS1 from which the hard sample descriptor HEDi was collected.

[0069] After obtaining an acceptance notification AN from the coordinator system, the local system LS1 can be configured to use hard examples in the federated learning of the machine learnable model. For example, as shown in the accompanying drawings, federated learning based on hard examples can be performed in one or more iterations, where the local system LS1 can determine the model update MUj 380 based on the hard example, and the coordinator system acting as a central server aggregates the model updates MUi received from the corresponding local systems LSi, and distributes the updated version MLM'370 of the machine learnable model to the local system for future use. In this example, the federated learning system can be a centralized federated learning system; however, the provided technology can be used in a decentralized federated learning system, or it can also be used in a centralized federated learning system in which the coordinator system CS is different from the central server. Federated learning can include verifying by the coordinator system CS whether the updated machine learnable model MLM' is sufficiently accurate, for example, having improved accuracy or accuracy exceeding a threshold, as measured against a reference data set RD. Provenance records can be retained for regulatory purposes.

[0070] Instead of or in addition to accepting hard examples for use in the federated learning itself, one or more hard examples may be accepted for inclusion in a reference dataset RD 330. The reference dataset RD may be maintained by a coordinator system CS and may be used to accept or reject future hard examples, and / or for use in federated learning, etc. In this case, the acceptance notification AN may indicate that the hard examples are for inclusion in the reference dataset, and the respective local systems LSi may provide the hard examples to the coordinator system CS upon receipt of the acceptance notification. Thus, anomalous data may be collected in a compliant manner, for example under an appropriate legal basis, and hard examples may be included in the reference dataset RD. Interestingly, using the provided techniques, only valuable / rare cases may need to be shared, performing much less data exchange while still ensuring high quality models and reference datasets.

[0071] The local system may use one or more hard examples in federated learning, and may provide one or more additional hard examples for inclusion in the reference dataset RD, but the same hard example is typically not used for both purposes.

[0072] The federated learning system 300 uses federated learning to address related privacy concerns. Following the "model to data" paradigm of federated learning, model updates MUj can be exchanged instead of collecting all training data at a centralized location. Any appropriate federated learning framework can be used. Compared to traditional deployments of federated learning, training can be extended in time, and there may not be a coordinator that selects the next local system for training. Instead, the local systems themselves can determine and share hard examples; then, these hard examples can be used in training only under certain conditions determined by the coordinator system CS (for example, only when enough similar hard examples are available).

[0073] Figure 4 A detailed but non-limiting example is shown of how to train a machine learnable model MLM 440 through federated learning.

[0074] The accompanying drawings show, for example, Figure 2 Local system 200 and / or Figure 3 The local system LS204 of the local system LSi is also shown. Figure 1 The coordinator system 100 and / or Figure 2 Coordinator system CS Coordinator system CS104.

[0075] The local system LS may be configured to apply Appl 420 (machine learnable model MLM) to the model input MI 410, thereby obtaining a model input MO 421. For example, the model input MI may be a medical image, and the model output MO may be a classification of the medical image.

[0076] The local system LS may determine one or more model inputs MI (e.g., model inputs to which the model is applied in the normal operation of the machine learnable model MLM) as hard examples. A hard example may be a model input, typically labeled and preferably manually labeled, that is determined to be potentially relevant to retraining the machine learnable model MLM, e.g., a model input may be determined to be a potentially rare case, or an erroneous or low-quality model input.

[0077] For example, based on the confidence score CO 422 calculated for the model input MI, the model input MI can be determined as a hard example. For example, if the confidence score is below a threshold, the model input MI can be selected as a hard example. The threshold can be higher than the classification threshold, for example, the model MLM can output a classification MO, while the model input MI is still considered a hard example. As shown in the figure, the confidence score CO can be determined by applying the Appl machine learnable model MLM. Many machine learnable models known per se can output a value that indicates the confidence of the model in its model output, such as class probability, etc., and can therefore be used as the confidence score CO.

[0078] However, the local system LS may also determine the confidence score by using a separate anomaly detection model. The confidence score may be calculated by the separate model based on a blind cross-check to avoid bias. For example, the confidence score may be based at least in part on information not included in the hard example descriptor, such as additional metadata. The separate model may use information not included in the model input for a more accurate assessment, such as metadata and / or historical records.

[0079] Other techniques may be used instead of or in addition to confidence scores to obtain hard examples. For example, one or more model inputs may be manually labeled as hard examples, e.g., of questionable quality. For example, considering the interpretation of CT scans as an application of the system, the radiology department may select CT images that have been judged to be of questionable quality or to have low confidence scores. One or more model inputs may also be selected as hard examples by random selection, e.g., to encourage variation in the set of hard examples or because in any case the labeling of such random cases may occur as part of a routine review of model performance. Hard examples are typically labeled manually, e.g., as part of a regular (e.g., monthly or quarterly) expert review to ensure that they are correct.

[0080] After selecting the model input MI as a hard example, the local system LS may be configured to determine metadata MD describing the hard example 462. In general, the metadata MD may include one or more features based on the model input MI and / or its label, and / or one or more features based neither on the model input nor on its label.

[0081] The metadata MD can include a descriptor of the labeler of the hard examples. This is useful because the input provided by human annotators may contain errors, sometimes referred to as "Type 1 errors".

[0082] When applied in clinical care practice, the metadata MD may further include information about the healthcare profession associated with the model input MI, e.g., the care provider involved, the person who made the diagnosis corresponding to the model input, etc. For example, the information may include the experience level of (multiple) healthcare professionals, or the degree of consistency of the labeling of hard examples, such as the STAPLE coefficients disclosed in: S. Warfield, "Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation", DOI: 10.1109 / TMI.2004.828354 (incorporated herein by reference). Such parameters are very useful for ranking and re-weighting hard examples for experts, and in particular for detecting, marking and rejecting "bad cases". The metadata may further include information about the patient associated with the hard example, e.g., information about the patient being imaged, such as patient diagnosis or laboratory test results. For example, such information can be extracted from the EMR.

[0083] The metadata MD may further include a timestamp of the model input MI, for example, a timestamp of when the medical image was captured or when the record was entered, allowing the similarity of the hard examples to be established based on this information. The metadata MD may further include a descriptor of the local system LS from which the hard examples originated. All of this information may be useful for determining clusters of similar hard examples and / or determining whether the hard examples may be abnormal. The set of metadata MD to be determined may be predetermined, for example, the set of metadata MD to be determined may be indicated in a metadata profile associated with a machine learnable model, as disclosed in patent application PCT / EP2020 / 051647 entitled “ASSOCIATING A POPULATION DESCRIPTOR WITH A TRAINED MODEL” (incorporated herein by reference with respect to the definition of the metadata profile of interest), or as disclosed in patent application US16 / 547,880 entitled “Determining features to be included in a risk assessment instrument” (incorporated herein by reference). For example, the number of metadata features may be at most or at least, at most or at least 20, or at most or at least 50.

[0084] The local system LS may further determine a model update MU 461 for updating model parameters of the machine learnable model MLM. To this end, the normal procedure of the federated learning system for determining the model update MU may be used. During federated learning, such model updates are typically exchanged in multiple iterations to update the model; in order to determine the model update MU, the technique used by the local system to determine the model update in one such iteration may be used. For example, the model update may include a gradient descent update to the model parameters. The model update MU may include an updated value or update for each trainable parameter of the machine learnable model, for example, it may include at least 1,000, at least 10,000, or at least 100,000 updated values ​​or updates. It is also possible to include only a subset of the updates or update values ​​in the model update MU: this may save storage, bandwidth, and computation time, but may still provide sufficient information for the purpose of accepting or rejecting hard examples, in particular for the purpose of comparing model updates with each other.

[0085] The local system LS may provide the coordinator system CS with hard example descriptors including the model updates MU and the metadata MD. The hard examples typically include manual labels, for example, determined as part of a review process. Therefore, the hard example descriptors may be provided to the coordinator system CS periodically, for example, after completing a review of a batch of hard examples, or after determining new hard examples one by one. The local system LS may retain the hard examples at least until it receives a notification from the coordinator system CS whether the hard examples are used for federated learning or provided as reference data.

[0086] The coordinator system CS may collect sets MU, MD of hard example descriptors from local systems of the federated learning system. The system CS may determine whether to use the collected hard examples and may therefore be referred to as an oracle platform for accepting hard examples. For example, the collection system CS may execute a procedure for determining which hard example descriptors to accept when a new hard example descriptor is received, when a predefined number of new hard example descriptors are received, or periodically. Figure 5 As discussed, this procedure may result in the acceptance of one or more hard example descriptors that have been previously processed and rejected, for example because no similar additional hard examples were available at the time. Therefore, the coordinator system CS typically retains the hard example descriptors that were not accepted at the time for later use.

[0087] Interestingly, it is possible to determine which hard example descriptors to accept or reject based on the hard example descriptors without having access to the hard examples themselves, such as model inputs and labels. This system could potentially be designed to collect rare and interesting diseases and filter out low-quality data, and thereby prevent the model's performance from degrading on a curated reference dataset.

[0088] The coordinator system LS may select a cluster HEC 451 of similar hard example descriptors from the collected set of hard example descriptors. The cluster may comprise a plurality of hard example descriptors. Clustering is typically based on metadata MD of the hard example descriptors and on model updates MU. For example, clusters may be obtained according to a clustering algorithm that partitions the set of hard example descriptors into clusters, but clusters may also be selected without clustering the remaining hard example descriptors, for example, by selecting hard example descriptors and including hard example descriptors that are within a given distance to the hard examples according to a distance metric. Figure 5 Detailed examples are discussed.

[0089] Given a selected cluster HEC, the coordinator system CS may determine whether the Acc 480 accepts the cluster based on the model updates MU and metadata MD of the hard instance descriptors of the cluster. Figure 5Detailed examples of this are discussed in If the cluster is accepted, hard examples of the accepted cluster AHEC 452 may be used in federated learning and / or included in a reference dataset maintained by a coordinator system CS (not shown in this figure).

[0090] In particular, as shown in the accompanying drawings, one or more hard examples of the accepted cluster AHEC can be used in a training operation Train 491 of the federated learning system. The training operation Train can be initiated, for example, by the coordinator system CS when the cluster AHEC has been accepted, or can be performed periodically using all cluster AHECs accepted during a past period, etc. Training Train can be performed in a known manner for federated learning itself. At least the local system LS that has access to the model input MI and the markers commonly used in federated learning can participate in the training operation. As shown in the accompanying drawings, depending on the federated learning technology used, the coordination system CS can also participate, for example, as a central system. Other local systems can also contribute corresponding hard examples and participate in the training operation Train. As is well known, in addition to newly accepted hard examples, training usually uses additional training data, such as previously accepted hard examples.

[0091] Although not shown in the figures, the coordinator system CS may select one or more further hard instances of the accepted cluster AHEC for inclusion in the reference data set, e.g. Figure 3 In this case, the coordinator system CS may provide an acceptance notification to the local system LS, which may then provide the model input MI and / or its label to the coordinator system CS.

[0092] Figure 5 A detailed but non-limiting example of how to accept a set of hard example descriptors for use in federated learning and / or for inclusion in a reference dataset is shown. For example, the techniques of this figure can be used to implement Figure 4 The clustering operation Clust 470 and the acceptance operation Acc 480 are performed.

[0093] The figure shows a collection 560 of collected hard example descriptors. The hard example descriptors may include metadata MD about the hard examples, and model updates MU from the hard examples for updating model parameters of the machine learnable model. The obtained metadata MD and model updates MU may be used to find similar hard examples distributed on the local system.

[0094] In particular, in this example, clusters of similar hard example descriptors can be determined based on determining pairwise distances between hard example descriptors Dist 571. Specifically, as shown in the figure, an affinity matrix 572 of pairwise distances, also called a similarity matrix, can be determined. To determine the pairwise distances, the metadata MD and the update MU can be combined to form feature vectors representing the hard examples, and the similarity S between these feature vectors can be calculated, for example, as the negative distance S=1-D between the vectors. For example, the distance can be L of an appropriate value n. n -distance, Levenshtein distance, Hamming distance, etc. The distance calculation Dist can also calculate different metrics of different groups of parameters and aggregate them or aggregate them. For example, the calculated distance can be expressed as a value between 0 and 1, where 1 indicates the minimum distance and the maximum correspondence.

[0095] Based on the pairwise distances 572, distance-based clustering Cl 573 can be performed to determine one or more clusters from the set 560 of hard example descriptors. As an illustration, a cluster 574 of five hard example descriptors; a cluster 575 of two hard example descriptors; and a cluster 576 of only one hard example descriptor are shown. For example, the distance-based clustering can be spectral clustering, density-based spatial clustering of applications with noise (DBSCAN) clustering, ordering points to identify cluster structures (OPTICS) clustering, etc. Non-distance-based clustering can also be used. It is also not necessary to complete the clustering, for example, if a sufficiently large cluster is found, clustering of the remaining clusters can be stopped, and a sufficiently large cluster may still be expanding through similar examples.

[0096] Given a cluster 574, it can be determined whether the CA 580 accepts or rejects the cluster. This process can be repeated for each determined cluster. Many factors can be considered in this determination. As shown in the figure, the determination can be implemented as a decision tree.

[0097] The first factor to determine may be a check of one or more cluster parameters of the cluster Cps? 581. Specifically, it may be checked whether the size of the cluster and / or the number of local systems contributing to the cluster are sufficient. A sufficiently large cluster and a sufficiently large number of contributing systems may both indicate that the likelihood of the presence of artifacts is low. The larger the number of contributing systems, the smaller the cluster size that can be tolerated. For example, if the number of local systems is at least a given threshold X (e.g., X=5), and / or if the size of the cluster is at least a given threshold Y (e.g., Y=20), the check may pass, Y. Otherwise, the check may fail, N, where the cluster may be directly rejected Rej 593.

[0098] Typically, the minimum cluster size depends on the model size and the nature of the data. The cluster size can be configured so that a sufficient number of examples are available for model retraining and to prevent overfitting. For example, the minimum cluster size can be the training set D 训练 Size of at least 10 -4 times, or at least 5 or 10. Another advantage of checking cluster parameters is that filtering small clusters can help prevent unexpected model changes that may lead to degradation of accuracy.

[0099] Another factor to determine may be an examination of the estimated impact of the cluster on the accuracy of the machine-learnable model Eff? 582. The impact may indicate the risk of model performance degradation after using the cluster for training. The impact L may result in one or more (preferably all) of the hard examples of the cluster being used for training Train 591.

[0100] To determine the estimated impact, a technique known per se can be used to estimate the impact of the training examples on the predictions made by the model (e.g., implicitly calculated from the model updates using the derived updates). For example, tracing influences can be used, as known from G. Pruthi et al., "Estimating Training Data Influence by Tracing Gradient Descent" (available at https: / / arxiv.org / abs / 2002.08484 and incorporated herein by reference), or influence functions can be calculated, as known from PW Koh et al., "Understanding Black-box Predictions via Influence Functions" (available at https: / / arxiv.org / abs / 1703.04730 and incorporated herein by reference). Using such techniques and based on a reference data set maintained by a coordinator system, the difference in model accuracy before and after the update measured for each test example can be estimated. Model updates that may degrade the performance of the model for a set of test cases can be postponed and marked as "suspicious".

[0101] Another factor in the determination may be a check of the extent to which the cluster represents the reference dataset Repr? 583. The degree of representativeness may indicate whether the cluster is likely to be a rare case. This check is preferably performed only on the metadata of the hard examples. Interestingly, the cluster may be accepted based on the fact that the cluster is sufficiently unrepresentative of the reference dataset, e.g., based on the representativeness being below a given threshold. If the cluster is sufficiently unrepresentative, N, one or more of the hard examples of the cluster may be used for training Train, and preferably one or more additional hard examples of the cluster may be included Val 592 in the validation dataset. If the cluster is not sufficiently unrepresentative, e.g., if its representativeness exceeds a threshold, it may be rejected, Rej.

[0102] If the cluster is sufficiently underrepresented, it may be an outlier and thus may be beneficial to the model, even if it causes the model's accuracy to degrade on the primary curated dataset, which may not yet contain representative cases. By using the cluster both for training and including in the reference dataset, both the model and its reference dataset may be improved.

[0103] The check Repr? can be performed by using techniques known per se to determine whether the hard examples of the cluster are in the distribution of the reference dataset. For example, a generative model of the reference dataset can be trained, wherein the check is performed by calculating the probability of hard examples generated according to the generative model. For example, the check can also be based on a feature-by-feature probability distribution of the corresponding features of the metadata. Before performing the check Repr?, the expected bounds for the meta-parameters can be derived from the reference dataset. When performing the check, it can be checked whether the hard examples are within the bounds (common cases, rejected Rej) or represent outliers (interesting cases, used for training Train and / or validation Val).

[0104] Checking Repr? can further include checking that the cluster is not an anomaly, in other words, whether it is sufficiently reasonable, for example, its anomaly score is indeed below a threshold. If the cluster is too unrepresentative of the reference dataset, it can be rejected as a possible anomaly Rej. For example, checking Repr? can check whether the representativeness of the cluster to the reference dataset is within a specific interval. This further reduces the possibility of low-quality data being accepted.

[0105] Although three separate decisions CPs?, Eff?, Repr? are shown as fixed decision trees, the corresponding acceptance / rejection methods can also be combined in different ways, for example, by applying a machine-learnable model that outputs the decision, such as a trained decision tree, a support vector machine, etc. For example, in addition to the described decision procedure, one or more custom rules can also be considered when deciding whether to accept or reject a cluster. For example, the analysis can be configured to accept or reject a cluster based on the type of medical staff and / or institution associated with the hard examples of the cluster, for example, hard examples involving top specialists or university medical centers are more likely to be rare diagnoses or procedures that have never been seen before, while cases annotated by students or at local hospitals are more likely to have quality issues.

[0106] Interestingly, based on the analysis of hard example descriptors as described herein, the coordinator system can also detect domain shift (also known as input shift). For example, it can be detected that the number of rejected hard example descriptors from a particular local system has increased. For example, the system may be faced with a consistent new (sub)population that the ML model has not previously seen (e.g., an Asian patient population present at a location within a Chinatown area, while the original model was trained only on a Caucasian population). In this case, an alert can be issued to the user of the coordinator system to take appropriate action, such as manually marking these hard examples for use in training Train and / or validation Val, or collecting more similar data.

[0107] Typically, the analysis steps Dist, Cl, CA? can be repeated, for example, each time a new batch of hard example descriptors is received, or periodically, etc. If the coordinator system accepts the cluster 574, for example, it passes the tests CPs?, Eff? and / or Repr? according to the decision procedure shown in the figure, then a federated retraining Train can be initiated. The coordinator system can trigger the model training procedure on the local system that has contributed hard examples to the accepted cluster. The local system can, for example, provide updates that can be aggregated by the coordinator system, for example, in one or more iterations.

[0108] The manner in which the analysis Dist, Cl, CA? is performed can be further refined based on feedback on the decision whether the cluster is accepted. For example, the decision to accept or reject a cluster may be manually reviewed or verified by later acquired data, such as later added to a reference data set. For example, conventional expert verification can be performed, wherein, for example, when the confidence level is below a threshold or on a routine basis, the acceptance / rejection decision can be evaluated by a human expert to ensure quality control. For example, feedback can be based on future events (e.g., for time series), later known benchmark truths (e.g., for pathology results), differences between inference results and automatically detected radiologist reports, etc. Feedback can be used to automatically adjust the criteria (in other words, parameters) of the analysis, such as the minimum cluster size, the minimum number of local systems, and / or the minimum or maximum non-representativeness used, or the parameters of the machine-learnable model used to make decisions.

[0109] Additionally, as discussed elsewhere, based on the analysis, the (multiple) hard examples of the accepted clusters can be used to train Train or include Val in the reference dataset via federated learning. Hard examples that are not in the accepted clusters, such as hard examples that were rejected Rej, or hard examples that were not included in the analyzed clusters at all, can be retained and used in later analysis. This may be helpful in expanding the training and test datasets in the future, for example, hard examples of clusters 576 that are considered too small by the inspection CP? can be combined in a later cluster Cl with other (e.g., newly obtained) hard example descriptors in a new cluster, which may be large enough and can then be accepted.

[0110] Figure 6 A block diagram of a computer-implemented method 600 of a local system operating a federated learning system is shown. The federated learning system is used for federated learning of machine learnable models. The federated learning system may include a coordinator system. The method 600 may correspond to Figure 2 System 200, Figure 3 System 201, or Figure 4 However, this is not limiting, as method 600 may also be performed using another system, apparatus, or device.

[0111] The method 600 may include accessing 610 model data representing model parameters of a machine learnable model in an operation titled “Access Model.” The method 600 may include obtaining 620 hard examples for the machine learnable model in an operation titled “Obtain Hard Examples.” The method 600 may include determining 630 model updates from the hard examples for updating model parameters of the machine learnable model in an operation titled “Determine Model Updates.” The method 600 may include providing 640 a hard example descriptor to a coordinator system in an operation titled “Provide Updates + Metadata,” the hard example descriptor including the model update and metadata describing the hard example. The method 600 may include using 650 the hard example in federated learning of the machine learnable model after obtaining an acceptance notification for the hard example descriptor from the coordinator system in an operation titled “Use Hard Examples.”

[0112] Figure 7 A block diagram of a computer-implemented method 700 of operating a coordinator system of a federated learning system is shown. The federated learning system may be used for federated learning of machine learnable models. The federated learning system may include multiple local systems. The method 700 may correspond to Figure 1 System 100, Figure 3 System 101, or Figure 4 However, this is not limiting, as method 800 may also be performed using another system, apparatus, or device.

[0113] The method 700 may include accessing 710 a set of hard example descriptors in an operation titled “Accessing Hard Example Descriptors.” The method 700 may include collecting 720 the set of hard example descriptors from a plurality of local systems in an operation titled “Collecting.” The method 700 may include selecting 730 clusters of similar hard example descriptors from the set of hard example descriptors in an operation titled “Clustering.” The method 700 may include accepting 740 the clusters for use in federated learning based on model updates and metadata included in the hard example descriptors of the clusters in an operation titled “Accepting.” The method 700 may include providing 750 an acceptance notification of the hard example descriptors for the clusters to the local systems from which the hard example descriptors were collected, in an operation titled “Providing Acceptance Notification.”

[0114] It should be understood that, generally speaking, Figure 6 Method 600 and Figure 7 The operations of the method 700 may be performed in any suitable order, such as serially, simultaneously, or a combination thereof, subject to, for example, input / output relationships requiring a particular order, where appropriate. Some or all of the methods may also be combined, for example, the method 600 of operating a local system and the method 700 of operating a coordinator system may be implemented to run simultaneously at a single system.

[0115] The method can be implemented on a computer as a computer-implemented method, as dedicated hardware, or as a combination of the two. Figure 8 As shown, computer instructions, such as executable code, can be stored on a computer readable medium 800, for example, in the form of a series of machine-readable physical marks 810 and / or as a series of elements having different electrical properties or values, such as magnetic properties or values, or optical properties or values. The medium 800 can be temporary or non-temporary. Examples of computer readable media include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Figure 8 An optical disc 800 is shown. Alternatively, the computer readable medium 800 may include model data 810 representing model parameters of a machine learnable model trained by a federated learning system as described herein.

[0116] Examples, embodiments or optional features, whether or not indicated as non-limiting, should not be construed as limiting the invention as claimed.

[0117] It should be noted that the embodiments mentioned above illustrate rather than limit the present invention, and those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference numerals placed between brackets should not be interpreted as limiting the claim. The use of the verb "comprise" and its inflections does not exclude the presence of elements or phases that are not those described in the claims. "One (a)" or "an" in front of an element does not exclude the presence of multiple such elements. When in front of a column of elements or a group of elements, expressions such as "at least one of ..." represent the selection of all elements or any subset of elements from the column or group. For example, the expression "at least one of A, B, and C" should be understood to include only A, only B, only C, both A and B, both A and C, both B and C, or all of A, B, and C. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a device claim that lists several means, several of these means can be embodied by the same hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A federated learning system (300) for federated learning of machine learnable models, wherein the federated learning system comprises a plurality of local systems (201, 202, 203) and a coordinator system (101), The local system (201) among the plurality of local systems is configured as follows: obtaining hard examples for the machine learnable model; determining a model update from the hard examples for updating model parameters (340) of the machine learnable model; providing a hard example descriptor to the coordinator system, the hard example descriptor comprising the model update and metadata describing the hard example; as well as upon obtaining an acceptance notification for the hard example descriptor from the coordinator system, using the hard example in the federated learning of the machine learnable model; The coordinator system (101) is configured as follows: collecting a set of hard example descriptors from the plurality of local systems; selecting a cluster of similar hard example descriptors from the set of hard example descriptors; accepting the cluster for use in the federated learning based on the model update and the metadata included in the hard example descriptor of the cluster; as well as An acceptance notification of the hard example descriptors for the cluster is provided to the local system from which the hard example descriptors were collected.

2. The system (300) of claim 1, wherein the local system is further configured to apply the machine learnable model to a model input, such as a medical image, the model input comprising healthcare data about a patient.

3. A system (300) according to any of the preceding claims, wherein the coordinator system is configured to accept the cluster based on one or more of: the size of the cluster, the number of local systems contributing to the cluster, the estimated impact of the cluster on the accuracy of the machine learnable model, and the extent to which the cluster represents a reference data set.

4. The system (300) of claim 3, wherein the coordinator system is configured to accept the cluster based on the cluster being insufficiently representative of the reference data set.

5. A system (300) according to any of the preceding claims, wherein the acceptance notification provided to the local system is for using the hard example in the federated learning, and wherein the coordinator system is configured to provide another acceptance notification for another hard example descriptor for another hard example of the cluster for including the another hard example in the reference data set.

6. A system (300) according to any of the preceding claims, wherein the local system is configured to obtain a model input for the machine learnable model; calculate a confidence score for the model input; and select the model input as the hard example based on the confidence score being below a threshold.

7. The system (300) of any of the preceding claims, wherein the metadata comprises data that is neither derived from model inputs nor from labels of the hard examples.

8. A system (300) according to any of the preceding claims, wherein the metadata includes one or more of the following: a timestamp, a descriptor of the local system, a descriptor of a labeler of the hard example, a degree of consistency in the labeling of the hard example, information about a healthcare professional associated with the hard example, information about a patient associated with the hard example, and information about image acquisition settings of the hard example.

9. The system (300) according to any of the preceding claims, wherein the coordinator system is configured to obtain feedback about accepted clusters and to update criteria for accepting clusters based on the feedback.

10. The system (300) according to any of the preceding claims, wherein the coordinator system is configured to retain hard example descriptors that are not included in the selected cluster or are included in the unaccepted cluster, and to use the retained hard example descriptors in another cluster selection and acceptance.

11. A local system (200) for use in a federated learning system according to any one of the preceding claims, wherein the federated learning system is for federated learning of machine learnable models, wherein the federated learning system comprises a coordinator system, wherein the local system comprises: a data interface (220) for accessing model data (040), the model data representing model parameters of the machine learnable model; The processor subsystem (240) is configured to: obtaining hard examples for the machine learnable model; determining a model update from the hard examples for updating the model parameters of the machine learnable model; providing a hard example descriptor to the coordinator system, the hard example descriptor comprising the model update and metadata describing the hard example; as well as After obtaining an acceptance notification for the hard example descriptor from the coordinator system, the hard example is used in the federated learning of the machine learnable model.

12. A coordinator system (100) for use in a federated learning system according to any one of claims 1 to 10, wherein the federated learning system is used for federated learning of machine learnable models, wherein the federated learning system comprises a plurality of local systems, wherein the coordinator system comprises: a data interface (120) for accessing a set of hard example descriptors; The processor subsystem (140) is configured to: collecting the set of hard example descriptors from the plurality of local systems; selecting a cluster of similar hard example descriptors from the set of hard example descriptors; accepting the cluster for use in the federated learning based on the model update and metadata included in the hard example descriptor of the cluster; as well as An acceptance notification of the hard example descriptors for the cluster is provided to the local system from which the hard example descriptors were collected.

13. A computer-implemented method (600) of operating a local system of a federated learning system according to any one of claims 1 to 10, wherein the federated learning system is for federated learning of machine-learnable models, wherein the federated learning system comprises a coordinator system, wherein the method comprises: accessing (610) model data, the model data representing model parameters of the machine learnable model; obtaining (620) hard examples for the machine learnable model; determining (630) a model update from the hard examples for updating the model parameters of the machine learnable model; providing (640) a hard example descriptor to the coordinator system, the hard example descriptor comprising the model update and metadata describing the hard example; as well as After obtaining an acceptance notification for the hard example descriptor from the coordinator system, the hard example is used (650) in the federated learning of the machine learnable model.

14. A computer-implemented method (700) of operating a coordinator system of a federated learning system according to any one of claims 1 to 10, wherein the federated learning system is used for federated learning of machine learnable models, wherein the federated learning system comprises a plurality of local systems, wherein the method comprises: accessing (710) a set of hard example descriptors; collecting (720) the set of hard example descriptors from the plurality of local systems; selecting (730) a cluster of similar hard example descriptors from the set of hard example descriptors; accepting ( 740 ) the cluster for use in the federated learning based on the model update and metadata included in the hard example descriptors of the cluster; as well as An acceptance notification of the hard example descriptors for the cluster is provided (750) to the local system from which the hard example descriptors were collected.

15. A transitory or non-transitory computer-readable medium (800) comprising data (810), the data representing: Instructions which, when executed by a processor system, cause the processor system to perform a computer-implemented method according to claim 13 and / or claim 14; and / or Model data, wherein the model data represents model parameters of a machine learnable model trained by the federated learning system according to any one of claims 1 to 10.

Images